[OSM-dev] API 0.5 is on the way

Frederik Ramm frederik at remote.org
Tue Sep 11 00:21:47 BST 2007


Hi,

(posting to talk & dev with reply-to set to dev, dunno if that works)

   those of you who also follow the "dev" list have seen that Gabriel 
Ebner has recently presented a working prototype of "OSM without 
segments", and I have created "OSM with relationships".

We have combined those two into a shiny new "OSM 0.5" which we hope to 
deploy in early October, provided that we don't encounter major obstacles
and provided of course we have the support of the community and those
who manage our essential servers (and take the blame if something 
breaks...).

The changeover from 0.4 to 0.5 will do away with segments, and at once
remove all complexity associated with them - "unordered ways", "unwayed
segments", and so on. The planet file will shrink by 40%, and API 
performance, especially on bounding box downloads, will be greatly 
improved as we're removing one level of indirection. Relationships
will be rather unimportant in the beginning but their existence allows
a great deal more tagging flexibility (turn restrictions, superways, 
all that stuff).

The changeover will also mean that almost every piece of software will
have to be modified to work with 0.5. You can get away with not under-
standing relationships for the time being (because there's not much
that actually relies on them), but the removal of segments affects 
almost everything. 

Nearly all widely-used applications have already been ported to 0.5
(the API of course, JOSM, osm2pgsql and with it Mapnik rendering, 
the planet file dump, and others; see wiki page for an up-to-date
list).

We are providing two kinds of migration scripts for the changeover.
One works on the current MySQL database schema and is included in the
Rails migration mechanism so that the database can be upgraded easily.
The other is a stand-alone tool to convert 0.4-style OSM files to
0.5-style (and, in a very limited fashion, vice versa) to allow you to 
use "legacy" applications until they natively support 0.5.

If you have written any piece of software that works with OSM data, 
either accessing the API or working on OSM XML files, please consider 
changing it to be compatible with 0.5. All you need should be on the
wiki page, including access information for a live API server running
0.5.

Also, if you have developed a JOSM plugin that deals with segments,
it will have to be modified to work with 0.5.

The changeover from 0.4 to 0.5 will cause a server downtime of probably
about one day. Due to the structural nature of the changes, it is un-
likely that you will have software that can handle 0.4 and 0.5 at the
same time; you will have to use one version up to the changeover day,
and another version after that.

The ongoing TIGER import will not be harmed (apart from having to
pause for a day); the OSM files that have been prepared can simply be
converted and then bulk-uploaded as before.

Nodes and their history remain unchanged, but the history information 
linked to segments and ways will be removed in the process. This is 
unavoidable since the new model is structurally different from the old 
model, and many historic ways simply cannot be represented in the new 
model. 

Current ways will be split into several parts where the new model cannot
accomodate their complexity, but converting historic ways would not 
only be a computational challenge (we would have review all historic
versions of a way and its segments and merge them into individual historic
ways), it would also falsify history because we would have to split 
ways that were not split in the past. (The only correct solution would
be to somehow return historic ways in 0.4 format and new ways in 0.5 
format but that would be a nightmare for clients and servers alike.) 

We will investigate ways to make the existing database available in a 
read-only fashion in order to allow access to pre-0.5 history: In any 
case the existing history data will be archived; nothing will be lost.

Almost everything you need to know about 0.5 is on the wiki page:

http://wiki.openstreetmap.org/index.php?title=OSM_Protocol_Version_0.5

You can also download a working 0.5 JOSM version from there (in fact,
you are encouraged to do so and find any bugs we've left over!) and
connect to our 0.5 test implementation with it. There's a database
schema and database dump in case you are running your own MySQL
database and want to try the 0.5 structure. If you need access to the
full MySQL data in order to test your software against it, you can
have an account on the machine hosting the test database. There's also
a 0.5 version of the latest planet file there for download.

We'd like to encourage everybody to play with what's already there, 
think about potential problems and how to deal with them, change
existing scripts and software, and ask questions if you are unsure
about details.

API 0.5 is now past the stage of a mere proposal; a lot of software
has already been written and tested. If someone found a serious flaw
in the concept we'd have to change it obviously, but other than that
we'd really like to get it off the ground without too much delay, to
free up our minds for some of the other interesting things to do ;-)

As I said initially, the goal is to iron out any problems we find,
adapt the remaining crucial applications, and then get it running
three or four weeks from now.

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frederik at remote.org  ##  N49°00.09' E008°23.33'





More information about the dev mailing list