[OSM-dev] HTTP OSM Server

[Dick Marinus]

Hi Folks,

I'm currently investigating if I could create an OSM server by only
using HTTP. The advantages are:
- There are a lot of fast HTTP mirrors
- The OSM server is quite busy and not reliable for large requests

I was thinking about generating indexes to the (~200 kB) bz2 blocks in
planet-osm.bz2 with the starting node id and way id.

Then create an index at zoom 17 tiles with way id's.

If you would like to download a certain "data tile" you'll lookup in the
tile-way index which way id's you'll need. The bz2 blocks will be looked
up in the bz2-index-ways and finally the nodes can be downloaded by
using the bz2-node-ways index.

The bz2 blocks can be downloaded by using HTTP range requests.

Currently I'm writing the index generator in C by using libbz2 and
libexpat. As it takes a while to parse an 5GB bz2 file I'm trying to be
able to resume the process half way. I'm thinking of using libcurl to do
the HTTP work for me.

Everything seems to work so far, I can extract the bz2 block boundaries
by using the libbz2 api by using a dirty trick and I can seek into the
planet.bz2 and continue decompressing.

Yesterday I've read on www.openstreetmap.nl that Oxilion (a generous OSM
hoster) will be hosting the planet.osm.bz2 with a date as filename.
Which is great for my OSM HTTP implementation.

A future request would be if the XML file could be exported in a
"special" order, ordered (sorted) by data tile... That might also
improve file compression.

Well, I don't have any questions but I'm open to suggestions :-)

Please let me know If you like the idea.

Dick