[OSM-dev] amazon AMIs for a full blown openstreetmap.org-like server

[Stefan de Konink]

Op 08-01-10 15:39, jamesmikedupont at googlemail.com schreef:
>> Tiny would have to be an osmtile in binary format, agreed?
>
> Yes, well if you sort the data properly then mapnik would just render
> the data in a sax callback,  right?

Jup, but you will never get better results of that storing in parts than 
having it stored in one file and having an index to it. So in that 
respect you have to mmap the file, for example a post/plane parser, 
extract the childs, send the childs to Mapnik.

I have tested this using MonetDB/XML, that wasn't really a succes ;) 
Either way you need something that stores that creates an index by a 
bounding box. You can use rtree's for that sure. But we already have 
shown that sideway cracking gives far better results. [google: sideway 
cracking databases]

>> You don't want to store it in XML, it will get huge... and requires
>> again the parsing overhead. And as I pointed out before, we prefer to
>> render 64 tiles at a time.
>
> So, you would have 64 tiles of data in one page.... How big is that?
> the size of a city.

All depending on the zoom level :)

> Well I have gotten the parsing down to very small using sax2, it is
> just a bit slower than reading the file.
> If you sort the xml data topologically and pack it all in, it could be
> very easy to process.

But how are you going to store this, for example if we take a *perfect* 
filesystem. Will you store everything by nodeid, wayid, relationid? OR 
will you store everything that is in a tile, in a file marking the tile, 
having the same structure as the tile server uses?


> I will have to work through the entire pipeline of processing to see
> if it fits my model, I risk having to put my foot in my mouth here....

Sure :) Just start with some smaller country and try to prevent 
bottlenecks. The worse thing that can happen is TomH saying you are a 
'Bonker' (whatever that may mean).


Stefan