[OSM-talk] Historic Mapping needs help Now!

[Nic Roets]

On Mon, Aug 17, 2009 at 11:00 PM, Mikel Maron <mikel_maron at yahoo.com> wrote:

>
> Look forward to some good ideas.
>

Here is my plan :
* Create a separate server that starts with a very old planet file and then
processes the diffs.
* Use Mercator projection, divide the map into tiles of fixed size of
approximately 2km x 2km. Enumerate the tiles according to the Hilbert curve.
This scheme works well for rendering cities areas at high zoom most queries
(which are the vast majority of queries). The lack of performance on other
queries will be more than compensated for (as gosmore proves).
* To prevent wasting disk space on ocean, desert and tiles, choose a
function that "maps" the many tiles unto few buckets (hash function).
* For each bucket allocate a few kilobytes of disk space. We need to mark it
as empty, but there is a nice Linux trick of marking parts of a file as zero
and delaying the actual writing until that part of the file is actually
modified.
* Now add all the data to the file i.e. write the info into the buckets. If
a bucket K gets full, use K+1. The structure(s) used for nodes have space to
point to the ways that use them. The ways are store in the buckets,
preferably next to one of the nodes used in it. Ways point to the nodes that
they depend on. Use a similar scheme for relations.
* Every node, way and relation must have a start time, end time (initially
infinite) and a pointer to its successor.
* Now process the diff files. Old nodes, ways & relations aren't removed.
Instead their end times and successors are updated.
* To find old nodes, ways and relations we need an index that maps their id
numbers to where they currently are in the file. These indices are not
stored in the file (RAM?). Note that I'm only interested in serving bounding
box queries for any given time T. I'm not interested in serving queries like
"Where was node N at time T ?" So old versions of this index is not
necessary.
* To serve a query, iterate over all the tiles and look in the buckets. If a
bucket K is full, look in K+1, until you found one with unused space.
* Over time many buckets will get full and this will mean that the server
will need to linearly search larger and larger parts of the file. When this
performance degradation happens (say after several months), we just create a
new file. The server then looks at the T values of the queries to choose the
correct file.

Then the T parameter will appear in the permalink, it will get passed to
mod_tile etc. and you will get a map of burning man 2008.

And the possibility of differential rendering also exist.


> Thanks
> Mikel
>
> _______________________________________________
> talk mailing list
> talk at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/talk/attachments/20090818/4cec8475/attachment.html>