[OSM-dev] AND upload almost ready

[Martijn van Oosterhout]

On 8/25/07, Marc Kessels <marc at kessels.name> wrote:
> another way to get this solved, is by subdividing the data in several
> pieces (each 10 MByte or so), and process them one-by-one. I am still
> planning to improve the and2osm/2AND program, such that it is much less
> memory consuming. This would also enable an output which is not
> overlapping, apart from the connecting nodes. These nodescould be
> flagged, and need some attention during/after the import in osm.

Hmm, I noticed that by using a different memory allocator (tcmalloc)
that the memory usage was significatly reduced. Note that for the
purposes of the AND import it's not all that useful for the program to
produce many output files, since that all have to be processed anyway
to exclude the protected areas, and if they share nodes but that isn't
known beforehand that somewhat increases complexity.

Once the datafiles have been produced it doesn't matter how long it
took to produce them. Similarly, the order we upload stuff isn't all
that relevent either, since we still have to upload all of it.

(Note: I've committed various changes to SVN, including a change to
make the output be in osmChange format. This is the format I've
settled on for handling the necessary changes).

Have a nice day,
-- 
Martijn van Oosterhout <kleptog at gmail.com> http://svana.org/kleptog/