[OSM-dev] planet_osm2txt with planet-060818?

[Joerg Ostertag (OSM Munich/Germany)]

On Monday 21 August 2006 14:37, Dean Earley wrote:
> >>> Was anyone successfull with
> >>> ./utils/osm-pdf-atlas/planet_osm2txt.pl planet-060818-a.osm.bz2
> >>> Any hints on how to do this on a machine with only 2GB of Memory?
> >>
> >> I guess the parsers will need to be rewritten to use SAX now rather
> >> than the DOM (read all and iterate) method. I happily parsed a
> >> 2.6GB planet.osm in about 2 minutes using MSXML (in VB)
> >
> > Would you mind sharing so more info on this approach. I'd be
> > interested in taking a look at some of the data in the planet file.
>
> Basically, it is an event based parser that keeps as little in memory as
> necessary.
>
> It notifies my code whenever a complete tag (start of and end of) has
> been found and any textual content.
>
> This is a sample of the VB code I used:

...

>
> It does require a syntactically valid XML file and unfortunately, this
> never completed as I ran out of disk space for the target db :)
>
> /me runs off to his flame proof bunker...

Well, this sounds like the same as it is done in planet_osm2txt.pl. It's also 
event driven. But reading all nodes into memory sucks a lot of memory for the 
current planet.osm File. What I'm testing is to only suck in all nodes for a 
given region. This already helped with processing the file in check_osm.pl to 
redure the memory imprint.

-

Joerg



-- 
Jörg (Germany, Munich)

http://www.ostertag.name/
TeamSpeak2: ts2.ostertag.name, user: tweety, Channel: "GPS Drive"
irc://irc.oftc.net/#osm