[OSM-dev] XAPI and other solutions

[Pierre-Alain Dorange]

Serge Wroclawski <emacsen at gmail.com> wrote:

> > Yes, but they are huge...
> > My original place=* for france is 50 MB (not compressed).
> 
> 50 meg doesn't seem so big.

Yes 50MB is fine. 2 GB (compressed) is huge.

> > France extract if 2 GB compressed, i don't think my python script was
> > able to handle such a huge data.
> 
> What kind of processing were you doing? I'm assuming some kind of XML
> (unless you were using the protobuffs). If thats the case, you can use
> a simple SAX parser. If you need an example, I'll send you one.

Yes i preprocess XML to extract city data (place=town, place=village,
etc...) to match them with an official source with population data.
For that i uses elementTree lib (python) that's fine.
I don't know about SAX, but i just see that contrary to DOM technique it
do not need to load the whole XML into memory, so it can be helpful if i
swtich to huge XML data file.
I'll check.

-- 
Pierre-Alain Dorange
OSM experiences : <http://www.leretourdelautruche.com/map/>