[OSM-talk] [OSM-dev] Managing large XML files aka osm.xml

[Artem Pavlenko]

On 2 Jan 2008, at 16:37, Stefan de Konink wrote:

> On Wed, 2 Jan 2008, Artem Pavlenko wrote:
>
>> David Siegel implemented support for libxml2 parser in Mapnik, check
>> it out :
>
> I was able to load the entiere planet.osm into MonetDB4 (XML), but
> incompare to the SQL version it is not optimal. And one may wonder  
> why he
> wants to use an text representation instead of a processed binary!

Sorry, if it was not clear. Mapnik has got three xml parsers to  
choose from (at compile time) : tinyxml, spirit and now libxml2.
Only libxml2 based supports advanced features like entities.  Mapnik  
is using XML parsers to load data/styles definition file,
not actual osm data.

>
> For plain XML processing people should probably stick with MSXML  
> for the
> time being. The performance of it is probably the best one out there.
I would doubt it. BTW, all three parsers mentioned above run happily  
on win32.

> We
> did some XSLT processing based on the tiles at home stuff, and it is not
> funny that an 8 times as powerful machine gets beaten.


> I wonder how much time it will take for each run on an 8GB machine
> processing the current planet.osm (61GB).

I understand but I never decompress planet :

:> osm2pgsql planet.osm.bz2


>
>
> Stefan
>

Artem