[OSM-dev] Problems with planet files and bz2

Eric Wolf ebwolf at gmail.com
Tue Mar 1 21:54:25 GMT 2011


Has anyone tried reading the bz2 compressed planet file?

I'm currently trying to figure out if there is something wrong in Python's
bz2 module, zlib, or the planet files themselves.

The python BZ2file module lets you read and write bz2 files like normal
files without decompressing the whole thing. This is key, especially when
dealing with full-planet which is 500GB+ uncompressed. However, I've found a
few instances where different people have had issues with the OSM planet
files and the bz2 module quitting after reading the first 900,000 bytes. Has
anyone had success with this?

FYI: The script works perfectly with a 165MB compressed file from Cloudmade.

-Eric

-=--=---=----=----=---=--=-=--=---=----=---=--=-=-
Eric B. Wolf                           720-334-7734
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20110301/f375363d/attachment.html>


More information about the dev mailing list