[OSM-dev] Optimal free compression algorithm for OSM XML data
Jon Burgess
jburgess777 at googlemail.com
Thu May 10 22:27:22 BST 2007
On Thu, 2007-05-10 at 15:53 +0100, Nick Hill wrote:
> Hello Shaun
>
> Thank you for the pointers for Mac users and 7-zip.
>
> I have uploaded a copy of the current planet.osm as 7z, where I have further
> increased compression using bigger dictionary etc.
>
> planet files are at:
> http://planet.openstreetmap.org/
>
> The URL for the current planet.osm in 7z format is:
> http://planet.openstreetmap.org/planet-070509.osm.7z
>
> The new file is 183Mb vs Bzip2 235Mb.
>
> You should find both downloading and unpacking the new file faster. Please try
> it and give feedback.
>
Works OK for me using osm2pgsql like this:
$ 7za -so x planet-070509.osm.7z | osm2pgsql /dev/stdin
I may be able to integrate a 7zip reader into the osm2pgsql input
routines so that the file can be read directly.
The CPU time to decompress .7z vs .bz2 is slightly better, though not by
nearly as much as your measurements. Perhaps the 4MB cache on my Core 2
Duo E6600 helps bzip2.
Time
Unpack
bzip2 86
7-zip 72
gzip 25
I think the increased compression ratio and faster performance are
definitely good reasons to choose .7z
Jon
More information about the dev
mailing list