[OSM-dev] Optimal free compression algorithm for OSM XML data

[Nick Hill]

After a brief discussion at the developers conference in Oxford regarding 
compression algorithms for planet.osm, I decided to perform a series of tests 
using different algorithms available in free software.

I chose the latest planet-070509.osm.

The following algorithms:
bzip2
gzip -6
zip (pkzip)
gzip -9
7z

I determined CPU time to compress, uncompress and compressed file size:


                   Time   Time     File Size
                   pack   Unpack   Mebibytes

bzip2             7144   347      246
7-zip             4436   114      218.6
gzip -6           186     36      345
gzip -9           474     36	  322
zip  Failed - file size too large

 From the above, the user will likely have the best experience using 7-zip. 
Smallest download size, whilst costing only 1/3rd of bzip2 to unpack. For OSM, 
7-zip is less costly, taking only 62% of the time it takes to compress the 
current bzip2.

gzip unpacked much faster than my hard drive could write, 7zip was a little 
faster than my hard drive.

These tests were performed with an AMD Athlon 2200+ 32 bit running at a clock 
speed of 1498Mhz from the year 2002.