[OSM-dev] New OSM binary fileformat implementation.

Scott Crosby scrosby06 at gmail.com
Mon Aug 9 18:40:12 BST 2010


On Wed, Apr 28, 2010 at 12:02 PM, Scott Crosby <scrosby06 at gmail.com> wrote:
> Hello!
>
> I would like to announce code implementing a binary OSM format that
> supports the full semantics of the OSM XML. It is 5x-10x faster at
> reading and writing and 30-50% smaller; an entire planet, including
> all metadata, can be read in about 12 minutes and written in about 50
> minutes on a 3 year old dual-core machine. I have implemented an
> osmosis reader and writer and have enhancements to the map splitter to
> read the format. Code is pure Java and uses Google protocol buffers
> for the low-level serialization.
>
> Comparing the file sizes:
>
>  8.2gb   planet-100303.osm.bz2
> 12  gb   planet-100303.osm.gz
>  5.2gb   planet-omitmeta.bin
>  6.2gb   planet.bin
>

Some newer results.

I have a modification to dense nodes to support storing tags. This
results in a 500mb smaller files for an entire planet. Sizes are now:

  4.7gb  planet-omitmeta.bin
  5.7gb  planet.bin

Results when dropping the resolution to 1m precision:
 3.8gb planet-granularity=10000-omitmeta.bin

This reduced resolution format may be a good choice for distributing
OSM snapshots to non-editors.

I have tested the new file on the cloudmade extract of rhode_island;
converting 122MB of uncompressed XML to and from binary format. The
result is bytewise identical to the source file except for the osmosis
version number at the top.

Scott



More information about the dev mailing list