[OSM-dev] Storing way/relation start offset in PBF file

[Scott Crosby]

> There's a way you could split it up a lot faster. There's no need to
> decompress each block. Just copy the blocks that contain entities that
> you're interested in.

Just a data point for doing this:

I implemented a quick Java program that CRC's every block in a PBF
file. It decompresses each block but does not parse them. On my 3 year
old laptop, It can run through a 4 gigabyte PBF file in 2.5 minutes.

Scott