[OSM-dev] [osmosis-dev] Proposal for a multithreaded PBF reader

[Paul Norman]

On 4/29/2015 9:55 AM, Martijn van Exel wrote:
> If osmosis is the reference implementation, is there a reason why it
> doesn't seem to leverage this block structure to speed up reading? Or
> does it?
Osmosis has the --read-pbf-fast task which allows multiple worker threads.

Personally, I've never considered osmosis to be a reference 
implementation of the OSM PBF format.

The "threading" branch of osm2pgsql included multi-thread parsing 
support. It never got merged as there were core dumps which occurred 
about 10% of the time when importing a full planet, and they never got 
isolated and it was felt the C++ rewrite would make memory management 
debugging less of a huge time sink. The real gains from the threading 
branch were not multi-threaded PBF reading, but more concurrency in the 
geometry processing and database parts.