[OSM-dev] [osmosis-dev] Proposal for a multithreaded PBF reader
penorman at mac.com
Wed Apr 29 17:27:13 UTC 2015
On 4/29/2015 9:55 AM, Martijn van Exel wrote:
> If osmosis is the reference implementation, is there a reason why it
> doesn't seem to leverage this block structure to speed up reading? Or
> does it?
Osmosis has the --read-pbf-fast task which allows multiple worker threads.
Personally, I've never considered osmosis to be a reference
implementation of the OSM PBF format.
The "threading" branch of osm2pgsql included multi-thread parsing
support. It never got merged as there were core dumps which occurred
about 10% of the time when importing a full planet, and they never got
isolated and it was felt the C++ rewrite would make memory management
debugging less of a huge time sink. The real gains from the threading
branch were not multi-threaded PBF reading, but more concurrency in the
geometry processing and database parts.
More information about the dev