[OSM-dev] OSM History Splitter

Peter Körner osm-lists at mazdermind.de
Wed Aug 10 11:03:34 BST 2011


Hi devs,

some months ago I announced my History Splitter and the availability of 
extracts. In the last months I've completed my work and implemented a 
second cutting-algorithm. While the --hardcut algorithm will stay the 
default as a, the --softcut algorithm features the following attributes:

- version-completeness (if one version is inside the bbox, all versions 
are included in the extract)
- way-completeness (ways are not cropped but included as they are in the 
database)
- way-reference-completeness (all nodes of ways touching the bbox are 
included)
- relation-completeness (relations are not cropped, they may however 
have children which are not included inside the extract)
- cascading-relations (all super-relations of relations touching the 
bbox are included in the extract)

those features are implemented using a two-pass process. It runs 
single-threaded, so to use multiple cores you are required to run 
multiple independent instances of the program. A python script that is 
able to control this parallelization is included, however multiple 
simultaneous splits increases the number of disk-seeks, which possibly 
outruns the speedup gained through distributing the CPU-Load to multiple 
cores.

The tool is, without changes, suitable for normal planet-extractions, 
too. By choosing the sequence and parallelization carefully, it's 
possible to accomplish the specific task of extracting a bbox or a 
polygon in 1/3 of the time required by osmosis.

The source and building instructions are available at github:
https://github.com/MaZderMind/osm-history-splitter

If you have bug reports or problems compiling or running the splitter, 
don't hesitate to mail me directly at osm at mazdermind.de.

Peter




More information about the dev mailing list