[osmosis-dev] --used-node performance and a possible way to improve it

[Frederik Ramm]

Hi,

Jochen Topf wrote:
> I have thoght about these things quite a lot, too. I agree with Igor that we
> are bumping into the limits of what the current Osmosis pipelining model can
> do.

We seem to be reaching a point where people are creating specialist 
programs to do things that are traditionally done with Osmosis - when 
until recently it was the other way round, you needed less and less 
specialist programs because Osmosis did it all.

For example, there's "osmconvert" 
(http://wiki.openstreetmap.org/wiki/Osmconvert), a tool that can 
read/write OSM, apply diffs, do bounding box, polygon, and tag 
filtering, and people published benchmarks in the forum where that 
program was almost 5 times as fast as Osmosis. The program is certainly 
not a marvel of software design (6000 lines of rather unreadable C code 
in one source file, and it uses NO LIBRARIES except lz, not even an XML 
parser or a geo library or Protobuf code - all hand-coded) - but still, 
it seems to have a growing user base because it is fast and can be 
easily installed/compiled everywhere. (Also, the tag filtering seems to 
be a bit easier than the tee/tf/used-way/used-node/merge orgy that you 
currently need to perform in Osmosis if you want to extract something 
like "all forests".)

It used to be that, compared to other means of achieving the same, 
Osmosis was perhaps minimally more clumsy and perhaps minimally slower, 
but I always used, and still use, Osmosis for everything because it was 
so convenient to have this one multi-tool. (Maybe except when I was 
using grep.) - But when others start to do the same in a fifth of the 
time then maybe it is really time to think;

> Split Osmosis into two parts, one
> is the low-level code for OSM objects, readers and writers etc. And one is
> the "control plane" code that stitches pipelines together etc.

... maybe the approach you are sketching would also allow one to somehow 
hack together "shortcuts" that would allow the building blocks to be 
used without anything not necessary for the task at hand.

Bye
Frederik

-- 
Frederik Ramm  ##  eMail frederik at remote.org  ##  N49°00'09" E008°23'33"