[osmosis-dev] Merge huge count of files

[Gubler, Ruediger]

Hi,

I have to merge a huge count of files. Doing this in one osmosis call creates thousands of threads and stops the rest of the system working well.
Is it possible and efficient to split the giant merge into smaller pieces?

e.g i have the following files

a1      a2      a3      a4
b1      b2      b3      b4
c1      c2      c3      c4
d1      d2      d3      d4

first iteration:        merge a1,a2,b1,b2 --> ab12 ....

ab12    ab34
cd12    cd34

second iteration: merge ab12,ab34,cd12,cd34 --> abcd1234


Or is it better to add always one file to the rest?

a1,a2 --> a12
a12,a3 --> a123
...


What is the best strategy to merge a huge count (e.g. 100x100 matrix) together with a minimum of needed memory?
Must the whole dataset fit in the memory?


Yours Rüdiger



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/osmosis-dev/attachments/20110929/2c268ec1/attachment.html>