[OSM-dev] osmium trouble

[Hermann Peifer]

On 21/11/2011 08:13, Jochen Topf wrote:
> On Sun, Nov 20, 2011 at 03:34:27PM -0700, Martijn van Exel wrote:
>> I also ran into trouble with files of around 40MB or larger. I still
>> get the same errors though, either using array or sparsetable for node
>> storage.
>> I'm on Ubuntu 10.04 64 bit with 16GB RAM.
>
> Can you post the script you are using? Maybe we can find out if its doing
> something that triggers a bug.
>

Just to confirm that osmjs works again for me after applying your fix 
from yesterday:

     commit 10ec36fbf104c5e1ae8ae1e0c5199cf072acb143
     Author: Jochen Topf <jochen at topf.org>
     Date:   Sun Nov 20 15:06:09 2011 +0100
             fix memory leak in Javascript glue code


I just can't confirm your comment from an earlier commit (22 Oct): "This 
change does not lead to much extra overhead..." [1]. Some more figures 
to illustrate the performance difference:

austria.osm.pbf	(164M)

Before:                 After:
real    0m58.012s       real    1m19.850s
user    0m52.899s       user    1m16.013s
sys     0m2.980s        sys     0m1.784s

france.osm.pbf (1.7G)

Before:                 After:
real    5m43.047s       real    11m4.719s
user    5m25.260s       user    10m49.733s
sys     0m13.881s       sys     0m14.349s

I was using again the same simple Javascript code [2].

Hermann

[1] 
https://github.com/joto/osmium/commit/17604decdd2f26d9fc831cc3c2412a8ca97dc2b0

[2]
Osmium.Callbacks.way = function() {
   if ( this.tags["highway"] ) {
     print(this.tags["highway"] + '\t' + this.geom.toWKT());
   }
}