[OSM-dev] osmjs says 'Killed'

[Martijn van Exel]

Jochen,

I compiled with the GC flag and can confirm that osmjs's memory usage
is definitely still growing - much slower, but that is probably due to
the garbage collection processes making the whole thing slow. I ran
that US pbf again and it croaked when its memory footprint was about
50% in top.

As you said without any further output it's hard to do any further
useful debugging.

Martijn

On Wed, Nov 9, 2011 at 1:37 AM, Jochen Topf <jochen at remote.org> wrote:
> On Wed, Nov 09, 2011 at 12:24:55AM -0700, Martijn van Exel wrote:
>> I'm testing my osmium setup, osmjs specifically. I'm feeding osmjs
>> small pbf planet extracts (city scale) and run the time.js script,
>> that goes OK. Then I feed it a US pbf planet extract, and osmjs says
>> 'Killed'  after about a minute or so. I tried -l none but that makes
>> no difference. -d gives me nothing.
>>
>> Osmosis says the pbf is OK (I ran $OSMOSIS --rb ../us.pbf --wn and it
>> did not give me any errors, I assume that means the file is OK).
>>
>> osmjs does not seem to have run out of memory. I don't have a lot
>> (2GB) but that should not prevent osmjs from running this, should it?
>
> If you don't use -l or -m osmjs should not use a lot of memory. Maybe there
> is a memory leak somewhere. Unfortunately thats hard to see, because the
> Javascript engine will take all the memory it gets (up to some limit) and
> only do garbage collection once that runs out. So in the "ps" output the
> process will always use lots of memory and suddenly crash, when it can't
> garbage collect any more. I don't remember what the message was in that
> case but thats probably the "Killed" you see.
>
> You can compile with -DOSMIUM_V8_FORCE_GC to force a garbage collection after
> each callback. This will make osmjs really slow, but you'll see the memory
> for the process growing over time, which it shouldn't do.
>
> I run osmjs regularly on a germany extract and sometimes others and don't have
> any problem like that. But it could be that the problem only shows up with, say
> very long ways, or objects with many tags or so. There could be something
> different about the US extract that triggers this. (Or maybe its just much
> bigger...)
>
> Jochen
> --
> Jochen Topf  jochen at remote.org  http://www.remote.org/jochen/  +49-721-388298
>
>



-- 
martijn van exel
geospatial omnivore
1109 1st ave #2
salt lake city, ut 84103
801-550-5815
http://oegeo.wordpress.com