[OSM-dev] osmjs says 'Killed'

[Jochen Topf]

On Wed, Nov 09, 2011 at 12:24:55AM -0700, Martijn van Exel wrote:
> I'm testing my osmium setup, osmjs specifically. I'm feeding osmjs
> small pbf planet extracts (city scale) and run the time.js script,
> that goes OK. Then I feed it a US pbf planet extract, and osmjs says
> 'Killed'  after about a minute or so. I tried -l none but that makes
> no difference. -d gives me nothing.
> 
> Osmosis says the pbf is OK (I ran $OSMOSIS --rb ../us.pbf --wn and it
> did not give me any errors, I assume that means the file is OK).
> 
> osmjs does not seem to have run out of memory. I don't have a lot
> (2GB) but that should not prevent osmjs from running this, should it?

If you don't use -l or -m osmjs should not use a lot of memory. Maybe there
is a memory leak somewhere. Unfortunately thats hard to see, because the
Javascript engine will take all the memory it gets (up to some limit) and
only do garbage collection once that runs out. So in the "ps" output the
process will always use lots of memory and suddenly crash, when it can't
garbage collect any more. I don't remember what the message was in that
case but thats probably the "Killed" you see.

You can compile with -DOSMIUM_V8_FORCE_GC to force a garbage collection after
each callback. This will make osmjs really slow, but you'll see the memory
for the process growing over time, which it shouldn't do.

I run osmjs regularly on a germany extract and sometimes others and don't have
any problem like that. But it could be that the problem only shows up with, say
very long ways, or objects with many tags or so. There could be something
different about the US extract that triggers this. (Or maybe its just much
bigger...)

Jochen
-- 
Jochen Topf  jochen at remote.org  http://www.remote.org/jochen/  +49-721-388298