[OSM-dev] osmium trouble
Martijn van Exel
mvexel at gmail.com
Tue Nov 22 17:47:37 GMT 2011
That's probably true, but I did not really see another way to
calculate the statistics I wanted without keeping all the users in
memory. I guess the alternative is to put the amounts of nodes, ways,
relations per user in a custom object, something like
function UserStats() {
this.uid = 0;
this.stats = [];
}
and avoid the array expansion.
I'll try something like that and report back.
Martijn
On Mon, Nov 21, 2011 at 11:54 AM, Jochen Topf <jochen at remote.org> wrote:
> Hi!
>
> You are keeping a lot of data around (users and ages array, nodes hash). Are
> you sure thats not the problem? There are about 500000 user IDs now, so if
> a user with id 500000 comes along the users array is expanded to that size.
> I have no idea how much memory Javascript will need for an empty array that
> size, but I could imagine that its quite a bit.
>
> Jochen
>
> On Mon, Nov 21, 2011 at 10:16:25AM -0700, Martijn van Exel wrote:
>> Date: Mon, 21 Nov 2011 10:16:25 -0700
>> From: Martijn van Exel <mvexel at gmail.com>
>> To: Jochen Topf <jochen at remote.org>
>> Cc: Hermann Peifer <peifer at gmx.eu>, ThomasB <toba0211 at yahoo.de>,
>> dev at openstreetmap.org
>> Subject: Re: [OSM-dev] osmium trouble
>>
>> Sure, it's on github --> https://github.com/mvexel/OSMQualityMetrics
>>
>> FYI I did compile against the latest v8 from svn.
>>
>> Martijn
>>
>> On Mon, Nov 21, 2011 at 12:13 AM, Jochen Topf <jochen at remote.org> wrote:
>> > On Sun, Nov 20, 2011 at 03:34:27PM -0700, Martijn van Exel wrote:
>> >> I also ran into trouble with files of around 40MB or larger. I still
>> >> get the same errors though, either using array or sparsetable for node
>> >> storage.
>> >> I'm on Ubuntu 10.04 64 bit with 16GB RAM.
>> >
>> > Can you post the script you are using? Maybe we can find out if its doing
>> > something that triggers a bug.
>> >
>> > Jochen
>> > --
>> > Jochen Topf jochen at remote.org http://www.remote.org/jochen/ +49-721-388298
>> >
>> >
>>
>>
>>
>> --
>> martijn van exel
>> schaaltreinen.nl
>>
>
> --
> Jochen Topf jochen at remote.org http://www.remote.org/jochen/ +49-721-388298
>
>
--
martijn van exel
schaaltreinen.nl
More information about the dev
mailing list