[OSM-dev] Run away Ruby and Postgres optimization

Eric Wolf ebwolf at gmail.com
Tue Sep 14 18:09:22 BST 2010


We have a relatively small dataset loaded and have 3-4 people editing
simultaneously. We aren't doing a bulk upload.

Everything is on one box, but it's a decent box: 4 fast CPUs, 6GB RAM, SATA
RAID.

I just did some tweaking. Set PasengerMaxPoolSize 20. Increased SHMMAX so I
can increase Postgres' buffer size. I am now seeing memory usage approach
100% (but not go to swap).

But it still seems to bog down after a few minutes of intense editing with
Potlatch (edit with save). Vector data is slow to return when scrolling
(getting the blinky triangle).

-Eric

-=--=---=----=----=---=--=-=--=---=----=---=--=-=-
Eric B. Wolf                           720-334-7734





On Mon, Sep 13, 2010 at 3:49 PM, Brendan Morley <morb.gis at beagle.com.au>wrote:

>  What parts of the API are you exercising?
>
> When setting up servers for CommonMap, I found that a 50,000 element upload
> takes 40 to 80 minutes to complete, depending on the type of server I used.
>
> Ruby on Rails is very inefficient for that kind of thing.
>
>
> Brendan
>
>
> On 14/09/2010 6:15 AM, Eric Wolf wrote:
>
> We finally have our system running and are doing some field tests with
> multiple editors. The system keeps getting bogged down. Looking at the
> process list, I'm seeing 7-8 ruby tasks, each eating 50%-100% of a CPU core
> (thank god for multiple CPUs & cores). They are persisting for 20+ minutes.
> Any idea what may be up? Should ruby tasks be hanging around that long?
>
>  I'm running a pretty much default Postgres install. Any hints on
> optimizing Postgres for OSM?
>
>  -Eric
>
> -=--=---=----=----=---=--=-=--=---=----=---=--=-=-
> Eric B. Wolf                           720-334-7734
>
>
>
>
> _______________________________________________
> dev mailing listdev at openstreetmap.orghttp://lists.openstreetmap.org/listinfo/dev
>
>
>
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20100914/9d9e966c/attachment.html>


More information about the dev mailing list