[OSM-talk] Slow TileMill rendering - Postgres using 1 core?

[Christian Quest]

As far as I know, Tilemill is using mapnik which is querying postgres.
Plain vanilla Mapnik is not doing more than one postgres query at a time
(not multithreading queries).
A patch made by mappy allows mapnik to multithread its pg queries. Are you
using the exact same version of Mapnik as before ?


2013/8/25 Steve Bennett <stevagewp at gmail.com>

> Hi all,
>
> I'm running TileMill on an 8 core Ubuntu VM with 32GB of memory, on an
> OpenStack cloud. Recently, my VM was destroyed, and I rebuilt it
> (identically, I thought) on slightly different hardware (same cloud, but
> different physical infrastructure).
>
> The new build is much slower at rendering - a screen worth of tiles at
> zoom 13 can take around a minute. That is, with virtually the same setup,
> same data, same styles. You can see some slow tiles here:
>
> http://emscycletours.site44.com/mel.html
>
> While panning around, the 'top' command shows mostly Postgres processes
> (different from last time I had performance problems[1], when the
> bottleneck was in Mapnik). Total CPU usage hangs around 12%: ie, exactly 1
> out of 8 cores is being used.
>
>
> https://dl.dropboxusercontent.com/u/767553/GIS/Screen%20shot%202013-08-25%20at%2011.15.01%20AM.png
>
> top - 11:10:32 up 3 days, 36 min,  1 user,  load average: 0.06, 0.17, 0.22
> Tasks: 133 total,   4 running, 129 sleeping,   0 stopped,   0 zombie
> %Cpu(s): 11.5 us,  0.1 sy,  0.0 ni, 88.4 id,  0.0 wa,  0.0 hi,  0.0 si,
>  0.0 st
> KiB Mem:  32950396 total,  7150132 used, 25800264 free,   117864 buffers
> KiB Swap:        0 total,        0 used,        0 free,  5221356 cached
>
>   PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND
>  2353 postgres  20   0 8510m 640m 635m S  41.2  2.0  23:36.57 postgres
>  2354 postgres  20   0 8510m 644m 639m S  40.2  2.0  23:24.26 postgres
>  2350 postgres  20   0 8510m 642m 638m S  14.0  2.0  23:19.19 postgres
>  2375 postgres  20   0 8510m 643m 639m S  14.0  2.0  23:17.80 postgres
> 13102 postgres  20   0 8508m 531m 527m S  13.6  1.7  13:03.21 postgres
>  2355 postgres  20   0 8508m 531m 526m S  13.3  1.7  13:45.15 postgres
>  2352 postgres  20   0 8510m 640m 636m S  10.0  2.0  23:31.17 postgres
>  2348 postgres  20   0 8510m 644m 639m S   9.3  2.0  23:41.88 postgres
> 12420 mapbox    20   0 3818m 1.0g 755m S   9.3  3.2  36:48.39 nodejs
>  2357 postgres  20   0 8508m 530m 526m S   7.3  1.7  13:38.57 postgres
>  2356 postgres  20   0 8508m 531m 526m R   6.3  1.7  13:42.52 postgres
>  2376 postgres  20   0 8508m 531m 527m S   6.0  1.7  13:35.51 postgres
> 13195 postgres  20   0 8508m 531m 527m S   5.3  1.7  12:33.65 postgres
>  3027 postgres  20   0 8508m 531m 527m R   3.3  1.7  13:29.06 postgres
>  2349 postgres  20   0 8508m 530m 526m S   3.0  1.6  13:38.19 postgres
>  2358 postgres  20   0 8508m 531m 527m S   3.0  1.7  13:44.59 postgres
>    26 root      20   0     0    0    0 S   0.3  0.0   0:08.64 ksoftirqd/5
>  2335 postgres  20   0 8489m 2732 1340 S   0.3  0.0   1:00.48 postgres
>
> So, wondering if anyone has any suggestions what the problem is, or how to
> fix it? Why is Postgres apparently using only one core, even though it has
> many processes? What tools could I use to further diagnose?
>
> My changed Postgres settings are as follows:
>
> shared_buffers = 8GB
> autovacuum = on
> effective_cache_size = 8GB
> work_mem = 128MB
> maintenance_work_mem = 64MB
> wal_buffers = 1MB
> checkpoint_segments = 10
>
> The server is set up as described here:
> http://steveko.wordpress.com/2013/05/08/tilemill-server/
>
> I'm not yet using any tile cache. I will do that next, but the problem I'm
> trying to solve at the moment is very slow tile generation, not slow
> serving of rendered tiles.
>
> Many thanks in advance,
> Steve
>
>
> [1]
> http://gis.19327.n5.nabble.com/TileMill-performance-td5751158.html
>
>
> _______________________________________________
> talk mailing list
> talk at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/talk
>
>


-- 
Christian Quest - OpenStreetMap France
Un nouveau serveur pour OSM... http://donate.osm.org/server2013/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/talk/attachments/20130825/9db23867/attachment-0001.html>