<div dir="ltr"><div>Hi Christian,<br></div> Thanks for the reply. After further investigation the actual problem was a missing index on planet_osm_polygon. (I'm really not sure why.)<br><br>Steve<br></div><div class="gmail_extra">
<br><br><div class="gmail_quote">On Sun, Aug 25, 2013 at 9:00 PM, Christian Quest <span dir="ltr"><<a href="mailto:cquest@openstreetmap.fr" target="_blank">cquest@openstreetmap.fr</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">As far as I know, Tilemill is using mapnik which is querying postgres.<div>Plain vanilla Mapnik is not doing more than one postgres query at a time (not multithreading queries).<br></div><div>A patch made by mappy allows mapnik to multithread its pg queries. Are you using the exact same version of Mapnik as before ?</div>
</div><div class="gmail_extra"><br><br><div class="gmail_quote">2013/8/25 Steve Bennett <span dir="ltr"><<a href="mailto:stevagewp@gmail.com" target="_blank">stevagewp@gmail.com</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div class="h5">
<div dir="ltr"><div><div><div><div>Hi all,<br><br>I'm running TileMill on an 8 core Ubuntu VM with 32GB of memory, on an OpenStack cloud. Recently, my VM was destroyed, and I rebuilt it (identically, I thought) on slightly different hardware (same cloud, but different physical infrastructure).<br>
<br>The new build is much slower at rendering - a screen worth of tiles at zoom 13 can take around a minute. That is, with virtually the same setup, same data, same styles. You can see some slow tiles here:<br><br><a href="http://emscycletours.site44.com/mel.html" target="_blank">http://emscycletours.site44.com/mel.html</a><br>
<br>While panning around, the 'top' command shows mostly Postgres processes (different from last time I had performance problems[1], when the bottleneck was in Mapnik). Total CPU usage hangs around 12%: ie, exactly 1 out of 8 cores is being used.<br>
<br><a href="https://dl.dropboxusercontent.com/u/767553/GIS/Screen%20shot%202013-08-25%20at%2011.15.01%20AM.png" target="_blank">https://dl.dropboxusercontent.com/u/767553/GIS/Screen%20shot%202013-08-25%20at%2011.15.01%20AM.png</a><br>
<span style="font-family:courier new,monospace"><br>
top - 11:10:32 up 3 days, 36 min, 1 user, load average: 0.06, 0.17, 0.22<br>Tasks: 133 total, 4 running, 129 sleeping, 0 stopped, 0 zombie<br>%Cpu(s): 11.5 us, 0.1 sy, 0.0 ni, 88.4 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st<br>
KiB Mem: 32950396 total, 7150132 used, 25800264 free, 117864 buffers<br>KiB Swap: 0 total, 0 used, 0 free, 5221356 cached<br><br> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND<br>
2353 postgres 20 0 8510m 640m 635m S 41.2 2.0 23:36.57 postgres<br> 2354 postgres 20 0 8510m 644m 639m S 40.2 2.0 23:24.26 postgres<br> 2350 postgres 20 0 8510m 642m 638m S 14.0 2.0 23:19.19 postgres<br>
2375 postgres 20 0 8510m 643m 639m S 14.0 2.0 23:17.80 postgres<br>13102 postgres 20 0 8508m 531m 527m S 13.6 1.7 13:03.21 postgres<br> 2355 postgres 20 0 8508m 531m 526m S 13.3 1.7 13:45.15 postgres<br>
2352 postgres 20 0 8510m 640m 636m S 10.0 2.0 23:31.17 postgres<br> 2348 postgres 20 0 8510m 644m 639m S 9.3 2.0 23:41.88 postgres<br>12420 mapbox 20 0 3818m 1.0g 755m S 9.3 3.2 36:48.39 nodejs<br>
2357 postgres 20 0 8508m 530m 526m S 7.3 1.7 13:38.57 postgres<br>
2356 postgres 20 0 8508m 531m 526m R 6.3 1.7 13:42.52 postgres<br> 2376 postgres 20 0 8508m 531m 527m S 6.0 1.7 13:35.51 postgres<br>13195 postgres 20 0 8508m 531m 527m S 5.3 1.7 12:33.65 postgres<br>
3027 postgres 20 0 8508m 531m 527m R 3.3 1.7 13:29.06 postgres<br> 2349 postgres 20 0 8508m 530m 526m S 3.0 1.6 13:38.19 postgres<br> 2358 postgres 20 0 8508m 531m 527m S 3.0 1.7 13:44.59 postgres<br>
26 root 20 0 0 0 0 S 0.3 0.0 0:08.64 ksoftirqd/5<br> 2335 postgres 20 0 8489m 2732 1340 S 0.3 0.0 1:00.48 postgres</span><br><br></div>So, wondering if anyone has any suggestions what the problem is, or how to fix it? Why is Postgres apparently using only one core, even though it has many processes? What tools could I use to further diagnose?<br>
<br></div>My changed Postgres settings are as follows:<br><br>shared_buffers = 8GB<br>autovacuum = on<br>effective_cache_size = 8GB<br>work_mem = 128MB<br>maintenance_work_mem = 64MB<br>wal_buffers = 1MB<br>checkpoint_segments = 10<br>
<br></div>The server is set up as described here:<br><a href="http://steveko.wordpress.com/2013/05/08/tilemill-server/" target="_blank">http://steveko.wordpress.com/2013/05/08/tilemill-server/</a><br><br></div>I'm not yet using any tile cache. I will do that next, but the problem I'm trying to solve at the moment is very slow tile generation, not slow serving of rendered tiles.<br>
<br>Many thanks in advance,<br>Steve<br><div><br><div><div><div><br>[1]<br><a href="http://gis.19327.n5.nabble.com/TileMill-performance-td5751158.html" target="_blank">http://gis.19327.n5.nabble.com/TileMill-performance-td5751158.html</a><br>
<br></div></div></div></div></div>
<br></div></div>_______________________________________________<br>
talk mailing list<br>
<a href="mailto:talk@openstreetmap.org" target="_blank">talk@openstreetmap.org</a><br>
<a href="http://lists.openstreetmap.org/listinfo/talk" target="_blank">http://lists.openstreetmap.org/listinfo/talk</a><br>
<br></blockquote></div><span class="HOEnZb"><font color="#888888"><br><br clear="all"><div><br></div>-- <br>Christian Quest - OpenStreetMap France<br>Un nouveau serveur pour OSM... <a href="http://donate.osm.org/server2013/" target="_blank">http://donate.osm.org/server2013/</a>
</font></span></div>
</blockquote></div><br></div>