[Tile-serving] [osm2pgsql] Slow going over pending ways... (#517)

[Paul Norman]

> @pnorman what information could I provide you with that would help us identify any bottlenecks? any commands I should run? thank you

Standard tools like top, iotop, iostat, htop, etc should all identify what is limiting. Odds are that you're limited by IO speed, but you want to confirm that.

------

> my advice 96GB effective cache is just nuts, postgres is heavly reliant on the cached part of linux memory, better off just allocating that memory to -C flag in the import process. Helper procsses tend lose effectiveness after 6 threads. That flat nodes flag is gonna slow you down alot also.

This is horrible advice. Flat nodes is a big performance increase, 8 processes is faster than 6 (though not 33% faster), cache above about 28GB (num nodes * 8 bytes / efficiency) is pointless, and effective_cache_size is not the memory postgres uses, but an estimate of total memory used for caching.

---
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/issues/517#issuecomment-164547519
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20151214/5ba727d0/attachment.html>