[Tile-serving] [osm2pgsql] Extremely slow database update (#128)

[jeffjanes]

I wasn't aware of vmtouch, thanks for the pointer.  I don't think I would use lock option.  You want to give the kernel hints, not orders.

Did it really get slower from 3.3knodes/s to 0.4knodes/s?  Or did you mean 0.4k/s for ways rather than nodes?

I don't know the command line for doing this, because I've never done this myself.  I don't have room to play with the entire planet, and I don't know how to apply diffs to just regions.  I'm just going based on my experience with other disk-bound systems.

Can you figure out who is doing the waiting?  Is osm2pgsql waiting directly for disk, or is it waiting for postgresql which is waiting for disk?  Perhaps what you need to prewarm is something other than flat-nodes, like one of the database tables.  In postgresql, if use the latest version and you turn on track_io_timing and use the extension pg_stat_statements, it will greatly help figure out what the bottleneck is.

---
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/issues/128#issuecomment-39385306
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20140402/026dd3b2/attachment.html>