[OSM-dev] osm2pgsql slow slim import

Paul Norman penorman at mac.com
Fri Dec 2 09:22:22 GMT 2011


> From: Kai Krueger [mailto:kakrueger at gmail.com]
> Subject: Re: [OSM-dev] osm2pgsql slow slim import
> 
> Anyway, as Frederik suggested, I have changed osm2pgsql to automatically
> set the parameter synchronous_commit to off for the import session.
> 
> Doing this should be safe. Unlike turning off fsync, turning of
> synchronous_commit can not lead to a corrupted database. All that can
> happen is that a few transactions that osm2pgsql though were processed
> might get lost on a database crash. During a full import that is "fine"
> as there is no way to recover from a partial import anyway and one needs
> to start from scratch. However, it should also be fine during diff
> imports, as it will simply mean some pending ways that were processed
> did not get mark as done and will be re-processed on the next diff
> import.
> 
> Hopefully this therefore fixes the performance regression introduced
> previously.
> 

I also researched this and came to the same conclusions that
synchronous_commit=off is fine for the similar application of pgsnapshot
(see
http://www.paulnorman.ca/blog/2011/11/loading-a-pgsnapshot-schema-with-a-pla
net-take-2/) for both the initial import and diff processing.

I also think fsync=off is safe for the initial import if you're prepared to
drop everything if you crash.




More information about the dev mailing list