[OSM-dev] osm2pgsql slow slim import
penorman at mac.com
Fri Dec 2 09:22:22 GMT 2011
> From: Kai Krueger [mailto:kakrueger at gmail.com]
> Subject: Re: [OSM-dev] osm2pgsql slow slim import
> Anyway, as Frederik suggested, I have changed osm2pgsql to automatically
> set the parameter synchronous_commit to off for the import session.
> Doing this should be safe. Unlike turning off fsync, turning of
> synchronous_commit can not lead to a corrupted database. All that can
> happen is that a few transactions that osm2pgsql though were processed
> might get lost on a database crash. During a full import that is "fine"
> as there is no way to recover from a partial import anyway and one needs
> to start from scratch. However, it should also be fine during diff
> imports, as it will simply mean some pending ways that were processed
> did not get mark as done and will be re-processed on the next diff
> Hopefully this therefore fixes the performance regression introduced
I also researched this and came to the same conclusions that
synchronous_commit=off is fine for the similar application of pgsnapshot
net-take-2/) for both the initial import and diff processing.
I also think fsync=off is safe for the initial import if you're prepared to
drop everything if you crash.
More information about the dev