[OSM-dev] rsync and osm2pgsql flat nodes
penorman at mac.com
Tue Apr 21 23:17:37 UTC 2015
osm2pgsql flat nodes is a binary file with the position of all nodes, in
a representation which is very efficient for full planet dumps, needing
only 8 bytes * max node ID, instead of the 40-60 bytes * number of nodes
that alternative representations would need. For the planet where the
number of nodes is about 81% of the max node ID, this takes much less
One significant downside for production environments is that this data
is not in the database and does not have the replication and redundancy
of in-DB nodes. This is a problem but should not be insurmountable.
Rather than needing to keep the flat nodes and database synchronized
with your replication method, the flat nodes needs to be no older than
the state file which also should be saved.
Has anyone used rsync to keep a copy of the flat nodes file up to date
on another standby machine which can then step in in case the first one
fails? My initial thought was to have another cron script that runs
async to the updates which rsyncs the flat nodes and then the state
file, in that order. I have no idea how the performance is, but most of
the flat nodes file should not have changed.
More information about the dev