[OSM-dev] replication database advice

Paul Norman penorman at mac.com
Tue Apr 22 19:02:10 UTC 2014


You want to run either 

 

-          Overpass + OSM3S

-          pyxapi/jxapi + pgsnapshot

-          cgimap + pgsnapshot

 

If you are editing (particularly with iD) you need minutely diffs, not
hourly, or you'll end up conflicting with yourself if you're not very
careful.

 

A strategy sometimes suggested is to have the planet saved locally and
extract from it, then take the diff from the current extract and the
previous extract and apply that diff to the database. Unfortunately, that's
too slow for editing.

 

What I'd probably do is have a larger than desired bounding box, apply
minutely diffs, and then periodically remove data outside that bounding box.

 

Keep in mind, iD has a bug where it ignores the data returned by the diff
upload (https://github.com/openstreetmap/iD/issues/1646), which will impact
you. After saving you will need to wait a minute then hit reload.

 

The last part of the puzzle is getting the client to use your local API for
reads and the remote one for writes and history calls.

 

If you're using JOSM you can use the mirrored download plugin to make map
calls from a remote API.

 

There's an apache config in the cgimap repo which directs supported queries
(map, get nodes/ways/relations, relation/full, etc) to the cgimap instance
and other calls to the main API. I'm not sure how well it would work with a
regional extract. It will also need minor changes for pyxapi/xapi and won't
work at all for overpass.

 

From: Mikel Maron [mailto:mikel_maron at yahoo.com] 
Sent: Tuesday, April 22, 2014 5:34 AM
To: dev at openstreetmap.org
Subject: [OSM-dev] replication database advice

 

Hi

 

I'm helping someone out. We're looking to set up a geographically filtered
replication from minutely or hourly planet diffs. The replicated, local
read-only db will be used to field large area map API queries locally, and
make extracts. Intention is to contribute changes back to OSM, so all
relevant information needs to be maintained. 

 

Is there a guide or best practice to setting something like this up?

 

Looking at the osmosis documentation, and seems like a pipeline of
--merge-replication-files  --bounding-box and --write-apidb-change tasks
would do it. But wonder if this is the most efficient way. Note, only need
the current state of features, not history, but don't see an option for that
in osmosis (only populateCurrentTables flag, nothing to turn off history).

 

Also, on the local API side, don't really want to keep the whole rails app
running. What are some of the alternative API servers out there?

 

Thanks

Mikel

 

* Mikel Maron * +14152835207 @mikel s:mikelmaron

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20140422/05cc4111/attachment.html>


More information about the dev mailing list