[OSM-dev] How to keep a geographic local part of OSM updated with minutely diffs?
Paul Norman
penorman at mac.com
Wed Dec 2 06:24:55 UTC 2015
On 12/1/2015 2:33 AM, tb wrote:
>> Option 1:
>>
>> Use bounding box option with osm2pgsql import, and run the cleanup
>> queries documented here:
>>
>> http://wiki.openstreetmap.org/wiki/User:Stephankn/knowledgebase#Cleanup_of_ways_outside_the_bounding_box
>>
>
> hm, this has some constraints:
> 1. use of "--flat-nodes" is not allowed, which IMHO is no problemo for
> a small database.
Flat nodes is not recommended for small databases anyways. It only
becomes useful for very large extracts (e.g. Europe) and if you're doing
that I'd probably jump up to the full planet for ease of diff application.
> 2. (at least to me) some unpredictable behavior in a case where an
> object location is changed from outside of the bounding box partially
> or fully inside the bounding box (partially also means no node inside
> the bounding box, but having a crossing edge). Also moving an object
> in two changes out of the bounding box and in again is questionable.
> Will this work reliable?
No, but it's not a big issue in practice. You use a larger bounding box
than you're interested in. No minutely diff methods that don't involve
keeping a full planet file will get this correct.
In summary, there's a few different ways to keep a region up to date
1. Use an extract from Geofabrik with their daily diffs. This is the
easiest and least resource-demanding option, but has a 24h update interval
2. Have a planet file that you keep up to date, create extracts from it,
compute the diffs between the extracts, and apply the diffs to the
database. This is option 1, except you're creating your own diffs. You
can probably get 1-2h update interval with 1-2h update lag.
3. Having a larger bounding box file, keeping it up to date, and
creating diffs to import. This would require periodic recreation of the
larger file for accumulated errors around the edges. Updates every few
minutes should be possible.
4. Putting minutely diffs directly into osm2pgsql, and cleaning as
needed. You'd also want to periodically reimport to clear errors. This
will let you get up to the minute.
Consider reimporting instead of updating for small extracts, because
reimporting is pretty quick with <1GB PBFs, and it will result in faster
rendering tables because they'll always be clustered and unbloated.
https://github.com/Komzpa/furry-sansa might also be worth looking at,
since it can automate some of this.
More information about the dev
mailing list