[OSM-dev] new Java OSM loading / storage library

[Andrew Byrd]

Hello OSM developers,

Conveyal has created an open source Java OSM library for use in our own projects (including OpenTripPlanner). It is now in a usable pre-release state, and some of you might be interested in trying it out. I would of course be very interested in any commentary or feedback that could help improve the library and make it more useful to the wider community. 

https://github.com/conveyal/osm-lib

This library provides:

0. A set of Java classes modeling OSM entities (Nodes, Ways, and Relations).

1. Disk-backed random access storage of OSM data of any size (up to an entire planet dump) without any external database server, using the excellent MapDB in-process storage engine.

2. Reading and writing of the PBF format, as well as a still-evolving implementation of the VEX format I proposed a few months back [1], and conversion between the two.

3. Spatial indexing of OSM data based on web Mercator tiles, which allows fetching tiles from anywhere in the world on demand. 

4. A web API built on top of that spatial index which retrieves arbitrary rectangles of PBF data on demand.

5. Continuous minutely updates while the API server is running. The OSM data can stay in sync only a few seconds behind the OSM replication server.

Points 3 and 4 are essentially a Java port of our Vanilla Extract project [2]. This has the potential to simplify workflows that produce many geographic extracts from a single source PBF, or that rely on fetching up-to-date geographic extracts on demand.

Spatial indexing is currently limited to ways and the nodes referenced by those ways, i.e. “loose” POI nodes are not indexed. This is simply because our main use case is routing along ways.

Regards,
Andrew Byrd

[1] https://lists.openstreetmap.org/pipermail/dev/2015-April/028546.html
[2] https://github.com/conveyal/vanilla-extract