[OSM-dev] New OGR driver to read OpenStreetMap .osm / .pbf files
jukka.rahkonen at latuviitta.fi
Sun Jul 15 15:34:40 BST 2012
I made a quick test with my old Windows laptop (2x1.2 GHz, 2 GB RAM) and
Finland.osm.pbf. OGR driver is a few times faster than osm2pgsql for me,
that is, 40 minutes vs. more than 2 hours on this slow computer. Osm2pgsql
spends a great deal of the extra time for indexing osm_nodes, osm_ways and
osm_relations even if I have no plans to do updates with diff files.
Nice thing is that layers can be selected individually like with any other
OGR multi-layer data source. This means that I can select to convert only
the POIs (Feature Count: 306297) and it goes in about one minute, or only
all the lines which takes about 10 minutes (Feature Count: 543777).
I need to play a bit more with this new toy. Right now I have only two
suggestions: osmconf.ini might have an option for renaming attributes.
Especially attribute names like "addr:city" are invalid in GML and
therefore cannot be published in WFS services before they are renamed to,
for example, "addr_city".
Another suggestion is to try if relations which are not polygon relations
could be converted into geometry collections. I am not sure if the result
would be usable for any real use but at least it should be possible to
visualise them with viewers like OpenJUMP.
Even Rouault wrote:
> This is a forward of an email I just send on the gdal dev mailing list :
> Following the recent brainstorming with Jukka, I've pushed into trunk a
> to read OpenStreetMap .osm / .pbf files .
> No particularly exotic dependencies : SQLite (and Expat for OSM XML files)
> See http://www.gdal.org/ogr/drv_osm.html for the details (will be
> available in
> a few hours).
> The performance to convert
> http://download.geofabrik.de/osm/europe/finland.osm.pbf into a Spatialite
> DB is
> the following one on my PC (Core i5 @ 2.67 GHz with 64bit GDAL) :
> $ time ogr2ogr finland.sqlite finland.osm.pbf -f SQLite -dsco
> SPATIALITE=YES -gt
> 10000 -progress --config OGR_SQLITE_SYNCHRONOUS OFF
> real 4m31.194s
> user 3m33.020s
> sys 0m46.070s
> Testing with larger areas, like whole France or Europe, shows sluggish
> performance when ways are built from nodes, but that's perhaps expected. I
> didn't compare with other tools to know if the indexing or request
> strategy is
> particularly bad.
> The data/osmconf.ini configuration file is pretty basic and its settings
> likely be improved with some tweaking. Contributions welcome.
> An improved version of the driver could allow specifying custom layers,
> instead of the 4 fixed ones.
> Happy testing,
> PS: I'm not sure if this is the right list @openstreetmap.org for that
> kind of
> dev mailing list
> dev at openstreetmap.org
More information about the dev