[OSM-dev] New OGR driver to read OpenStreetMap .osm / .pbf files

Jukka Rahkonen jukka.rahkonen at latuviitta.fi
Sun Jul 15 15:34:40 BST 2012


Hi,

I made a quick test with my old Windows laptop (2x1.2 GHz, 2 GB RAM) and 
Finland.osm.pbf. OGR driver is a few times faster than osm2pgsql for me,
that is, 40 minutes vs. more than 2 hours on this slow computer. Osm2pgsql
spends a great deal of the extra time for indexing osm_nodes, osm_ways and
osm_relations even if I have no plans to do updates with diff files.

Nice thing is that layers can be selected individually like with any other
OGR multi-layer data source. This means that I can select to convert only
the POIs (Feature Count: 306297) and it goes in about one minute, or only
all the lines which takes about 10 minutes (Feature Count: 543777).

I need to play a bit more with this new toy. Right now I have only two
suggestions: osmconf.ini might have an option for renaming attributes.
Especially attribute names like "addr:city" are invalid in GML and
therefore cannot be published in WFS services before they are renamed to,
for example, "addr_city".
Another suggestion is to try if relations which are not polygon relations
could be converted into geometry collections. I am not sure if the result
would be usable for any real use but at least it should be possible to
visualise them with viewers like OpenJUMP.

-Jukka Rahkonen-

Even Rouault wrote:

> Hi,
>
> This is a forward of an email I just send on the gdal dev mailing list :
>
> Following the recent brainstorming with Jukka, I've pushed into trunk a
> driver
> to read OpenStreetMap .osm / .pbf files .
>
> No particularly exotic dependencies : SQLite (and Expat for OSM XML files)
>
> See http://www.gdal.org/ogr/drv_osm.html for the details (will be
> available in
> a few hours).
>
> The performance to convert
> http://download.geofabrik.de/osm/europe/finland.osm.pbf into a Spatialite
> DB is
> the following one on my PC (Core i5 @ 2.67 GHz with 64bit GDAL) :
>
> $ time ogr2ogr finland.sqlite finland.osm.pbf -f SQLite -dsco
> SPATIALITE=YES -gt
> 10000 -progress --config OGR_SQLITE_SYNCHRONOUS OFF
>
> real    4m31.194s
> user    3m33.020s
> sys 0m46.070s
>
> Testing with larger areas, like whole France or Europe, shows sluggish
> performance when ways are built from nodes, but that's perhaps expected. I
> didn't compare with other tools to know if the indexing or request
> strategy is
> particularly bad.
>
> The data/osmconf.ini configuration file is pretty basic and its settings
> could
> likely be improved with some tweaking. Contributions welcome.
>
> An improved version of the driver could allow specifying custom layers,
> instead of the 4 fixed ones.
>
> Happy testing,
>
> Even
>
> PS: I'm not sure if this is the right list @openstreetmap.org for that
> kind of
> news.
>
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/dev
>





More information about the dev mailing list