[Talk-ca] Geobase NRN import script

Jason Reid osm at bowvalleytechnologies.com
Thu Jan 1 23:17:54 GMT 2009


Thanks to some patches from Steve Singer, I've committed a new version 
of the NRN import script to the SVN repository. This version uses the 
python OGR bindings to better process the source data, and Steve also 
worked out a system to merge junction nodes together (earlier versions 
of the script ended up with multiple identical nodes at a junction, 1 
per way). The script can be found at 
http://svn.openstreetmap.org/applications/utils/import/geobase2osm/ if 
you are interested.

For those who are not familiar with what has been mentioned in the past 
about the script, it takes the GML files provided by Geobase and 
converts them to OSM. The conversion is as close to a 1 to 1 as we can 
get it, if there is a linestring in the GML it becomes a way in OSM, a 
point in the GML it becomes a node in OSM. All the relevant tags (nrn id 
#'s, etc) are assigned to the proper OSM entities. The script could 
theoretically be modified in the future to use the shapefiles provided 
by Geobase, however I've had some discussion with individuals at Geobase 
and was assured that the only difference between the gml and shape files 
is the format, the data inside is identical.

The script will generate a single OSM file per GML file, so 1 per 
province currently. What I had envisioned when I first drafted the 
program in October 2007 (following the discussions at FOSS4G 2007, well 
before we had official permission to use the data) was to take these 
files and split them into smaller chunks to upload, either by a certain 
size or some smaller jurisdictions, and likely upload them in a process 
similar to how the TIGER data was incorporated into OSM (a dedicated 
upload script running and uploading each file one after another). The 
biggest issue that I can forsee with multiple people doing the upload is 
the possibility of two people uploading the same section (which is not 
fun to clean up, especially in areas where the geobase data is not the 
only data), and the sheer size of the upload. The GML files alone are 
nearly 8gb before any processing, and that is with the current files 
that only a select few have street names, etc, included.

The mapping between the Geobase tags and OSM tags is a work in progress 
still, it roughly follows whats on the wiki, both on the geobase import 
and the Canadian tagging guidelines. This is the largest area of the 
script that needs refinement yet.

You can see some of the initial converted data rendered on the map at 
http://openstreetmap.ca/map/. This map will be updated periodically as 
things progress, as a test bed to make sure that things are working. So 
not all provinces are there yet.

-Jason Reid




More information about the Talk-ca mailing list