[Talk-ca] Geobase NRN import script
Jason Reid
osm at bowvalleytechnologies.com
Thu Jan 1 23:17:54 GMT 2009
Thanks to some patches from Steve Singer, I've committed a new version
of the NRN import script to the SVN repository. This version uses the
python OGR bindings to better process the source data, and Steve also
worked out a system to merge junction nodes together (earlier versions
of the script ended up with multiple identical nodes at a junction, 1
per way). The script can be found at
http://svn.openstreetmap.org/applications/utils/import/geobase2osm/ if
you are interested.
For those who are not familiar with what has been mentioned in the past
about the script, it takes the GML files provided by Geobase and
converts them to OSM. The conversion is as close to a 1 to 1 as we can
get it, if there is a linestring in the GML it becomes a way in OSM, a
point in the GML it becomes a node in OSM. All the relevant tags (nrn id
#'s, etc) are assigned to the proper OSM entities. The script could
theoretically be modified in the future to use the shapefiles provided
by Geobase, however I've had some discussion with individuals at Geobase
and was assured that the only difference between the gml and shape files
is the format, the data inside is identical.
The script will generate a single OSM file per GML file, so 1 per
province currently. What I had envisioned when I first drafted the
program in October 2007 (following the discussions at FOSS4G 2007, well
before we had official permission to use the data) was to take these
files and split them into smaller chunks to upload, either by a certain
size or some smaller jurisdictions, and likely upload them in a process
similar to how the TIGER data was incorporated into OSM (a dedicated
upload script running and uploading each file one after another). The
biggest issue that I can forsee with multiple people doing the upload is
the possibility of two people uploading the same section (which is not
fun to clean up, especially in areas where the geobase data is not the
only data), and the sheer size of the upload. The GML files alone are
nearly 8gb before any processing, and that is with the current files
that only a select few have street names, etc, included.
The mapping between the Geobase tags and OSM tags is a work in progress
still, it roughly follows whats on the wiki, both on the geobase import
and the Canadian tagging guidelines. This is the largest area of the
script that needs refinement yet.
You can see some of the initial converted data rendered on the map at
http://openstreetmap.ca/map/. This map will be updated periodically as
things progress, as a test bed to make sure that things are working. So
not all provinces are there yet.
-Jason Reid
More information about the Talk-ca
mailing list