[OSM-dev] TIGER import.rb

Brandon Martin-Anderson badhill at gmail.com
Wed Jun 13 21:13:16 BST 2007

Hey Dave,

It seems I was plugging away at the same problem at the same time. The
attached ruby code does the same thing - converts a TIGER/line dataset into
an osm.xml file for use in JOSM. I was waiting until the county I uploaded
to the OSM has been rendered before announcing to the list, and it seems my
tendency to release conservatively has cost us some duplicated work. It's
just now starting to show up on Mapnik, so here it is:

San Juan County, WA:
A little closer, with residential street names:

For bonus points, check out the metadata on each OSM feature. Includes
address ranges and alternate names.

The code that produced this is attached. Here's what each file does:
tiger.rb - a model of TIGER data
osm_model.rb - a model of OSM data
attr_map.rb - a mapping of TIGER CFCC data to OSM tags
to_osm.rb - extends the TigerLine::Dataset class with to_osm and to_osm_xml
tiger_to_osm.rb - a command-line function to convert a set of TIGER files
into an osm.xml file.


Let's put our heads together, combine the features of our two importers, and
work on some of the outstanding issues like the _incredible slowness_ of the
upload. The county in the above map took about 10 hours to upload. And it's
a small, rural county.


On 6/13/07, Dave Hansen <dave at sr71.net> wrote:
> I've hacked up the tiger import code a bit.  Instead of having it upload
> to osm directly, I decided to have it produce .osm files that JOSM could
> open.  I bet this might be a good approach in the future for other
> people, too.  It allows you to make sure that the map "looks right", and
> to run things like the JOSM validator plugin on it. At this point, the
> only validator warnings that it produces are for untagged and unnamed
> ways.  I need to verify that the TIGER db even _has_ names for these.
> I've also created a couple of ruby classes to make handing the node,
> segment, and way classes a bit easier.  This is my first coding in ruby,
> ever, so please be gentle. :)
> The node creation code will detect close nodes and merge them.  This
> uses raw lat/lon and pretends they're actual distance units for now.
> I'm sure this can be easily fixed up to use real distances
> This will create ordered ways, and will coalesce all adjacent ways with
> the same name into a single way.  It will flip segments and ways as
> necessary to make them fit.  I had some performance problems with this,
> but I feel like it's running at a workable speed now.
> One last thing...  Do we really need the mysql database?  After
> importing a single zip code, the database only looks to me to be ~2MB.
> I also don't see any SELECT statements which look too horribly complex.
> Any chance we could just build the DB's contents in-memory?  We'd be
> left with scripts that take TIGER .zip files and produce .osms.  We
> could post those individually somewhere and have people familiar with
> the area go over them before actually uploading them.
> -- Dave
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20070613/6fee5c4c/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: attr_map.rb
Type: application/x-ruby
Size: 5250 bytes
Desc: not available
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20070613/6fee5c4c/attachment.rb>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: osm_model.rb
Type: application/x-ruby
Size: 1351 bytes
Desc: not available
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20070613/6fee5c4c/attachment-0001.rb>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tiger.rb
Type: application/x-ruby
Size: 10087 bytes
Desc: not available
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20070613/6fee5c4c/attachment-0002.rb>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tiger_to_osm.rb
Type: application/x-ruby
Size: 178 bytes
Desc: not available
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20070613/6fee5c4c/attachment-0003.rb>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: to_osm.rb
Type: application/x-ruby
Size: 3499 bytes
Desc: not available
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20070613/6fee5c4c/attachment-0004.rb>

More information about the dev mailing list