<html><head><style type="text/css"><!-- DIV {margin:0px;} --></style></head><body><div style="font-family:times new roman,new york,times,serif;font-size:12pt"><div style="font-family: times new roman,new york,times,serif; font-size: 12pt;">My recommendation for the next try at importing TIGER has been to make it more manual,<br>
that counties should be imported by individuals interested in mapping in the area. All other<br>
data in OpenStreetMap is built under stewardship, and until we're extremely confident in the <br>quality of the import, I would suggest Tiger be closely handled as well.<br><br>-Mikel<br><br><div style="font-family: times new roman,new york,times,serif; font-size: 12pt;">----- Original Message ----<br>From: Andy Robinson <Andy_J_Robinson@blueyonder.co.uk><br>To: Thomas Lunde <tlunde@gmail.com>; dev@openstreetmap.org<br>Sent: Wednesday, March 21, 2007 10:32:33 AM<br>Subject: Re: [OSM-dev] Effort in the US wasted until TIGER import is complete?<br><br><div>Thomas,<br><br>The issue that caused the data to be deleted was that each pair of nodes for<br>each TIGER segment was placed in the database as unique ID's. As a result<br>there was no connection between adjacent segments making up a street. That<br>is, there would be two nodes at the end of two adjacent segments rather than<br>one common node.<br><br>The reason that was given why this was the way the original
script imported<br>the data was that the TIGER data is not that precise and therefore there was<br>a question as to how the common node position could be identified if the two<br>adjacent segments did not have the exact same end co-ordinates. It was also<br>suggested that the error of position for the supposedly common data in TIGER<br>varies depending on whether you were in a rural or urban location.<br><br>Before the data was deleted I took a very close look at several places where<br>data had been imported. I could not find a discernable difference in<br>duplicate node position.<br><br>Therefore probably the best approach to take is to modify the original<br>import script so that it looks for and reuses the closest existing node (if<br>one exists) lying within say a 1 metre radius of the required new node<br>position. That should cover the vast majority of the data without too many<br>issues.<br><br>It really needs a coder to revisit the original script and modify it
to<br>reuse nodes that have already been created by the import. I also believe<br>that it should also seek to join up the adjacent segments that carry the<br>same tags into ways, the original import did not create whole street ways.<br>Obviously for very long streets it would be beneficial to limit the length<br>of any individual way.<br><br>There should also be some more consideration as to what tag data should be<br>imported. As far as I recall, the original import covered the street name<br>and the start zip and end zip for each segment. Is there other data in the<br>TIGER set that would be useful to import at the same time?<br><br>Once a modified script is ready then a small set of sample county TIGER data<br>needs to be imported, both for an urban location and then for a rural one.<br>This should iron out the remaining issues. Picking a couple of counties with<br>the smallest data set should enable an import and deletion cycle if required<br>without taking up too much
time and resource.<br><br>For the full roll out of TIGER data there needs to be some thought given to<br>how the script should run. Originally the script imported I think at 1<br>second intervals. This was downgraded to 3 sec intervals when server load<br>was a problem. If we wanted to import the data quickly, then we could take a<br>tiles@home type approach where counties are doled out and uploaded by<br>different machines running the same script. However for this approach to<br>work we may have to consider what platform improvements are necessary to<br>without making the platform too slow for other users.<br><br>The original approach on one machine at 1 sec insert cycle produces a huge<br>amount of data quite quickly (it by far swamped the volume of data for the<br>rest of the world put together in the time it ran) it will still take a<br>considerable time to import everything, many many months rather than weeks.<br><br>Obviously all of this needs to acknowledge that
some US users have already<br>contributed their own data and we would not want to see their hard work<br>wasted or corrupted. Imports to counties where data already exists should<br>automatically fall over until the users can be contacted so that a community<br>decision can be made as to whether the county will be initially mapped from<br>TIGER data or developed from GPS & aerial imagery data.<br><br>Cheers<br><br>Andy<br><br>Andy Robinson<br>Andy_J_Robinson@blueyonder.co.uk <br><br>>-----Original Message-----<br>>From: dev-bounces@openstreetmap.org [mailto:dev-bounces@openstreetmap.org]<br>>On Behalf Of Thomas Lunde<br>>Sent: 21 March 2007 12:09 AM<br>>To: dev@openstreetmap.org<br>>Subject: [OSM-dev] Effort in the US wasted until TIGER import is complete?<br>><br>>Hello -<br>><br>>The TIGER page [1] indicates that the initial effort was removed from<br>>the database because of data corruption.<br>><br>>Other postings seem to
indicate that a new TIGER import will overwrite<br>>existing US data.[2]<br>><br>>The last message on the mailing lists I can find about the TIGER<br>>import is from Jan 16, 2007 and indicates that the import is on an<br>>indefinite hold.[3]<br>><br>>The status page[4] seems to confirm this, as it shows the last header as:<br>>TIGER -> OSM Import Status<br>>(as of Thu Nov 30 14:47:41 +0000 2006)<br>><br>><br>>Is there anything a US-oriented would-be OSM participant can do to<br>>help with the import? Is the project stalled by a lack of CPU, disk,<br>>person-hours for coding, or something else?<br>><br>><br>>Thanks for any pointers you can provide to ways that I can help.<br>><br>>Thomas<br>><br>><br>><br>><br>>[1] <a target="_blank" href="http://wiki.openstreetmap.org/index.php/Tiger">http://wiki.openstreetmap.org/index.php/Tiger</a><br>><br>>[2] Sorry, I can't find this
again... I think it was on the Talk list.<br>><br>>[3] <a target="_blank" href="http://lists.openstreetmap.org/pipermail/talk/2007-January/010246.html">http://lists.openstreetmap.org/pipermail/talk/2007-January/010246.html</a><br>><br>>[4] <a target="_blank" href="http://svn.openstreetmap.org/utils/tiger_import/status">http://svn.openstreetmap.org/utils/tiger_import/status</a><br>><br>>_______________________________________________<br>>dev mailing list<br>>dev@openstreetmap.org<br>><a target="_blank" href="http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev">http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev</a><br><br><br><br><br><br>E-mail message checked by PC Tools Spyware Doctor (5.0.0.169)<br>Database version: 5.06900<br><a target="_blank" href="http://www.pctools.com/spyware-doctor/">http://www.pctools.com/spyware-doctor/</a><br><br><br>_______________________________________________<br>dev mailing
list<br>dev@openstreetmap.org<br><a target="_blank" href="http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev">http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev</a><br></div></div><br></div></div></body></html>