[OSM-dev] TIGER import.rb

Brandon Martin-Anderson badhill at gmail.com
Thu Jun 14 17:07:12 BST 2007


At present the Node#from_xml (as well as Segment#from_xml and Way#from_xml)
method in the rails port iterate through all <node> tags in the posted XML
request, parse _every one_, and _only return the last one parsed_. It would
be nigh trivial to return an array of parsed Nodes, and the corresponding
controller can supervise the insertion of each one into the DB, which should
be a fairly cheap operation serverside. This would speed up the import of
counties by orders of magnitude.

I'm very new here, so what's the best way I can help make that happen?

-B

On 6/14/07, Andy Robinson <Andy_J_Robinson at blueyonder.co.uk> wrote:
>
> snip
>
> >> and work on some of the outstanding issues like the _incredible
> >> slowness_ of the upload. The county in the above map took about 10
> >> hours to upload. And it's a small, rural county.
> >
> >I thought that the original tiger import ran on the server and didn't
> >even go through the normal API.  But, that was before my time, and I
> >guess it was a bit of a fiasco.
> >
> >I actually don't think 10 hours is too bad. :)
> >
> >I think it would be really nice if all of the people interested in the
> >tiger data could use the same conversion programs to get the data into
> >OSM.  We're all going to have the same issues, and I would imagine we'll
> >need to come up with a few new tags and conventions for handling things.
> >Do you know of any other areas that are using the TIGER data?
> >
>
>
> Just to clarify on this point, the original TIGER import ran via the API
> on
> a one second insert cycle. That was causing problems for other users so it
> was backed off to a 3 second insert cycle.
>
>
> The problem now is that we have even more users pressing for the resources
> and thus I believe that while the process is slower than it might be if we
> dumped straight to the OSM database the ability for users to control and
> take ownership of the process via JOSM has a significant appeal to our
> community based project. I'm sure that you will find many seasoned OSMers
> who do not live in the USA will still be interested in "mapping" in the
> USA
> in the same way some have been mapping cities in other countries using Y!
> imagery.
>
> Cheers
>
> Andy
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20070614/6c3d0e1b/attachment.html>


More information about the dev mailing list