[OSM-talk] (Easter?) TIGER Import

Ben Gimpert ben at somethingmodern.com
Fri Apr 21 10:12:33 BST 2006


My Fellow Alaska Enthusiasts,

Since the import's exponential backoff intentionally fails after a three
hour delay, I've been having to manually restart the TIGER import
process.

This is lame, so yesterday I wrote a second layer of process management,
a "watcher" that -- every five minutes -- checks if the import demon is
still running.  If not, it parses the log file for the last properly
imported street and restarts the import appropriately.  Alaska has been
especially slow due to the delay between the manual restarts (a.k.a.
"Ben's sleeping"), so I'll be interested to see how things chug along
now.

Dig Anchorage, yo:

http://www.openstreetmap.org/index.html?lat=61.121586&lon=-149.8265075&zoom=11

I'm also thinking about completely dropping the "sleep(1)" call between
streets... Worried about the inadvertant OSM DoS, though.

		Cheers,
		Ben


On Wed, Apr 19, 2006 at 09:49:29AM +0100, Etienne Cherdlu wrote:
> The import appears to have been doing Alaska for several days.  So
> assuming Alaska as a typical state (which is isn't), then 5 x 50 = 500
> days.  So, yes, it could easily take 1000 days.  Maybe it should be
> speeded up a bit else it will be 3 years before Wyoming is done.
> 
> BTW you can see which state it is doing by looking at
> http://brainoff.com/osm/change.
> 
> Etienne
> 
> 
> On 4/19/06, Mikel Maron <mikel_maron at yahoo.com> wrote:
> >
> >
> >
> > If I understand, a "complete chain" is a single street, so my estimate is
> > off by a factor of 10 .. so over 1000 days for the upload! Can that be
> > right?
> >
> > 1 road for ever about every 9 people .. that sounds like the US! ;)
> >
> >
> > ----- Original Message ----
> > From: David Blasby <dblasby at openplans.org>
> > Cc: talk at openstreetmap.org
> > Sent: Monday, April 17, 2006 6:51:42 PM
> > Subject: Re: [OSM-talk] (Easter?) TIGER Import
> >
> >
> > >I can help by asking how many streets in Tiger? ;)
> > >
> > >Have no real idea what this number might be. Say there's 1 US street for
> > every 100 people, so about 3 million streets. At 3 second per street, it's
> > just over 100 days .. which actually pretty reasonable .. though completely
> > unreliable ;)
> > >
> > >
> > I just finished loading TIGER 2005fe and found that there were
> > 35,809,110 road segments ("complete chains").
> >
> > dave
> > ps.
> > http://docs.codehaus.org/display/GEOSDOC/Loading+TIGER+data
> >
> >
> > _______________________________________________
> > talk mailing list
> > talk at openstreetmap.org
> > http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
> >
> > _______________________________________________
> > talk mailing list
> > talk at openstreetmap.org
> > http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
> >
> >
> >
> 
> 
> --
> Etienne
> (Forgot to reply to the list again)
> 
> _______________________________________________
> talk mailing list
> talk at openstreetmap.org
> http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk




More information about the talk mailing list