[OSM-dev] Effort in the US wasted until TIGER import is complete?
dcsmith at gmail.com
Fri Mar 23 04:39:50 GMT 2007
No, I'm currently not familliar with the data model so I should look
As for a 1 sec/insert cycle, if we don't do it on the primary db, it
makes no immediate sense, and I'd be interested in timings without
it. I have no idea how many inserts are going on but, I would guess
from a ballpark on the size of tiger that you'd be right.
Again I'll look at the ruby code tonight, and if someone has a schema
for osm that'd be nice. If whoever did the original script could
outline their thinking that would be helpful as well.
I am subscribed to the dev list.
On Mar 23, 2007, at 12:06 AM, Thomas Lunde wrote:
> On 3/22/07, Don Smith <dcsmith at gmail.com> wrote:
>> do you have a machine setup? I'll look at the code tonight, but you
>> seem to have a better grasp of the operations going on. Any ideas in
>> what specifically needs to be done?
>> Right now, the two tasks appear to be aggregation of related segments
>> into ways (?) and the 1 sec insert cycle. I would suggest instead, if
>> possible that we load the data into mysql, and then when it's ready,
>> batch import the sql into the master db(does this make sense? Instead
>> of running the script twice, run it once, dump the db, and then
>> import the dump?).
> Don -
> I do have a server that could be used, but it sounds like Nathan has a
> better one. He is/has downloading/downloaded the latest TIGER data.
> Both of us need to have a better understanding of the OSM current data
> model than at present. Pointers at particular documentation would be
> helpful, otherwise I'll just look around the site and the code.
> What I think I understand is that the 1 sec insert cycle of the old
> Ruby code would still take weeks/months to do an import. Is that
> If so, it seems that there's got to be a better way. I agree with you
> that using seperate servers to do a higher speed import and then to
> dump the data from DB to DB directly would seem to be the smarter
> Are you already familiar with the OSM data model and/or with the old
> Ruby import code?
> Nathan,Don -- if y'all are subscribed to the Dev list, let me know and
> I shan't cc: you directly.
More information about the dev