[OSM-dev] Effort in the US wasted until TIGER import is complete?
tlunde at gmail.com
Wed Mar 21 16:42:04 GMT 2007
It was previously written (but I can't find by who in my email chain):
> > The original approach on one machine at 1 sec insert cycle produces
> > a huge
> > amount of data quite quickly (it by far swamped the volume of data
> > for the
> > rest of the world put together in the time it ran) it will still
> > take a
> > considerable time to import everything, many many months rather
> > than weeks.
Oy! There must be a way to avoid months of waiting.
On 3/21/07, Frederik Ramm <frederik at remote.org> wrote:
> Would it not be sensible - for this special case, where such a large
> amount of data is imported - to import directly into the central
> MySQL database? Of course this would require some admin cooperation
> and oversight but for an amont of data that will (my guess) instantly
> double what we already have, one shouldn't do anything without admin
> cooperation or oversight anyway;-)
If an environment is a problem, I can provide one for this testing
purpose. I'm thinking of a dump of the current MySQL database which
is then imported into a new database on a server that is neither
production nor the main dev environment, if there are concerns on the
impact of doing so there. Obviously, if this is the sort of thing
that can be done in dev, so much the better.
I don't speak Ruby either, but I've heard so much about its
readability that perhaps the current scripts can be used as
"inspiration" for a import script that is more widely grokked. (I do
_not_ want to turn this into a language war, and I'm well aware of the
reaction to an outsider who wanders into a project and just starts
firing off "Why don't you use language X?" questions. I'm merely
trying to help in any way that I can. Again: I'm not trying to start
a debate on the use of Ruby.)
In addition to kick-starting the TIGER project again, I am hopeful
that someone can answer my original question:
Would pre-TIGER-import mapping efforts in the US be wasted/blown-away
by the eventual import?
More information about the dev