[Talk-ca] Quick update

[Dale Atkin]

Hey Sam,

Couple of quick points:

The mp files for the topo dataset are *massive* (IIRC they are ~50GB for the
whole dataset... even zipped that's a lot of data). Once I generate them,
there is no way I can get them re-uploaded without a *huge* amount of work,
heck, even burning them to DVDs would be a hell of a lot of work. The big
leap forward in my processing was when I got a version of cgpsmapper I could
run locally (it's the 'fully routable version' on a special license from
Stan that specifies that it goes to the 'public good'). 

The other problem coming from mp files, is we'd loose all the NIDs, which is
essential if we want to be sure that the dataset remains updatable. 

I really think the whole problem has been overcomplicated. The solution (in
my mind) is simple. 

1. Backup everything that is online now (doesn't the server keep a backup
anyways?).
2. Wipe out the Canadian dataset (probably leave the POIs intact)
3. Upload the Geobase/Geogratis data, and include tags for NIDs, and all the
other numerous tags that are available. This should get you about 99% of the
way there.
4. Run some filters on the data you downloaded, and look for roads etc that
have additional data on them (names, speed limits etc).
5. Manually merge this with the dataset you have from Geobase so that
most/all of the data has an associated NID on it.
6. Updates to the dataset are then handled by making reference to data that
has an NID associated with it (anything that has an NID that isn't in the
new database gets deleted, any new NIDs get added). The downside, is if
someone manually adds a road or two, it will temporarily be duplicated until
it is manually cleaned up, the upside is, if someone cares enough to
manually add this road, they'll likely notice relatively quickly when it
gets added to the dataset.

Step 2 is probably the most contentious, as people have put a lot of work in
to the data that is there, but given that a huge percentage of this data is
duplicated effort, I really think it's the most effective way to get a
useful, complete map out of it. 

Dale




-----Original Message-----
From: samvekemans at gmail.com [mailto:samvekemans at gmail.com] On Behalf Of Sam
Vekemans
Sent: Monday, December 29, 2008 12:09 AM
To: Talk-ca at openstreetmap.org; Dale Atkin
Subject: Quick update

Yuppee, lots more activity :)

a tim-bit that i thought i mentioned but maybe didnt :|

The idea of importing the next version of the ibycus topo.
 (and helping to make sure its right-but probably will be)

we would just need a list of what IS'NT included.

Creating a script to import each feature type (that users want/from
each tile) might be actually easier, since it is (will be) the most
current data.

The objection i had was about license; which is understandable.

I'll ask: What if we got explicit approval from both, Dale (ibycus)
and Stan (cgpsmapper)  (used to create ibycus img tiles)?

-we did dout geobase :-)

i'll go ahead and ask anyway, just to get a definate answer.

Cheers,
Sam