Hi John, <br><br>Sorry for the delay... and thanks everyone else for chiming in. Others can (as you saw) explain it better. :-)<br><br>This message is directed to the talk-ca list, as it serves as a summery for the latest and greatest. In a month or so, I'll be able to summarize in 1 page. But for now, I've put a lot of thought into the below message, so, although long and rambley... it's the best answer i got :) <br>
<br><br>First off, let me introduce the team... lets call it 'the GeoBase Road Crew'. These folks all know how to use python scripting and know how to use a 'postGIG' database. And are brilliant! And going a super awesome job! From west to east, we have;<br>
<br>Steve Singer - who loaded 092c area for me - And kicked of the whole province of Alberta & Most of Ontario... doing a super job .. and we have to thank for all the hard work done. .. paving the way for the others to follow.<br>
<br>Jason Reid - who started the geobase2osm script (i think getting the initial geobase tag matching done) revising the script from shp2osm <br><br>user: Austin Henry - Who has converted All of Vancouver Island. And for the 092B (Victoria area) he made the complete.osm area file available. As it was easier to manually add in roads.. than to pick-and-choose from RoadMatcher. (Since there's a lot of mappers around, this makes sense). And for the rest of the island, he manually checked out roadmatchers results and tried to attach as many geobase roads to OSM roads as he could.<br>
<br>user: Michael Barbanov - Who is working on the 092G area (Vancouver area) where 092G07 (port Coquitlam) is awesome. <br> <br>User: Neskie - who is in the 082L area and did some roads first, then went on to be in charge of the Aboriginal Lands Database..<br>
<br>user: Adam Dunn - who is now going full at it, and getting (what looks like) the rest of BC done. Starting with 092H Wow, awesome!<br><br>user: Steve Singer - again did all of Alberta, and most of Ontario<br><br>user: John Peterson - who did some of southwestern Ontario (and used the Stats Can roadnames) before roadnames became available in Ontario, using that cool script to match it up. Awesome!<br>
<br>user: Frank Steggink - who is doing a super job in Quebec - getting it all done. <br><br>user: Yan Morin - who did the Mont Laurier area . and also started the geobase NHN data, and make a cool python version of the script.<br>
<br>.... and of course, the entire talk-ca list, as anyone who has ideas, chimes in when they can :)<br><br>... and me user:Acrosscanadatrails, who doesn't know a think about python except that it works great, and we all like the results. :) I'm the one who's keeping track of the progress on a national level using the 'Canada Data Import Chart' [1].<br>
... I'm also the one who is managing the conversion process for the 'CanVec data (all the other map features). For this conversion I'm using Ian Dees's script shp-to-osm (which is simple java script and command line function with a rules.txt file). Who's helping along.<br>
<a href="http://wiki.openstreetmap.org/wiki/Canvec2osm">http://wiki.openstreetmap.org/wiki/Canvec2osm</a><br><br>.. as well, I'm working he geobaseNHN data (that user: Yan Morn started) and found errors). Frank is helping right now get the details on that.(considering that the data can also be found on canvec. So I'm sorting all that out.<br>
<br><br>We now have a imports@ discussion list[2], where the folks around the world who have conducted (or will be) or interested in. dealing with large amounts of data, and how it effects the community. etc. Brilliant minds, where I'm amazed everyday. How ALL questions can be answered.<br>
<br>And so, (recently) I've been using the IRC #osm channel[3], where there are on average 150 awesome mappers/programmers/GIS professionals/architects... and more who basically can answer any question asked. ... and debate about that, with fun and sometimes heated discussions. ... always ending up in some kind of change for the better :-)<br>
<br>Also, the talk-au & the talk-us list, where in the USA, there dealing with the post-Tiger trauma... and in Australia where they now have access to all this data and working it all out. And Also, new Zealand, where they are working on converting .mp files converting to OSM (LINZ data).[4]<br>
<br><div class="gmail_quote">On Sun, Oct 25, 2009 at 9:43 AM, john whelan <span dir="ltr"><<a href="mailto:jwhelan0112@gmail.com" target="_blank">jwhelan0112@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Yes I'm new and my background is red tape. I understand there is a<br>
lot going on and I appreciate the work that has been done.<br>
<br>
My background is in databases etc. If you ask clients what they want<br>
they always rate reliability above anything else. To me the data from<br>
geobase is good high quality data with input from multiple sources<br>
including municipal governments. Talk to them nicely and we can get<br>
all sorts of high quality things like bus stops, speed limits etc<br>
direct from the municipalities.<br></blockquote><div><br> </div><div>So ya, right now i have a stack of data available,, and ready.. I DO think that the community would appreciate me just doing one thing at a time. I volunteer to get it done. Others can help if they like. But the community would agree that getting all the roads in is 1st priority, then all of CanVec data in is next... then the national protected areas... then the aboriginal lands... then any Land Information Ontario that got missed from CanVec. ... then the Nanaimo Data thats available.. then the Vancouver bus stops that are available. <br>
As thats a logical order :) (and geobaseNHNrivers are mixed in there with canvec, but some geobaseNHN stuff too, depending on the back-end details for that)<br><br>.. and once municipalities get wind of whats going on, they'll probably pop up :-) and the CRD (Capital regional district, is on it's way, but probably a few years away... might be because they feel left out since Vancouver opened up there data :-)<br>
<br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
What appears to have happened is where the data has been merged roads<br>
that are in the geobase database no longer connect to roads that have<br>
been put in via potlatch. I think the end point is dropped</blockquote><div> The older potlatch roads do not have the same depth of tags as the geobase<br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
data. Not only that but roads that run close to the old roads appear<br>
to be deleted for the section close to the potlatch inputted roads.<br></blockquote><div><br>Here's the low-down. (social impact)<br>We respect the integrity of the local area mapper who spent a considerable amount of time either tracing from imagery, or tracing from there own GPS tracks.... and place this on a HIGHER priority than that of geobase/canvec data.<br>
So, again... this is Openstreetmap, where its a collaborative community who builds the map. ... we respect the integrity of the local area mapper who spent a
considerable amount of time either tracing from imagery, or tracing
from there own GPS tracks.... and place this on a HIGHER priority than
that of geobase/canvec data.<br>
<br>
<br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Oh and my WAAS enabled GPS thinks at least one Potlatch inputted road<br>
is in the wrong place.<br></blockquote><div><br>.. the more GPS tracks along the road the more probability of accuracy :-)<br> .. and i use garmin too ...that device is technically the best IMO :)<br><a href="http://wiki.openstreetmap.org/wiki/Garmin" target="_blank">http://wiki.openstreetmap.org/wiki/Garmin</a><br>
<br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br></blockquote><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
My personal view is that we should go with the geobase database and<br>
see if we can layer on tags etc. That way we can update the database<br>
from time to time by replacing the entire geobase data layer.<br>
<br></blockquote><div><br>So by having this .osm file available in JOSM as under the current data.osm layer. You can manually copying moving the ways over, and join what you like. (selecting a whole residential suburb, if it was missed)<br>
<br>If there are no other mappers around, YOU are now the 'expert' of this area, you can make it as detailed as you like.<br>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
I think OSM can add value in Canada basically by tagging and enhancing<br>
data already there, if a client can get geobase data elsewhere that<br>
actually has connecting roads and complete roads they may prefer that<br>
to OSM where the data quality isn't so consistent.<br>
<br></blockquote><div><br>We don't provide data for 'clients', rather, were shellfish in that sense (i live by the ocean, its shellfish), as we make the map better for ourselves. ... and as a result, YES our map is superior to other maps of similar quality. (but. of course, inferior to engineering drawings)<br>
<br>For all the other map features... here's the low-down (unlike, the US TIGER data, this is different). I learned from Dave Hansen who did the initial Tiger Import, and he shared the experiences, and ideas. ... as well as from when Michal Gilbert (who did the province borders) as well as from Steve Singer (who did the initial Alberta import, and Ontario import). <br>
<br>As a community, we decided (back in December 08) that we need to do what we can to preserve the integrity of the local area mappers who spent the previous years driving the thousands of km all over the country. and local cities of course too, where lots of mapping has been done.<br>
... so the end result and the SOLUTION is that we make these .osm files available and local people can do what they like. ... they can have the geobase_road.osm file available (download the file and open it) and move the osm roads over a bit to better align where needed. <br>
<br>AND it's those folks who i mentioned in the beginning of this letter, that are working hard and will get it all done. And so, for those areas that are slightly more complicated, they will make the complete.osm file available for local people (like yourself) to manually copy in what you like, and manually moving the existing data slightly (but only if your adding more features the the map).<br>
<br>So anyway,<br>I make these .osm files available and they will be stored on the NRCan ftp site (thanks NRCan :-) ... basically, a direct copy-translation of the canvec SHP files, and converted to the OSM equivalent. The files are made available in bit-sized files (so the API can handle 2000 points max at a time). This allows you to open up your local area (NTS tile) and open up 1 file at a time, and pan around the area and load the osm data wherever the features area.<br>
<br>BTW, if possible, i will ALSO make the canvec roads. (This is a direct copy of GeoBase roads) available, along with the .zip package. So then they can be used as a reference. (They are available in the next version of canvec-to-osm, as i run the script, but with the duplicate nodes as a KNOWN ERROR) (i labeled the file to indicate that) :-)<br>
<br>
... ie loading the 'railways.osm' file you would download all the data along the railways.... and find out what has/hasn't been copied in. (you can ALSO open up a file that is the 'NTS boundary area rectangle'.osm so you can use that as a guide. (but don't upload the rectangle, as its not needed)<br>
<br>... so,<br>You then need to find out WHO it was who put those railways in there that were there already, and contact them. (as they might want to help you, and as a courtesy to the local other mappers ... as thats how we map :-). .. so if the railway is off by more than 10 meters, its fine to adjust them. if MORE than 10 meters. then you'll need to keep that original in there until someone can manually go out there and verify. (alot of times the data available might be wrong or old).<br>
<br>here's an example... take a look at the contours. .. it's kinda hard to tell whats better.. as there might be a 'spur' that goes off.. i don't know for sure..<br><a href="http://www.openstreetmap.org/?lat=48.5124&lon=-123.5481&zoom=14&layers=00B0FTFT" target="_blank">http://www.openstreetmap.org/?lat=48.5124&lon=-123.5481&zoom=14&layers=00B0FTFT</a><br>
<br> Just because it came from CanVec government data... doesn't always mean the accuracy is there. (ortho-rectifying) from paper maps at a different zoom, may make a BIG difference. ... and this railway data also included the bridges, which might have not been mapped by others. (so an easy way is to break the way at either end of the bridge.. and add the tag 'bridge=yes' to it.<br>
<br>...and another option... lets say you want to run bulk_import and OpenJUMP . .. well, i make a .zip file available in this same .zip package that includes the original shp.zip fileset that came from the NRCan site. so you can then load the local .osm area using mySQLpostGISthing (or whatever)... and get a DIFF file. .. then convert that to osm. a derivative of geobase2osm can do the trick.. or if it's a SHP file, the basic shp-to-osm would work. (if the area your working on has no data to begin with in the 1st place, then direct API load is fine). ... as its IDENTICAL to the way JOSM works... accept using the command line for it.<br>
<br>... IMO (from experience) Bulk_uploading data is a 'blind' way of importing data. So to just convert DIR|ECT from the SHP file to OSM and to the API.... without creating .osm files 1st, and actually looking at them to verify what it actually is that you are loading.... this is why i STRONGLY recommend taking the slow approach, and opening up 1 file at a time... <br>
<br>BTW, to load a full NTS area of your working area.... the zoom is set that it takes just 6 spots panning around the rectangle to grab the whole NTS tile area. (not much).<br>... and hey, if your file is to 'spotty' where there are bits all over the place, you can just hide the current OSM area, and manually just select the data you don't want to load right now, and delete it, and then re open up the file and select the other half. (if that makes sense) <br>
<br>... So anyway, the point is this. since YOU are the one who is copying in the data to OSM, you have the responsibility to ensure that the quality of the data you are dropping in.. is the best that it can be. ... and if it's NOT... well, you shouldn't be coping it in.<br>
<br>see specifically <br><a href="http://wiki.openstreetmap.org/wiki/Import/Guidelines#Create_a_community">http://wiki.openstreetmap.org/wiki/Import/Guidelines#Create_a_community</a><br><br>
... as 'plopping-in' data just because it's available is NOT that great. (for Canada, OpenJUMP with Automatch is the best solution. .. however, having the roads as NTS tiles and dropped in locally might have been a little better.... but overall, since the size of the country is so great.. automatch did a super job, and didn't interfere with the local data, which is what the goal was.<br>
<br> </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Oh and I prefer JOSM to potlatch for some reason.<br>
<br></blockquote><div>For loading .osm files... JOSM is the tool to use. I use potlatch when adding relations, because its a bit easier. .. and tracing off of yahoo imagery is easier also with potlatch... and simple small edits too. <br>
<br><br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Thoughts?<br>
<br></blockquote><div><br>Yup... thats good for now.<br><br>Cheers,<br>Sam<br> <br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Cheerio John<br>
<br>
2009/10/25 Sam Vekemans <<a href="mailto:acrosscanadatrails@gmail.com" target="_blank">acrosscanadatrails@gmail.com</a>>:<br>
<div><div></div><div>> Hi,<br>
> just use potlatch & connect the roads. No harm done :-)<br>
><br>
> The OSM changeset history keeps track of attribution and such.<br>
><br>
> If you want a copy of where the geobase version of that road is, i can<br>
> provide that.<br>
><br>
> The system that imported the roads didnt want to interfeer with what<br>
> work previous people did.<br>
><br>
> Cheers,<br>
> Sam<br>
><br>
><br>
> On 10/25/09, john whelan <<a href="mailto:jwhelan0112@gmail.com" target="_blank">jwhelan0112@gmail.com</a>> wrote:<br>
>> Looking at my local map in Orleans Ontario noticed that Merkley Drive<br>
>> does not connect up to Charlemagne. Merkley Drive is tagged<br>
>> Geobase_import_2009 and all sorts of interesting things. Charlemagne<br>
>> is just tagged potlach and residential road.<br>
>><br>
>> Is there an easy way to extend Merkley Drive so it does connect? ie<br>
>> add another node but copy the attributes of the existing road?<br>
>><br>
>> Should I simply hold off until I know that the Geobase_import_2009 has<br>
>> been completed?<br>
>><br>
>> Is there a way to submit a GPS trace into Geobase to get their base<br>
>> map corrected?<br>
>><br>
>> Thanks John<br>
>><br>
>> _______________________________________________<br>
>> Talk-ca mailing list<br>
>> <a href="mailto:Talk-ca@openstreetmap.org" target="_blank">Talk-ca@openstreetmap.org</a><br>
>> <a href="http://lists.openstreetmap.org/listinfo/talk-ca" target="_blank">http://lists.openstreetmap.org/listinfo/talk-ca</a><br>
>><br>
><br>
><br>
> --<br>
> Twitter: @Acrosscanada<br>
> Blog: <a href="http://Acrosscanadatrails.blogspot.com" target="_blank">http://Acrosscanadatrails.blogspot.com</a><br>
> Facebook: <a href="http://www.facebook.com/sam.vekemans" target="_blank">http://www.facebook.com/sam.vekemans</a><br>
> OpenStreetMap IRC: <a href="http://irc.openstreetmap.org" target="_blank">http://irc.openstreetmap.org</a><br>
> @Acrosscanadatrails<br>
><br>
</div></div></blockquote></div><br>[1] <a href="http://spreadsheets.google.com/ccc?key=0Am70fsptsPF2dG1ZN1YwMmZCVDhDOHZpbUNmOGlvWGc&hl=en">http://spreadsheets.google.com/ccc?key=0Am70fsptsPF2dG1ZN1YwMmZCVDhDOHZpbUNmOGlvWGc&hl=en</a><br>
<br>[2] <a href="mailto:Import@openstreetmap.org">Import@openstreetmap.org</a> - discussion list<br><br>[3] <a href="http://irc.openstreetmap.org/irc.cgi">http://irc.openstreetmap.org/irc.cgi</a><br><br>[4] <a href="mailto:talk-us@openstreetmap.org">talk-us@openstreetmap.org</a>; <a href="mailto:talk-au@openstreetmap.org">talk-au@openstreetmap.org</a>; <a href="http://wiki.openstreetmap.org/wiki/LINZ">http://wiki.openstreetmap.org/wiki/LINZ</a> <br>