Hi James, talk-ca (and imports list, because this is an international issue that is true for all imports) (and Joe for the LINZ data)<br><br>Great to hear from ya. :)<br><br>Although your question (see full message below) is in regards to GeoBase NRN, im not the best person to answer that question (because i dont personally do the actual conversion).<br>
<br>However, since i am now providing the data from the CanVec set, and (most reciently) started providing the data for the GeoBaseNHN data. Perhaps i can shed some light. ... um, ya i will anyway :-)<br><br>To get back to basics the purpose of osm is to "Mission: To map the world and give the data away for free". <br>
<br>Taken, into context, we are building the maps at 1st by hand, and THEN as data becomes available, we can use the available data to 'improve on the existing map'. (how i interpret that is to 1 - provide the data in .osm format, then 2 - make this data available on a ftp server somewhere (ie. mediafire), and 3 - claim the area your working on, and use whatever tools that are best suited to bring this data into osm.<br>
<br>To explain further,<br>That said, I am of FIRM belief, that we need to treat 'importing government data' (of all types), as an "addition-to" rather than "importing it all half-hazardly, then fixing it, later". basically spending more time fixing it, than the time it would have taken to go out and actually map it.<br>
<br>When mapping in general we are always concerned about: Map Quality & Map Quanity. However, having map Quantity isn't that great, when the quality is poor. and it's my FIRM belief that it's better to have empty spaces, than having to go around and 'fix' everything afterwords. We know that the data source of 'accuracy in meters' (from geobase in particular) is a mix of 'acceptable' less than 10meters, to 'could be better' greater than 10meters. And the age of the data, as well as the acquisition technique needs to be taken into account.<br>
<br>So, as i mentioned above, for this 'data import process', we need to focus on the step 1 & 2. So when new data becomes available, we simply do step 1 and 2, and let the local area mappers view what is available, and they can use what ever method they choose. Here's a few examples of methods to use, once the data is available on a server somewhere.<br>
<br>Note: - This data that is available (should) contain a copy of the origional source files (ie. .SHP .GML .mp), as well as a README.txt file (which explains what the data is and how to use it), as well as a CHANGELOG.txt file which lists what has changed since the last conversion-version. And of course the .osm file(s), which are appropriatly named. (and in my case, the RULES.txt files that were used for the conversion) (and in others the shp2osm.py script that was used, which contains the source tags 'to' osm tags)<br>
<br>1 - Using JOSM, open up the .osm file (if shp-to-osm was used) it would be less than 2000 points) and view it, and download as a new layer all the osm area around the features, to ensure no duplication. and upload it.<br>
<br>2 - Using JOSM, open up the .osm files and download as a new layer the osm area around the features, and merge the layers. and saveAS a newname .osm file.<br>2b - use some funkey script to find matches, then use some nifty way to upload direct to the API<br>
2c - using some funkey de-duplifier script would help if needed. :)<br><br>3 - using the source .SHP or .GML or .mp file (that is available within the zip file of converted data), use OpenJUMP and some funkey magic to compare it with existing OSM data and abra-cadabra & upload. ;-)<br>
<br>4 - using those .osm files upload to a tileserver somewhere out-there (like <a href="http://osm.ca" target="_blank">osm.ca</a>) and make a cool map. That anyone can use. and add in select OSM data as you please (thanks to ODbl) :) .. and maybe make it into a slippy map that you can trace over too?<br>
<br>5 - using gpsbabel (i'll do that step for ya). convert the canvecroads.osm file into .gpx and open up the gpx file in JOSM and have fun tracing. (especially handy for when your out there with walking papers, as you can write in the streetnames, and when you get home you'll already have GPX tracks that 'could' be better than what you have.. but it's a 2nd guide for ya.)<br>
<br>6 - using OSMCUT with some funkey magic, you can take those .osm files and slice them up even further (into 2, 4 or 16 or whatever pieces), and work with what you like. <br><br>7 - and finally 7, (provided you are the ONLY one who drew in the map features) and provided you have the DIRECT consent from local mappers (good idea to make a meet-up-mapping-party for this),<br>
- load the .osm file in JOSM o(or using part of the above methods)<br> - download a new layer of the current osm data.<br> - delete the current osm data that you intend to replace (search select) or manual select and remove. <br>
-hide the layer of to-be-imported.osm file<br> - upload changes (removing map features)<br> - show that to-be-imported.osm file and either merge with other, or upload it<br><br>PLEASE (but remember to note (on the wiki for small countries) or on the GoogleDocs Chart for Canada) what tile your working on, so there is no toe-stepping-on. And please if your going to load an area, please copy in the WHOLE tile area, as that helps others when working on neighbouring tiles.<br>
<br><br>So anyway James, as to try to answer your question, because we are building our OWN map, and using the governent data as 'EXTRA', we are not depending on this data, nor are we trying to make our map look like the government data map. If there are areas where the government data can help, and using on of the above methods will work just fine, then we will 'copy-in'. (in-other-words) because the source .shp files are available in the zip folder on a server somewhere, The UUID, is UU-ISless (not needed). When new data is available it can be Again (step 1-converted & 2-shared), and then compared to the older data. and with some magic, compared with existing .osm data. Making us all :-) politely :)<br>
<br>And in summery, FYI, what i am doing is converting the canvec features and the GeoBaseNHN features, and making these .osm files (and a copy of the source) available on a server (for now it's <a href="http://mediafire.com" target="_blank">mediafire.com</a>) and will ALSO be copying in the roads.osm files that those others are making. and also making these files available. (if anyone has areas they want done 1st, just ask)<br>
<br>We did estimate 1 year for the process, and so far so good :)<br><br>By november, i should have a majority of the .osm files available from both canvec & geobase NHN, so they will be abailable to put into the OSM database taking most of next year. (with whatever method locals feel happy with). (being sure what whoever is copying-in) makes contact (as much as possable, to inform others of whats going on, so they can help too if they want). (since i cant do everything) someone else can make the roads.osm files available as that requres geobase2osm, super powers, that i dont have) (ie. a basic PostGIS understanding)<br>
<br>Maybe (James) it doesn't answer your question, but should help the imports@ list understand more what's going on, as well as whoever is following along on the talk-ca list :)<br><br>Cheers,<br>Sam<br><br>P.S. Hopefully by the next conference call, i'll be able to explain it in MUCH less words :) (so i can understand the positive & negative feedback on this) :-)<br>
<br><div class="gmail_quote">On Sun, Oct 11, 2009 at 10:06 AM, James Ewen <span dir="ltr"><<a href="mailto:ve6srv@gmail.com" target="_blank">ve6srv@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
We did a bulk import of GeoBase data provided by the Canadian<br>
government a while ago. I've been pondering and procrastinating on how<br>
to go about modifying the data without destroying it.<br>
<br>
The GeoBase import has a lot of information included with the ways.<br>
One piece of information is the UUID, a unique identifier for that<br>
particular way. We are told that the UUID needs to be kept with the<br>
way for future cross reference for future updates, as well as so that<br>
the Canadian government can find places where the OSM community fixes<br>
GeoBase problems.<br>
<br>
So, there are a number of issues that I am struggling with...<br>
<br>
1) Modifying a way to add more detail/realign to reality.<br>
<br>
This one is easy, as you won't be destroying any data, only<br>
adding/moving nodes so that the way represents real world data better.<br>
<br>
2) Deleting a way that does not exist anymore.<br>
<br>
Not too hard here either, if new construction has destroyed an old<br>
way, it needs to be removed.<br>
<br>
3) Adding new ways.<br>
<br>
New construction, or areas missed by GeoBase need to be added.<br>
<br>
4) Modification of existing ways that may destroy or highly modify data.<br>
<br>
This is where the quandary occurs. Things like chopping up a way to<br>
add bridges, or to realign an intersection, or other such<br>
modifications... do we just go about our editing with little regard<br>
for the existing data, or should we attempt to keep as much GeoBase<br>
data as possible? If we are to keep as much data as possible, what's<br>
the procedure?<br>
<br>
Maybe I have answered my question with 1, 2 and 3 above. Maybe I keep<br>
part of the existing way where it describes the way properly, or<br>
modifying it so it does, delete the rest of it, and then treat it all<br>
like new construction.<br>
<br>
James<br>
VE6SRV<br>
<br>
_______________________________________________<br>
Talk-ca mailing list<br>
<a href="mailto:Talk-ca@openstreetmap.org" target="_blank">Talk-ca@openstreetmap.org</a><br>
<a href="http://lists.openstreetmap.org/listinfo/talk-ca" target="_blank">http://lists.openstreetmap.org/listinfo/talk-ca</a><br>
</blockquote></div><br>