<br><div class="gmail_quote">On Fri, Oct 9, 2009 at 4:06 AM, <span dir="ltr"><<a href="mailto:marcus.wolschon@googlemail.com">marcus.wolschon@googlemail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div class="im">On Thu, 8 Oct 2009 10:56:27 -0700, Sam Vekemans<br>
<<a href="mailto:acrosscanadatrails@gmail.com">acrosscanadatrails@gmail.com</a>> wrote:<br>
> Hi,<br>
> in the next week, i'll update the wiki 'importing government (bulk<br>
> provider) data' and list the 3-step approach.<br>
<br>
</div>I'm looking forward to it.<br>
But please don't limit your self too much to shape-files.<br>
<div class="im"><br></div></blockquote><div>The concept is the same.<br>1. convert <br>2. share and <br>3. import manual (josm) / automatic (bulk_upload) / semi-automatic (road_matcher/automatch)<br> <br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div class="im">
> If i get through on the conference call today, i'll try to explain it,<br>
> and maybe someone else who is a better wikipedian can fix it. :-)<br>
<br>
</div>Sorry. I'll not be on any of the conference-calls at all.<br>
(<a href="http://wiki.openstreetmap.org/wiki/Foundation/Import_Support_Working_Group" target="_blank">http://wiki.openstreetmap.org/wiki/Foundation/Import_Support_Working_Group</a>)<br>
<div class="im"><br>
> Step 1: Data conversion<br>
> -using eithor 'shp-to-osm.jar' or 'shp2osm.py' or 'mp2osm' or 'other'<br>
> we make a conversion script<br>
> -1 person is in charge of editing this script (as users will submit<br>
> errors) (just as for other programs)<br>
<br>
</div>Yes, my parser for the common interchange-format of LCLs for TMC-services<br>
exists for a while now. No known errors, well structured and commented<br>
(code quality is very important to me).<br>
<div class="im"><br>
<br>
> Step 2: data sharing<br>
> -the person that is running the conversion script would make these<br>
> .osm files available to the community so people can see it.<br>
<br>
</div>Well. Guess what I did quite a few days ago. ;)<br>
<div class="im"><br>
><br>
> 3 -data import<br>
> eithor a.1 use josm and copy the features over<br>
<br>
</div>That's what we do for everything that cannot be automatically imported.<br>
I had to add even more warnings as despite quite a lot of fat warnings<br>
people DID upload some of these reference .osm -files directly.<br>
<div class="im"><br></div></blockquote><div>the nature of the beast :). ... which is why i want to be making the rivers (geobaseNHN data) available at the same time, and only for those areas that have roads already imported. (so to save people work)<br>
<br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div class="im">
> 3.2 use <a href="http://bulk_import.pl" target="_blank">bulk_import.pl</a> for direct api load (for large areas,<br>
> if given permission to access the API at non-peak hours, and on que)<br>
<br>
"que"?<br></div></blockquote><div><br>On the status chart i note the progress, and list what is next to load.<br>With bulk import, you can set it so that it uploaded to the API only at non-peak hours, the sys admins, can set it so that changesets would get held back, to make room for the regular OSM traffic flow. <br>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div class="im">
<br>
> 3.3 use a postGIS database and use OpenJUMP Automatch to<br>
> compare datasets.<br>
<br>
</div>Never heard of that but the LCL-map if extremely simplified. I don't think<br>
the complex matching needed can be done by a generic tool.<br>
<div class="im"><br></div></blockquote><div>It just helps by finding exact matches, and highlights where it couldn't match right based on a set of rules.<br>.... thats why i prefer having the .osm files available, and letting local people copy what they like. (avoiding openJUMP all together)<br>
<br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div class="im">
> The facts that remain are;<br>
> 1 -the available data is constantly changing<br>
<br>
</div>Not much changes to LCLs expected here. But many other countries with<br>
lots of TMC-providers may follow. That's why I took so much care my tools<br>
are well written and easy to understand for a developer.<br>
<div class="im"><br></div></blockquote><div> </div><div>In Canada it's speratic, and somethings would be 6 months for a partial update and 1 year for full update. and sometimes the source features tagging gets changed too.<br>
<br>Because it's a federal holding house, they provide data for other people too, not just OSM :) so they want to keep their dataset uptodate also.<br> </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div class="im">
> utility tool, and having the 'tag matching list' be maintained by 1<br>
> person, will make it easier for everyone to work with the data.<br>
<br>
</div>What do you mean by 'tag matching list'?<br>
<div class="im"><br></div></blockquote><div> </div><div>with shp-to-osm the rules.txt lists the tag matches (source=osm) key/value comparisions.<br><br>from the google docs chart. (or wiki chart if it's not that big)<br>
<br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div class="im">
> By making it available, we can let the users decide what they want to<br>
> add, or omit. and put the emphasis on simply making all available data<br>
> available to use when and if we want.<br>
<br>
</div>Sounds less like hints for people doing an import then general advice<br>
for people offering datasets to import.<br>
<br></blockquote><div>lost me there, it's learning for everyone IMO :) <br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Marcus<br>
</blockquote></div><br>re: next message<br>><br>
>> The "Merge layers" option in JOSM is also not really usable because it<br>
>> can take really long (several days) for two layers with 100k nodes.<br>
<br>
>Well I don't think he was talking about single .osm -files with that<br>
many elements.<br>
. The .osm -files I generate (@topic)never have THAT kind of node-count.<br>
<br>Ya actually it was refering to how lakes can be a higher node count, and the shp-to-osm script has a hard time with that. So the solution is to manually split up the big lake to different polygons.<br><br>
>Your script is VERY "without fancy logic". I wouldn't use that<br>
at all if there is any already existing data in the area you are doing<br>
an import in. Besides there can be elements with the same id in your case.<br><br> You dont need to use it :) Its designed to only handle this data. (and im not an expert programmer eithor)<br><br>... i just ask that they make the .osm files available rather than direct upload so local area mappers can copy in the data. .. so where elements are existing, its up to the local area mapper to decide on what to add, or not to add, and how or if they want to merge it. That decision shouldn't be my own to make remotly. (not having being physically been in the area).<br>
<br>And a last point, it's fine if the converted .osm files remain on a server for a year before getting added, as long as people know it's there. I don't see a point in rushing, 'just to get it copied in' . <br>
<br>Cheers,<br>Sam<br>
<br>