[Talk-ca] Multipolygon problems

Sat Jul 1 08:30:35 UTC 2017

Hi Jochen,

Maybe a MapRoulette challenge might even not be necessary. Yesterday I 
started to clean up a bit in Québec, but since it was already past 
midnight for me, I wanted to continue this morning. To my surprise 
Pierre has done a lot of work and now the entire province of Québec 
looks to be free from these errors. I just could find three errors, and 
fixed them. Bon travail, Pierre!

The OSM inspector will still be a good idea, in order to spot future errors.

To all, this is the procedure I used yesterday, and probably something 
similar also by Pierre.
* Not sure if it is a requirement, but it's better to use 64 bit Java.
* You'll need JOSM with the remote control plugin. You'll also need to 
start JOSM.
* Use Pierre's Overpass query (http://overpass-turbo.eu/s/q5K), and 
zoom/pan to the area of interest.
* Press Run and let Overpass do its work. Don't be afraid when Overpass 
mentions that the result is huge if you have a modern computer. Last 
night I wasn't experiencing any problems with 30MB data.
* Press Export, and choose JOSM. Press "Repair query" when Overpass asks 
you to decide.
* JOSM starts downloading the data. Note that you're only getting the 
outer rings.
* Go to these rings one by one, and load data (at least you'll need the 
relationship itself).
* Remove the natural=wood tag from the outer ring.
* Eventually JOSM starts looking cluttered, because of all the extra 
data. You can use the search query "type:way natural=wood role:outer" to 
see if there are still rings needing work.
* Upload your changes. Be prepared to handle conflicts if someone else 
is working near you on this issue as well.
* After a while, check Overpass Turbo again.

I'm not sure what the update frequency is, but Pierre's changes from 4 
hours ago were already processed.

Good luck!

Frank

On 01-07-2017 09:52, Jochen Topf wrote:
> On Fri, Jun 30, 2017 at 11:47:36PM +0200, Frank Steggink wrote:
>> On 30-06-2017 21:21, Jochen Topf wrote:
>>> On Fri, Jun 30, 2017 at 08:16:40PM +0200, Frank Steggink wrote:
>>>> Maybe I'm not understanding it, but in the OSM inspector [1] I just see one
>>>> case of old style multipolygon, in Manitoba. Last week, when you posted your
>>>> original message, I just saw one case in New Brunswick. IIRC, it was a park,
>>>> not even from the Canvec import.
>>> The types of problems I am talking about don't show up in the OSM
>>> inspector. This is not old-style multipolygons (where tags are on the
>>> outer ways and not on the relation), but multipolygons where the tags
>>> are on the relation AND on the ways.
>> Ah, ok, now I understand. Since there was a lot of discussion about old
>> style multipolygon tagging, and since this type of problem hasn't been added
>> to OSM inspector,  this wasn't immediately obvious.
>>>> In the OSM inspector other errors can be seen, but the most prevalent one is
>>>> "Touching rings". Maybe indeed a case of suboptimal mapping, but nothing
>>>> which seems urgent to me.
>>>>
>>>> Here is an example of a forest multipolygon, imported by me
>>>> (canvec_fsteggink). It is still version 1, but it has tags on the relation,
>>>> not on the rings (except for the quarries): [2]
>>>> This is from Canvec v7.0. IIRC, we started at v6.0, and the last version I
>>>> know of is v10.0. Maybe v6.0 had wrong tagging, but I'm not seeing any such
>>>> cases in the OSM inspector.
>>>>
>>>> So, I'd like to ask you to give a couple of examples where data imported
>>>> from Canvec is clearly wrong with regard to old style multipolygon tagging.
>>> Here are all cases in Canada (not only those from the imports):
>>> https://tmp.jochentopf.com/954226a3acab882d28d8500ddef8203d/same-tags-ca.pbf
>>>
>>> Here is one example where you can clearly see the problem:
>>> http://www.openstreetmap.org/relation/541821
>> How difficult would it be to add this to OSM inspector? Not everybody has
>> Postgres running, and is able to use osm2pgsql. Yes, there is documentation,
>> but it requires some technical skills. Also, it would be very convenient to
>> have this updated daily.
> It is not that difficult to add to the OSM Inspector and if I have the
> time I'll work on that together with the Geofabrik people.
>
>>>> When we have clear examples, then it might be easier to come up with a plan
>>>> how to fix it. But so far, I see absolutely no reason why Canada stands out
>>>> in a negative way. Yes, we all acknowledge that Canvec data is suboptimal,
>>>> but as others already have pointed out, mapping everything by hand in
>>>> especially remote areas is nearly impossible.
>>> Canada stands out in a negative way, because
>>> a) there are so many problems. Nearly a third of the cases worldwide are in
>>>      Canada and
>>> b) most of these problems are probably caused by one little program, the
>>>      program used to convert/import the CanVec data.
>> As you might have noticed, later imports, like the example I provided, don't
>> have that issue anymore. I'm mentioning this to express that not _all_
>> Canvec data is at fault! Only the first couple of versions. However, for
>> some reason this was never noticed up until a point that collaborative
>> action was done to have it fixed. Probably because the rendering pipeline of
>> the slippy map was accepting this kind of tagging up until recently.
> Okay, that is a big relief already. At least we are not making this
> problem worse by new imports that might happen in the future.
>
>>> Mapping Canada "by hand" might be difficult because it is such a huge
>>> country and there aren't that many mappers. But the same arguments goes
>>> for why you have to be extra careful importing data. If you break
>>> something, there are not enough people to fix it manually. And, yes,
>>> errors do happen. And if we find them, we fix them and move on. But
>>> errors from imports can be so huge there aren't enough people there to
>>> fix them manually.
>> The world is so huge that there aren't enough people to create and maintain
>> a global world map. However, OSM exists. Fixing errors can also be
>> crowdsourced. Martijn van Exel is really doing a great job with MapRoulette,
>> for instance. Although fixing errors (cleaning up the mess left behind by
>> others) is not nearly as rewarding as mapping, it might be easier to do,
>> especially since there is no need for a lot of creativity when fixing the
>> same kind of errors.
> You might have seen that I spent a lot of time in the last months to
> create more than 60 Maproulette challenges for all sorts of different
> multipolygon problems in different communities. And the community worked
> tirelessly on all these problems. (http://area.jochentopf.com/fixed.html)
>
> If the Canadian community steps up and is willing to do this work
> manually, I'd he happy to provide such Maproulette challenges. I have
> challenges running at this moment for this exact problem for other parts
> of the world (http://area.jochentopf.com/fixing.html). But I wanted to
> give the Canadian community the chance for some input first, because of
> the unique situation here.
>
>> Personally, I think that, although things were far from perfect, they were
>> done with the best intentions and with the support of the majority of the
>> Canadian OSM community. We have to deal with this situation now. A much more
>> cooperative tone would have been very welcome, especially since you would
>> like to see us coming off our lazy butts and fix our mess.
>>
>> There is really nothing to gain by threatening to contact the DWG in order
>> to have those imports removed. They already exist for about 7 years! And if
>> the Canadian community at large wouldn't have welcomed it, this would have
>> come to the surface way sooner.  So, why has this suddenly become such a
>> huge problem because of the way how the slippy map is rendered?
> Well, I tried writing a nice mail informing you all about the problem. A
> week later, when nobody had even acknowledged the problem, I wrote the
> next mail. And that did exactly what it was supposed to do: It got you
> all off your butts and discuss this problem. Now to the next step:
>
>> We can much better focus on getting the job done, than criticizing each
>> other. If 150 people are fixing 100 multipolygons each, this is doable! We
>> could do it with the help of OSM inspector, and eventually a MapRoulette
>> task.
> Thank you. That is the first time somebody from the Canadian community
> is actually addressing the issue at hand and proposing a way forward.
> Lets see whether other people in the community see it the same way and
> you can come to a solution together.
>
> I can help by providing Maproulette challenges and OSMI views and
> downloads with lists of affected relations etc. But I can't decide how
> you want to address this problem. The Canadian community has to do this.
>
>> p.s. Are you still wearing your t-shirt with Lake Manicouagan on it, based
>> on OSM data? I hope it doesn't contain wrong tagging or imported data. ;)
> Unfortunately the lake has faded a lot on that t-shirt so I don't wear
> it much any more. I should make a new one! :-)
>
> Jochen