<div dir="ltr"><div dir="auto"><div dir="auto">Quick update here: Andy is right, this specific incident with the trees was not part of any organized editing, the mapper was not a novice and likely from the region. We discussed it a bit more on Telegram. Explicitly, the quoted example is *not* part of any HOT or YouthMappers organized editing, and this example in particular makes no sense talking about improving the skill of human validators.<div dir="auto"><br></div><div dir="auto">From my initial comment, it was just an real world example of an individual using RapID and AI generated suggestions (not an example of organized editing, so by current practices, no need to review).</div><div dir="auto"><br></div><div dir="auto">Looking at the photos, it's on the borderline of maybe be or not something, but a human using other editors (without suggestions) very likely would simply ignore. Also, the problem is not misaligned lines (like maybe we would see on organized editing) but creating entire features that don't exist which would be well aligned.</div><div dir="auto"><br></div><div dir="auto">I don't know when RapiD was allowed to be used outside strictly trained teams outside Facebook or even when the algorithms increased these kinds of false positives which are harder to review, however the current situation resembles a lot what happened in Egypt/India cited as reference on Wiki page about the MapWithAI <a href="https://news.ycombinator.com/item?id=17856687" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank">https://news.ycombinator.com/item?id=17856687</a> . Some parts of the text of the link</div><div dir="auto"><br>> (...)<br>> Eventually we all gathered up and I just pushed the button to contribute a random 1km / 1km square in Egypt (we'd already computed edits across the country accounting for over 100% increase but things could always get better). Then we waited. No one ever reached out so we started contributing a little more at a steady rate to see if anyone else working in the country noticed (including improving the original 1km / 1km box as the models improved).<br>> (...)<br>> I ended up gathering a few too many people and myself along with the core team was too hasty in communicating the quality bar for submission. We planned to shadow edits as the week went on to make sure the new members were up to speed but things unfolded much more quickly. Within a few hours we accidentally submitted a few bad roads. A local mapper noticed as he'd just driven the area on his motorcycle the day before. I immediately left a meeting where we were negotiating buying more satellite imagery and jumped on a bike back to the war room. No harm done, we had scripts ready to undo edits.<br>> When I got into the war room things were more problematic. Quite a few of the new folks had made similar mistakes so we paused everything. There was now a small group of local mappers in an IRC channel worried about large scale vandalism (though they quickly realized that wasn't the case). They noticed the breadth of the edits and tracked down the accounts of most everyone in the room. The map community in India is one of the better communities but still a room of this many people making edits at a such a scale was unlike anything you'd normally expect since the ML made editing 10-100x faster than hand tracing imagery.<br>> (...)</div><div dir="auto"><br></div><div dir="auto">The rapid editor can be accessed on this link <<a href="https://rapideditor.org/" rel="noreferrer noreferrer" target="_blank">https://rapideditor.org/</a>> (it is possible to preview suggestions without logging with OSM credentials, so others here can check how bad it is in your region). Despite past issues with MapWithAI being restricted to always require validation, the current site has zero warnings for mappers (and we know non-novices can upload the bad suggestions). The MapWithAI use license for its users (link here <a href="https://mapwith.ai/doc/license/MapWithAILicense.pdf">https://mapwith.ai/doc/license/MapWithAILicense.pdf</a>) try to clearly assume no responsibility caused by its services (which would be implied AI suggestions) in case user make any error, but the end data would still be uploaded to OSM.</div><div dir="auto"><br></div><div>What to do?</div><div dir="auto"><br></div><div dir="auto">Att.</div><div>Rocha</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">---------- Forwarded message ---------<br>De: <strong class="gmail_sendername" dir="auto">john whelan</strong> <span dir="auto"><<a href="mailto:jwhelan0112@gmail.com" rel="noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">jwhelan0112@gmail.com</a>></span><br>Date: seg, 15 de mai de 2023 19:25<br>Subject: Re: [Osmf-talk] Mapping trees as buildings (was: Re: Alternative Strategic Plan)<br>To: Damilola Olufemi <<a href="mailto:olufemidamilola263@gmail.com" rel="noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">olufemidamilola263@gmail.com</a>><br>Cc: OSMF Talk <<a href="mailto:osmf-talk@openstreetmap.org" rel="noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">osmf-talk@openstreetmap.org</a>><br></div><br><br><div dir="auto"><div>Unfortunately that raises all sorts of questions.<div dir="auto"><br></div><div dir="auto">First how do you know something has been validated. Second what quality standards do you suggest? Lines up with Bing or other imagery?</div><div dir="auto"><br></div><div dir="auto">Then you get to the quality of validation. I've seen project managers in HOT tag something as invalid when it wasn't.</div><div dir="auto"><br></div><div dir="auto">HOT has done a lot of work on validation and even they struggle with the subject.</div><div dir="auto"><br></div><div dir="auto">I think the best approach is to say that the majority of mapping is accurate and if anyone notices any problems they can correct it.</div><div dir="auto"><br></div><div dir="auto">I think in general the first pass will probably be armchair mappers mapping major highways, then HOT mapping settlements, followed locals taking ownership and that's when you'll often see the quality improve.</div><div dir="auto"><br></div><div dir="auto">There are bottom feeders such as myself that look for obvious errors, such as a two km motorway connecting two small settlements, or a building mapped three times and correct them but I don't mark anything in a positive way to say I have formally validated it.</div><div dir="auto"><br></div><div dir="auto">Cheerio John</div><br><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, May 15, 2023, 17:45 Damilola Olufemi <<a href="mailto:olufemidamilola263@gmail.com" rel="noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">olufemidamilola263@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hello, <div><br></div><div>I had a similar experience with a Remote Sensing organization here in Nigeria. I was overwhelmed with questions that I was not prepared enough to give answers to about data quality on OpenStreetMap. My suggestion would be that imageries on OpenStreetMap get reviewed more often and be sure that the imageries are up to date with a good resolution. I once mentioned that data quality on OpenStreetMap is easily questionable, considering the fact that it is being contributed by volunteers that have other activities to focus on irrespective of the amount of training received. Only a few will be keen on delivering the best. Perhaps volunteers could map while trained staffs validate. My opinion. </div><div><br></div><div>Regards</div><div>Damilola </div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, May 15, 2023 at 12:30 PM Andy Townsend <<a href="mailto:ajt1047@gmail.com" rel="noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">ajt1047@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 15/05/2023 05:47, Emerson Rocha via osmf-talk wrote:<br>
> Let me give a real world example. Weeks ago on unofficial <br>
> OpenStreetMap Telegram channel (message <br>
> <a href="https://t.me/OpenStreetMapOrg/101372" rel="noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">https://t.me/OpenStreetMapOrg/101372</a> ) a mapper from Colombia <br>
> complained that the buildings added in an area (near volcano Nevado <br>
> del Ruiz) was visited by local civil defense. The building=yes (added <br>
> by a human mapper, but geometry suggested <br>
> Microsoft/buildingFootprints) actually was a... tree. Guayacanes and <br>
> Yarumos to be more exact. He was pissed off on the chat, saying it <br>
> already complained in the past.<br>
<br>
<br>
For the avoidance of doubt this was a mistake by a human mapper <br>
apparently from the same country (and with a bit of experience - 1000 <br>
edits or so) using "RapiD 1.1.9". It wasn't a new contributor to OSM <br>
attending a mapathon organised by an NGO abroad. There appear to be no <br>
changeset comments on their changes suggesting that people have noticed <br>
problems with "trees as buildings" or similar. If a complaint was made, <br>
it doesn't appear to have been made to this mapper. Put bluntly, <br>
they're not going to know how problematic tools like "RapiD" can be <br>
unless someone tells them. That's not to say that RapiD aren't really <br>
useful at finding missing buildings; just that they will also find some <br>
false positives too (a quick scan locally (UK) finds around 20 missing <br>
sheds and small agricultural buildings that no-one has bothered to add - <br>
and one that appears to be a bit of scrap metal, that isn't.<br>
<br>
I'd suggest (and will suggest on Telegram) that a changeset comment <br>
helping them understand how to use tools such as this would be the best <br>
initial approach.<br>
<br>
Best Regards,<br>
<br>
Andy<br>
<br>
(as usual writing here in a personal capacity, and apologies from <br>
dragging this thread even further from "strategic" things)<br>
<br>
<br>
<br>
_______________________________________________<br>
osmf-talk mailing list<br>
<a href="mailto:osmf-talk@openstreetmap.org" rel="noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">osmf-talk@openstreetmap.org</a><br>
<a href="https://lists.openstreetmap.org/listinfo/osmf-talk" rel="noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">https://lists.openstreetmap.org/listinfo/osmf-talk</a><br>
</blockquote></div>
_______________________________________________<br>
osmf-talk mailing list<br>
<a href="mailto:osmf-talk@openstreetmap.org" rel="noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">osmf-talk@openstreetmap.org</a><br>
<a href="https://lists.openstreetmap.org/listinfo/osmf-talk" rel="noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">https://lists.openstreetmap.org/listinfo/osmf-talk</a><br>
</blockquote></div></div></div>
_______________________________________________<br>
osmf-talk mailing list<br>
<a href="mailto:osmf-talk@openstreetmap.org" rel="noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">osmf-talk@openstreetmap.org</a><br>
<a href="https://lists.openstreetmap.org/listinfo/osmf-talk" rel="noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">https://lists.openstreetmap.org/listinfo/osmf-talk</a><br>
</div></div>
</div>