[Osmf-talk] Mapping trees as buildings (was: Re: Alternative Strategic Plan)

Tue May 16 06:34:05 UTC 2023

It seems the best approach to Emerson's concerns is to use Daylight Map
Distribution for any end user case because it removes errors and graffiti
from OpenStreetMap that the community does not miss. Otherwise the data is
not reliable as Emerson emphasizes, there is risk of getting buildings on
the map that do not exist and an end user could be puzzled by this and try
to visit the building, only finding a tree there.

As Emerson points out, OSM users are prone to error so the map itself will
be full of cases where POIs are outdated by years, roads may be wrong, one
way tags may be missing, lack of speed limits exist, bridges and streams
are not added, and much of it is not visible without ground survey.

Going to the extreme, the only solution to ensure no errors ever reach
OpenStreetMap seems to me to only allow people to edit the map who are
highly trained and certified maybe by OSMF, and to only allow edits whose
source is from ground survey. Google Maps has a good model for this, where
it professionally maps everything and the end users of it's app can suggest
changes or edits but ultimately do not have completely free "write access".

Of course, the alternative is to be an open and welcoming community and
encourage users to be careful in their map editing activity, and encourage
others to help validate. As Steve suggested, things could have a verified
date tag which can help when there is low trust in the community and their
data sources. But any data with only one human as a primary source is
likely to be used in a basemap, in navigation and routing, or other uses
cases of OSM without validation, and users who find errors should be
encouraged to edit the map and correct them.

Personally I make several edits a month in Switzerland which has high
quality, that correct wrong data. Last month I moved a statue which was
about 10 meters offset from reality, and I left a note for myself to delete
a path that does not exist. I added another path that isn't visible in
satellite images, so was questionable, but I ground surveyed it, where the
path is a key route for a self guided tour of orchards.

On Tue, May 16, 2023, 05:43 Emerson Rocha via osmf-talk <
osmf-talk at openstreetmap.org> wrote:

> Quick update here: Andy is right, this specific incident with the trees
> was not part of any organized editing, the mapper was not a novice and
> likely from the region. We discussed it a bit more on Telegram. Explicitly,
> the quoted example is *not* part of any HOT or YouthMappers organized
> editing, and this example in particular makes no sense talking about
> improving the skill of human validators.
>
> From my initial comment, it was just an real world example of an
> individual using RapID and AI generated suggestions (not an example of
> organized editing, so by current practices, no need to review).
>
> Looking at the photos, it's on the borderline of maybe be or not
> something, but a human using other editors (without suggestions) very
> likely would simply ignore. Also, the problem is not misaligned lines (like
> maybe we would see on organized editing) but creating entire features that
> don't exist which would be well aligned.
>
> I don't know when RapiD was allowed to be used outside strictly trained
> teams outside Facebook or even when the algorithms increased these kinds of
> false positives which are harder to review, however the current situation
> resembles a lot what happened in Egypt/India cited as reference on Wiki
> page about the MapWithAI https://news.ycombinator.com/item?id=17856687 .
> Some parts of the text of the link
>
> > (...)
> > Eventually we all gathered up and I just pushed the button to contribute
> a random 1km / 1km square in Egypt (we'd already computed edits across the
> country accounting for over 100% increase but things could always get
> better). Then we waited. No one ever reached out so we started contributing
> a little more at a steady rate to see if anyone else working in the country
> noticed (including improving the original 1km / 1km box as the models
> improved).
> > (...)
> > I ended up gathering a few too many people and myself along with the
> core team was too hasty in communicating the quality bar for submission. We
> planned to shadow edits as the week went on to make sure the new members
> were up to speed but things unfolded much more quickly. Within a few hours
> we accidentally submitted a few bad roads. A local mapper noticed as he'd
> just driven the area on his motorcycle the day before. I immediately left a
> meeting where we were negotiating buying more satellite imagery and jumped
> on a bike back to the war room. No harm done, we had scripts ready to undo
> edits.
> > When I got into the war room things were more problematic. Quite a few
> of the new folks had made similar mistakes so we paused everything. There
> was now a small group of local mappers in an IRC channel worried about
> large scale vandalism (though they quickly realized that wasn't the case).
> They noticed the breadth of the edits and tracked down the accounts of most
> everyone in the room. The map community in India is one of the better
> communities but still a room of this many people making edits at a such a
> scale was unlike anything you'd normally expect since the ML made editing
> 10-100x faster than hand tracing imagery.
> > (...)
>
> The rapid editor can be accessed on this link <https://rapideditor.org/>
> (it is possible to preview suggestions without logging with OSM
> credentials, so others here can check how bad it is in your region).
> Despite past issues with MapWithAI being restricted to always require
> validation, the current site has zero warnings for mappers (and we know
> non-novices can upload the bad suggestions). The MapWithAI use license for
> its users (link here https://mapwith.ai/doc/license/MapWithAILicense.pdf)
> try to clearly assume no responsibility caused by its services (which would
> be implied AI suggestions) in case user make any error, but the end data
> would still be uploaded to OSM.
>
> What to do?
>
> Att.
> Rocha
>
> ---------- Forwarded message ---------
> De: john whelan <jwhelan0112 at gmail.com>
> Date: seg, 15 de mai de 2023 19:25
> Subject: Re: [Osmf-talk] Mapping trees as buildings (was: Re: Alternative
> Strategic Plan)
> To: Damilola Olufemi <olufemidamilola263 at gmail.com>
> Cc: OSMF Talk <osmf-talk at openstreetmap.org>
>
>
> Unfortunately that raises all sorts of questions.
>
> First how do you know something has been validated.  Second what quality
> standards do you suggest?  Lines up with Bing or other imagery?
>
> Then you get to the quality of validation.  I've seen project managers in
> HOT tag something as invalid when it wasn't.
>
> HOT has done a lot of work on validation and even they struggle with the
> subject.
>
> I think the best approach is to say that the majority of mapping is
> accurate and if anyone notices any problems they can correct it.
>
> I think in general the first pass will probably be armchair mappers
> mapping major highways, then HOT mapping settlements, followed locals
> taking ownership and that's when you'll often see the quality improve.
>
> There are bottom feeders such as myself that look for obvious errors, such
> as a two km motorway connecting two small settlements, or a building mapped
> three times and correct them but I don't mark anything in a positive way to
> say I have formally validated it.
>
> Cheerio John
>
>
> On Mon, May 15, 2023, 17:45 Damilola Olufemi <olufemidamilola263 at gmail.com>
> wrote:
>
>> Hello,
>>
>> I had a similar experience with a Remote Sensing organization here in
>> Nigeria. I was overwhelmed with questions that I was not prepared enough to
>> give answers to about data quality on OpenStreetMap. My suggestion would be
>> that imageries on OpenStreetMap get reviewed more often and be sure that
>> the imageries are up to date with a good resolution. I once mentioned that
>> data quality on OpenStreetMap is easily questionable, considering the fact
>> that it is being contributed by volunteers that have other activities to
>> focus on irrespective of the amount of training received. Only a few will
>> be keen on delivering the best. Perhaps volunteers could map while trained
>> staffs validate. My opinion.
>>
>> Regards
>> Damilola
>>
>> On Mon, May 15, 2023 at 12:30 PM Andy Townsend <ajt1047 at gmail.com> wrote:
>>
>>> On 15/05/2023 05:47, Emerson Rocha via osmf-talk wrote:
>>> > Let me give a real world example. Weeks ago on unofficial
>>> > OpenStreetMap Telegram channel  (message
>>> > https://t.me/OpenStreetMapOrg/101372 ) a mapper from Colombia
>>> > complained that the buildings added in an area (near volcano Nevado
>>> > del Ruiz) was visited by local civil defense. The building=yes (added
>>> > by a human mapper, but geometry suggested
>>> > Microsoft/buildingFootprints) actually was a... tree. Guayacanes and
>>> > Yarumos to be more exact. He was pissed off on the chat, saying it
>>> > already complained in the past.
>>>
>>>
>>> For the avoidance of doubt this was a mistake by a human mapper
>>> apparently from the same country (and with a bit of experience - 1000
>>> edits or so) using "RapiD 1.1.9".  It wasn't a new contributor to OSM
>>> attending a mapathon organised by an NGO abroad.  There appear to be no
>>> changeset comments on their changes suggesting that people have noticed
>>> problems with "trees as buildings" or similar.  If a complaint was made,
>>> it doesn't appear to have been made to this mapper.  Put bluntly,
>>> they're not going to know how problematic tools like "RapiD" can be
>>> unless someone tells them.  That's not to say that RapiD aren't really
>>> useful at finding missing buildings; just that they will also find some
>>> false positives too (a quick scan locally (UK) finds around 20 missing
>>> sheds and small agricultural buildings that no-one has bothered to add -
>>> and one that appears to be a bit of scrap metal, that isn't.
>>>
>>> I'd suggest (and will suggest on Telegram) that a changeset comment
>>> helping them understand how to use tools such as this would be the best
>>> initial approach.
>>>
>>> Best Regards,
>>>
>>> Andy
>>>
>>> (as usual writing here in a personal capacity, and apologies from
>>> dragging this thread even further from "strategic" things)
>>>
>>>
>>>
>>> _______________________________________________
>>> osmf-talk mailing list
>>> osmf-talk at openstreetmap.org
>>> https://lists.openstreetmap.org/listinfo/osmf-talk
>>>
>> _______________________________________________
>> osmf-talk mailing list
>> osmf-talk at openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/osmf-talk
>>
> _______________________________________________
> osmf-talk mailing list
> osmf-talk at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/osmf-talk
> _______________________________________________
> osmf-talk mailing list
> osmf-talk at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/osmf-talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/osmf-talk/attachments/20230516/111b3166/attachment.htm>