[Talk-us] [bot] Proposal to fix issue with duplicated inner polygons in the United States

Tue Feb 22 19:45:02 UTC 2022

Hi, Andy,
Thanks for all of this—there was plenty to think about.
It’s true that the duplicate polygons are not the biggest problem here. Our intent is just to help by cleaning up the data so that it is easier for editors to correct the features. We did this work in Canada, too, in collaboration with the community and learned a lot from them. (Here’s the conversation thread<https://lists.openstreetmap.org/pipermail/talk-ca/2021-December/010184.html>.)
As to our methods, our plan is to fix the violations as they appear in Osmose<http://osmose.openstreetmap.fr/en/issues/open?item=1170&source=&class=1&username=&bbox=>.  For instance, this is the link for the features to be fixed in the Louisiana state: http://osmose.openstreetmap.fr/en/issues/open?country=usa_louisiana&item=1170&source=&class=1&username=&bbox=; this is an example of Illinois: http://osmose.openstreetmap.fr/en/issues/open?country=usa_illinois&item=1170&source=&class=1&username=&bbox=; and so on.
The count shown in the table<https://wiki.openstreetmap.org/wiki/Automated_edits/TTmechanicalupdates/Fix_issue_with_duplicated_inner_polygons_in_United_States#Bot_runs> on the wiki is the one taken from Osmose<http://osmose.openstreetmap.fr/en/issues/open?item=1170&source=&class=1&username=&bbox=>, and dates back to 8 February 2022, when we first published this documentation. We've just added a few words of explanation and a general link to this Osmose violation so that hopefully there are no more misunderstandings.

It is true, as you say, that a double inner polygon might appear on features that have incorrect geometry or incorrect tags or are generally badly captured so that even if we fix the violation as logged by Osmose, it will still leave the feature itself incorrect. Still, in cleaning up the data, we are moving things along in the right direction. Our aim is to make small contributions that we know will be good for the map.

We would like to go ahead with the plan as outlined below, but if you have further concerns, please let me know.
Courtney

From: Andy Townsend <ajt1047 at gmail.com>
Date: Monday, February 21, 2022 at 6:58 AM
To: talk-us at openstreetmap.org <talk-us at openstreetmap.org>
Subject: Re: [Talk-us] [bot] Proposal to fix issue with duplicated inner polygons in the United States

Hi everyone,

First - a caveat - I've never been to Louisiana, and much of this email is based on tidying up some similar issues in New Jersey at the weekend (at https://www.openstreetmap.org/changeset/117636871<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.openstreetmap.org%2Fchangeset%2F117636871&data=04%7C01%7Cosm%40tomtom.com%7C14d1f96f4afe422a142108d9f5315c32%7C374f80267b544a3ab87d328fa26ec10d%7C0%7C0%7C637810415392632010%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=xFmPYlFLUS2zZEC9dEElwLrZPEYqqd5SUxZdIabypYA%3D&reserved=0> and following).  However, the "duplicate inner polygon" issue was very much present there.

The biggest import issues (apart from the actual "golf fixing" that I was trying to do) that I had wasn't "duplicate inner polygons" (in any editor it's easy to move them together), they were instead:

  *   In some cases the landuse imports no longer represented reality (a recreation ground that wasn't; woods that had been completely been remodeled for a golf course)
  *   In other cases it looked like the import misclassified what was not normally or not currently wetland as "wetland" in OSM - I couldn't see any break in the trees.  Maybe the "wetland" areas just had some sort of legal protection on them or some sort of flood risk?
  *   Some of the relations involved were very, very silly.  https://www.openstreetmap.org/relation/5884686<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.openstreetmap.org%2Frelation%2F5884686&data=04%7C01%7Cosm%40tomtom.com%7C14d1f96f4afe422a142108d9f5315c32%7C374f80267b544a3ab87d328fa26ec10d%7C0%7C0%7C637810415392632010%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=YVG4NWNJmvM8DTGS76QxVbDuM1OkKXuHYeX%2FdsngmrQ%3D&reserved=0> for example is clearly NOT one feature, and (as noted previously) not, or at least not always, marsh.

Given all that, duplicate inner polygons is the last problem that I'd worry about here.

All that said, LA is clearly not NJ, and wetland areas on the coast are much more likely to be actual wetland.  Also some areas of LA (at random, https://www.openstreetmap.org/#map=16/29.4707/-89.9440<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.openstreetmap.org%2F%23map%3D16%2F29.4707%2F-89.9440&data=04%7C01%7Cosm%40tomtom.com%7C14d1f96f4afe422a142108d9f5315c32%7C374f80267b544a3ab87d328fa26ec10d%7C0%7C0%7C637810415392632010%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=kdx1rPZq9itcolhtdt3h%2BmNb4LghXdpUVmYCfkJajGk%3D&reserved=0> ) clearly have been tidied up, mostly match the available imagery, and the available imagery sources mostly agree amongst themselves.

Can you publish a list of issues in addition to the count on https://wiki.openstreetmap.org/wiki/Automated_edits/TTmechanicalupdates/Fix_issue_with_duplicated_inner_polygons_in_United_States<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.openstreetmap.org%2Fwiki%2FAutomated_edits%2FTTmechanicalupdates%2FFix_issue_with_duplicated_inner_polygons_in_United_States&data=04%7C01%7Cosm%40tomtom.com%7C14d1f96f4afe422a142108d9f5315c32%7C374f80267b544a3ab87d328fa26ec10d%7C0%7C0%7C637810415392632010%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=nYp5NvAFD5M4OXs2Wvse5UjvzkFnNggVhdXN2l6t9YQ%3D&reserved=0> so that people can assess what you are actually trying to "fix"?  That way, people will be able to see whether you'd just be changing "easily detectable broken data" into "less easily detectable broken data, or actually doing anything worthwhile.

Best Regards,

Andy

On 21/02/2022 11:10, TomTom OSM wrote:

Hello everyone!

Let us know if you have any more feedback.

If there are no more suggestions from your side, we will happily start the test run in the state of Louisiana as originally planned. After this, you’ll be able to take a look at all the changesets and see the outcome of our automated action. We will also keep you updated on the progress of our actions in this thread.

We are ready to start on Thursday 2/24/2022. Does this timing work for you?

Looking forward to hearing from you.
Marjan

From: Eric Ladner <eric.ladner at gmail.com><mailto:eric.ladner at gmail.com>
Sent: Thursday, February 17, 2022 5:06 PM
To: Eric Patrick <txemt1 at gmail.com><mailto:txemt1 at gmail.com>
Cc: TomTom OSM <OSM at tomtom.com><mailto:OSM at tomtom.com>; Minh Nguyen <minh at nguyen.cincinnati.oh.us><mailto:minh at nguyen.cincinnati.oh.us>; talk-us at openstreetmap.org<mailto:talk-us at openstreetmap.org>
Subject: Re: [Talk-us] [bot] Proposal to fix issue with duplicated inner polygons in the United States

You don't often get email from eric.ladner at gmail.com<mailto:eric.ladner at gmail.com>. Learn why this is important<http://aka.ms/LearnAboutSenderIdentification>
It's a daunting task to update thousands of miles of coastline, bays, rivers, etc., after a weather event, esp. when the satellite imagery lags by months or years.

Probably a "best-effort" thing.

On Wed, Feb 16, 2022 at 7:17 PM Eric Patrick <txemt1 at gmail.com<mailto:txemt1 at gmail.com>> wrote:
Will these be updated after every hurricane comes through and changes the coastline?

On Tue, Feb 15, 2022 at 2:24 PM Eric Ladner <eric.ladner at gmail.com<mailto:eric.ladner at gmail.com>> wrote:
Having experience with editing some of the Lousiana coastal areas, it's several failed, incomplete, and possibly overlapping NHD imports with a mess of invalid multi-polygons, unclosed features, and any other error you can think of.

I would recommend against turning the bot loose on that geometry until simpler test cases are attempted.

On Tue, Feb 15, 2022 at 5:20 AM TomTom OSM <OSM at tomtom.com<mailto:OSM at tomtom.com>> wrote:
Hi Minh,

Thank you for your kind words, and valuable feedback!

I passed your suggestions to my team. They will take a closer look at them, and I'll get back with more details in a while.

Kind regards,
Marjan

-----Original Message-----
From: Minh Nguyen <minh at nguyen.cincinnati.oh.us<mailto:minh at nguyen.cincinnati.oh.us>>
Sent: Friday, February 11, 2022 6:58 PM
To: talk-us at openstreetmap.org<mailto:talk-us at openstreetmap.org>
Subject: Re: [Talk-us] [bot] Proposal to fix issue with duplicated inner polygons in the United States

[You don't often get email from minh at nguyen.cincinnati.oh.us<mailto:minh at nguyen.cincinnati.oh.us>. Learn why this is important at http://aka.ms/LearnAboutSenderIdentification.]

Vào lúc 04:25 2022-02-08, TomTom OSM đã viết:
> We propose to do a test run by updating invalid polygons in the
> Louisiana state, where 5,541 cases are logged by the Osmose rule. Per
> the Automated Edits code of conduct, we will wait for your feedback
> before we begin.

Hi Marjan, at a glance, it looks like many of the issues involve multipolygons imported from NHD, since Louisiana has a very complex coastline and system of bayous. [1][2] I think many of us would welcome help cleaning up old NHD imports, particularly regarding topology issues.

If you later move on to other states, you may need to pay additional attention to occurrences of this warning that don't only involve natural multipolygons. For example, it's possible that a natural feature might be coincident to an enclave of a boundary relation. I'm unsure if mappers would prefer the boundary relation to be conflated with the natural feature or kept separate from it.

[1] https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.openstreetmap.org%2Fwiki%2FNational_Hydrography_Dataset&data=04%7C01%7Cosm%40tomtom.com%7Ca2065442e7e14a5f7f7408d9ed886316%7C374f80267b544a3ab87d328fa26ec10d%7C0%7C0%7C637801992953769114%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=DdeDNtg2FTWBWHq5B3nViP%2BQi7M8OqEb5kylsBUBcEg%3D&reserved=0<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.openstreetmap.org%2Fwiki%2FNational_Hydrography_Dataset&data=04%7C01%7Cosm%40tomtom.com%7C14d1f96f4afe422a142108d9f5315c32%7C374f80267b544a3ab87d328fa26ec10d%7C0%7C0%7C637810415392632010%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=JpiV1bex9Z0W%2BANMbqT92ltJYbN3q9yBq5bEAv23rKA%3D&reserved=0>
[2]
https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fosmose.openstreetmap.fr%2Fen%2Fissues%2Fopen%3Fsource%3D10139%26item%3D1170%26class%3D1%26limit%3D500&data=04%7C01%7Cosm%40tomtom.com%7Ca2065442e7e14a5f7f7408d9ed886316%7C374f80267b544a3ab87d328fa26ec10d%7C0%7C0%7C637801992953769114%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=wmlkJzpsSzLtPa36I%2B6TXqXJyG%2BPMH%2Brqn4eA2%2B13X0%3D&reserved=0<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fosmose.openstreetmap.fr%2Fen%2Fissues%2Fopen%3Fsource%3D10139%26item%3D1170%26class%3D1%26limit%3D500&data=04%7C01%7Cosm%40tomtom.com%7C14d1f96f4afe422a142108d9f5315c32%7C374f80267b544a3ab87d328fa26ec10d%7C0%7C0%7C637810415392632010%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=OSejkkxEkqiXXca89RWInmRrAvwEzonCc2CfP4FIHfM%3D&reserved=0>

--
minh at nguyen.cincinnati.oh.us<mailto:minh at nguyen.cincinnati.oh.us>

_______________________________________________
Talk-us mailing list
Talk-us at openstreetmap.org<mailto:Talk-us at openstreetmap.org>
https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.openstreetmap.org%2Flistinfo%2Ftalk-us&data=04%7C01%7Cosm%40tomtom.com%7Ca2065442e7e14a5f7f7408d9ed886316%7C374f80267b544a3ab87d328fa26ec10d%7C0%7C0%7C637801992953769114%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=BfQWDedsfE4orzbv6PdH3uLA%2B36F2KqD6RaGyJkqt%2FE%3D&reserved=0<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.openstreetmap.org%2Flistinfo%2Ftalk-us&data=04%7C01%7Cosm%40tomtom.com%7C14d1f96f4afe422a142108d9f5315c32%7C374f80267b544a3ab87d328fa26ec10d%7C0%7C0%7C637810415392788254%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=XYpqRYUreqvYmvM8RYtTFCc%2FCc%2BslpJyFPxjCeXaLyI%3D&reserved=0>
_______________________________________________
Talk-us mailing list
Talk-us at openstreetmap.org<mailto:Talk-us at openstreetmap.org>
https://lists.openstreetmap.org/listinfo/talk-us<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.openstreetmap.org%2Flistinfo%2Ftalk-us&data=04%7C01%7Cosm%40tomtom.com%7C14d1f96f4afe422a142108d9f5315c32%7C374f80267b544a3ab87d328fa26ec10d%7C0%7C0%7C637810415392788254%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=XYpqRYUreqvYmvM8RYtTFCc%2FCc%2BslpJyFPxjCeXaLyI%3D&reserved=0>

--
Eric Ladner
_______________________________________________
Talk-us mailing list
Talk-us at openstreetmap.org<mailto:Talk-us at openstreetmap.org>
https://lists.openstreetmap.org/listinfo/talk-us<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.openstreetmap.org%2Flistinfo%2Ftalk-us&data=04%7C01%7Cosm%40tomtom.com%7C14d1f96f4afe422a142108d9f5315c32%7C374f80267b544a3ab87d328fa26ec10d%7C0%7C0%7C637810415392788254%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=XYpqRYUreqvYmvM8RYtTFCc%2FCc%2BslpJyFPxjCeXaLyI%3D&reserved=0>

--
Eric Ladner

_______________________________________________

Talk-us mailing list

Talk-us at openstreetmap.org<mailto:Talk-us at openstreetmap.org>

https://lists.openstreetmap.org/listinfo/talk-us<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.openstreetmap.org%2Flistinfo%2Ftalk-us&data=04%7C01%7Cosm%40tomtom.com%7C14d1f96f4afe422a142108d9f5315c32%7C374f80267b544a3ab87d328fa26ec10d%7C0%7C0%7C637810415392788254%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=XYpqRYUreqvYmvM8RYtTFCc%2FCc%2BslpJyFPxjCeXaLyI%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/talk-us/attachments/20220222/a669c0a3/attachment-0001.htm>