[HOT] Data Quality comments

Sun Jul 31 16:17:03 UTC 2022

Thank you for your observations John. As usual many of the problems you
highlight are valid but your assumptions are not. First please observe
that HOT has a policy of not being condescending towards others so your
comment "Let the locals add more, it will help them feel as if they own
the map" They do own the map, all of us contributors "own the map".
Secondly your comment "Adding levels from imagery just contaminates the
map" does not "contaminate" the map. It may not be correct, I agree. But
this adding a layer to buildings has occurred because, in their
questionable wisdom, the iD Developer team added that in the issues
function as one of the solutions to overlapping buildings. They have also
created other issues with this "suggesting" fixes to beginners like a
highway crossing a waterway does suggest a bridge or ford but neglects to
include a culvert. But some mappers have learnt from the buildings layers
and took to adding -1 to waterways as well so that they do not have to
deal with these crossing point issues. As to the accurate positioning of
the imagery you are once again correct in that it is a problem. We have
talked about OSM allowing "fixed" features on the map that cannot be
moved or changed so that we have something to anchor the ever changing
imagery to. The OSM UK has managed to breach this oversight by getting
permission to add a layer in JOSM that is fixed and we can move whichever
imagery to line up with this. It is a surveyed land parcel layer that
covers the whole of the UK. I am in discussion with other groups to see
whether this method can be used in other areas where similar data is
available. Then we have something to anchor the imagery to. Your
observation that HOT mapper's first language will not be English is also
correct but may I point you to LearnOSM https://learnosm.org/en/, which,
by the way is managed and maintained by the HOT Training Working Group,
which is being translated into 23 languages and more are in the process
of being added. Also, HOT has created 3 hubs with a 4th in the process,
to take mapping closer to the local mappers and encouraging them to add
local detail to the map and is successfully onboarding a lot more
non-English language groups. As you noted, bandwidth at a mapathon is a
problem and I recall that we blew the server on at least one occasion
which changed the way we run mapathons. But we are not just working with
high income western mappers as we reach more and more local mappers who
do not have the luxury of top end equipment available, or do not have the
technical ability to play around with a Rasberry Pi, and internet
connections can be dodgy at the best of times. For buildings we have
approached the iD Developers on a number of occasion over the past 5
years asking for a JOSM buildings style tool to be added to the iD
Editor, but to no avail. Either it is technically not possible or they do
not have someone with the skills to create it or....? Never the less it
has not yet happened. As you will now observe that HOT is not just
ignoring things, they are actively looking for ways to reduce the problem
and have already found work arounds and solutions to some of the
problems. We are now giving training to new organisations in the best
ways to create projects that are clear and concise. By the way, it was
local mappers in Africa that created the Highway Tag Africa Wiki
document. https://wiki.openstreetmap.org/wiki/Highway_Tag_Africa . I
would also bring to your attention that there is a category of road that
is missing. That is a minor rural road, the equivalent of the residential
road in towns and cities. This road does not link villages and towns but
traverses the rural area connecting loose knit and wide spread
communities in small holdings and on farms. These are not unclassified
roads linking villages and hamlets and they are not tracks either. This
is what is causing a lot of confusion in the mapping of highways in rural
areas. I hope this helps you to understand that while there may be many
unsquared buildings on the map the reality is that those people are on
the map and being noticed. Errors are being addressed, not ignored. As an
exercise I ran the JOSM Validation function on an area of London just
north of the Thames and got 87 errors and 2217 warnings that includes
duplicate nodes, overlapping railways, tree inside building, amenity
inside amenity, crossing barrier/waterway and building, building inside
building, and so on. Downtown New York produced 272 errors and 2135
warnings that read pretty much the same. Central Tokyo was not so bad
with 38 errors and 860 warnings. Central Paris was 104 errors and 865
warnings. So we can ask what level is good mapping and how can we
realistically maintain every part of this world map? Please do not stop
bringing problems to our attention, we are just as dedicated in cleaning
up bad mapping and improving the accuracy as you are. Thank you Ralph
Aytoun

On Sun, 31 Jul 2022 10:13:22 -0400, John Whelan wrote:

> I notein weeklyosmthat ngumenawesamson has made some comments about
data
> quality in HOT mapping.
>
> I've just finished going through Ghana and doing a bit of clean up. I
> tend to map in Africa these days and often end up cleaning up after HOT
> mappers and I've been around a long time.
>
> I think we need to split the mapping into armchair from imagery and on
> the ground.
>
> A couple of comments on the armchair side.
>
> First buildings, from imagery realistically I think all you can do is
> draw a building and say building=yes. I see untagged building outlines,
> I see them labelled house, with levels, I see them labelled as layer -1
> etc. The NGOs aren't really interested in knowing exactly where the
> building is to the nearest centimeter they're more interested in
knowing
> it exists so they can estimate population. Adding levels from imagery
> just contaminates the map.
>
> Second highways, realistically in Africa we need to map highways
between
> settlements or between settlements and a highway and these can be
> labelled unclassified. Let the locals sort out any other classification
> including paths etc. This allows routing. One highway connection per
> settlement at least provides a route to it. Let the locals add more, it
> will help them feel as if they own the map.
>
> Personally I'd ignore paths and tracks going to fields. They tend to
> clutter the map and take up limited mapping resources. They only other
> highway type I'd map in HOT would be highway=residential.
>
> It also means that in JOSM you can just draw in the highways without
> tags. Then JOSM validation will select them as being untagged. Add
> highway unclassified and you're done.
>
> I think you need to understand a bit about your target audience before
> planning your project. Sometime ago I worked with a group of six
> American university geography students. Their lecturer gave them the
> task of mapping one tile each. I was validating on the project and gave
> quite a lot of feedback about four times a day. Only one student
> completely mapped a tile and that took two weeks. The others dropped
> out after a week. The lecturer's comment was she hadn't realised it was
> so complex and wouldn't have asked them to do it if she had realised it
> was so much work. None of those six mappers mapped again in OSM.
>
> The instructions on that project needed to be fleshed out. You needed
> to go to other places to find out exactly how something should be
tagged
> and the project asked for everything to be mapped.
>
> My expectation is an American University student should have a good
> command of English. Realistically HOT Mappers first language will not
> be English so instructions need to be simple.
>
> Also HOT projects tend to appeal to high self monitors, ( think of a
> group of people who go out on a Friday night and together do crazy
> things because it's fun.) They don't have time or interest to read
> boring instructions they just want to map so remove their choices as
> much as possible and make the instructions simple such as:
>
> Raw satellite imagery is accurate to 60 meters. To get better accuracy
> we align it to existing buildings etc. However expect the different
> imagery to be aligned slightly differently and when you map a building
> don't place it across a highway. Put it to the side even if the imagery
> says put it in the middle of the highway.
>
> For buildings I and other experienced mappers running mapathons have
> found if you give them JOSM and the buildings_tool plugin you get a lot
> more buildings mapped by your mappers and they are more accurately
> drawn. It takes two or three mouse clicks using the buildings_tool
> compared to five or more in iD. I have yet to see a validator
invalidate
> a building mapped with this tool. iD gives too many choices of tags.
>
> JOSM using Microsoft's OpenJDK is fine and is simple to use for new
> mappers. It does take some planning though. Have your mappers install
> it before arriving. Yes it takes bandwidth but on a laptop you can set
> up the laptop to connect to the internet at off peak hours.
>
> Bandwidth for a mapathon can be a problem. Technically the image tiles
> can be cached on a local server. A Raspberry Pi running SAMBA and using
> an SSD works well. However one laptop can be used as a server to the
> other machines. I'll leave it to HOT's technical team to sort out the
> details and create a set of clear instructions. Just remember JOSM
> holds the data locally and it can be uploaded at a later date so
> technically you could download the tile beforehand into JOSM, work with
> off line imagery and uploaded at a later date. The whole mapathon need
> not be connected online which could be useful in places with poor or
> expensive internet connections.
>
> On the ground mapping is different and needs a different approach.
>
> Cheerio John
>
>
> --
> Sent from Postbox
> _______________________________________________
> HOT mailing list
> HOT at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/hot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/hot/attachments/20220731/fcbf3449/attachment.htm>