[HOT] Data Quality comments

[john whelan]

My comments are aimed at the bulk of the problems I see and I suspect these
are from new mappers.

When I say locals perhaps I should have said those with better local
knowledge.  If a new mapper adds everything in as unclassified then I think
that is about the limit.  Above that you can't tell from imagery you need
to take into account other factors and explaining them to a new mapper
takes time and is error prone.

On iD and buildings, my suggestion would be a version or mode that only
drew buildings.  It would only allow an area to be drawn and any areas
would automatically be tagged building=yes with no other option.  It would
save time when mapping.  Coding it would be remove the interface options
other than area.  When area is finished have the program tag it
building=yes.  If possible I'd keep it in the main program but enter the
mode either by menu or <Crtl>B or some such.

This morning I looked at a bit of Africa.  I think I've added tags to a few
hundred untagged ways, mainly buildings.  I must confess I didn't check
each one to see if it was a HOT project or not but it is a waste of the
mappers time if what they mapped isn't usable and I do have other things to
do.  I suspect London has fewer untagged buildings as a percentage of
buildings mapped.

Raspberry Pi with an SSD and a usb battery pack should be around $50.
Cheaper than a laptop.  If you're using smartphones, CX file explorer can
be used to connect to a server using Wi-Fi.  If you're using Windows
machines one of the laptops can be configured as a tile server and do it in
the background.  What is needed is the documentation on how to set it up.

Things such as tagging health centres etc are best dealt with at a
different level.  It needs boots on the ground of some description.
Whether that is HOT mappers or importing good quality open data it is a
different task and needs different training to the new mapper who will be
with us for a dozen edits.

Cheerio John

On Sun, Jul 31, 2022, 12:17 Ralph Aytoun <ralph.aytoun at runbox.com> wrote:

> Thank you for your observations John. As usual many of the problems you
> highlight are valid but your assumptions are not. First please observe that
> HOT has a policy of not being condescending towards others so your comment
> "Let the locals add more, it will help them feel as if they own the map"
> They do own the map, all of us contributors "own the map". Secondly your
> comment "Adding levels from imagery just contaminates the map" does not
> "contaminate" the map. It may not be correct, I agree. But this adding a
> layer to buildings has occurred because, in their questionable wisdom, the
> iD Developer team added that in the issues function as one of the solutions
> to overlapping buildings. They have also created other issues with this
> "suggesting" fixes to beginners like a highway crossing a waterway does
> suggest a bridge or ford but neglects to include a culvert. But some
> mappers have learnt from the buildings layers and took to adding -1 to
> waterways as well so that they do not have to deal with these crossing
> point issues. As to the accurate positioning of the imagery you are once
> again correct in that it is a problem. We have talked about OSM allowing
> "fixed" features on the map that cannot be moved or changed so that we have
> something to anchor the ever changing imagery to. The OSM UK has managed to
> breach this oversight by getting permission to add a layer in JOSM that is
> fixed and we can move whichever imagery to line up with this. It is a
> surveyed land parcel layer that covers the whole of the UK. I am in
> discussion with other groups to see whether this method can be used in
> other areas where similar data is available. Then we have something to
> anchor the imagery to. Your observation that HOT mapper's first language
> will not be English is also correct but may I point you to LearnOSM
> https://learnosm.org/en/, which, by the way is managed and maintained by
> the HOT Training Working Group, which is being translated into 23 languages
> and more are in the process of being added. Also, HOT has created 3 hubs
> with a 4th in the process, to take mapping closer to the local mappers and
> encouraging them to add local detail to the map and is successfully
> onboarding a lot more non-English language groups. As you noted, bandwidth
> at a mapathon is a problem and I recall that we blew the server on at least
> one occasion which changed the way we run mapathons. But we are not just
> working with high income western mappers as we reach more and more local
> mappers who do not have the luxury of top end equipment available, or do
> not have the technical ability to play around with a Rasberry Pi, and
> internet connections can be dodgy at the best of times. For buildings we
> have approached the iD Developers on a number of occasion over the past 5
> years asking for a JOSM buildings style tool to be added to the iD Editor,
> but to no avail. Either it is technically not possible or they do not have
> someone with the skills to create it or....? Never the less it has not yet
> happened. As you will now observe that HOT is not just ignoring things,
> they are actively looking for ways to reduce the problem and have already
> found work arounds and solutions to some of the problems. We are now giving
> training to new organisations in the best ways to create projects that are
> clear and concise. By the way, it was local mappers in Africa that created
> the Highway Tag Africa Wiki document.
> https://wiki.openstreetmap.org/wiki/Highway_Tag_Africa . I would also
> bring to your attention that there is a category of road that is missing.
> That is a minor rural road, the equivalent of the residential road in towns
> and cities. This road does not link villages and towns but traverses the
> rural area connecting loose knit and wide spread communities in small
> holdings and on farms. These are not unclassified roads linking villages
> and hamlets and they are not tracks either. This is what is causing a lot
> of confusion in the mapping of highways in rural areas. I hope this helps
> you to understand that while there may be many unsquared buildings on the
> map the reality is that those people are on the map and being noticed.
> Errors are being addressed, not ignored. As an exercise I ran the JOSM
> Validation function on an area of London just north of the Thames and got
> 87 errors and 2217 warnings that includes duplicate nodes, overlapping
> railways, tree inside building, amenity inside amenity, crossing
> barrier/waterway and building, building inside building, and so on.
> Downtown New York produced 272 errors and 2135 warnings that read pretty
> much the same. Central Tokyo was not so bad with 38 errors and 860
> warnings. Central Paris was 104 errors and 865 warnings. So we can ask what
> level is good mapping and how can we realistically maintain every part of
> this world map? Please do not stop bringing problems to our attention, we
> are just as dedicated in cleaning up bad mapping and improving the accuracy
> as you are. Thank you Ralph Aytoun
>
> On Sun, 31 Jul 2022 10:13:22 -0400, John Whelan wrote:
>
> > I notein weeklyosmthat ngumenawesamson has made some comments about data
> > quality in HOT mapping.
> >
> > I've just finished going through Ghana and doing a bit of clean up.  I
> > tend to map in Africa these days and often end up cleaning up after HOT
> > mappers and I've been around a long time.
> >
> > I think we need to split the mapping into armchair from imagery and on
> > the ground.
> >
> > A couple of comments on the armchair side.
> >
> > First buildings, from imagery realistically I think all you can do is
> > draw a building and say building=yes.  I see untagged building outlines,
> > I see them labelled house, with levels, I see them labelled as layer -1
> > etc. The NGOs aren't really interested in knowing exactly where the
> > building is to the nearest centimeter they're more interested in knowing
> > it exists so they can estimate population.  Adding levels from imagery
> > just contaminates the map.
> >
> > Second highways, realistically in Africa we need to map highways between
> > settlements or between settlements and a highway and these can be
> > labelled unclassified.  Let the locals sort out any other classification
> > including paths etc.  This allows routing.  One highway connection per
> > settlement at least provides a route to it.  Let the locals add more, it
> > will help them feel as if they own the map.
> >
> > Personally I'd ignore paths and tracks going to fields.  They tend to
> > clutter the map and take up limited mapping resources.  They only other
> > highway type I'd map in HOT would be highway=residential.
> >
> > It also means that in JOSM you can just draw in the highways without
> > tags.  Then JOSM validation will select them as being untagged.  Add
> > highway unclassified and you're done.
> >
> > I think you need to understand a bit about your target audience before
> > planning your project.  Sometime ago I worked with a group of six
> > American university geography students.  Their lecturer gave them the
> > task of mapping one tile each.  I was validating on the project and gave
> > quite a lot of feedback about four times a day.  Only one student
> > completely mapped a tile and that took two weeks.  The others dropped
> > out after a week.  The lecturer's comment was she hadn't realised it was
> > so complex and wouldn't have asked them to do it if she had realised it
> > was so much work.  None of those six mappers mapped again in OSM.
> >
> > The instructions on that project needed to be fleshed out.  You needed
> > to go to other places to find out exactly how something should be tagged
> > and the project asked for everything to be mapped.
> >
> > My expectation is an American University student should have a good
> > command of English.  Realistically HOT Mappers first language will not
> > be English so instructions need to be simple.
> >
> > Also HOT projects tend to appeal to high self monitors, ( think of a
> > group of people who go out on a Friday night and together do crazy
> > things because it's fun.)  They don't have time or interest to read
> > boring instructions they just want to map so remove their choices as
> > much as possible and make the instructions simple such as:
> >
> > Raw satellite imagery is accurate to 60 meters.  To get better accuracy
> > we align it to existing buildings etc.  However expect the different
> > imagery to be aligned slightly differently and when you map a building
> > don't place it across a highway. Put it to the side even if the imagery
> > says put it in the middle of the highway.
> >
> > For buildings I and other experienced mappers running mapathons have
> > found if you give them JOSM and the buildings_tool plugin you get a lot
> > more buildings mapped by your mappers and they are more accurately
> > drawn. It takes two or three mouse clicks using the buildings_tool
> > compared to five or more in iD. I have yet to see a validator invalidate
> > a building mapped with this tool. iD gives too many choices of tags.
> >
> > JOSM using Microsoft's OpenJDK is fine and is simple to use for new
> > mappers. It does take some planning though.  Have your mappers install
> > it before arriving.  Yes it takes bandwidth but on a laptop you can set
> > up the laptop to connect to the internet at off peak hours.
> >
> > Bandwidth for a mapathon can be a problem.  Technically the image tiles
> > can be cached on a local server.  A Raspberry Pi running SAMBA and using
> > an SSD works well. However one laptop can be used as a server to the
> > other machines.  I'll leave it to HOT's technical team to sort out the
> > details and create a set of clear instructions.  Just remember JOSM
> > holds the data locally and it can be uploaded at a later date so
> > technically you could download the tile beforehand into JOSM, work with
> > off line imagery and uploaded at a later date.  The whole mapathon need
> > not be connected online which could be useful in places with poor or
> > expensive internet connections.
> >
> > On the ground mapping is different and needs a different approach.
> >
> > Cheerio John
> >
> >
> > --
> > Sent from Postbox
> > _______________________________________________
> > HOT mailing list
> > HOT at openstreetmap.org
> > https://lists.openstreetmap.org/listinfo/hot
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/hot/attachments/20220731/f9c5a955/attachment.htm>