[HOT] How to become a validator (a suggestion)

[john whelan]

I tend to think in shades of grey rather than black and white.  In OSM
there are different mappers, each mapper interprets things slightly
differently so two mappers will rarely give exactly the same result with
the same inputs.

For example one might like to add large buildings, the other will ignore
them.  Both are correct but are different interpretations.  Which to my
mind means a validator's role is more to ensure that the major requirements
of the project are met and guidelines followed.

One difficulty is the size of huts.  For many mappers <crtl>c and <crtl>v
is a quick way to map these.  However there are ways to analyse the size of
the hut then estimate how many people are living in it.  Do we expect
validators to check the size of the hut mapped is the same as the size in
the image?  Then do we invalidate or remap?

A particular project may have a deadline to meet, NGO staff flying out for
example.  In which case they may wish to reduce the information requested
to the bare essential minimum in order to get the project completed before
the deadline.  However many NGOs etc will make use of the map at some point
in time in the future so additional information may be useful and in any
event there will be some economic advantages to the locals in having the
area mapped.

We use different standards for mapping in different parts of the world and
whilst local knowledge helps it is not essential for HOT mapping.  What is
essential is following the local agreed standards ie
http://wiki.openstreetmap.org/wiki/Highway_Tag_Africa and some very
experienced mappers may not be aware of them.

JOSM / iD / Potlatch / OSMand amongst others are all valid methods for
entering map data.  JOSM does some validation which catches a fair number
of basic errors before uploading and its building plugin is unbeaten for
quickly and accurately drawing in buildings.  It is my editor of choice.
However it needs to be installed and that includes JAVA.  JAVA has been
listed in the past as being a security hole by US government so for some
corporate machines installing it is not an option.  Also there can be some
issues with Apple computers and JAVA.  Additionally installing it is more
complex than just opening up a browser and using iD.  So in the same way
that some people freeze when asked to add two numbers together so some have
problems even thinking about installing JOSM.  JOSM is also very rich and I
don’t use all its features so when you train with JOSM you need to train
people just to use the features they need.  If its just highways and
settlements these are fairly minimal.  It does have the advantage of being
able to tag anything so you can point them back to
http://wiki.openstreetmap.org/wiki/Map_Features which helps with
consistency and standard ways of tagging.

Different people think in different ways about validation.  To me in order
to reduce the number of errors being made in the first place I prefer to
validate as the project is done.  This is a form of coaching the mappers.
Others take the view of map it all first then validate.  Fine except if
you’re coming in to validate two months after the mapping its not really
worthwhile giving feedback to a mapper about something they did two months
ago.

The worst thing I’ve seen is a new mapper validating other’s work, select a
task at random would occasionally suggest this but then you really don’t
know if the mapping is good or not.  In Nepal it was so bad we just
invalidated everything in sight and revalidated.

I often work with others validating and if its working against time then we
delegate, check this etc and any queries bug me.  I’ll go in and pull in
much of the project into JOSM afterwards and give it a more through check.
Are the correct tags being used?  A common problem building=yes being used
for a group of buildings. Highway=pedestrian and there are a few others.
<crtl>i in JOSM can give you a fair bit of information about the item.

How do we get more validators?  I’m not sure, in general most mappers with
300 plus edits are ready to validate.  However I’ve seen some mappers when
I validate their work consistently miss 5% of settlements.  I’ve seen
others some of whom have a background in GIS and I’m happy if they validate
after seeing them map a dozen tiles.  I think we need to value them more
and give them some sort of recognition for the work they do.  On a couple
of projects now we’ve validated as the mapping was done and the project
gets completed very quickly.  You’ve probably noticed that a project gets
about two weeks in the limelight and if it hasn’t made good progress in
those two weeks it lingers on unfinished.  Get a validator in there early
and you get more mapping with the feedback and you get more of a team
spirit to complete the project.

To my mind all projects could benefit from a validator but we don’t have
enough and coaxing them onto new projects is the most effective use we can
make of the ones we have.

Cheerio John

On 5 April 2016 at 22:07, graham <graham at klunky.co.uk> wrote:

> Thanks for you feedback John and Blake,
>
>
> I have tried to pull out some of the points raised here, for my
> understanding and perhaps for others:
>
>
> There are different standards of mapping and to some extent different
> standards of validation needed, depending on the project.
>
> In-country knowledge of the country can effect the quality of
> mapping/validation.
>
> A good mapper does not mean that their are a good validator, different
> skill sets are needed.
>
> The capacity to map/validate is also dependant to some degree on the the
> quality of the video system and screen one is using.
>
> There seems to be a difference of quality of mapping depending on the
> software used: JOSM vs. iD. Is this an issue to be addressed?
>
> Was it suggested that coaching is a good way for people to become good
> validators?
>
> Certain projects could do with a certified validator but naturally to have
> a "badge" is not for everybody.
>
> A link to validating guidelines
> http://wiki.openstreetmap.org/wiki/OSM_Tasking_Manager/Validating_data
>
> Please say if I missed and misrepresented your points.
>
>
> Regards,
> Graham
>
>
>
> On 5/4/2016 21:49, john whelan wrote:
>
> I think the point was that that there are different standards of mapping
> and to some extent validation as well.  In Nepal we had time and resource
> constraints and I must confess I took some short cuts and didn't do a
> through a job as could have been done.  Sometimes it's a judgement call and
> I think as you and I have discussed sometimes some validators do an
> excellent job but check rather more than either of us would when validating.
>
> I've even heard a whisper of a validator taking one look at someone's work
> and zapping it and just remapping as it was quicker but of course that
> never happens in OSM and would never be documented.
>
> Cheerio John
>
> On 5 April 2016 at 09:36, Blake Girardot <bgirardot at gmail.com> wrote:
>
>>
>>
>> On 4/5/2016 3:15 PM, john whelan wrote:
>>
>> Blake thought my methods of validation in Nepal left much to be
>>> desired.
>>>
>>
>> That sounds a little strong :) I forget the question I had or the
>> comment, but after talking with you I was good with it and agreed is all I
>> remember.
>>
>> >I agreed but given that 70% of the mappers were new even the
>>
>>> basic validation I did improved the data quality quickly.
>>>
>>
>> This I 100% agree with.
>>
>> Cheers,
>> Blake
>>
>>
>> I also had a
>>
>>> couple of mappers who were visually checking tiles and finding 30% more
>>> buildings sometimes.  Which comes back to the quality of the video
>>> system and screen you’re using.  “Why does your laptop show the image in
>>> JOSM better than mine in iD?”  The lap top I was using was an old Dell
>>> professional grade one and my desktop screen at home shows an even
>>> clearer image.  So the equipment the validator has available might be an
>>> important factor on the quality of the validation.
>>>
>>> To me validation is a form of coaching being good at something doesn’t
>>> mean you make a good coach.  To me Maperthons are a source of a dozen
>>> new mappers and really questionable data.  The faster we can get in and
>>> give feedback the better.  When you need to add 50 settlements to a tile
>>> it takes resources and to do this I’ve used sensible mappers with a
>>> month’s experience and delegated.  If its just highway=unclassified and
>>> landuse=residential that’s fine.  I’ve also seen mappers with a thousand
>>> edits to their name who don’t make good validators, the project asks for
>>> settlements and connecting highways, they like to map all the tracks as
>>> well.  I’ve seen tiles invalidated for missing things that were not
>>> requested in the project instructions.
>>>
>>> It also needs tact, a European mapper who has been mapping locally will
>>> almost certainly use the wrong tags in Africa for highways.  They’re
>>> high quality mappers of the type Africa needs but invalidating the tile
>>> because the tags are wrong may not help with the retention rates.
>>>
>>> http://wiki.openstreetmap.org/wiki/OSM_Tasking_Manager/Validating_data
>>> is a start, I’m not certain I agree with every statement but it’s a good
>>> start.
>>>
>>> There is a problem with iD mappers they tend to have more crossing ways
>>> and highways that almost meet than others.  Probably because there is no
>>> easy way to check for these in iD. So whilst I would comment JOSM
>>> validation detected six crossing ways normally, if I know they’re an iD
>>> editor I just correct and don’t comment.
>>>
>>> If it’s a more complex project, map and tag everything in sight I don’t
>>> even bother validating these days.  I’ll let someone else with more
>>> experience than I go and do it.  I only have 8,000+ edits to my name.
>>> These projects certainly could do with a certified validator and to be
>>> honest I have no interest in getting a badge.
>>>
>>> Cheerio John
>>>
>>> On 5 April 2016 at 05:37, graham < <graham at klunky.co.uk>
>>> graham at klunky.co.uk
>>> <mailto:graham at klunky.co.uk>> wrote:
>>>
>>>     Dear Hi,____
>>>
>>>     __ __
>>>
>>>     With the subject of validators in that last few emails, and past
>>> emails thread discussing how to know when some one can start validating or
>>> not. How much experience does one need? One may think that they are good,
>>> but other may not etc..
>>>     ____
>>>
>>>     __ __
>>>
>>>     So, I thought to just suggest an idea. I am not sure what people
>>> might think about it, maybe it has already been discussed before.
>>>     Maybe the issue has been solved already.____
>>>
>>>     __ __
>>>
>>>     The idea:____
>>>
>>>     I think that maybe if there was a process to become officially
>>> recognised as an"Validator", then it would be a "position" worth achieving,
>>> it would make it a challenge. For now, I do not think that there is any
>>> solid pre-requisite to be a validator?____
>>>
>>>     __ __
>>>
>>>     Below I am suggesting a potential avenue to become an official
>>> validator:____
>>>
>>>     __ __
>>>
>>>     1) To complete a minimum number of tiles, as proof of commitment to
>>> HOT and direct experience with HOT mapping (regardless of one's
>>> professional career), then 2) to take a short test of proficiency.____
>>>
>>>     __ __
>>>
>>>     Details:____
>>>
>>>     1) Can members the HOT team determine the number of tiles each user
>>> has completed? I know that more that one user can complete one tile, but
>>> perhaps if one user completes a high percentage of a tile (of all the
>>> points, line, and polygons), that would be considered a "complete tile" for
>>> this purpose. As for another users then reworking the tile, this is another
>>> discussion point.____
>>>
>>>     __ __
>>>
>>>     2)The test might be to review a number of tiles from previous
>>> projects and they need to comment on the quality of the digitalisation and
>>> complete any reworking that is needed. A minimum number of tiles need to be
>>> correct, (to a certain standard), in order to pass the test. In each test,
>>> a random subset of tiles for a complete selection of tiles could be used,
>>> so that not cheating could occur.____
>>>
>>>     __ __
>>>
>>>     __ __
>>>
>>>     Additionally, these official validators could have a symbol next to
>>> their users names. This way, when a mapper asks for advise regarding their
>>> mapping, the mapper would know if a "validator" is responding or not.____
>>>
>>>     __ __
>>>
>>>     Just an idea...____
>>>
>>>     __ __
>>>
>>>     Regards,____
>>>
>>>     __ __
>>>
>>>     Graham____
>>>
>>>
>>>     _______________________________________________
>>>     HOT mailing list
>>>     HOT at openstreetmap.org <mailto:HOT at openstreetmap.org>
>>>     https://lists.openstreetmap.org/listinfo/hot
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> HOT mailing list
>>> HOT at openstreetmap.org
>>> https://lists.openstreetmap.org/listinfo/hot
>>>
>>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/hot/attachments/20160406/d2463cdc/attachment.html>