[HOT] How to become a validator (a suggestion)
graham
graham at klunky.co.uk
Thu Apr 7 13:34:24 UTC 2016
Hi John,
I guess that there is a clear point that you
mentioned:"a validator's role is more to ensure that the major requirements of the project are met and guidelines followed."
The issue of the huts size is a tricky one, and as you say, it depends on how quickly the maps are needed by the relief agencies. For simply the number of huts,inaccurate sizes may be acceptable
for a fast
delivery but if populations are to be estimated from the size of the huts, data
would then
been lost. For the validator to either invalidate or remap, is problem, but I guess that this relies on the mapper understanding the task descriptions. Should not the task description explicitly say
something like "correct size and shape of the dwellings to be
interpreted", for example. The other issue here is the scale at which the images are interpreted. If someone maps at a too smaller scale (perhaps for a fast delivery), then potential accuracy could be lost.
Thanks for highlighting these two links:
http://wiki.openstreetmap.org/wiki/Highway_Tag_Africa
<http://wiki.openstreetmap.org/wiki/Highway_Tag_Africa>
http://wiki.openstreetmap.org/wiki/Map_Features
Validating as the project goes along from your point of view seems the good way. This is particularly
relevant with mentoring, and
valuable feed back to mappers at the time. Without feedback, new mapper might lose interest, as they want to know if they have done a good job or not, or if they have a question, they need an answer. As you say, this also gets the projects done faster.
Getting more validators, you suggest some form of recognition, this is whyI thought for that "badge" idea. Still, perhaps someone else can think of a better idea.
My final idea might be, if the
HOT team knew of interested and capable validators, perhaps they could invite them to support some projects
as
validators. Being asked for support due to one's skills, is often a reward in itself.
Thanks for your views and ideas on this subject, it has opened my eyes to the issues and challenges of validating, as well as other issues of mapping in general.
Regards,
Graham
On 6/4/2016 19:55, john whelan wrote:
> I tend to think in shades of grey rather than black and white. In OSM
> there are different mappers, each mapper interprets things slightly
> differently so two mappers will rarely give exactly the same result
> with the same inputs.
>
> For example one might like to add large buildings, the other will
> ignore them. Both are correct but are different interpretations.
> Which to my mind means a validator's role is more to ensure that the
> major requirements of the project are met and guidelines followed.
>
> One difficulty is the size of huts. For many mappers <crtl>c and
> <crtl>v is a quick way to map these. However there are ways to analyse
> the size of the hut then estimate how many people are living in it.
> Do we expect validators to check the size of the hut mapped is the
> same as the size in the image? Then do we invalidate or remap?
>
> A particular project may have a deadline to meet, NGO staff flying out
> for example. In which case they may wish to reduce the information
> requested to the bare essential minimum in order to get the project
> completed before the deadline. However many NGOs etc will make use of
> the map at some point in time in the future so additional information
> may be useful and in any event there will be some economic advantages
> to the locals in having the area mapped.
>
> We use different standards for mapping in different parts of the world
> and whilst local knowledge helps it is not essential for HOT mapping.
> What is essential is following the local agreed standards ie
> http://wiki.openstreetmap.org/wiki/Highway_Tag_Africa and some very
> experienced mappers may not be aware of them.
>
> JOSM / iD / Potlatch / OSMand amongst others are all valid methods for
> entering map data. JOSM does some validation which catches a fair
> number of basic errors before uploading and its building plugin is
> unbeaten for quickly and accurately drawing in buildings. It is my
> editor of choice. However it needs to be installed and that includes
> JAVA. JAVA has been listed in the past as being a security hole by US
> government so for some corporate machines installing it is not an
> option. Also there can be some issues with Apple computers and JAVA.
> Additionally installing it is more complex than just opening up a
> browser and using iD. So in the same way that some people freeze when
> asked to add two numbers together so some have problems even thinking
> about installing JOSM. JOSM is also very rich and I don’t use all its
> features so when you train with JOSM you need to train people just to
> use the features they need. If its just highways and settlements
> these are fairly minimal. It does have the advantage of being able to
> tag anything so you can point them back to
> http://wiki.openstreetmap.org/wiki/Map_Features which helps with
> consistency and standard ways of tagging.
>
> Different people think in different ways about validation. To me in
> order to reduce the number of errors being made in the first place I
> prefer to validate as the project is done. This is a form of coaching
> the mappers. Others take the view of map it all first then validate.
> Fine except if you’re coming in to validate two months after the
> mapping its not really worthwhile giving feedback to a mapper about
> something they did two months ago.
>
> The worst thing I’ve seen is a new mapper validating other’s work,
> select a task at random would occasionally suggest this but then you
> really don’t know if the mapping is good or not. In Nepal it was so
> bad we just invalidated everything in sight and revalidated.
>
> I often work with others validating and if its working against time
> then we delegate, check this etc and any queries bug me. I’ll go in
> and pull in much of the project into JOSM afterwards and give it a
> more through check. Are the correct tags being used? A common
> problem building=yes being used for a group of buildings.
> Highway=pedestrian and there are a few others. <crtl>i in JOSM can
> give you a fair bit of information about the item.
>
> How do we get more validators? I’m not sure, in general most mappers
> with 300 plus edits are ready to validate. However I’ve seen some
> mappers when I validate their work consistently miss 5% of
> settlements. I’ve seen others some of whom have a background in GIS
> and I’m happy if they validate after seeing them map a dozen tiles. I
> think we need to value them more and give them some sort of
> recognition for the work they do. On a couple of projects now we’ve
> validated as the mapping was done and the project gets completed very
> quickly. You’ve probably noticed that a project gets about two weeks
> in the limelight and if it hasn’t made good progress in those two
> weeks it lingers on unfinished. Get a validator in there early and
> you get more mapping with the feedback and you get more of a team
> spirit to complete the project.
>
> To my mind all projects could benefit from a validator but we don’t
> have enough and coaxing them onto new projects is the most effective
> use we can make of the ones we have.
>
> Cheerio John
>
> On 5 April 2016 at 22:07, graham <graham at klunky.co.uk
> <mailto:graham at klunky.co.uk>> wrote:
>
> Thanks for you feedback John and Blake,
>
>
> I have tried to pull out some of the points raised here, for my
> understanding and perhaps for others:
>
>
> There are different standards of mapping and to some extent
> different standards of validation needed, depending on the project.
>
> In-country knowledge of the country can effect the quality of
> mapping/validation.
>
> A good mapper does not mean that their are a good validator,
> different skill sets are needed.
>
> The capacity to map/validate is also dependant to some degree on
> the the quality of the video system and screen one is using.
>
> There seems to be a difference of quality of mapping depending on
> the software used: JOSM vs. iD. Is this an issue to be addressed?
>
> Was it suggested that coaching is a good way for people to become
> good validators?
>
> Certain projects could do with a certified validator but naturally
> to have a "badge" is not for everybody.
>
> A link to validating guidelines
> http://wiki.openstreetmap.org/wiki/OSM_Tasking_Manager/Validating_data
>
> Please say if I missed and misrepresented your points.
>
>
> Regards,
> Graham
>
>
>
> On 5/4/2016 21:49, john whelan wrote:
>> I think the point was that that there are different standards of
>> mapping and to some extent validation as well. In Nepal we had
>> time and resource constraints and I must confess I took some
>> short cuts and didn't do a through a job as could have been
>> done. Sometimes it's a judgement call and I think as you and I
>> have discussed sometimes some validators do an excellent job but
>> check rather more than either of us would when validating.
>>
>> I've even heard a whisper of a validator taking one look at
>> someone's work and zapping it and just remapping as it was
>> quicker but of course that never happens in OSM and would never
>> be documented.
>>
>> Cheerio John
>>
>> On 5 April 2016 at 09:36, Blake Girardot <bgirardot at gmail.com
>> <mailto:bgirardot at gmail.com>> wrote:
>>
>>
>>
>> On 4/5/2016 3:15 PM, john whelan wrote:
>>
>> Blake thought my methods of validation in Nepal left much
>> to be
>> desired.
>>
>>
>> That sounds a little strong :) I forget the question I had or
>> the comment, but after talking with you I was good with it
>> and agreed is all I remember.
>>
>> >I agreed but given that 70% of the mappers were new even the
>>
>> basic validation I did improved the data quality quickly.
>>
>>
>> This I 100% agree with.
>>
>> Cheers,
>> Blake
>>
>>
>> I also had a
>>
>> couple of mappers who were visually checking tiles and
>> finding 30% more
>> buildings sometimes. Which comes back to the quality of
>> the video
>> system and screen you’re using. “Why does your laptop
>> show the image in
>> JOSM better than mine in iD?” The lap top I was using
>> was an old Dell
>> professional grade one and my desktop screen at home
>> shows an even
>> clearer image. So the equipment the validator has
>> available might be an
>> important factor on the quality of the validation.
>>
>> To me validation is a form of coaching being good at
>> something doesn’t
>> mean you make a good coach. To me Maperthons are a
>> source of a dozen
>> new mappers and really questionable data. The faster we
>> can get in and
>> give feedback the better. When you need to add 50
>> settlements to a tile
>> it takes resources and to do this I’ve used sensible
>> mappers with a
>> month’s experience and delegated. If its just
>> highway=unclassified and
>> landuse=residential that’s fine. I’ve also seen mappers
>> with a thousand
>> edits to their name who don’t make good validators, the
>> project asks for
>> settlements and connecting highways, they like to map all
>> the tracks as
>> well. I’ve seen tiles invalidated for missing things
>> that were not
>> requested in the project instructions.
>>
>> It also needs tact, a European mapper who has been
>> mapping locally will
>> almost certainly use the wrong tags in Africa for
>> highways. They’re
>> high quality mappers of the type Africa needs but
>> invalidating the tile
>> because the tags are wrong may not help with the
>> retention rates.
>>
>> http://wiki.openstreetmap.org/wiki/OSM_Tasking_Manager/Validating_data
>> is a start, I’m not certain I agree with every statement
>> but it’s a good
>> start.
>>
>> There is a problem with iD mappers they tend to have more
>> crossing ways
>> and highways that almost meet than others. Probably
>> because there is no
>> easy way to check for these in iD. So whilst I would
>> comment JOSM
>> validation detected six crossing ways normally, if I know
>> they’re an iD
>> editor I just correct and don’t comment.
>>
>> If it’s a more complex project, map and tag everything in
>> sight I don’t
>> even bother validating these days. I’ll let someone else
>> with more
>> experience than I go and do it. I only have 8,000+ edits
>> to my name.
>> These projects certainly could do with a certified
>> validator and to be
>> honest I have no interest in getting a badge.
>>
>> Cheerio John
>>
>> On 5 April 2016 at 05:37, graham <graham at klunky.co.uk
>> <mailto:graham at klunky.co.uk>
>> <mailto:graham at klunky.co.uk
>> <mailto:graham at klunky.co.uk>>> wrote:
>>
>> Dear Hi,____
>>
>> __ __
>>
>> With the subject of validators in that last few
>> emails, and past emails thread discussing how to know
>> when some one can start validating or not. How much
>> experience does one need? One may think that they are
>> good, but other may not etc..
>> ____
>>
>> __ __
>>
>> So, I thought to just suggest an idea. I am not sure
>> what people might think about it, maybe it has already
>> been discussed before.
>> Maybe the issue has been solved already.____
>>
>> __ __
>>
>> The idea:____
>>
>> I think that maybe if there was a process to become
>> officially recognised as an"Validator", then it would be
>> a "position" worth achieving, it would make it a
>> challenge. For now, I do not think that there is any
>> solid pre-requisite to be a validator?____
>>
>> __ __
>>
>> Below I am suggesting a potential avenue to become an
>> official validator:____
>>
>> __ __
>>
>> 1) To complete a minimum number of tiles, as proof of
>> commitment to HOT and direct experience with HOT mapping
>> (regardless of one's professional career), then 2) to
>> take a short test of proficiency.____
>>
>> __ __
>>
>> Details:____
>>
>> 1) Can members the HOT team determine the number of
>> tiles each user has completed? I know that more that one
>> user can complete one tile, but perhaps if one user
>> completes a high percentage of a tile (of all the points,
>> line, and polygons), that would be considered a "complete
>> tile" for this purpose. As for another users then
>> reworking the tile, this is another discussion point.____
>>
>> __ __
>>
>> 2)The test might be to review a number of tiles from
>> previous projects and they need to comment on the quality
>> of the digitalisation and complete any reworking that is
>> needed. A minimum number of tiles need to be correct, (to
>> a certain standard), in order to pass the test. In each
>> test, a random subset of tiles for a complete selection
>> of tiles could be used, so that not cheating could occur.____
>>
>> __ __
>>
>> __ __
>>
>> Additionally, these official validators could have a
>> symbol next to their users names. This way, when a mapper
>> asks for advise regarding their mapping, the mapper would
>> know if a "validator" is responding or not.____
>>
>> __ __
>>
>> Just an idea...____
>>
>> __ __
>>
>> Regards,____
>>
>> __ __
>>
>> Graham____
>>
>>
>> _______________________________________________
>> HOT mailing list
>> HOT at openstreetmap.org <mailto:HOT at openstreetmap.org>
>> <mailto:HOT at openstreetmap.org <mailto:HOT at openstreetmap.org>>
>> https://lists.openstreetmap.org/listinfo/hot
>>
>>
>>
>>
>> _______________________________________________
>> HOT mailing list
>> HOT at openstreetmap.org <mailto:HOT at openstreetmap.org>
>> https://lists.openstreetmap.org/listinfo/hot
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/hot/attachments/20160407/2a29f4b0/attachment.html>
More information about the HOT
mailing list