[OSM-talk] Who is a good mapper? Who isn't?

[Richard Weait]

On Sat, Oct 6, 2012 at 11:42 PM, mick <bareman at tpg.com.au> wrote:
> On Sat, 6 Oct 2012 23:04:27 -0400
> Richard Weait <richard at weait.com> wrote:

>> What on Earth are you doing, Richard?
>
> Make sure you are wearing your fireproof jocks,

Thanks, Mick.  Cup and burn suit in place.  :-)

Awesome.  Only a few hours in and already I'm getting great, specific
help on this topic.  Details at the end.  Wall o' text follows.

I'm also hearing concerns that perhaps I'm mean-spirited, have lost my
mind and that the survey is a bad idea.  If you feel that way, thank
you for saying so.  I appreciate your concerns and the suggestions
that you have offered.  I've replied in private to those concerns.
I'll try to clarify, here, for other who may have just scratched their
heads and moved on. :-)

I may have taken too "flip" an approach in creating and advertising
the survey.  I would like to reassure you that I'm not Hell bent on
causing embarrassment for real mappers.  That isn't my style.  I
prefer to offer public praise or private advice.  We advocate for that
in the broader OSM community and ask mappers to discuss their
disagreements.

Mapping is fun and it can be hard.

Mapping quality is hard to determine and quantify.  Let's presume that
the quality of a specific mapper can be described with several
parameters.  That's crass, because we are all wonderful individuals.
But here I go.

Mapper skill:
uninformed to knowledgeable

Mapper experience:
inexperienced to experienced

Mapper productivity:
Maps seldom to Maps frequently

Mapper orthodoxy:
"unusual mapping" to "typical mapping"

Mapper motivation:
malicious to helpful

There are probably more.  And all of the scales are fuzzy.  Great.

At this point, we detect malicious edits via reports from other
mappers.  That's how spammers are found in the diaries and how we find
mappers who vandalize the map through ignorance or malice.  There are
a few tools for basic, automatic evaluation.

I'm working on a new one.  When we can detect, more easily, the
accounts that need more attention, then we can give them that
attention.  That might be to block and revert a spammer / vandal, or
to educate a misinformed newcomer.  You can help by building for me a
corpus of accounts that you've classified as good or bad, so I can
test my test.  Right now, I can get every mapper ranked on a scale.
Checking them is a big job and a complicated one because I don't know
every mapper and every local custom.  If you have a list of your
favourite mappers, I want to see them on the far right tail of my
scale.  If there is a mapper who does a poor job, I'd hope to find
them on the left end of the scale.

Results so far:
Most of you like to point out good mapping rather than poor.  That's
very nice, and no surprise.  Thank you.

Early results are encouraging for my test.  If you don't want to blame
a (few) specific account(s), consider describing that good / bad
aspects of their edits.

But really, help me find the errors in my scoring system before I go
public with it.  Help me fix it. :-)

http://www.surveymonkey.com/s/WPXKCPS

Best regards,
Richard.

PS: To the mapper who nominated themselves as a good mapper - my test
seems to agree with you.  :-)  Happy mapping!