[OSM-talk] Helping mappers feel comfortable about their contributions / quality control

[Jochen Topf]

Hi!

On Tue, Sep 20, 2011 at 08:57:22AM -0500, Jaakko Helleranta.com wrote:
> Some thoughts regarding various discussions about quality control /
> mechanisms in OSM:
> 
> We (Seb and I) had a brief discussion yesterday with one of the advanced
> Haitian mappers about what improvements the locals would like to see in OSM
> tools to make their contributing to the project easier / more fluent.
> 
> One of the first things that came out was a request to have some sort of
> edit review system for anyone who doesn't feel super comfortable about their
> edits when sending them (or for those mappers' edits that the group would
> like to have reviewed before uploading).
> 
> This is pretty much exactly what Ian suggested in the "Barriers of Entry"
> thread the other day.
> 
> Interestingly I got a personal experience in using such a tool just a moment
> ago when I read a system message from Google Map Maker regarding my approved
> edit in Map Maker. I've attached the message below for reference.

I have seen blog posts complaining about the Google system where local
knowledge was ignored by Google admins sitting thousands of miles away who
did know a lot less about some edit than the originator. 

> Now, I obviously don't know how the Google system works (more than what I
> can see). But what I saw when making the edit is that there is a part (that
> I think all must fill - for all edited/added features, afaik) that asks the
> contributor how confident they are about the edit(s): Very much so, pretty
> confident, somewhat confident, "I'm just guessing" - or something along
> those lines.
> 
> I think this makes all the sense in the world.

In theory this is a good idea, in practice I don't expect this to work. New
mappers who are unsure about the work they do will be unsure about which
confidence levels to use, too. Experienced mappers will be annoyed by the
system and, if forced to use it, always click on "I am totally sure".

If everybody is not forced to use it, most data will not have this confidence
level information. But if I don't see what the confidence level of existing
data is, how am I supposed to judge it in relation to my confidence level of
some new data I have? I have to look at the data itself. And thats what we
are doing anyway.

A editorial process and confidence level thing is one *more* thing people have
to learn how to use. OSM is hard enough, lets not make it more complicated to
use.

Metadata never works. Metadata is never updated properly. Thats because there
is no intrinsic value in keeping it current. It doesn't show up on the map.
And a confidence level is just one kind of metadata.

We can see this effect with the "source" tag. In theory it should be updated
to always reflect the source or sources of some piece of data. In practice
it is hard to tell how to update this properly, so most people don't do it,
and it looses all its value for everybody, because even if it is maintained
you can't tell and so can't rely on it.

> So, in short:
> 1) It would be really great if we could cater to the Haitian mappers and all
> others who would rather have someone review their edits before they are
> uploaded (or at least have a flag system for review requests). I'd love to
> use such system for my own edits, too, at times!
> 
> 2) A "confidence level" indicator either on node/element edit level or
> perhaps on changeset level (or something else?) would also be nice --
> combined with / linked to above-mentioned (optional) review system.
> 
> I think having something like this will become imperative if OSM wants to /
> ends up expanding its contributor base significantly.

There is no way around it: Mapper have to learn how to judge the data thats
there and their own data in relation to it. Mappers will make mistakes, thats
part of the process. A reviewing process tied to the main OSM database would
be very hard to implement, you'll need to keep track of multiple versions of
everything etc.

I think "the system" works fine. We do have a lot of platforms for people to
ask questions and to discuss changes. There are web forums and mailing lists
and bug trackers (like opestreetbugs which allows comments). People can come
to local meetups and discuss particular changes there. Lets work on these
kind of platforms to make them more accessible and easier to use.

Jochen
-- 
Jochen Topf  jochen at remote.org  http://www.remote.org/jochen/  +49-721-388298