[Talk-GB] Measuring the Quality of Mapping

[SK53]

There's quite an industry of academics doing this sort of thing (sorry few
links or I'd be all night):

* Muki Hakaly was one of the first to look at OSM, and he's usually had one
or more students looking at some aspect of this.
* Alexander Zipf's group (HeiGIT at Heidelberg do a whole range of work
with OSM, and provide tools such as OHSome to do statistical queries on OSM
data.
* Stefano de Sabbata, Leicester, had an approach
<https://www.sciencedirect.com/science/article/pii/S1353829217305804> for
assessment which, from what I recall, lent itself to being used at a local
or personal level.

Other names of academic PIs (Principal Investigators) who do a broad range
of research on or involving OSM which occur to me are: Stefan Keller
(Rapperswil), Maurizio Neapolitano (Merano), Peter Mooney (Maynooth) and Marco
Minghini (EC JRC, Ispra). Papers at various recent State of the Map may
yield other people/ideas. But beware this often ends up being a discussion
of the modifiable areal unit problem
<https://en.wikipedia.org/wiki/Modifiable_areal_unit_problem>.

* Disaster Ninja
<https://disaster.ninja/live/#id=be0cccbf-434a-4ff1-bb5d-de8a2aee1ab7;position=-1.4542041967563364,52.21596649719362;zoom=6.177361590284047>
has an "OSM quality index" background layer

Broadly, most studies indicate the following rough proxies for quality:

* Multiple mappers in an area (IIRC about 10 in one of Hakaly studies)
* Regular contributions and updates

Specific quality measures tend to be mostly done a thematic level, with
highway completeness being the earliest studied. Usually these require some
external data source for comparison purposes. We already have tools along
these lines for Food Hygiene & Solar Power. The most interesting
possibility is creating quality metrics internally derived from OSM. One
approach
<http://sk53-osm.blogspot.com/2018/07/can-we-identify-completeness-of.html>
I've advocated is looking for saturation in the mapping of a given thematic
group (note I didn't find much evidence for this with mapping shops). Peter
Reed had a very detailed thread about how retail was mapped in the UK,
which ended
<http://tlatet.blogspot.com/2015/08/osm-retail-survey-conclusions-1.html>
by deriving a fairly simple metric based on comparing OSM & FHRS data.

Something which I don't think has been particularly touched on w.r.t
quality metrics is up-to-dateness (particularly relevant because it's
harder to keep stuff up-to-date than create it). We tend to be pretty good
at getting big new roads, bridges and railway lines in as they open, but
are worse at updating new residential developments, and fairly poor at
keeping data on schools & similar things current.

tldr; there are lots of approaches and some indices are already available.

Jerry

On Tue, 30 Nov 2021 at 19:29, Chris Andrew <cjhandrew at gmail.com> wrote:

> Hi, all.
>
> This is probably a really stupid idea.
>
> Thinking about the quality of OSM mapping, is it currently possible to
> assess the mapping in a given area, and rate it with a score, based on
> quality criteria? Just thinking that in the spirit of 'continuous
> improvement' and the Deming Cycle (Plan > Do > Check > Act (repeat...)), we
> could use this method to identify areas that need improving, and use this
> to organise future efforts.
>
> I guess we could use stats that may already be available, eg missing/
> incorrect tags, where available, x-ref OSM data with other permissive data
> sources and identify potentially missing information. I know that
> Robert Whittaker has done some great work that is relevant to this
> question, but having some sort of published 'league table' of good/ bad
> areas, that can be understood by most people, would surely be a good thing.
>
> I know this is a vague idea, but as they (correctly?) say, if we can't
> measure it, we can't improve it.
>
> Am I barking up the wrong tree?
>
> Cheers,
>
> Chris
> chris_debian
>
>
> _______________________________________________
> Talk-GB mailing list
> Talk-GB at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-gb
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/talk-gb/attachments/20211130/32bd6150/attachment.htm>