[HOT] Live stats for mapathons / RedCross API / JOSM plugin (was: Mapathon stats analysis)

Sat Jun 17 11:42:38 UTC 2017

Dear friends,

I looked into https://github.com/AmericanRedCross/osm-stats a little, and
while I can't quite get overpass results to match with what the API
outputs, it does have very useful features.

Is anybody interested in (or already working on) getting (live) stats for
mapathons, via enhancements to osm-stats or otherwise?

E.g. you could
1. Use osm-stats to get users making changes with specific hastags
2. Use overpass to load objects changed by those users since start of
mapathon

You could then run some stats (similar to the /user endpoint in osm-stats).
These stats could also include results from running queries on bounding
boxes, or on user-lists. This could all be done in combination with
osm-stats and overpass, but there may be a case for integrating it into
osm-stats, see https://github.com/AmericanRedCross/osm-stats/issues/45

Either with the process above, or an extended osm-stats, you could also
systematically check for common issues, e.g. untagged single nodes, ways
with four nodes which are not squared, etc. I.e. you could automatically
check for such common issues across the whole mapathon, with output like:
"OSM user ABC added X untagged nodes and Y non-squared buildings." rather
than wait for a validator to spot the issues manually / later.

Of course, some of this this can be done via MapPaint/MapCSS-styles in JOSM
too (e.g. single nodes), but I am not sure how you could do squared
buildings. Also across a large area, the above could also be useful for an
overview (and giving an indication of somebody making systematic mistake
vs. one-off slips). The JOSM plugin could also take you to those places
where there are issues, to inspect this (or even within tasks to fix it).
Definitely going beyond what you can do with MapCSS or validation rule, you
could use the JOSM plugin to visit new buildings (and whatever other
objects) to check the tracing was done accurately.

Of course, there are different approaches, that give you almost the same
(i.e. using MapCSS or validation rules, To Do plugin etc), but a dedicated
"MM mapathon validation" plugin would be easy to set up and could be made
user friendly. It wouldn't replace the usual validation process, but would
enable a "key validator" to keep an overview during a mapathon.

I'd be keen to hear people's views, and whether anybody else feels this
would be worth-while and useful.

What do you think?
Bjoern

On 16 June 2017 at 17:07, Bjoern Hassler <bjohas+mw at gmail.com> wrote:

> Hi Mike,
>
> Thanks for that. Yeah, at the moment I'm using overpass queries with
> (changed:"2017-06-15T17:00:00Z","2017-06-15T21:00:00Z"); to work out
> changes during the mapathon. You could go by bbox (and perhaps catch some
> other MM mappers who were mapping there independently or, as you say
> unlikely, some other mappers). Or you can go by user (no bbox) in case
> users contributed to several projects, or by accident picked another
> project.
>
> Is there a simple way of getting the bbox from the tasking manager? That
> would be great. (Of course, it can be computed, e.g. by downloading the gpx
> box from the NE and SW tiles... I know it's simple... but not as simple as
> clicking a button on the tasking manager that takes you from a task to the
> overpass query with the bbox filled in :)
>
> The leaderboard uses this https://github.com/AmericanRedCross, which is
> helpful, as the Overpass API cannot query on the changeset comments, but
> the redcross api can (and so you don't need to use the main OSM API).
>
> Bjoern
>
> On 16 June 2017 at 16:27, Mike Thompson <miketho16 at gmail.com> wrote:
>
>> One can use Overpass to get all of the OSM elements which were added,
>> changed or deleted during the time of your mapathon in your AOI.  These
>> will be tagged with the user who last touched them, so you can get stats by
>> user.  The disadvantage is that you will include edits by users who were
>> not part of your mapathon but who happened to made edits to OSM in that AOI
>> during the time period of your mapathon.  However, unless you are working
>> on a crisis task, this is not likely to skew your results too much in my
>> experience.
>>
>> I sometimes make a Carto map of the results, e.g.:
>> https://tekim.carto.com/viz/baf5c3a2-2aa8-11e7-88b2-0ef24382
>> 571b/embed_map
>>
>> I have more detailed instructions should someone be interested.
>>
>> Mike
>>
>>
>> On Fri, Jun 16, 2017 at 9:15 AM, Donal Hunt <donal.hunt at gmail.com> wrote:
>>
>>> *[I don't speak for the team (I'm just a volunteer contributor to
>>> missing maps) so someone with more context may chime in with better
>>> info...]*
>>>
>>> I'm not familiar with the implementation of the leaderboard at all.
>>> Someone from the HOTOSM web team probably knows. Or you can file a feature
>>> request here I think: https://github.com/Miss
>>> ingMaps/missingmaps.github.io/issues
>>>
>>> For the features specific to the tasking manager, I suspect that filing
>>> them here <https://github.com/hotosm/osm-tasking-manager2/issues> (
>>> https://github.com/hotosm/osm-tasking-manager2/issues) is the right
>>> thing to do. That way one of the regular maintainers or a volunteer
>>> developer can see the priority of the feature being developed and
>>> contribute to it being implemented.
>>>
>>> For the stats related requests, I suspect something probably exists
>>> already within the OSM community / ecosystem that could be tweaked for your
>>> needs. I don't have enough context right now to suggest something (maybe in
>>> the future).
>>>
>>> Regards
>>>
>>> Donal
>>>
>>> On Fri, Jun 16, 2017 at 3:38 PM, Bjoern Hassler <bjohas+mw at gmail.com>
>>> wrote:
>>>
>>>> Dear Jan, dear Donal, dear friends,
>>>>
>>>> Is there an instance of https://github.com/tgrippa/
>>>> Mapathon_HOT_OSM_WhatWeMapped running somewhere?
>>>>
>>>> This is great: http://www.missingmaps.org/leaderboards/#/HASHTAG,
>>>> though there's some discrepancy between the data it shows and what I can
>>>> see from the overpass api directly and from
>>>> http://resultmaps.neis-one.org/. How does the leaderboards tool
>>>> collect the information?
>>>>
>>>> Ideally, what I would like is:
>>>> - all users who contributed to tasking managers under id #1234 (from
>>>> date/time-date/time), including incomplete tasks
>>>> - all users who contributed to the bbox of project #1234  (from
>>>> date/time-date/time)
>>>> - all users who used hashtag in a variety of spellings (#hashtag |
>>>> #Hashtag | etc)  (from date/time-date/time)
>>>> (- all users who attended a mapathon.... see below)
>>>>
>>>> Ideally as GET parameters or similar, i.e.
>>>> http://someservice/?project=1234|1235|1236&includeincomplete
>>>> tasks=yes&includeprojectareas=yes&hastag=hashtag|hashtag2&ha
>>>> stagcasesensitive=no&start=...&end=...
>>>>
>>>> which would return a list of users, with projects contributed to and
>>>> how that fact was determined (via project id1/2/3, project area for project
>>>> id1/2/3 or hashtag1, hashtag2, etc), plus number of contributions per user
>>>> in different categories (nodes, ways, way[building], way[highway]), plus
>>>> (one can but dream!) the age of their OSM account and total changesets.
>>>>
>>>> Clearly there is redundancy in the query - but that's on purpose. It
>>>> would e.g. find people who are mapping in an area, but not working through
>>>> the task manager.
>>>>
>>>> It would also be amazing if on the tasking manager, people can register
>>>> their attendance at a mapathon. E.g. as you go to project page it says: "A
>>>> mapathon for this project is in progress. Click here if you are
>>>> participating in this mapathon in person or remotely."
>>>>
>>>> (E.g. In terms of the query, add includeattendeelist=yes
>>>> http://someservice/?....&includeattendeelist=yes )
>>>>
>>>> If you had such an attendee list tool, maybe you could even put in your
>>>> name and table number. Then we'd immediately know who is there, and could
>>>> start looking at their edits, and support them. I sometimes wander round a
>>>> mapathon trying to find a user who needs a bit of extra support.
>>>>
>>>> Having said all of this, I'd be very happy to help build such tools,
>>>> but would prefer to do this as part of a small team!
>>>> Bjoern
>>>>
>>>>
>>>> On 16 June 2017 at 14:09, Jan Martinec <jan at martinec.name> wrote:
>>>>
>>>>> Dne 16.6.2017 v 14:13 Bjoern Hassler napsal(a):
>>>>>
>>>>>> Dear friends,
>>>>>>
>>>>>>
>>>>>> What tools do we normally use to get statistics on a mapathon?
>>>>>>
>>>>>> Clearly the tasking manager provides contributors to the task in the
>>>>>> 'stats'
>>>>>> section, and also I can run overpass to look for changes made by
>>>>>> those users.
>>>>>>
>>>>>> I don't think it's possible to get changesets by #hashtag? You'd have
>>>>>> to use the
>>>>>> main API to get all change sets for the period of the event, and then
>>>>>> select the
>>>>>> ones that have the right hashtag(s)?
>>>>>>
>>>>>> Does anybody have some tools they could point me to?
>>>>>>
>>>>>> (Something like this http://resultmaps.neis-one.org/osm-changesets
>>>>>> would be
>>>>>> great, if it listed the changesets and users...)
>>>>>>
>>>>>> Many thanks!
>>>>>> Bjoern
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> HOT mailing list
>>>>>> HOT at openstreetmap.org
>>>>>> https://lists.openstreetmap.org/listinfo/hot
>>>>>>
>>>>>>
>>>>> Hello,
>>>>>
>>>>> would this be useful? I think it provides a numerical output as well
>>>>> as pretty pictures:
>>>>> https://github.com/tgrippa/Mapathon_HOT_OSM_WhatWeMapped
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Jan "Piskvor" Martinec
>>>>>
>>>>> _______________________________________________
>>>>> HOT mailing list
>>>>> HOT at openstreetmap.org
>>>>> https://lists.openstreetmap.org/listinfo/hot
>>>>>
>>>>
>>>>
>>>
>>> _______________________________________________
>>> HOT mailing list
>>> HOT at openstreetmap.org
>>> https://lists.openstreetmap.org/listinfo/hot
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/hot/attachments/20170617/544de195/attachment.html>