[HOT] Live stats for mapathons / RedCross API / JOSM plugin (was: Mapathon stats analysis)

Tue Jun 20 06:48:54 UTC 2017

Instead of using changed, I run a query [0]: using diff and filter the
edits made by only by users who are present in the mapathon:

Caveat: there's an existing bug when querying dates with UTC offsets, so
use the actual offset time instead.

Also, instead of filtering by "uid:", you may use "user:" for usernames.
However, some new users eventually change their names to something else, so
I prefer to use the former.

[0]: https://gist.github.com/govvin/8cc3cfff204314536d744fe82c52e6f3

On Sat, Jun 17, 2017 at 7:45 PM Bjoern Hassler <bjohas+mw at gmail.com> wrote:

> Dear friends,
>
> I looked into https://github.com/AmericanRedCross/osm-stats a little, and
> while I can't quite get overpass results to match with what the API
> outputs, it does have very useful features.
>
> Is anybody interested in (or already working on) getting (live) stats for
> mapathons, via enhancements to osm-stats or otherwise?
>
> E.g. you could
> 1. Use osm-stats to get users making changes with specific hastags
> 2. Use overpass to load objects changed by those users since start of
> mapathon
>
> You could then run some stats (similar to the /user endpoint in
> osm-stats). These stats could also include results from running queries on
> bounding boxes, or on user-lists. This could all be done in combination
> with osm-stats and overpass, but there may be a case for integrating it
> into osm-stats, see
> https://github.com/AmericanRedCross/osm-stats/issues/45
>
> Either with the process above, or an extended osm-stats, you could also
> systematically check for common issues, e.g. untagged single nodes, ways
> with four nodes which are not squared, etc. I.e. you could automatically
> check for such common issues across the whole mapathon, with output like:
> "OSM user ABC added X untagged nodes and Y non-squared buildings." rather
> than wait for a validator to spot the issues manually / later.
>
> Of course, some of this this can be done via MapPaint/MapCSS-styles in
> JOSM too (e.g. single nodes), but I am not sure how you could do squared
> buildings. Also across a large area, the above could also be useful for an
> overview (and giving an indication of somebody making systematic mistake
> vs. one-off slips). The JOSM plugin could also take you to those places
> where there are issues, to inspect this (or even within tasks to fix it).
> Definitely going beyond what you can do with MapCSS or validation rule, you
> could use the JOSM plugin to visit new buildings (and whatever other
> objects) to check the tracing was done accurately.
>
> Of course, there are different approaches, that give you almost the same
> (i.e. using MapCSS or validation rules, To Do plugin etc), but a dedicated
> "MM mapathon validation" plugin would be easy to set up and could be made
> user friendly. It wouldn't replace the usual validation process, but would
> enable a "key validator" to keep an overview during a mapathon.
>
> I'd be keen to hear people's views, and whether anybody else feels this
> would be worth-while and useful.
>
> What do you think?
> Bjoern
>
>
> On 16 June 2017 at 17:07, Bjoern Hassler <bjohas+mw at gmail.com> wrote:
>
>> Hi Mike,
>>
>> Thanks for that. Yeah, at the moment I'm using overpass queries with
>> (changed:"2017-06-15T17:00:00Z","2017-06-15T21:00:00Z"); to work out
>> changes during the mapathon. You could go by bbox (and perhaps catch some
>> other MM mappers who were mapping there independently or, as you say
>> unlikely, some other mappers). Or you can go by user (no bbox) in case
>> users contributed to several projects, or by accident picked another
>> project.
>>
>> Is there a simple way of getting the bbox from the tasking manager? That
>> would be great. (Of course, it can be computed, e.g. by downloading the gpx
>> box from the NE and SW tiles... I know it's simple... but not as simple as
>> clicking a button on the tasking manager that takes you from a task to the
>> overpass query with the bbox filled in :)
>>
>> The leaderboard uses this https://github.com/AmericanRedCross, which is
>> helpful, as the Overpass API cannot query on the changeset comments, but
>> the redcross api can (and so you don't need to use the main OSM API).
>>
>> Bjoern
>>
>> On 16 June 2017 at 16:27, Mike Thompson <miketho16 at gmail.com> wrote:
>>
>>> One can use Overpass to get all of the OSM elements which were added,
>>> changed or deleted during the time of your mapathon in your AOI.  These
>>> will be tagged with the user who last touched them, so you can get stats by
>>> user.  The disadvantage is that you will include edits by users who were
>>> not part of your mapathon but who happened to made edits to OSM in that AOI
>>> during the time period of your mapathon.  However, unless you are working
>>> on a crisis task, this is not likely to skew your results too much in my
>>> experience.
>>>
>>> I sometimes make a Carto map of the results, e.g.:
>>>
>>> https://tekim.carto.com/viz/baf5c3a2-2aa8-11e7-88b2-0ef24382571b/embed_map
>>>
>>> I have more detailed instructions should someone be interested.
>>>
>>> Mike
>>>
>>>
>>> On Fri, Jun 16, 2017 at 9:15 AM, Donal Hunt <donal.hunt at gmail.com>
>>> wrote:
>>>
>>>> *[I don't speak for the team (I'm just a volunteer contributor to
>>>> missing maps) so someone with more context may chime in with better
>>>> info...]*
>>>>
>>>> I'm not familiar with the implementation of the leaderboard at all.
>>>> Someone from the HOTOSM web team probably knows. Or you can file a feature
>>>> request here I think:
>>>> https://github.com/MissingMaps/missingmaps.github.io/issues
>>>>
>>>> For the features specific to the tasking manager, I suspect that filing
>>>> them here <https://github.com/hotosm/osm-tasking-manager2/issues> (
>>>> https://github.com/hotosm/osm-tasking-manager2/issues) is the right
>>>> thing to do. That way one of the regular maintainers or a volunteer
>>>> developer can see the priority of the feature being developed and
>>>> contribute to it being implemented.
>>>>
>>>> For the stats related requests, I suspect something probably exists
>>>> already within the OSM community / ecosystem that could be tweaked for your
>>>> needs. I don't have enough context right now to suggest something (maybe in
>>>> the future).
>>>>
>>>> Regards
>>>>
>>>> Donal
>>>>
>>>> On Fri, Jun 16, 2017 at 3:38 PM, Bjoern Hassler <bjohas+mw at gmail.com>
>>>> wrote:
>>>>
>>>>> Dear Jan, dear Donal, dear friends,
>>>>>
>>>>> Is there an instance of
>>>>> https://github.com/tgrippa/Mapathon_HOT_OSM_WhatWeMapped running
>>>>> somewhere?
>>>>>
>>>>> This is great: http://www.missingmaps.org/leaderboards/#/HASHTAG,
>>>>> though there's some discrepancy between the data it shows and what I can
>>>>> see from the overpass api directly and from
>>>>> http://resultmaps.neis-one.org/. How does the leaderboards tool
>>>>> collect the information?
>>>>>
>>>>> Ideally, what I would like is:
>>>>> - all users who contributed to tasking managers under id #1234 (from
>>>>> date/time-date/time), including incomplete tasks
>>>>> - all users who contributed to the bbox of project #1234  (from
>>>>> date/time-date/time)
>>>>> - all users who used hashtag in a variety of spellings (#hashtag |
>>>>> #Hashtag | etc)  (from date/time-date/time)
>>>>> (- all users who attended a mapathon.... see below)
>>>>>
>>>>> Ideally as GET parameters or similar, i.e.
>>>>>
>>>>> http://someservice/?project=1234|1235|1236&includeincompletetasks=yes&includeprojectareas=yes&hastag=hashtag|hashtag2&hastagcasesensitive=no&start=...&end=.
>>>>> ..
>>>>>
>>>>> which would return a list of users, with projects contributed to and
>>>>> how that fact was determined (via project id1/2/3, project area for project
>>>>> id1/2/3 or hashtag1, hashtag2, etc), plus number of contributions per user
>>>>> in different categories (nodes, ways, way[building], way[highway]), plus
>>>>> (one can but dream!) the age of their OSM account and total changesets.
>>>>>
>>>>> Clearly there is redundancy in the query - but that's on purpose. It
>>>>> would e.g. find people who are mapping in an area, but not working through
>>>>> the task manager.
>>>>>
>>>>> It would also be amazing if on the tasking manager, people can
>>>>> register their attendance at a mapathon. E.g. as you go to project page it
>>>>> says: "A mapathon for this project is in progress. Click here if you are
>>>>> participating in this mapathon in person or remotely."
>>>>>
>>>>> (E.g. In terms of the query, add includeattendeelist=yes
>>>>> http://someservice/?....&includeattendeelist=yes )
>>>>>
>>>>> If you had such an attendee list tool, maybe you could even put in
>>>>> your name and table number. Then we'd immediately know who is there, and
>>>>> could start looking at their edits, and support them. I sometimes wander
>>>>> round a mapathon trying to find a user who needs a bit of extra support.
>>>>>
>>>>> Having said all of this, I'd be very happy to help build such tools,
>>>>> but would prefer to do this as part of a small team!
>>>>> Bjoern
>>>>>
>>>>>
>>>>> On 16 June 2017 at 14:09, Jan Martinec <jan at martinec.name> wrote:
>>>>>
>>>>>> Dne 16.6.2017 v 14:13 Bjoern Hassler napsal(a):
>>>>>>
>>>>>>> Dear friends,
>>>>>>>
>>>>>>>
>>>>>>> What tools do we normally use to get statistics on a mapathon?
>>>>>>>
>>>>>>> Clearly the tasking manager provides contributors to the task in the
>>>>>>> 'stats'
>>>>>>> section, and also I can run overpass to look for changes made by
>>>>>>> those users.
>>>>>>>
>>>>>>> I don't think it's possible to get changesets by #hashtag? You'd
>>>>>>> have to use the
>>>>>>> main API to get all change sets for the period of the event, and
>>>>>>> then select the
>>>>>>> ones that have the right hashtag(s)?
>>>>>>>
>>>>>>> Does anybody have some tools they could point me to?
>>>>>>>
>>>>>>> (Something like this http://resultmaps.neis-one.org/osm-changesets
>>>>>>> would be
>>>>>>> great, if it listed the changesets and users...)
>>>>>>>
>>>>>>> Many thanks!
>>>>>>> Bjoern
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> HOT mailing list
>>>>>>> HOT at openstreetmap.org
>>>>>>> https://lists.openstreetmap.org/listinfo/hot
>>>>>>>
>>>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> would this be useful? I think it provides a numerical output as well
>>>>>> as pretty pictures:
>>>>>> https://github.com/tgrippa/Mapathon_HOT_OSM_WhatWeMapped
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Jan "Piskvor" Martinec
>>>>>>
>>>>>> _______________________________________________
>>>>>> HOT mailing list
>>>>>> HOT at openstreetmap.org
>>>>>> https://lists.openstreetmap.org/listinfo/hot
>>>>>>
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> HOT mailing list
>>>> HOT at openstreetmap.org
>>>> https://lists.openstreetmap.org/listinfo/hot
>>>>
>>>>
>>>
>>
> _______________________________________________
> HOT mailing list
> HOT at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/hot
>
-- 

/Erwin Olario

e: erwin at ngnuity.xyz | v/m: https://t.me/GOwin | s: https://mstdn.io/@GOwin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/hot/attachments/20170620/f3b15c91/attachment.html>