[HOT] Live stats for mapathons / RedCross API / JOSM plugin(was: Mapathon stats analysis)
Violaine Doutreleau
v_doutreleau at cartong.org
Tue Jun 27 08:02:13 UTC 2017
Hi,
Good to hear, don't hesitate to give us feedbacks afterwards !
For the download error, changing the overpass server can help in some
cases, but then you won't be able to use the end date option (but still
usefull when you do that just at the end of the mapathon)...
Bests,
V
Le 27/06/2017 à 09:16, kiggudde deogratias a écrit :
> I have also tried it out and it's a good one. Am going to take it out
> on a test run on the next Mapathon.
>
> On Tue, Jun 27, 2017 at 9:19 AM, Erwin Olario <govvin at gmail.com
> <mailto:govvin at gmail.com>> wrote:
>
> Boxbe <https://www.boxbe.com/overview> This message is eligible
> for Automatic Cleanup! (govvin at gmail.com
> <mailto:govvin at gmail.com>) Add cleanup rule
> <https://www.boxbe.com/popup?url=https%3A%2F%2Fwww.boxbe.com%2Fcleanup%3Fkey%3DG6sasPvMKhaW%252BLYmSQrH6gO95zP%252BX1q5hjW5Y42VI14%253D%26token%3DtXiqykbpCfGK67Lsld621pcdxcWVJu5dBfVGPmb6S8I1ogfAASQujlF0j3%252Ba4o9zB0w7M%252FZmokseFEcypoI%252F0fbFu%252F8Im%252BxOKAVleGuN736ZuKD%252Fcdx8aeQ2MrB05spnh4J28VSp%252BxQ%253D&tc_serial=31125916218&tc_rand=1828961458&utm_source=stf&utm_medium=email&utm_campaign=ANNO_CLEANUP_ADD&utm_content=001>
> | More info
> <http://blog.boxbe.com/general/boxbe-automatic-cleanup?tc_serial=31125916218&tc_rand=1828961458&utm_source=stf&utm_medium=email&utm_campaign=ANNO_CLEANUP_ADD&utm_content=001>
>
>
> This tool from CartONG looks very interesting. I just tried it on
> older TMs we've worked in the past but it's throwing me download
> errors.
>
> I'll give that script a run one of these days, Bjoern.
>
>
> On Thu, Jun 22, 2017, 22:57 Rebecca Firth
> <rebecca.firth at hotosm.org <mailto:rebecca.firth at hotosm.org>> wrote:
>
> Thanks so much for sharing that Violaine, really useful for
> mapathons especially if there is an issue with the overall
> leaderboard - thank you to the volunteer who made it too!
>
> On Thu, Jun 22, 2017 at 5:31 AM, Violaine Doutreleau
> <v_doutreleau at cartong.org <mailto:v_doutreleau at cartong.org>>
> wrote:
>
> Hi all,
>
> Here is the tool we use, developed by one of our
> volunteers : http://mapathon.cartong.org/
>
> It works pretty well in area where there is no much
> mappers external to the TM project, directly editing in
> OSM, as we are trying to figure out how to best take into
> account changeset comments (for now it works on
> bbox+timestamps). Thanks Bjoern for the ref., we will have
> a deeper look into this !
>
> Hope this can help !
>
> Bests,
> Violaine for Missing Maps
> CartONG
>
> Le 20/06/2017 à 21:09, Bjoern Hassler a écrit :
>> Hi Erwin, dear friends,
>>
>> I have some nodejs code here:
>> https://github.com/OSM-Utilities/JOSM-Scripts-HOT/blob/master/lib/mmstats.js
>> <https://github.com/OSM-Utilities/JOSM-Scripts-HOT/blob/master/lib/mmstats.js>.
>> It shows users contributing under a hashtag (using the
>> osmstats api) and then shows edits by those users
>> (buildings, roads, but also other objects), as well as
>> common issues (e.g. isolated untagged notes, untagged
>> ways, non-square 4-node buildings). E.g. you'll see that
>> for some users the non-square buildings are a few % of
>> their edits, which means they are just omissions. For
>> other users, it's more like 50%, which means they are not
>> squaring systematically.
>>
>> On a technical note: In terms of changed and diff: That
>> works for stats, but obviously also interested in
>> validation. What I'd really like is a query that gives me
>> the "current state of everything that changed between
>> these times". In the above code, you'll see some
>> recursion, that should do that (though I have to do some
>> more testing, and there may be some more recursion
>> required). I was trying to keep it all within one query
>> as well, as the queries do take time.
>>
>> Let me know if you have any questions or comments!
>> Bjoern
>>
>> On 20 June 2017 at 07:48, Erwin Olario <govvin at gmail.com
>> <mailto:govvin at gmail.com>> wrote:
>>
>>
>> Instead of using changed, I run a query [0]: using
>> diff and filter the edits made by only by users who
>> are present in the mapathon:
>>
>> Caveat: there's an existing bug when querying dates
>> with UTC offsets, so use the actual offset time instead.
>>
>> Also, instead of filtering by "uid:", you may use
>> "user:" for usernames. However, some new users
>> eventually change their names to something else, so I
>> prefer to use the former.
>>
>> [0]:
>> https://gist.github.com/govvin/8cc3cfff204314536d744fe82c52e6f3
>> <https://gist.github.com/govvin/8cc3cfff204314536d744fe82c52e6f3>
>>
>>
>> On Sat, Jun 17, 2017 at 7:45 PM Bjoern Hassler
>> <bjohas+mw at gmail.com <mailto:bjohas%2Bmw at gmail.com>>
>> wrote:
>>
>> Dear friends,
>>
>> I looked into
>> https://github.com/AmericanRedCross/osm-stats
>> <https://github.com/AmericanRedCross/osm-stats> a
>> little, and while I can't quite get overpass
>> results to match with what the API outputs, it
>> does have very useful features.
>>
>> Is anybody interested in (or already working on)
>> getting (live) stats for mapathons, via
>> enhancements to osm-stats or otherwise?
>>
>> E.g. you could
>> 1. Use osm-stats to get users making changes with
>> specific hastags
>> 2. Use overpass to load objects changed by those
>> users since start of mapathon
>>
>> You could then run some stats (similar to the
>> /user endpoint in osm-stats). These stats could
>> also include results from running queries on
>> bounding boxes, or on user-lists. This could all
>> be done in combination with osm-stats and
>> overpass, but there may be a case for integrating
>> it into osm-stats, see
>> https://github.com/AmericanRedCross/osm-stats/issues/45
>> <https://github.com/AmericanRedCross/osm-stats/issues/45>
>>
>> Either with the process above, or an extended
>> osm-stats, you could also systematically check
>> for common issues, e.g. untagged single nodes,
>> ways with four nodes which are not squared, etc.
>> I.e. you could automatically check for such
>> common issues across the whole mapathon, with
>> output like: "OSM user ABC added X untagged nodes
>> and Y non-squared buildings." rather than wait
>> for a validator to spot the issues manually / later.
>>
>> Of course, some of this this can be done via
>> MapPaint/MapCSS-styles in JOSM too (e.g. single
>> nodes), but I am not sure how you could do
>> squared buildings. Also across a large area, the
>> above could also be useful for an overview (and
>> giving an indication of somebody making
>> systematic mistake vs. one-off slips). The JOSM
>> plugin could also take you to those places where
>> there are issues, to inspect this (or even within
>> tasks to fix it). Definitely going beyond what
>> you can do with MapCSS or validation rule, you
>> could use the JOSM plugin to visit new buildings
>> (and whatever other objects) to check the tracing
>> was done accurately.
>>
>> Of course, there are different approaches, that
>> give you almost the same (i.e. using MapCSS or
>> validation rules, To Do plugin etc), but a
>> dedicated "MM mapathon validation" plugin would
>> be easy to set up and could be made user
>> friendly. It wouldn't replace the usual
>> validation process, but would enable a "key
>> validator" to keep an overview during a mapathon.
>>
>> I'd be keen to hear people's views, and whether
>> anybody else feels this would be worth-while and
>> useful.
>>
>> What do you think?
>> Bjoern
>>
>>
>> On 16 June 2017 at 17:07, Bjoern Hassler
>> <bjohas+mw at gmail.com
>> <mailto:bjohas+mw at gmail.com>> wrote:
>>
>> Hi Mike,
>>
>> Thanks for that. Yeah, at the moment I'm
>> using overpass queries with
>> (changed:"2017-06-15T17:00:00Z","2017-06-15T21:00:00Z");
>> to work out changes during the mapathon. You
>> could go by bbox (and perhaps catch some
>> other MM mappers who were mapping there
>> independently or, as you say unlikely, some
>> other mappers). Or you can go by user (no
>> bbox) in case users contributed to several
>> projects, or by accident picked another project.
>>
>> Is there a simple way of getting the bbox
>> from the tasking manager? That would be
>> great. (Of course, it can be computed, e.g.
>> by downloading the gpx box from the NE and SW
>> tiles... I know it's simple... but not as
>> simple as clicking a button on the tasking
>> manager that takes you from a task to the
>> overpass query with the bbox filled in :)
>>
>> The leaderboard uses this
>> https://github.com/AmericanRedCross
>> <https://github.com/AmericanRedCross>, which
>> is helpful, as the Overpass API cannot query
>> on the changeset comments, but the redcross
>> api can (and so you don't need to use the
>> main OSM API).
>>
>> Bjoern
>>
>> On 16 June 2017 at 16:27, Mike Thompson
>> <miketho16 at gmail.com
>> <mailto:miketho16 at gmail.com>> wrote:
>>
>> One can use Overpass to get all of the
>> OSM elements which were added, changed or
>> deleted during the time of your mapathon
>> in your AOI. These will be tagged with
>> the user who last touched them, so you
>> can get stats by user. The disadvantage
>> is that you will include edits by users
>> who were not part of your mapathon but
>> who happened to made edits to OSM in that
>> AOI during the time period of your
>> mapathon. However, unless you are working
>> on a crisis task, this is not likely to
>> skew your results too much in my experience.
>>
>> I sometimes make a Carto map of the
>> results, e.g.:
>> https://tekim.carto.com/viz/baf5c3a2-2aa8-11e7-88b2-0ef24382571b/embed_map
>> <https://tekim.carto.com/viz/baf5c3a2-2aa8-11e7-88b2-0ef24382571b/embed_map>
>>
>> I have more detailed instructions should
>> someone be interested.
>>
>> Mike
>>
>>
>> On Fri, Jun 16, 2017 at 9:15 AM, Donal
>> Hunt <donal.hunt at gmail.com
>> <mailto:donal.hunt at gmail.com>> wrote:
>>
>> /[I don't speak for the team (I'm
>> just a volunteer contributor to
>> missing maps) so someone with more
>> context may chime in with better
>> info...]/
>>
>> I'm not familiar with the
>> implementation of the leaderboard at
>> all. Someone from the HOTOSM web team
>> probably knows. Or you can file a
>> feature request here I think:
>> https://github.com/MissingMaps/missingmaps.github.io/issues
>> <https://github.com/MissingMaps/missingmaps.github.io/issues>
>>
>> For the features specific to the
>> tasking manager, I suspect that
>> filing them here
>> <https://github.com/hotosm/osm-tasking-manager2/issues> (https://github.com/hotosm/osm-tasking-manager2/issues
>> <https://github.com/hotosm/osm-tasking-manager2/issues>)
>> is the right thing to do. That way
>> one of the regular maintainers or a
>> volunteer developer can see the
>> priority of the feature being
>> developed and contribute to it being
>> implemented.
>>
>> For the stats related requests, I
>> suspect something probably exists
>> already within the OSM community /
>> ecosystem that could be tweaked for
>> your needs. I don't have enough
>> context right now to suggest
>> something (maybe in the future).
>>
>> Regards
>>
>> Donal
>>
>> On Fri, Jun 16, 2017 at 3:38 PM,
>> Bjoern Hassler <bjohas+mw at gmail.com
>> <mailto:bjohas+mw at gmail.com>> wrote:
>>
>> Dear Jan, dear Donal, dear friends,
>>
>> Is there an instance of
>> https://github.com/tgrippa/Mapathon_HOT_OSM_WhatWeMapped
>> <https://github.com/tgrippa/Mapathon_HOT_OSM_WhatWeMapped>
>> running somewhere?
>>
>> This is great:
>> http://www.missingmaps.org/leaderboards/#/HASHTAG
>> <http://www.missingmaps.org/leaderboards/#/HASHTAG>,
>> though there's some discrepancy
>> between the data it shows and
>> what I can see from the overpass
>> api directly and from
>> http://resultmaps.neis-one.org/
>> <http://resultmaps.neis-one.org/>.
>> How does the leaderboards tool
>> collect the information?
>>
>> Ideally, what I would like is:
>> - all users who contributed to
>> tasking managers under id #1234
>> (from date/time-date/time),
>> including incomplete tasks
>> - all users who contributed to
>> the bbox of project #1234 (from
>> date/time-date/time)
>> - all users who used hashtag in a
>> variety of spellings (#hashtag |
>> #Hashtag | etc) (from
>> date/time-date/time)
>> (- all users who attended a
>> mapathon.... see below)
>>
>> Ideally as GET parameters or
>> similar, i.e.
>> http://someservice/?project=1234|1235|1236&includeincompletetasks=yes&includeprojectareas=yes&hastag=hashtag|hashtag2&hastagcasesensitive=no&start=...&end=.
>> <http://someservice/?project=1234%7C1235%7C1236&includeincompletetasks=yes&includeprojectareas=yes&hastag=hashtag%7Chashtag2&hastagcasesensitive=no&start=...&end=.>..
>>
>> which would return a list of
>> users, with projects contributed
>> to and how that fact was
>> determined (via project id1/2/3,
>> project area for project id1/2/3
>> or hashtag1, hashtag2, etc), plus
>> number of contributions per user
>> in different categories (nodes,
>> ways, way[building],
>> way[highway]), plus (one can but
>> dream!) the age of their OSM
>> account and total changesets.
>>
>> Clearly there is redundancy in
>> the query - but that's on
>> purpose. It would e.g. find
>> people who are mapping in an
>> area, but not working through the
>> task manager.
>>
>> It would also be amazing if on
>> the tasking manager, people can
>> register their attendance at a
>> mapathon. E.g. as you go to
>> project page it says: "A mapathon
>> for this project is in progress.
>> Click here if you are
>> participating in this mapathon in
>> person or remotely."
>>
>> (E.g. In terms of the query, add
>> includeattendeelist=yes
>> http://someservice/?....&includeattendeelist=yes
>> <http://someservice/?....&includeattendeelist=yes>
>> )
>>
>> If you had such an attendee list
>> tool, maybe you could even put in
>> your name and table number. Then
>> we'd immediately know who is
>> there, and could start looking at
>> their edits, and support them. I
>> sometimes wander round a mapathon
>> trying to find a user who needs a
>> bit of extra support.
>>
>> Having said all of this, I'd be
>> very happy to help build such
>> tools, but would prefer to do
>> this as part of a small team!
>> Bjoern
>>
>>
>> On 16 June 2017 at 14:09, Jan
>> Martinec <jan at martinec.name
>> <mailto:jan at martinec.name>> wrote:
>>
>> Dne 16.6.2017 v 14:13 Bjoern
>> Hassler napsal(a):
>>
>> Dear friends,
>>
>>
>> What tools do we normally
>> use to get statistics on
>> a mapathon?
>>
>> Clearly the tasking
>> manager provides
>> contributors to the task
>> in the 'stats'
>> section, and also I can
>> run overpass to look for
>> changes made by those users.
>>
>> I don't think it's
>> possible to get
>> changesets by #hashtag?
>> You'd have to use the
>> main API to get all
>> change sets for the
>> period of the event, and
>> then select the
>> ones that have the right
>> hashtag(s)?
>>
>> Does anybody have some
>> tools they could point me to?
>>
>> (Something like this
>> http://resultmaps.neis-one.org/osm-changesets
>> <http://resultmaps.neis-one.org/osm-changesets>
>> would be
>> great, if it listed the
>> changesets and users...)
>>
>> Many thanks!
>> Bjoern
>>
>>
>>
>> _______________________________________________
>> HOT mailing list
>> HOT at openstreetmap.org
>> <mailto:HOT at openstreetmap.org>
>> https://lists.openstreetmap.org/listinfo/hot
>> <https://lists.openstreetmap.org/listinfo/hot>
>>
>>
>> Hello,
>>
>> would this be useful? I think
>> it provides a numerical
>> output as well as pretty
>> pictures:
>> https://github.com/tgrippa/Mapathon_HOT_OSM_WhatWeMapped
>> <https://github.com/tgrippa/Mapathon_HOT_OSM_WhatWeMapped>
>>
>> Cheers,
>>
>> Jan "Piskvor" Martinec
>>
>> _______________________________________________
>> HOT mailing list
>> HOT at openstreetmap.org
>> <mailto:HOT at openstreetmap.org>
>> https://lists.openstreetmap.org/listinfo/hot
>> <https://lists.openstreetmap.org/listinfo/hot>
>>
>>
>>
>>
>> _______________________________________________
>> HOT mailing list
>> HOT at openstreetmap.org
>> <mailto:HOT at openstreetmap.org>
>> https://lists.openstreetmap.org/listinfo/hot
>> <https://lists.openstreetmap.org/listinfo/hot>
>>
>>
>>
>>
>> _______________________________________________
>> HOT mailing list
>> HOT at openstreetmap.org <mailto:HOT at openstreetmap.org>
>> https://lists.openstreetmap.org/listinfo/hot
>> <https://lists.openstreetmap.org/listinfo/hot>
>>
>> --
>>
>> /Erwin Olario
>>
>> e: erwin at ngnuity.xyz <mailto:erwin at ngnuity.xyz> |
>> v/m: https://t.me/GOwin | s: https://mstdn.io/@GOwin
>>
>>
>>
>>
>> _______________________________________________
>> HOT mailing list
>> HOT at openstreetmap.org <mailto:HOT at openstreetmap.org>
>> https://lists.openstreetmap.org/listinfo/hot
>> <https://lists.openstreetmap.org/listinfo/hot>
> --
> *Violaine Doutreleau*Coordinatrice Missing Maps CartONG
> mobile : 06.95.02.42.44 skype : doutreleau.violaine _P
> Help save paper - do you need to print this email?_
> _______________________________________________ HOT
> mailing list HOT at openstreetmap.org
> <mailto:HOT at openstreetmap.org>
> https://lists.openstreetmap.org/listinfo/hot
> <https://lists.openstreetmap.org/listinfo/hot>
>
> --
> *Rebecca Firth*
> Community Partnerships Manager
> rebecca.firth at hotosm.org <mailto:tyler.radford at hotosm.org>
> @RebeccaFirthy
> Skype: rebeccafirth
> *Humanitarian OpenStreetMap Team*
> *Using OpenStreetMap for Humanitarian Response & Economic
> Development*
> web <http://hotosm.org/> | twitter
> <https://twitter.com/hotosm> | facebook
> <https://www.facebook.com/hotosm> | donate
> <http://donate.hotosm.org/>
>
> --
>
> /Erwin Olario
>
> e: erwin at ngnuity.xyz <mailto:erwin at ngnuity.xyz> | v/m:
> https://t.me/GOwin | s: https://mstdn.io/@GOwin
>
> _______________________________________________ HOT mailing list
> HOT at openstreetmap.org <mailto:HOT at openstreetmap.org>
> https://lists.openstreetmap.org/listinfo/hot
> <https://lists.openstreetmap.org/listinfo/hot>
>
> --
> *Humanitarian OpenStreetMap Team *Mapping Trainer, Uganda Email:
> kigguddedeogratias at gmail.com
> <mailto:kigguddedeogratias at gmail.com>Skype: deogratius kiggudde
> Twitter: @KigguddeDUG Mobile: +256 703 143 225
> *Using OpenStreetMap for Humanitarian Response & Economic Development*
> web <http://hotosm.org/> | twitter <https://twitter.com/hotosm> |
> facebook <https://www.facebook.com/hotosm> | donate
> <http://hotosm.org/donate>
-- Fwd: Re: Disponibilité de votre lieu pour un mapathon Missing Maps
*Violaine Doutreleau*Coordinatrice Missing Maps CartONG
<file:///%5C%5Cmedia%5Cviolaine%5CData%5C0.CartONG%5Cwww.cartong.org>
mobile : 06.95.02.42.44 skype : doutreleau.violaine _P Help save paper -
do you need to print this email?_
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/hot/attachments/20170627/b10b4a58/attachment.html>
More information about the HOT
mailing list