[OSM-talk] Pictures of opening hours signs for machine learning purposes
Lukas.Toggenburger at fhgr.ch
Sun Apr 4 20:02:14 UTC 2021
I have made about 800 to 1000 pictures of opening hours sign of shops which I am willing to share, e.g. for machine learning purposes.
Once upon a time I envisioned that there should be a smartphone app (either standalone or part of e.g. Vespucci or StreetComplete) that simplifies the process of mapping opening hours of shops and the like when being on the go. The process would have been that a mapper can take a picture of such a sign and it would be automatically converted (using machine learning techniques) to an OSM opening hours string ( https://wiki.openstreetmap.org/wiki/Key:opening_hours ) and/or be displayed in a suitable opening hours display/editor (e.g. the one from Vespucci: https://github.com/simonpoole/OpeningHoursFragment ). Motivated by a Masters student willing to work on this as one of his semester theses I took a lot of pictures of opening hours signs, post box collection times, event announcements and the like. Unfortunately neither this student nor I find the time necessary to further work on this. In order to not let these hours of work go to waste, I search for one or multiple parties interested in either continuing to work on this or use these pictures for something else that is useful.
So what do we have?
- 2763 JPEG files (I often took multiple shots of the same sign)
- 13.5 GB
- Approx. 800 to 1000 files when deduplicated (I did only partially do this, so I don't know the exact number)
- Most pictures are shot around Lake Zürich in (german-speaking) Switzerland, a small amount of them in other cantons and countries
- Almost all pictures were shot using a Fairphone 2 smartphone (4096x3072 px) with above-average JPEG quality, most of them should be geotagged
- Almost all stem from 2018 (so probably too old to be directly useful for mapping)
- Most signs are shot with an angle to prevent myself being visible in the pictures
Here are my proposed next steps:
- Remove duplicates
- Maybe remove some of the pictures according to some criteria, e.g. language of the sign (most of them are german)
- Invent some kind of normalized opening hours scheme (so that one "meaning" of opening hours has exactly one textual representation). This is necessary since almost all opening hours instances can be represented using several strings while having the same meaning.
- Decide on an annotation scheme (in-file, out-of-file, etc.)
- Annotate the pictures with this normalized opening hours scheme
- Maybe add annotated pictures from other sources
- Train an OCR/machine learning system using the annotated pictures (maybe use fancy techniques like data augmentation)
- Implement your model as part of Vespucci, StreetComplete or a standalone app and simplify the process of mapping opening hours. Note that achieving especially high accuracy is NOT necessary since mappers still can (and probably should) check and adjust the recognized opening hours before uploading them.
- Publish the annotated pictures under a liberal licence to boost interest in improving the state-of-the-art for this kind of machine learning problem
I'm also open for usage ideas outside of OpenStreetMap.
Make sure to either write me directly or CC me when replying to the mailing list, so I won't miss your mail.
More information about the talk