[Imports] Facebook's AI-Assisted Road Tracing for OSM
osm
osm at fb.com
Tue Mar 21 21:02:53 UTC 2017
Hello OSM Enthusiasts,
Firstly, thank you for all the feedback. We really appreciate it. Sorry for the delay in responding. I have spent the last few days gathering all the questions and have come up with a list of answers that I hope brings some clarity. I tried to send this e-mail yesterday but had too many example pics so it didn’t go through. Please click on the “Example” links to see pictures we shared to help clarify some of the question around process. ☺
Can you share DG imagery?
Our license does not currently permit this. However, to reiterate what Kevin from Digital Globe said, earlier on this thread, this will not be an issue in a few weeks when all of OSM will get an imagery refresh.
Facebook License with Digital Globe
This license is necessary to comply with our obligations to Digital Globe, and we think the benefits of supplying this data outweigh the licensing restraints. The data will be under ODBL once it has been contributed to OSM in compliance with the terms.
There is no contradiction between the OSM terms and the Facebook license with Digital Globe. Our license terms do not apply to data once contributed to OSM and termination of the agreement with Digital Globe would not impact the data that has already been contributed to OSM.
Is the Imagery free of Cloud Cover?
The intended output of +Vivid is cloud free, and in most cases, DG meets this. There are some highly cloudy parts of the world where it’s unavoidable. On average though, this product maintains less than 1% cloud cover.
Who are your editors?
Our team consists of 3 engineers and 14 editors, and usernames can be found on the Import wiki. All editors have also updated their profile to say they map for Facebook. You can see their names on their profile. All our mappers are qualified for editing with backgrounds in GIS, geography and are extremely technically savvy. We have also spent a considerable amount of time over the last 5 months training specifically for OSM.
The mappers are not bound by time limits and metrics as the focus is primarily on quality. We do however track time taken to edit, density of tile and who mapped what tile to use this information to plan for future mapping.
Facebook does not control these accounts and all users manage their own profiles and passwords. Once we start mapping we also expect each mapper to respond to questions from the community. All feedback is discussed as a group to ensure we are all aware of the feedback and can learn as team. No other accounts other than the ones listed here and the wiki have been used to make edits for Facebook.
Our team includes:
Jaclyn<https://www.openstreetmap.org/user/VLD001> - VLD001
Annie<https://www.openstreetmap.org/user/VLD002> - VLD002
Alexandra<https://www.openstreetmap.org/user/VLD003> - VLD003
Sheffield<https://www.openstreetmap.org/user/VLD004> - VLD004
Mei<https://www.openstreetmap.org/user/VLD005> - VLD005
Joseph<https://www.openstreetmap.org/user/VLD006> - VLD006
Kurt<https://www.openstreetmap.org/user/VLD007> - VLD007
Pablo Presti-Mohr<https://www.openstreetmap.org/user/VLD008> VLD008
Goerge<https://www.openstreetmap.org/user/RVR001> - RVR001
Jeff<https://www.openstreetmap.org/user/RVR002> - RVR002
David<https://www.openstreetmap.org/user/RVR003> - RVR003
Yunzhi<https://www.openstreetmap.org/user/RVR004> - RVR004
Adrian<https://www.openstreetmap.org/user/RVR005> - RVR005
Stefani<https://www.openstreetmap.org/user/RVR006> - RVR006
Mohamed<https://www.openstreetmap.org/user/RVR007> - RVR007
Who is behind osm at fb.com<mailto:osm at fb.com>?
There are 4 people behind this email. This helps us collaborate as a team and is the most efficient way for everyone to be on the same page. More people also ensures that no emails are missed. I (Drishtie Patel) as the Program Manager primarily watch this email and respond to queries. The other 3 people are our Engineers namely, Ming Gao and Saikat Basu and the Product Manager for Maps at Facebook, Sadi Khan.
Can the Chageset be more specific?
Yes we can absolutely change that to import=facebook-ai-____. We don't specifically have versions but can include the area or task number for example. Open to suggestions.
As for adding more details to our edits: there were no changes to current OSM data in this sample and we plan to add notes when we upload as would normally happen when using the Tasking Manager and iD. The sample shared has not been uploaded yet ☺
Can you Publish Source Code for ML
We cannot share the source code for Machine Learning at this time, but we can and will share our internal tools.
Can you Publish Source Code for internal tools?
Yes. We are currently using a version of iD Editor<https://github.com/openstreetmap/iD> and the HOT Tasking Manager<https://github.com/hotosm/osm-tasking-manager2>. We are absolutely going to share our tooling. We are currently working on creating a Github where we will post soon.
Can Facebook Share other data?
This is clear violation of privacy so we cannot share this kind of information. We are exploring ways to crowdsource road names for OSM and will share that as we have a more concrete plan.
Are you going to Map other countries?
Yes. We are moving slowly to focus efforts on one country at a time to make sure our process is accepted by the OSM community out of respect for the process.
Yes we made a mistake
We've learned from a number of missteps in our initial efforts in Egypt and Thailand last year. We have since refined our processes, hired a team of well experienced mappers and are working to follow all the processes and requests being asked of us.
What is the Overall Process?
PHASE 1 - Generating Road Masks
1. Training data is created by editors. Example<http://scontent.xx.fbcdn.net/v/t39.10482-6/16781235_1170760256386350_6655657355411193856_n.jpg?oh=2a84c16359c99bec01f9d1d23b9241ee&oe=595CF8FC>
2. Training data used by engineers in ML to produce road masks. Example<http://scontent.xx.fbcdn.net/v/t39.10482-6/17365429_1592481874110291_3999181234856001536_n.png?oh=df901589c651edc1883bf0472eb1d4ca&oe=59670662>
3. Road masks are processed to remove low confidence predictions and add connections between short breaks.
PHASE 2 - Creating Road Vectors (.osm files)
1. Road masks are then processed using an algorithm from a black and white image to a vector format specifically a .osm file. Example<http://scontent.xx.fbcdn.net/v/t39.10482-6/16781103_1267857896629755_8804482991712632832_n.png?oh=cb25f91d0e6159b08f6ec342353ae77a&oe=593F3547>
2. Our .osm file is then merged with current OSM changesets. During this process the following things take place
· All current OSM data for an area is merged, keeping full history.
· No changes are made to the current OSM data.
· Duplicated roads are deleted from the ML generated data. Current OSM data is always taken over facebook generated data.
· New ML identified roads are connected to current OSM roads.
3. We then have a locally stored file containing both our generated roads and the current OSM data set for a specific area.
PHASE 3 - Human Validation
1. We use the Tasking manager to divide up tasks and create a flow of editing and validation for the Facebook mapping team. Example<http://scontent.xx.fbcdn.net/v/t39.10482-6/17365380_1448559118508805_536801470968758272_n.jpg?oh=3cb38653fa004f2a20eeaabc2cfa583c&oe=5971703D>
2. The Mapper picks a task and loads the locally stored .osm file generated from the post processing steps outlined above. Example<http://scontent.xx.fbcdn.net/v/t39.10482-6/17433416_1595081350521384_2351112230801506304_n.jpg?oh=304c8670744ee2baf27c78043cc6943a&oe=595ECAF0>
3. The task is opened up in our enhanced version of iD. (This includes both the roads generated by us and the current OSM data). Example<http://scontent.xx.fbcdn.net/v/t39.10482-6/16686335_1118257054950682_8271679018604953600_n.jpg?oh=b033e8c22c7b7248a1def85d1733fadc&oe=596EA8D8>
4. Our generated roads are highlighted in a different color so editors can inspect it for issues like crossing highways, disconnected roads, incorrect intersections, short road stubs, road types, etc., and fix these issues manually one by one, using Digital Globe satellite imagery background.
5. We do not delete current OSM data, but make typical OSM editing changes where necessary to ensure high quality of the data (more precisely follow imagery, consistent tagging, etc).
6. Our modified iD tool is equipped with data validation functionalities similar to JOSM and osmlint https://github.com/osmlab/osmlint/. This allows us to check for quality and conflicts with current OSM data. Some examples of what we check for include:
· isolated roads that are not connected to rest of the map. Example-Before<http://scontent.xx.fbcdn.net/v/t39.10482-6/16781243_1457031487701473_198753992489041920_n.jpg?oh=a3413fae5079f7ce38519d9977b28a9b&oe=5971B86C>. Example-After<http://scontent.xx.fbcdn.net/v/t39.10482-6/17433411_1244723028968011_867537480591605760_n.jpg?oh=f5a7cbeebec559fc6c80bfbfeef2d1b6&oe=59255F3A>
· roads crossing waterways.
· roads with repeating node IDs.
· roads with possible connections after a short extension. Example-Before<http://scontent.xx.fbcdn.net/v/t39.10482-6/17365393_1561733437200741_5520973609088057344_n.jpg?oh=486151bca24ba30880738c15a1770f87&oe=596A8307>. Example-After<http://scontent.xx.fbcdn.net/v/t39.10482-6/17351734_1279516578799625_2079794241860534272_n.jpg?oh=5eb3c2118ae35ab40ddae75f225c7ad8&oe=595C2079>
7. In case of conflicts between our newly created roads and other OSM editor's mapping, our mapper will almost always choose “keep their edits”. If we think our newly added roads are better aligned with the latest version of DG satellite imagery, we'll contact the other mapper offline to reach resolution. If we end up changing other mapper's edits, we will leave detailed notes to explain why.
8. Until all errors are fixed mappers will not be able to save. Here is an example<http://scontent.xx.fbcdn.net/v/t39.10482-6/16685255_1430090083699471_1532498821462884352_n.png?oh=8438b1e92957b809f59b874507521bbc&oe=5973316B> of a highlighted issue prompting the mapper to fix it.
9. After fixing all the issues detected, our mapper clicks the “Save Local” button in the iD tool to save their edited roads locally for validation.
10. A second person, the validator then goes into the same task to verify that the data looks correct, makes necessary changes as needed and clicks “Submit” to finally upload the tile to OSM.
Can you share more data samples?
We have attached a few samples that we are sharing based on the 3 Phases explained above. We do these in small batches and are not sitting on a bunch of data for the whole country but are happy to share what we have created over the last couple of days.
1. Phase 1 - Generating Road Masks. These are the masks of the initial output from machine learning. These are not processed to a vector format. We have offered to share our tile service<https://wiki.openstreetmap.org/wiki/AI-Assisted_Road_Tracing> to anyone. Sample 1<http://scontent.xx.fbcdn.net/v/t39.10482-6/16685603_1205005339598346_4564649276063350784_n.jpg?oh=b2619991eef35541c7a0fb1fd7d3e19a&oe=59276BD6> | Sample 2<http://scontent.xx.fbcdn.net/v/t39.10482-6/17351717_1626376154057536_1688258658015641600_n.jpg?oh=ac96567463786eb6047cae69fb9a6c19&oe=59277824> | Sample 3<http://scontent.xx.fbcdn.net/v/t39.10482-6/16686305_1494502210621428_618761533014409216_n.jpg?oh=3316b9f6798e3367c678fb8b93cb808d&oe=595F35AB> | Sample 4<http://scontent.xx.fbcdn.net/v/t39.10482-6/17119819_1384749424908927_2865149829640617984_n.jpg?oh=a812255978a6b91933392451b693f5ae&oe=59255A6B>
2. Phase 2 - Creating Road Vectors. This is the processed road vectors merged with current OSM data locally - These .osm files that have NOT been edited or validated by people and would not be uploaded as is. (Please see attached)
3. Phase 3 - Human Validation. This is the final output validated by mappers twice in order to be uploaded to OSM. While we are happy to share this please note the time taken to validate, share and wait for feedback would be counter productive because the current OSM data may be changed during this time. This increases the possibility for conflicts. We plan to map small areas at a time to ensure we are working with current OSM data and validating conflicts quickly and smoothly as we add our generated roads. Sample 1<http://scontent.xx.fbcdn.net/v/t39.10482-6/17351465_1228329023883542_5416323939192274944_n.jpg?oh=d34ccfb8c092940bdadf4b322a9bb33f&oe=59720FD0> | Sample 2<http://scontent.xx.fbcdn.net/v/t39.10482-6/17365486_1399125510131415_7405155781655396352_n.jpg?oh=786797c91760722e701eabb66ad37a65&oe=5962161B> | Sample 3<http://scontent.xx.fbcdn.net/v/t39.10482-6/17365310_1444333455611506_7249929757997924352_n.jpg?oh=a5f90f3d7f0ec4c6f5224d766da0be5c&oe=596697D0> | Sample 4<http://scontent.xx.fbcdn.net/v/t39.10482-6/17351405_1379847552085633_543677743674425344_n.jpg?oh=6f9f2c7c4e18736c28a7da8e5f2f46ea&oe=59581CCC>
For the attached samples, please note these are not edited and would not in any way be uploaded to OSM as is. We have also provided screenshots for Phase 1 and Phase 3 as it would appear in our internal tools with the DG imagery before being edited.
Look forward to more feedback :)
Best,
Drishtie Patel on behalf of the OSM at Facebook Team
OSM Profile<https://www.openstreetmap.org/user/DrishT>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/imports/attachments/20170321/cc548028/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Sample Data - Phase 2.zip
Type: application/zip
Size: 227564 bytes
Desc: Sample Data - Phase 2.zip
URL: <http://lists.openstreetmap.org/pipermail/imports/attachments/20170321/cc548028/attachment-0001.zip>
More information about the Imports
mailing list