[HOT] Building Detection using Machine Learning

Thu Jan 5 19:43:58 UTC 2017

Hi Philip

On Mon, Dec 19, 2016 at 3:17 PM, Philip Hunt <phunt123 at icloud.com> wrote:
> I attended my first Humanitarian OpenStreetMap Team (HOT) mapping event a few months ago
> and was interested to see how successful machine learning would be at detecting buildings in
> satellite images. The results look promising but I wanted to know if it could be useful to the
> community and if it’s worth pursuing further.

Looks nice!

Extracting buildings from satellite imagery has been approached quite
some times before. I think that object recognition algos. now get
close to 98%. But even if would be 99.8 percent it's important for OSM
that a human integrates the data into the database.

Your project is very similar to my OSMDeepOD project [1]. We use OSM
and Bing satellite imagery which we feed to the open source software
library TensorFlow with a retrained inception V3 convolution neural
network ("deep learning").

We concentrated on missing crosswalks and are now switching to
building outline, roof orientation and building levels.

As proposed by Rory we take the output and hand it over to MapRoulette
[2]. Actually there are currently 14 MapRoulette challenges ongoing
like e.g. the "Missing Crosswalks of Lucerne in Switzerland". Even if
you don't read German, you can see the progress here [3].

I'm keen to see how your Viola+Jones/Haar-Cascade algorithm performs
versus our deep learning!

We're open to collaborate and accept GitHub pull requests.

:Stefan

[1] https://github.com/geometalab/OSMDeepOD
[2] http://maproulette.org/
[3] http://zebrastreifen-safari.osm.ch/

2017-01-05 16:58 GMT+01:00 Denis Carriere <carriere.denis at gmail.com>:
> I agree with Rory with the rotated buildings.
>
> However, these building outlines would be great to detect missing buildings
> within an area. It's usually very hard to find "missing" buildings when your
> HOT Tasking Manager is at 100% completed.
>
> This workflow/tool can definitely be used in the validation process of a HOT
> task or adding buildings to an entire city by comparing the existing OSM
> data with the machine learning buildings and find the ones that don't
> overlap.
>
> I don't ever see this ever being imported, but it's a great tool and you
> should continue developing this!
>
> Cheers,
>
> ~~~~~~
> Denis Carriere
> GIS Software & Systems Specialist
>
> On Thu, Jan 5, 2017 at 10:41 AM, Rory McCann <rory at technomancy.org> wrote:
>>
>> You've noticed how your algoritm isn't able to get properly rotated
>> buildings. And this might be an advantage! All buildings being
>> non-rotated is *obviously* incorrect, so people aren't gonna want to
>> import them, they'll realise that they have to have a human to
>> review/correct it.
>>
>> Perhaps you should load all your "buildings" into MapRoullette or ToFix
>> and people can hope from one spot to the next, mapping buildings? Your
>> "buildings" will ensure that some of the harder work is done, and people
>> can map much faster. It also provides a good measure of how many
>> buildings are in an area, and hence whether the OSM building mapping is
>> complete or some is missing.
>>
>>
>> On 19/12/16 15:17, Philip Hunt wrote:
>> > Hi all,
>> >
>> > I attended my first Humanitarian OpenStreetMap Team (HOT) mapping event
>> > a few months ago and was interested to see how successful machine learning
>> > would be at detecting buildings in satellite images. The results look
>> > promising but I wanted to know if it could be useful to the community and if
>> > it’s worth pursuing further. I thought I would post a sample of the results
>> > and then quickly explain the process and issues.
>> >
>> >
>> > Results
>> > ———
>> >
>> > These are the results of a test I ran on project 2101 (Rongo, Kenya -
>> > PMI/USAID) on 1 November 2016. These images show the buildings detected by
>> > the algorithm on the first six unstarted tasks from the project. Potential
>> > buildings are marked with green rectangles:
>> >
>> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_4.png
>> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_5.png
>> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_9.png
>> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_12.png
>> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_13.png
>> > https://s3-eu-west-1.amazonaws.com/hot-osm-ml-test-data/2101_14.png
>> >
>> > As you can see the initial results look promising - most of the
>> > buildings have been detected and the false positive rate is pretty low.
>> >
>> >
>> > Process
>> > ————
>> >
>> > I’ve been using the Viola–Jones machine learning algorithm, which
>> > requires training to know what is and isn’t a building. Once the algorithm
>> > is trained, it can be used to detect buildings in new images in a few
>> > seconds.
>> >
>> > The whole process looks like this:
>> >
>> > - Get the HOT project and task data using the HOT API
>> > - Get the satellite imagery of the area from OSM
>> > - Get the nearby existing buildings from the OSM API
>> > - Find the existing buildings in the satellite imagery and use these to
>> > train the algorithm
>> > - Run through each incomplete task in the HOT project and detect
>> > buildings
>> > - Output the results as OSM XML
>> > - Load the output into JOSM, validate and upload to OSM
>> >
>> >
>> > Issues
>> > ———
>> >
>> > I loaded the output of the algorithm into JOSM and completed tasks 1 and
>> > 2 of project 2101. However it still took a bit of work to make sure the data
>> > is good enough for OSM and I think an experienced mapper would have taken
>> > roughly the same amount of time starting from scratch.
>> >
>> > The main issue is the algorithm can’t rotate the detected rectangle to
>> > fit the building shape (as you can see from the example images above, none
>> > of the rectangles are rotated). I’ve tried using methods such as line
>> > detection to detect the building and rotate and crop the rectangle around
>> > the edges - this worked well some of the time and other times went horribly
>> > wrong.
>> >
>> > The second issue is false positives. While the examples above we’re
>> > generally clean, sometimes the algorithm would think a field was a building.
>> > Because data uploaded to OSM needs to be accurate it can take some time
>> > checking each potential building in JOSM.
>> >
>> > Another potential issue could be training samples. When testing I
>> > trained a new algorithm for each project, using local existing building data
>> > from OSM as training data. The assumption here is that nearby buildings will
>> > look like buildings in the project area and that nearby building data is
>> > available and accurate.
>> >
>> >
>> > Next Steps
>> > —————
>> >
>> > The Viola–Jones objection detection research paper was first published
>> > in 2001 so the algorithm has been around for a while. Machine learning has
>> > improved since then and neural networks are showing a lot of promise - using
>> > these could increase the reliability and also allow the detected rectangle
>> > to be fixed around the edge of the building - meaning a lot less editing in
>> > JOSM. I’m also aware of similar projects, but I haven’t found anything
>> > that’s able to detect buildings or ready for use yet:
>> >
>> > https://github.com/trailbehind/DeepOSM (find misconfigured roads in OSM)
>> > https://github.com/patrick-dd/landsat-landstats (predicts population
>> > size)
>> > https://github.com/larsroemheld/OSM-HOT-ConvNet
>> >
>> >
>> > I'm not suggesting this could replace volunteers (since algorithms will
>> > never be completely accurate), but maybe this could help speed things up or
>> > be used to quickly estimate building locations over large areas.
>> >
>> > Please let me know your thoughts. Could this could be a useful tool for
>> > HOT or any other volunteer organisations and is it worth taking any further?
>> >
>> > Thanks,
>> >
>> > Philip Hunt
>> >
>> >
>> > _______________________________________________
>> > HOT mailing list
>> > HOT at openstreetmap.org
>> > https://lists.openstreetmap.org/listinfo/hot
>> >
>>
>>
>>
>> _______________________________________________
>> HOT mailing list
>> HOT at openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/hot
>>
>
>
> _______________________________________________
> HOT mailing list
> HOT at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/hot
>