[Talk-ca] Importing buildings in Canada
Nate Wessel
bike756 at gmail.com
Sun Jan 5 16:40:10 UTC 2020
The task size, and even shape is totally customizable. I've set up a
couple that are entirely based on the density of the data:
http://tasks.osmcanada.ca/project/168
https://tasks.openstreetmap.us/project/107
Best,
Nate Wessel, PhD
Planner, Cartographer, Transport Nerd
NateWessel.com <https://www.natewessel.com>
On 2020-01-04 12:40 p.m., Daniel @jfd553 wrote:
>
> Bonjour groupe
>
> Looks like we're going in the same direction so far :-)
>
> I agree with Nate regarding the implementation of the task manager. In
> my experience, a size of a few blocks would be better in urban areas,
> but boring in rural areas. Is it something that can be adjusted?
>
> Daniel
>
> *From:*Nate Wessel [mailto:bike756 at gmail.com]
> *Sent:* Saturday, January 04, 2020 10:09
> *To:* talk-ca at openstreetmap.org
> *Subject:* Re: [Talk-ca] Importing buildings in Canada
>
> Hi Daniel,
>
> Thank you for all the work you've put into this. I'd like to offer a
> couple suggestions and/or clarifications for your proposed import
> process, overview though it is.
>
> First, I think it is very important that a tasking manager is set up
> on a city/by city basis only, and that only AFTER consensus is
> achieved that the import should proceed in that area. I would really
> like to avoid seeing the massive nationwide tasking that was set up
> the first time around. We should be making it hard for people to go
> rogue in regions where consensus for an import doesn't (yet) exist.
>
> Related to this, though important enough to be a second point in it's
> own right, the tasking squares need to be small enough that a single
> user can manage them and inspect every single building in a task. The
> first round of import used task squares that were massive, and which
> couldn't be divided any further past a certain point. Even in rural
> areas, it is likely inappropriate to import areas larger than 1km^2.
> In central Toronto it would be (and was) idiotic. An import that
> doesn't take local scale into account shouldn't proceed. "Too big to
> load into JOSM" is about 100x too big to import in my opinion and is
> not a good enough benchmark for import batch sizing.
>
> That is, each import needs to be local, and not just in a superficial
> sense.
>
> I'll also add that the issue of conflation doesn't seem to have been
> worked out yet except to note that it is an issue. What will we do
> with the millions of buildings which will substantially
> overlap/duplicate existing buildings or imports? This needs to be
> worked out in detail before anything starts up again.
>
> And what needs to be done about already existing low quality imports?
> It's good to acknowledge their existence, but what will be done about
> them? We've set up a task to clean up some of the mess in Toronto (
> http://tasks.osmcanada.ca/project/168 ) but this is only the tip of
> the iceberg.
>
> Again, I thank everyone for their time and effort on this - we can get
> this done if we go slow and do it right :-)
>
> Best,
>
> Nate Wessel, PhD
> Planner, Cartographer, Transport Nerd
> NateWessel.com <https://www.natewessel.com>
>
> On 2020-01-03 3:40 p.m., Daniel @jfd553 wrote:
>
> Bonjour groupe, mes excuses pour ce très long courriel !-)
>
> I have reviewed everything that has been written on the ODB import
> (aka Canada Building Import) in Talk-ca and the wiki. I proposed
> changes to some wiki pages (via talk tabs) to ease the discussions
> about this import and the following. Now, in order to restart the
> import, here are some thoughts and a proposal on how to proceed to
> complete the task.
>
> *1. Issues with the ODB Data Import*
>
> Many concerns were raised about the import. One major concern was
> to obtain local communities’ buy-in in the Canadian context.
> Another concern was to improve the quality of the data prior the
> import. The following paragraphs intend to clear most of these
> concerns.
>
> *1.1. Which data import project?*
>
> According to the import guidelines (steps 3 & 4), a data import
> explicitly refers to a single data source (ODB in our case).
> Discussions about the availability and quality of Microsoft or
> ESRI data, while interesting, are not relevant as they should be
> dealt with as other import projects.
>
> *1.2. What has been imported so far?*
>
> According to what I found [1], the ODB import is completed for 21
> municipalities. These imports seem to have kept OSM content’s
> history, at least for the samples checked, but many problems were
> found. In some case, the imports brought swimming pools in OSM
> because they were included in the dataset (e.g. Moncton). In other
> cases, importing buildings with accurate locations (XY) over
> content mapped from less accurate imagery resulted in buildings
> that now overlap the street network (e.g. Squamish). It means that
> all these 21 imports need to be carefully re-examined and
> corrected as required.
>
> For 12 other municipalities, the import is partial, either
> suspended as requested, or because previous imports had already
> provided most of the buildings (often from the same municipal
> provider). That said the import will definitely improve OSM
> accuracy and completeness if done properly.
>
> *2. How should ODB Data be imported?*
>
> I will copy the following paragraphs in the “Canada Building
> Import” wiki page [3] for a detailed discussion…
>
> Since the data (ODB, OSM and imagery) differ from one municipality
> to another, there can be no imports at the national or provincial
> level. We have to work on a municipal basis and make sure to
> identify all the problems and the corrective measures to apply
> when dealing with issues like those I identified [1].
>
> *2.1 Importing Locally*
>
> According to the import guidelines (step 2), we must not import
> the data without local buy-in. However, and contrarily to some
> European country, there is usually no such thing as a local OSM
> community in each municipality. However, we may find a few local
> mappers from time to time. Working on a municipal basis should
> allow identifying these local mappers before doing the import. I
> often use this tool [2] to identify and contact local mappers.
> Once identified, I suggest that…
>
> - We contact them to explain our intents by referring to
> appropriate wiki pages.
>
> - We wait a week or two to let them respond nothing, that they
> have concerns, or wish to help.
>
> - Without negative answers we could proceed to the import.
>
> I first suggest that when a contributor wishes to import ODB for a
> given municipality, he first identifies himself as responsible for
> the import (we need to create an entry for each municipality
> somewhere in the wiki). He can then contact local mappers, as
> explain above, and go ahead with the import once everything
> settled. For those who already made the import, I suggest that
> they review their work since many issues were detected with some
> of these imports.
>
> Since there are only a few local OSM communities in Canada, and
> because Canada is large, I suggest not limiting the import of a
> given municipality to the people of the concerned province or region.
>
> *2.2 Pre-processing*
>
> Once local mappers have agreed, some pre-processing can be done if
> required.
>
> A few months ago, I developed a tool that could be used to process
> the data [4]. Concerns were raised because the application was
> developed using proprietary software. So I documented the whole
> process and algorithms in order to see courageous coders
> converting it in open source software. In the meantime, and as
> long as I have access to an FME licence, I could process the data,
> when necessary, prior to make it available through the task manager.
>
> Proposed pre-processing [4] includes:
>
> - Reading of original ODB data,
>
> - Removal of near collinear nodes (simplification),
>
> - Orthogonalization of buildings (for corners having near right
> angles),
>
> - Tagging of building footprints,
>
> - Providing files in OSM format.
>
> /Proposed tagging:/ In addition to the tags produced by the
> orthogonalization process [4] and the source tag (source
> <https://wiki.openstreetmap.org/wiki/Key:source>=Statistics Canada
> - Open Building Database), the name of the Census Subdivision
> provided in ODB data [5] is used to add the addr:city tag to each
> building.
>
> The pre-processing requires parameters that are specific to the
> data to process. These parameters were estimated on a municipal
> basis using actual ODB data. The processing time increases
> exponentially according to the number of buildings so, it may take
> a couple of days before the data is available for a given
> municipality. Currently, the proposed pre-processing does not
> convert terrace buildings into individual houses nor it tags
> topological errors.
>
> *2.3. Import Process*
>
> After the local mappers, if any, agreed to the import, the
> pre-processing completed when required, we can proceed to the import.
>
> 1- Do not bulk import the data! Always use the task manager
> (http://tasks.osmcanada.ca/). Select and open a task square in
> JOSM. If it’s too big (e.g. too much work or request is too big to
> load in JOSM), go back to the task manager and split the task into
> smaller squares.
>
> 2- Load imagery layer (Bing or ESRI World Imagery) and align the
> imagery with ODB data (i.e. create a new image offset) if
> necessary because, unless proven otherwise, ODB should be more
> accurate (XY) than most available images especially in hilly areas.
>
> 3- Align the existing OSM content to the image (i.e. after the new
> offset is applied) if required.
>
> 4- Currently step 2 and following as described in the wiki [2]. I
> suggest merging the Conflation section [6] here and reviewing
> everything to take into account the current proposal.
>
> *References*
>
> [1] https://wiki.openstreetmap.org/wiki/The_Open_Database_of_Buildings
>
> [2]
> https://wiki.openstreetmap.org/wiki/Canada_Building_Import#Import_process
>
> [3] http://resultmaps.neis-one.org(“Overview of OpenStreetMap
> Contributors aka who’s around me?”)
>
> [4] https://github.com/jfd553/OrthogonalizingBuildingFootprint
>
> [5]
> https://www150.statcan.gc.ca/n1/pub/92-195-x/2011001/geo/csd-sdr/csd-sdr-eng.htm
>
> [6]
> https://wiki.openstreetmap.org/wiki/Canada_Building_Import#Conflation
>
> Let’s move ahead!
>
> Daniel
>
>
>
> _______________________________________________
>
> Talk-ca mailing list
>
> Talk-ca at openstreetmap.org <mailto:Talk-ca at openstreetmap.org>
>
> https://lists.openstreetmap.org/listinfo/talk-ca
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/talk-ca/attachments/20200105/cb7275f2/attachment-0001.htm>
More information about the Talk-ca
mailing list