[Talk-ca] Importing buildings in Canada

Fri Jan 3 21:13:23 UTC 2020

Sounds sane to me.

Thanks John

On Fri, 3 Jan 2020 at 16:05, Daniel @jfd553 <jfd553 at hotmail.com> wrote:

> Hi John,
>
> To comply with the import directive, I would like the discussion to
> continue on the current ODB import only. Could we talk about other sources
> of potential imports at another time? -)
>
>
>
> Thanks
>
> Daniel
>
> *From:* john whelan [mailto:jwhelan0112 at gmail.com]
> *Sent:* Friday, January 03, 2020 15:57
> *To:* Daniel @jfd553
> *Cc:* talk-ca at openstreetmap.org
> *Subject:* Re: [Talk-ca] Importing buildings in Canada
>
>
>
> Sounds sane to me.
>
>
>
> Are we thinking of setting up a second import to handle Microsoft building
> outlines on a similar basis? Or extending this one?  I only ask since if we
> are doing it municipality by municipality it might be an idea to identify
> those municipalities that do not have suitable open data through stats
> canada.  I can see us with a list of municipalities and a data source and I
> think it probably should be in one place.
>
>
>
> Cheerio John
>
>
>
> On Fri, 3 Jan 2020 at 15:42, Daniel @jfd553 <jfd553 at hotmail.com> wrote:
>
> Bonjour groupe, mes excuses pour ce très long courriel !-)
>
> I have reviewed everything that has been written on the ODB import (aka
> Canada Building Import) in Talk-ca and the wiki. I proposed changes to some
> wiki pages (via talk tabs) to ease the discussions about this import and
> the following. Now, in order to restart the import, here are some thoughts
> and a proposal on how to proceed to complete the task.
>
>
>
> *1. Issues with the ODB Data Import*
>
> Many concerns were raised about the import. One major concern was to
> obtain local communities’ buy-in in the Canadian context. Another concern
> was to improve the quality of the data prior the import. The following
> paragraphs intend to clear most of these concerns.
>
> *1.1. Which data import project?*
>
> According to the import guidelines (steps 3 & 4), a data import explicitly
> refers to a single data source (ODB in our case). Discussions about the
> availability and quality of Microsoft or ESRI data, while interesting, are
> not relevant as they should be dealt with as other import projects.
>
> *1.2. What has been imported so far?*
>
> According to what I found [1], the ODB import is completed for 21
> municipalities. These imports seem to have kept OSM content’s history, at
> least for the samples checked, but many problems were found. In some case,
> the imports brought swimming pools in OSM because they were included in the
> dataset (e.g. Moncton). In other cases, importing buildings with accurate
> locations (XY) over content mapped from less accurate imagery resulted in
> buildings that now overlap the street network (e.g. Squamish). It means
> that all these 21 imports need to be carefully re-examined and corrected as
> required.
>
> For 12 other municipalities, the import is partial, either suspended as
> requested, or because previous imports had already provided most of the
> buildings (often from the same municipal provider). That said the import
> will definitely improve OSM accuracy and completeness if done properly.
>
> *2. How should ODB Data be imported?*
>
> I will copy the following paragraphs in the “Canada Building Import” wiki
> page [3] for a detailed discussion…
>
> Since the data (ODB, OSM and imagery) differ from one municipality to
> another, there can be no imports at the national or provincial level. We
> have to work on a municipal basis and make sure to identify all the
> problems and the corrective measures to apply when dealing with issues like
> those I identified [1].
>
> *2.1 Importing Locally*
>
> According to the import guidelines (step 2), we must not import the data
> without local buy-in. However, and contrarily to some European country,
> there is usually no such thing as a local OSM community in each
> municipality. However, we may find a few local mappers from time to time.
> Working on a municipal basis should allow identifying these local mappers
> before doing the import. I often use this tool [2] to identify and contact
> local mappers. Once identified, I suggest that…
>
> - We contact them to explain our intents by referring to appropriate wiki
> pages.
>
> - We wait a week or two to let them respond nothing, that they have
> concerns, or wish to help.
>
> - Without negative answers we could proceed to the import.
>
> I first suggest that when a contributor wishes to import ODB for a given
> municipality, he first identifies himself as responsible for the import (we
> need to create an entry for each municipality somewhere in the wiki). He
> can then contact local mappers, as explain above, and go ahead with the
> import once everything settled. For those who already made the import, I
> suggest that they review their work since many issues were detected with
> some of these imports.
>
> Since there are only a few local OSM communities in Canada, and because
> Canada is large, I suggest not limiting the import of a given municipality
> to the people of the concerned province or region.
>
> *2.2 Pre-processing*
>
> Once local mappers have agreed, some pre-processing can be done if
> required.
>
> A few months ago, I developed a tool that could be used to process the
> data [4]. Concerns were raised because the application was developed using
> proprietary software. So I documented the whole process and algorithms in
> order to see courageous coders converting it in open source software. In
> the meantime, and as long as I have access to an FME licence, I could
> process the data, when necessary, prior to make it available through the
> task manager.
>
> Proposed pre-processing [4] includes:
>
> - Reading of original ODB data,
>
> - Removal of near collinear nodes (simplification),
>
> - Orthogonalization of buildings (for corners having near right angles),
>
> - Tagging of building footprints,
>
> - Providing files in OSM format.
>
> *Proposed tagging:* In addition to the tags produced by the
> orthogonalization process [4] and the source tag (source
> <https://wiki.openstreetmap.org/wiki/Key:source>=Statistics Canada - Open
> Building Database), the name of the Census Subdivision provided in ODB data
> [5] is used to add the addr:city tag to each building.
>
> The pre-processing requires parameters that are specific to the data to
> process. These parameters were estimated on a municipal basis using actual
> ODB data. The processing time increases exponentially according to the
> number of buildings so, it may take a couple of days before the data is
> available for a given municipality. Currently, the proposed pre-processing
> does not convert terrace buildings into individual houses nor it tags
> topological errors.
>
> *2.3. Import Process*
>
> After the local mappers, if any, agreed to the import, the pre-processing
> completed when required, we can proceed to the import.
>
> 1- Do not bulk import the data! Always use the task manager (
> http://tasks.osmcanada.ca/). Select and open a task square in JOSM. If
> it’s too big (e.g. too much work or request is too big to load in JOSM), go
> back to the task manager and split the task into smaller squares.
>
> 2- Load imagery layer (Bing or ESRI World Imagery) and align the imagery
> with ODB data (i.e. create a new image offset) if necessary because, unless
> proven otherwise, ODB should be more accurate (XY) than most available
> images especially in hilly areas.
>
> 3- Align the existing OSM content to the image (i.e. after the new offset
> is applied) if required.
>
> 4- Currently step 2 and following as described in the wiki [2]. I suggest
> merging the Conflation section [6] here and reviewing everything to take
> into account the current proposal.
>
> *References*
>
> [1] https://wiki.openstreetmap.org/wiki/The_Open_Database_of_Buildings
>
> [2]
> https://wiki.openstreetmap.org/wiki/Canada_Building_Import#Import_process
>
> [3] http://resultmaps.neis-one.org (“Overview of OpenStreetMap
> Contributors aka who’s around me?”)
>
> [4] https://github.com/jfd553/OrthogonalizingBuildingFootprint
>
> [5]
> https://www150.statcan.gc.ca/n1/pub/92-195-x/2011001/geo/csd-sdr/csd-sdr-eng.htm
>
> [6] https://wiki.openstreetmap.org/wiki/Canada_Building_Import#Conflation
>
>
>
>
>
>
>
> Let’s move ahead!
>
> Daniel
>
> _______________________________________________
> Talk-ca mailing list
> Talk-ca at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-ca
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/talk-ca/attachments/20200103/53d1e123/attachment.html>