<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>John, <br>
Those links were just to indicate what can be done with task
setup. I don't think we're close to setting up actual import tasks
yet, though we do seem to be on the road toward getting things
sorted out :-)<br>
</p>
<div class="moz-signature">
<p> Nate Wessel<small>, PhD<br>
Planner, Cartographer, Transport Nerd<br>
<a href="https://www.natewessel.com"
style="text-decoration:none;">NateWessel.com</a></small>
</p>
</div>
<div class="moz-cite-prefix">On 2020-01-05 12:20 p.m., john whelan
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAJ-Ex1Em_Y26KLu=pAZLtX0raUy4a_O961R6XKT40GvDv7Z2tg@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="auto">I'm getting lost as to who is organising what and
who is taking responsibility for defining and setting up the
tasks and where they are being set up.
<div dir="auto"><br>
</div>
<div dir="auto">Are your two examples of what can be done? Or
are they to be used by mappers to add buildings?</div>
<div dir="auto"><br>
</div>
<div dir="auto">My understanding was Daniel would identify those
areas that looked the most interesting then these would be set
up after the process to see if any local mappers were
available and what their input would be.</div>
<div dir="auto"><br>
</div>
<div dir="auto">Thanks John</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Sun, Jan 5, 2020, 11:41 AM
Nate Wessel, <<a href="mailto:bike756@gmail.com"
moz-do-not-send="true">bike756@gmail.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<p>The task size, and even shape is totally customizable.
I've set up a couple that are entirely based on the
density of the data:<br>
<a href="http://tasks.osmcanada.ca/project/168"
target="_blank" rel="noreferrer" moz-do-not-send="true">http://tasks.osmcanada.ca/project/168</a><br>
<a href="https://tasks.openstreetmap.us/project/107"
target="_blank" rel="noreferrer" moz-do-not-send="true">https://tasks.openstreetmap.us/project/107</a></p>
<p>Best,<br>
</p>
<div>
<p> Nate Wessel<small>, PhD<br>
Planner, Cartographer, Transport Nerd<br>
<a href="https://www.natewessel.com"
style="text-decoration:none" target="_blank"
rel="noreferrer" moz-do-not-send="true">NateWessel.com</a></small>
</p>
</div>
<div>On 2020-01-04 12:40 p.m., Daniel @jfd553 wrote:<br>
</div>
<blockquote type="cite">
<div>
<p class="MsoNormal"><span style="color:#1f497d">Bonjour
groupe</span></p>
<p class="MsoNormal"><span style="color:#1f497d"> </span></p>
<p class="MsoNormal"><span style="color:#1f497d">Looks
like we're going in the same direction so far :-)</span></p>
<p class="MsoNormal"><span style="color:#1f497d">I agree
with Nate regarding the implementation of the task
manager. In my experience, a size of a few blocks
would be better in urban areas, but boring in rural
areas. Is it something that can be adjusted?</span></p>
<p class="MsoNormal"><span style="color:#1f497d"
lang="FR-CA"> </span></p>
<p class="MsoNormal"><span style="color:#1f497d">Daniel</span></p>
<p class="MsoNormal"><span style="color:#1f497d"> </span></p>
<div>
<div style="border:none;border-top:solid #b5c4df
1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span
style="font-size:10.0pt;font-family:"Tahoma","sans-serif""
lang="EN-US">From:</span></b><span
style="font-size:10.0pt;font-family:"Tahoma","sans-serif""
lang="EN-US"> Nate Wessel [<a
href="mailto:bike756@gmail.com"
target="_blank" rel="noreferrer"
moz-do-not-send="true">mailto:bike756@gmail.com</a>]
<br>
<b>Sent:</b> Saturday, January 04, 2020 10:09<br>
<b>To:</b> <a
href="mailto:talk-ca@openstreetmap.org"
target="_blank" rel="noreferrer"
moz-do-not-send="true">talk-ca@openstreetmap.org</a><br>
<b>Subject:</b> Re: [Talk-ca] Importing
buildings in Canada</span></p>
</div>
</div>
<p class="MsoNormal"> </p>
<p>Hi Daniel, </p>
<p>Thank you for all the work you've put into this. I'd
like to offer a couple suggestions and/or
clarifications for your proposed import process,
overview though it is. </p>
<p>First, I think it is very important that a tasking
manager is set up on a city/by city basis only, and
that only AFTER consensus is achieved that the import
should proceed in that area. I would really like to
avoid seeing the massive nationwide tasking that was
set up the first time around. We should be making it
hard for people to go rogue in regions where consensus
for an import doesn't (yet) exist. </p>
<p>Related to this, though important enough to be a
second point in it's own right, the tasking squares
need to be small enough that a single user can manage
them and inspect every single building in a task. The
first round of import used task squares that were
massive, and which couldn't be divided any further
past a certain point. Even in rural areas, it is
likely inappropriate to import areas larger than
1km^2. In central Toronto it would be (and was)
idiotic. An import that doesn't take local scale into
account shouldn't proceed. "Too big to load into JOSM"
is about 100x too big to import in my opinion and is
not a good enough benchmark for import batch sizing. </p>
<p>That is, each import needs to be local, and not just
in a superficial sense. </p>
<p>I'll also add that the issue of conflation doesn't
seem to have been worked out yet except to note that
it is an issue. What will we do with the millions of
buildings which will substantially overlap/duplicate
existing buildings or imports? This needs to be worked
out in detail before anything starts up again. </p>
<p>And what needs to be done about already existing low
quality imports? It's good to acknowledge their
existence, but what will be done about them? We've set
up a task to clean up some of the mess in Toronto ( <a
href="http://tasks.osmcanada.ca/project/168"
target="_blank" rel="noreferrer"
moz-do-not-send="true">http://tasks.osmcanada.ca/project/168</a>
) but this is only the tip of the iceberg. </p>
<p>Again, I thank everyone for their time and effort on
this - we can get this done if we go slow and do it
right :-)</p>
<p>Best,</p>
<div>
<p>Nate Wessel<span style="font-size:10.0pt">, PhD<br>
Planner, Cartographer, Transport Nerd<br>
<a href="https://www.natewessel.com"
target="_blank" rel="noreferrer"
moz-do-not-send="true"><span
style="text-decoration:none">NateWessel.com</span></a></span>
</p>
</div>
<div>
<p class="MsoNormal">On 2020-01-03 3:40 p.m., Daniel
@jfd553 wrote:</p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"
style="margin-bottom:10.0pt;line-height:115%"><span
lang="FR-CA">Bonjour groupe, mes excuses pour ce
très long courriel !-)</span></p>
<p class="MsoNormal" style="line-height:115%">I have
reviewed everything that has been written on the ODB
import (aka Canada Building Import) in Talk-ca and
the wiki. I proposed changes to some wiki pages (via
talk tabs) to ease the discussions about this import
and the following. Now, in order to restart the
import, here are some thoughts and a proposal on how
to proceed to complete the task.</p>
<p class="MsoNormal" style="line-height:115%"> </p>
<p class="MsoNormal" style="line-height:115%"><b><span
style="font-size:14.0pt;line-height:115%">1.
Issues with the ODB Data Import</span></b></p>
<p class="MsoNormal"
style="margin-bottom:10.0pt;line-height:115%">Many
concerns were raised about the import. One major
concern was to obtain local communities’ buy-in in
the Canadian context. Another concern was to improve
the quality of the data prior the import. The
following paragraphs intend to clear most of these
concerns.</p>
<p class="MsoNormal" style="line-height:115%"><b>1.1.
Which data import project?</b></p>
<p class="MsoNormal"
style="margin-bottom:10.0pt;line-height:115%">According
to the import guidelines (steps 3 & 4), a data
import explicitly refers to a single data source
(ODB in our case). Discussions about the
availability and quality of Microsoft or ESRI data,
while interesting, are not relevant as they should
be dealt with as other import projects.</p>
<p class="MsoNormal" style="line-height:115%"><b>1.2.
What has been imported so far?</b></p>
<p class="MsoNormal"
style="margin-bottom:10.0pt;line-height:115%">According
to what I found [1], the ODB import is completed for
21 municipalities. These imports seem to have kept
OSM content’s history, at least for the samples
checked, but many problems were found. In some case,
the imports brought swimming pools in OSM because
they were included in the dataset (e.g. Moncton). In
other cases, importing buildings with accurate
locations (XY) over content mapped from less
accurate imagery resulted in buildings that now
overlap the street network (e.g. Squamish). It means
that all these 21 imports need to be carefully
re-examined and corrected as required. </p>
<p class="MsoNormal"
style="margin-bottom:10.0pt;line-height:115%">For 12
other municipalities, the import is partial, either
suspended as requested, or because previous imports
had already provided most of the buildings (often
from the same municipal provider). That said the
import will definitely improve OSM accuracy and
completeness if done properly.</p>
<p class="MsoNormal" style="line-height:115%"><b><span
style="font-size:14.0pt;line-height:115%">2. How
should ODB Data be imported?</span></b></p>
<p class="MsoNormal"
style="margin-bottom:10.0pt;line-height:115%">I will
copy the following paragraphs in the “Canada
Building Import” wiki page [3] for a detailed
discussion…</p>
<p class="MsoNormal"
style="margin-bottom:10.0pt;line-height:115%">Since
the data (ODB, OSM and imagery) differ from one
municipality to another, there can be no imports at
the national or provincial level. We have to work on
a municipal basis and make sure to identify all the
problems and the corrective measures to apply when
dealing with issues like those I identified [1]. </p>
<p class="MsoNormal" style="line-height:115%"><b>2.1
Importing Locally</b></p>
<p class="MsoNormal"
style="margin-bottom:10.0pt;line-height:115%">According
to the import guidelines (step 2), we must not
import the data without local buy-in. However, and
contrarily to some European country, there is
usually no such thing as a local OSM community in
each municipality. However, we may find a few local
mappers from time to time. Working on a municipal
basis should allow identifying these local mappers
before doing the import. I often use this tool [2]
to identify and contact local mappers. Once
identified, I suggest that…</p>
<p class="MsoNormal">- We contact them to explain our
intents by referring to appropriate wiki pages.</p>
<p class="MsoNormal">- We wait a week or two to let
them respond nothing, that they have concerns, or
wish to help.</p>
<p class="MsoNormal" style="margin-bottom:6.0pt">-
Without negative answers we could proceed to the
import.</p>
<p class="MsoNormal"
style="margin-bottom:10.0pt;line-height:115%">I
first suggest that when a contributor wishes to
import ODB for a given municipality, he first
identifies himself as responsible for the import (we
need to create an entry for each municipality
somewhere in the wiki). He can then contact local
mappers, as explain above, and go ahead with the
import once everything settled. For those who
already made the import, I suggest that they review
their work since many issues were detected with some
of these imports.</p>
<p class="MsoNormal"
style="margin-bottom:10.0pt;line-height:115%">Since
there are only a few local OSM communities in
Canada, and because Canada is large, I suggest not
limiting the import of a given municipality to the
people of the concerned province or region.</p>
<p class="MsoNormal" style="line-height:115%"><b>2.2
Pre-processing</b></p>
<p class="MsoNormal"
style="margin-bottom:10.0pt;line-height:115%">Once
local mappers have agreed, some pre-processing can
be done if required. </p>
<p class="MsoNormal"
style="margin-bottom:10.0pt;line-height:115%">A few
months ago, I developed a tool that could be used to
process the data [4]. Concerns were raised because
the application was developed using proprietary
software. So I documented the whole process and
algorithms in order to see courageous coders
converting it in open source software. In the
meantime, and as long as I have access to an FME
licence, I could process the data, when necessary,
prior to make it available through the task manager.</p>
<p class="MsoNormal"
style="margin-bottom:10.0pt;line-height:115%">Proposed
pre-processing [4] includes: </p>
<p class="MsoNormal">- Reading of original ODB data,</p>
<p class="MsoNormal">- Removal of near collinear nodes
(simplification), </p>
<p class="MsoNormal">- Orthogonalization of buildings
(for corners having near right angles),</p>
<p class="MsoNormal">- Tagging of building footprints,</p>
<p class="MsoNormal"
style="margin-bottom:10.0pt;line-height:115%">-
Providing files in OSM format.</p>
<p class="MsoNormal"
style="margin-bottom:10.0pt;line-height:115%"><i>Proposed
tagging:</i> In addition to the tags produced by
the orthogonalization process [4] and the source tag
(<a
href="https://wiki.openstreetmap.org/wiki/Key:source"
target="_blank" rel="noreferrer"
moz-do-not-send="true"><span
style="color:windowtext;text-decoration:none">source</span></a>=Statistics
Canada - Open Building Database), the name of the
Census Subdivision provided in ODB data [5] is used
to add the addr:city tag to each building.</p>
<p class="MsoNormal"
style="margin-bottom:10.0pt;line-height:115%">The
pre-processing requires parameters that are specific
to the data to process. These parameters were
estimated on a municipal basis using actual ODB
data. The processing time increases exponentially
according to the number of buildings so, it may take
a couple of days before the data is available for a
given municipality. Currently, the proposed
pre-processing does not convert terrace buildings
into individual houses nor it tags topological
errors.</p>
<p class="MsoNormal" style="line-height:115%"><b>2.3.
Import Process</b></p>
<p class="MsoNormal"
style="margin-bottom:10.0pt;line-height:115%">After
the local mappers, if any, agreed to the import, the
pre-processing completed when required, we can
proceed to the import. </p>
<p class="MsoNormal"
style="margin-bottom:10.0pt;line-height:115%">1- Do
not bulk import the data! Always use the task
manager (<a href="http://tasks.osmcanada.ca/"
target="_blank" rel="noreferrer"
moz-do-not-send="true">http://tasks.osmcanada.ca/</a>).
Select and open a task square in JOSM. If it’s too
big (e.g. too much work or request is too big to
load in JOSM), go back to the task manager and split
the task into smaller squares.</p>
<p class="MsoNormal"
style="margin-bottom:10.0pt;line-height:115%">2-
Load imagery layer (Bing or ESRI World Imagery) and
align the imagery with ODB data (i.e. create a new
image offset) if necessary because, unless proven
otherwise, ODB should be more accurate (XY) than
most available images especially in hilly areas.</p>
<p class="MsoNormal"
style="margin-bottom:10.0pt;line-height:115%">3-
Align the existing OSM content to the image (i.e.
after the new offset is applied) if required.</p>
<p class="MsoNormal"
style="margin-bottom:10.0pt;line-height:115%">4-
Currently step 2 and following as described in the
wiki [2]. I suggest merging the Conflation section
[6] here and reviewing everything to take into
account the current proposal.</p>
<p class="MsoNormal"
style="margin-bottom:10.0pt;line-height:115%"><b><span
style="font-size:14.0pt;line-height:115%">References</span></b></p>
<p class="MsoNormal"><span style="font-size:9.0pt">[1]
</span><a
href="https://wiki.openstreetmap.org/wiki/The_Open_Database_of_Buildings"
target="_blank" rel="noreferrer"
moz-do-not-send="true"><span
style="font-size:9.0pt">https://wiki.openstreetmap.org/wiki/The_Open_Database_of_Buildings</span></a></p>
<p class="MsoNormal"><span style="font-size:9.0pt">[2]
</span><a
href="https://wiki.openstreetmap.org/wiki/Canada_Building_Import#Import_process"
target="_blank" rel="noreferrer"
moz-do-not-send="true"><span
style="font-size:9.0pt">https://wiki.openstreetmap.org/wiki/Canada_Building_Import#Import_process</span></a></p>
<p class="MsoNormal"><span style="font-size:9.0pt">[3]
</span><a href="http://resultmaps.neis-one.org"
target="_blank" rel="noreferrer"
moz-do-not-send="true"><span
style="font-size:9.0pt">http://resultmaps.neis-one.org</span></a><span
style="font-size:9.0pt"> (“Overview of
OpenStreetMap Contributors aka who’s around me?”)</span></p>
<p class="MsoNormal"><span style="font-size:9.0pt">[4]
</span><a
href="https://github.com/jfd553/OrthogonalizingBuildingFootprint"
target="_blank" rel="noreferrer"
moz-do-not-send="true"><span
style="font-size:9.0pt">https://github.com/jfd553/OrthogonalizingBuildingFootprint</span></a></p>
<p class="MsoNormal"><span style="font-size:9.0pt">[5]
</span><span style="font-size:10.0pt"><a
href="https://www150.statcan.gc.ca/n1/pub/92-195-x/2011001/geo/csd-sdr/csd-sdr-eng.htm"
target="_blank" rel="noreferrer"
moz-do-not-send="true"><span
style="font-size:9.0pt">https://www150.statcan.gc.ca/n1/pub/92-195-x/2011001/geo/csd-sdr/csd-sdr-eng.htm</span></a></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt">[6]
<a
href="https://wiki.openstreetmap.org/wiki/Canada_Building_Import#Conflation"
target="_blank" rel="noreferrer"
moz-do-not-send="true">
https://wiki.openstreetmap.org/wiki/Canada_Building_Import#Conflation</a></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt"> </span></p>
<p class="MsoNormal"><span style="font-size:9.0pt"> </span></p>
<p class="MsoNormal"
style="margin-bottom:10.0pt;line-height:115%"> </p>
<p class="MsoNormal"
style="margin-bottom:10.0pt;line-height:115%">Let’s
move ahead!</p>
<p class="MsoNormal"
style="margin-bottom:10.0pt;line-height:115%">Daniel</p>
<p class="MsoNormal"><span><br>
<br>
</span></p>
<pre>_______________________________________________</pre>
<pre>Talk-ca mailing list</pre>
<pre><a href="mailto:Talk-ca@openstreetmap.org" target="_blank" rel="noreferrer" moz-do-not-send="true">Talk-ca@openstreetmap.org</a></pre>
<pre><a href="https://lists.openstreetmap.org/listinfo/talk-ca" target="_blank" rel="noreferrer" moz-do-not-send="true">https://lists.openstreetmap.org/listinfo/talk-ca</a></pre>
</blockquote>
</div>
</blockquote>
</div>
_______________________________________________<br>
Talk-ca mailing list<br>
<a href="mailto:Talk-ca@openstreetmap.org" target="_blank"
rel="noreferrer" moz-do-not-send="true">Talk-ca@openstreetmap.org</a><br>
<a href="https://lists.openstreetmap.org/listinfo/talk-ca"
rel="noreferrer noreferrer" target="_blank"
moz-do-not-send="true">https://lists.openstreetmap.org/listinfo/talk-ca</a><br>
</blockquote>
</div>
</blockquote>
</body>
</html>