<div dir="auto">I can run all the shapefiles through qgis simplify tool if this resolves the issue...</div><br><div class="gmail_quote"><div dir="ltr">On Fri., Jan. 18, 2019, 4:08 p.m. Nate Wessel <<a href="mailto:bike756@gmail.com">bike756@gmail.com</a> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<p>With default settings in JOSM, sure. In the import I was working
on, we used a Douglas-Peucker algorithm with a 20cm threshold
(before the import started) and it worked beautifully. We had many
points that seemed to have been introduced in the shapefiles as
some kind of data artifact - they didn't add any detail to the
shape at all. This procedure removed almost all of them with no
discernible reduction in quality. <br>
</p>
<div class="m_7176009870857990757moz-signature">Nate Wessel<br>
<span style="font-size:10px;color:#777">Jack of all trades, Master
of Geography, PhD candidate in Urban Planning<br>
<a href="http://natewessel.com" target="_blank" rel="noreferrer">NateWessel.com</a></span>
<br>
<br>
</div>
<div class="m_7176009870857990757moz-cite-prefix">On 1/18/19 4:03 PM, James wrote:<br>
</div>
<blockquote type="cite">
<div dir="auto">dare you to run simplify tool on anything remotely
round, it will make it look like garbage</div>
<br>
<div class="gmail_quote">
<div dir="ltr">On Fri., Jan. 18, 2019, 3:49 p.m. John Whelan
<<a href="mailto:jwhelan0112@gmail.com" target="_blank" rel="noreferrer">jwhelan0112@gmail.com</a> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="font-family:Verdana;font-size:12pt" bgcolor="#FFFFFF" text="#000000">
<div style="font-size:12pt;font-family:Verdana">The import
mailing list was pointed to the correct page of the wiki.
The initial post was to say this is what we were thinking
of and there was a comment saying we needed to change the
comment line.<br>
<br>
>There is no mention of this proposed import on the
import catalogue<br>
<br>
<br>
The import process was reviewed by the person who set up
the Ottawa import did we miss that step on the Ottawa
import as well? Neither was it raised as a concern on the
import mailing list. I think this is very minor and can be
corrected.<br>
<br>
We learnt a fair bit on the Ottawa import and my
expectation is since we are using experienced mappers to
do the import conflation would be either handled by them
or the building not imported. We aren't using new mappers
in a mapathon here and with experienced mappers then I
think you have to trust them. The world isn't perfect.
Think in terms of service level.<br>
<br>
>There are 2X more nodes than needed to represent the
building accurately.<br>
<br>
The problem with correcting this is you are introducing
approximations. This will vary according to the source
and this can be simplified or corrected once its in OSM. I
think this is a different issue of a mechanical edit that
needs to be considered separately.<br>
<br>
If we are concerned with database size then I suggest we
change the instructions to say put the source comment on
the change set rather than on the building outline.<br>
<br>
Cheerio John<br>
<br>
<br>
<span>Nate Wessel wrote on 2019-01-18 3:06 PM:</span><br>
<blockquote type="cite">
<p>John, <br>
</p>
<p>You seem to be playing the long game with this data -
it sounds like you've been working with this a lot
longer than I have, and you've put in the time and
effort to help make this actually-quite-incredible
dataset available to us. I don't want to stop the
import from happening - quite the opposite. I just
want to make sure that the time is taken to do this
right. OSM deserves that. Your (our) long awaited
victory will be the sweeter for our patience now. <br>
</p>
<p>There are several specific issues I see where the I's
are not crossed, nor the t's dotted. I've mentioned
several already, so I'll try to be brief (I really
need to get back to working on my dissertation).</p>
<p>1) There was extremely limited discussion on the
imports mailing list. The initial email did not make
clear the scope of the project. I read the email and
did not think twice at it, thinking it was entirely
about Ottawa. The link in that email was actually to
the Ottawa import, and not this one, which seems to
have been only in draft at the time.
<a class="m_7176009870857990757m_887281844982557008moz-txt-link-freetext" href="https://lists.openstreetmap.org/pipermail/imports/2018-November/005812.html" rel="noreferrer noreferrer" target="_blank">https://lists.openstreetmap.org/pipermail/imports/2018-November/005812.html</a><br>
As such, this project has NOT been reviewed by the
imports list, which is a requirement for proceeding
with the import.<br>
</p>
<p>2) There is no mention of this proposed import on the
import catalogue (<a class="m_7176009870857990757m_887281844982557008moz-txt-link-freetext" href="https://wiki.openstreetmap.org/wiki/Import/Catalogue" rel="noreferrer noreferrer" target="_blank">https://wiki.openstreetmap.org/wiki/Import/Catalogue</a>)<br>
which is required in the imports guidelines. I suspect
many other guidelines have not been followed. <br>
</p>
<p>3) The wiki page describing the import is not
adequate to assess the quality of the data or of the
proposed import. See for example:
<a class="m_7176009870857990757m_887281844982557008moz-txt-link-freetext" href="https://wiki.openstreetmap.org/wiki/WikiProject_Canada/Canada_Stats_Canada_Building_Outlines_Import/Plan#Risks" rel="noreferrer noreferrer" target="_blank">https://wiki.openstreetmap.org/wiki/WikiProject_Canada/Canada_Stats_Canada_Building_Outlines_Import/Plan#Risks</a><br>
The import guidelines call for a description of how
conflation will be handled. The fact that two of the
major importers seem to have a substantial
disagreement about how to handle existing data
indicates this was not well discussed and I can see
that it isn't well documented. <br>
</p>
<p>4) The buildings need to be simplified, quite a bit
actually. Most buildings have multiple nodes
representing straight lines. This bloats the database
and makes things harder to edit by hand later. There
are probably 2x more nodes than are needed to
represent the data accurately, making it harder for
editors and data consumers to work with down the
road.This is a simple fix that will save countless
hours later.<br>
</p>
<p>... I could go on, but I think this is plenty
sufficient to justify pressing pause on all this. <br>
</p>
<p>Again, I don't in any way want to disrespect the work
that has gone into this effort already. We're all
volunteers here and I know how much time this all
takes. However. importing all/most of the buildings in
Canada is a monstrously large task, which will have to
dance around a lot of people's toes. We should expect
this to take a really damn long time if we're going to
do it right. We need to have the patience to learn
from experience, from critique, and from the wisdom of
the people who've learned from flawed imports in the
past and have devised guidelines and processes so that
we can have better experiences with this in the
future. <br>
</p>
<div class="m_7176009870857990757m_887281844982557008moz-signature">Nate
Wessel<br>
<span style="font-size:10px;color:#777">Jack of all
trades, Master of Geography, PhD candidate in Urban
Planning<br>
<a href="http://natewessel.com" rel="noreferrer noreferrer" target="_blank">NateWessel.com</a></span>
<br>
<br>
</div>
<div class="m_7176009870857990757m_887281844982557008moz-cite-prefix">On
1/18/19 2:24 PM, john whelan wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small">My
background is I'm a retired civil servant who has
written and overseen procurement documents and
fairly large procurements. Dotting the is and
crossing the Ts are my speciality.</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small"><br>
</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small">There
are two parts to an import. The first part is the
part played by the import mailing group. They
confine themselves to is the license correct and
do you have a reasonable plan. In this case the
license is one of the few that has been confirmed
by the Legal Working Group of OpenStreetMap and as
such no questions were raised about it on the
import mailing list. We have methodology that has
been used before successfully with the Ottawa
building outline import. There were major
discussions both on talk-ca and the import mailing
group before that import took place and we took
note of the issues raised and addressed them. The
licensing issue goes back about eight years to
when I was talking to Federal Government Treasury
Board and explaining their Open Data license did
not align with OSM. That is why their license is
now known as 2.0.</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small"><br>
</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small">The
second part is the local group makes the decision
to import they are the authority no one else.</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small"><br>
</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small">Apparently
you were not part of the talk-ca when the
discussions took place which would have been the
time and place to raise concerns.</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small"><br>
</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small">When
the Ottawa import was done there were one or two
places where the existing buildings and the import
overlapped. In the instructions on the import
there are instructions to cover this. Specifically
there is a validation step. I seem to recall the
error rate was of the order of 1% and I expect
this latest batch to be roughly the same.<br>
</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small"><br>
</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small">If
you can identify which municipalities data is of
poor quality then I'm sure we can remove these.
For the most part these are from the foundation
plans recorded by the municipality using
professional surveying techniques.</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small"><br>
</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small">Would
you like to clarify exactly where I failed to dot
the Is and cross the Ts please.</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small"><br>
</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small">Many
Thanks</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small"><br>
</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small">John<br>
</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small"><br>
</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small"><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="m_7176009870857990757m_887281844982557008gmail_attr">On Fri, 18
Jan 2019 at 13:37, Nate Wessel <<a href="mailto:bike756@gmail.com" rel="noreferrer noreferrer" target="_blank">bike756@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>Hi John, <br>
</p>
<p>As Steve has said, you seem to be the only
one suggesting that thousands of import
committees might need to be formed. Certainly
I'm not suggesting that.</p>
<p>My understanding of OSM import procedure (and
wiki-style projects more generally) is that
imports should operate in an essentially
consensual way where possible. The goal is to
build consent and bring people on board with a
project or a change by addressing their
concerns in a meaningful and respectful way. <br>
</p>
<p>I think that I have made some substantive and
troubling claims about the quality of the data
being imported. I've pointed out that this
project has not followed the import procedures
that were produced by a community of mappers
larger than just those in Canada. <br>
</p>
<p>So to respond to your implication, I am in
some sense the one reviewing the project, just
as I would welcome you to find ways that my
own contributions could be better. If you want
my credentials for reviewing your work, here
they are:</p>
<p>1) I am an active contributor to OSM in
Toronto, where I live (and elsewhere)<br>
</p>
<p>2) I am currently helping to lead a building
import in Hamilton County Ohio that has better
addressed some of the issues I see this import
struggling with. I can help you do the same.<br>
</p>
<p>3) I've been doing research in GIS for a long
time now, though I don't need that to tell you
that the issues I've described are hardly
insurmountable technically or even all that
difficult to fix. It would take maybe one
day's hard work to get the technical side of
this right. <br>
</p>
<p> I think Canadian OSMers will agree that we
can take a pause to get things right on such a
massive import. If they don't - if I'm shouted
down or better, if my critiques are adequately
addressed, then I will leave you to finish the
project in peace. I might even lend a hand if
all goes well, as I sincerely hope it does :-)</p>
<p>Best,<br>
</p>
<div class="m_7176009870857990757m_887281844982557008gmail-m_4868754124937657174moz-signature">Nate
Wessel<br>
<span style="font-size:10px;color:rgb(119,119,119)">Jack
of all trades, Master of Geography, PhD
candidate in Urban Planning<br>
<a href="http://natewessel.com" rel="noreferrer noreferrer" target="_blank">NateWessel.com</a></span>
<br>
<br>
</div>
<div class="m_7176009870857990757m_887281844982557008gmail-m_4868754124937657174moz-cite-prefix">On
1/18/19 1:11 PM, john whelan wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small">I
know of no other way to contact him but he
made an interesting comment that the
project is on hold in the wiki pending
review.</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small"><br>
</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small">Would
he care to comment on who is supposed to
be reviewing the project?</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small"><br>
</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small">My
understanding is that the import was
raised in talk-ca before it commenced for
comment and these were generally
favourable. I took that as the local
mappers to Canada had been consulted and
they are the "local mappers" authority in
this case.</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small"><br>
</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small">I
understand he has concerns about local
mappers making decisions but in Canada we
have been importing similar data through
CANVEC for some time. CANVEC data comes
from a number of sources including
municipal data.</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small"><br>
</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small">Is
he suggesting that each of the 3,700
municipalities in Canada should form a
group of local mappers who can make
individual decisions on whether their
municipal data should be imported and we
should end up with 3,700 import plans?</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small"><br>
</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small">Thanks
John</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small"><br>
</div>
<div class="gmail_default" style="font-family:verdana,sans-serif;font-size:small"><br>
</div>
</div>
<br>
<fieldset class="m_7176009870857990757m_887281844982557008gmail-m_4868754124937657174mimeAttachmentHeader"></fieldset>
<pre class="m_7176009870857990757m_887281844982557008gmail-m_4868754124937657174moz-quote-pre">_______________________________________________
Talk-ca mailing list
<a class="m_7176009870857990757m_887281844982557008gmail-m_4868754124937657174moz-txt-link-abbreviated" href="mailto:Talk-ca@openstreetmap.org" rel="noreferrer noreferrer" target="_blank">Talk-ca@openstreetmap.org</a>
<a class="m_7176009870857990757m_887281844982557008gmail-m_4868754124937657174moz-txt-link-freetext" href="https://lists.openstreetmap.org/listinfo/talk-ca" rel="noreferrer noreferrer" target="_blank">https://lists.openstreetmap.org/listinfo/talk-ca</a>
</pre>
</blockquote>
</div>
_______________________________________________<br>
Talk-ca mailing list<br>
<a href="mailto:Talk-ca@openstreetmap.org" rel="noreferrer noreferrer" target="_blank">Talk-ca@openstreetmap.org</a><br>
<a href="https://lists.openstreetmap.org/listinfo/talk-ca" rel="noreferrer noreferrer noreferrer" target="_blank">https://lists.openstreetmap.org/listinfo/talk-ca</a><br>
</blockquote>
</div>
</blockquote>
</blockquote>
<br>
<div class="m_7176009870857990757m_887281844982557008moz-signature">-- <br>
<div>Sent from <a href="https://www.postbox-inc.com/?utm_source=email&utm_medium=siglink&utm_campaign=reach" rel="noreferrer noreferrer" target="_blank"><span style="color:rgb(0,157,247)">Postbox</span></a></div>
</div>
</div>
</div>
_______________________________________________<br>
Talk-ca mailing list<br>
<a href="mailto:Talk-ca@openstreetmap.org" rel="noreferrer noreferrer" target="_blank">Talk-ca@openstreetmap.org</a><br>
<a href="https://lists.openstreetmap.org/listinfo/talk-ca" rel="noreferrer noreferrer noreferrer" target="_blank">https://lists.openstreetmap.org/listinfo/talk-ca</a><br>
</blockquote>
</div>
</blockquote>
</div>
</blockquote></div>