[Imports] Ongoing Canadian building import needs to be stopped, possibly reverted

john whelan jwhelan0112 at gmail.com
Thu Jan 17 19:22:30 UTC 2019


This is just to address some of Steve's concerns.

First if you look at the 2020 wiki page history you'll see there is a lot
of input from Steve.  My concern with this very detailed input is it made
it hard for a new person to quickly locate relevant information, an
overview if you like.

I will confess that there have been small groups in face to face meetings
in small cafes where you need a password to logon to the internet.  He was
not specifically invited to them all.

I confess we have used conference calls and other methods of communication
without notifying hundreds of people first.  There have even been meetings
that I was unaware of.  For example I haven't even communicated directly
with the mappers who are doing most of the import at the moment.

There has even been at least one mapathon that Stats Canada only found out
about after the event.

Personally I'm not convinced that OpenStreetMap really needs every building
in the planet mapped in detail.

The history was I was after the bus stops in Ottawa which meant I needed
them with an open data license we could use.  I used to work at Stats
Canada and the corporate culture is very different to OSM.  In Canada we
have fewer mappers on the ground and more places to map than in many parts
of Europe.  We have a history of importing CANVEC data which comes from a
number of sources including Municipalities.  So I acted in a coordinating
role.  We managed to persuade the City of Ottawa to change it's open data
license to align with the federal one.  I got my bus stops.  The local
mappers were very much involved and there were at least half a dozen face
to face meetings that took place.  I drifted down to one of them.

Stats was very pleased with the added tags on the building outlines in
Ottawa. This is information they felt could not be easily obtained in any
other way.

I am very aware that this data is important to many.  This includes Federal
government departments and agencies.  They were very vocal at a meeting at
Stats Canada during the HOT summit in Ottawa.  It was open and at least
half a dozen OpenStreetMappers were present, three or four were from
European or other out of town locations.  Having the building data in one
place makes it much easier for the ed users than having to handle different
formats and open data licenses.  Currently one municipal social agency is
very interested in mapping places where fresh food can be obtained.  I
forget some of the other interests but they were quite legitimate.  We have
seen considerable interest by high schools and students in OpenStreetMap
and using streetcomplete with building outlines is one way that they can
add value without causing too much havoc.

After we imported Ottawa a group of mappers decided that we needed more
buildings.  They organised mapathons with new mappers and mapped buildings
with iD.  The results were not good and the data quality side was raised in
talk-ca.  I was involved in one where I set up new mappers with JOSM and
the buildings_tool plugin and that went much better as far as accuracy was
concerned.

The result of these mapathons and the community reaction was to convince
Stats Canada that releasing more building outlines as was done in Ottawa
under an Open Data license was a way forward.  Kingston in particular was
keen to release its building outlines and get them into OpenStreetMap.
Obtaining them and making them available was a Stats Canada decision and
was made in their time frame.

Given that Stats Canada released the data under an acceptable Open Data
license I thought and still think the best way forward was to set up a plan
and a process to import the data.  The alternative was probably going to be
Ad-Hoc importing.
I suspect that talk-ca is probably the most appropriate mailing list for
this sort of discussion which is why I emailed Nate directly.

Cheerio John

On Thu, 17 Jan 2019 at 12:47, OSM Volunteer stevea <
steveaOSM at softworkers.com> wrote:

> Yaro Shkvorets <shkvorets at gmail.com> wrote
> > 1) There was a discussion going on in the import list (starting with
> Ottawa
> > import a year ago) and in the slack channel. If you have any concerns
> let's
> > talk.
>
> I know nothing about slack channel discussion (a proprietary communication
> methodology I will not use in an open data project), but there has been
> discussion (in talk-ca, in wiki...) far longer than that.  I noticed
> problems with this "import" (approach at having problematic Canadian
> building data) as data began to enter OSM:  it was rife with legal,
> technical and quality problems in 2017.  On talk-ca, in many personal
> emails and with a genuine attempt at politeness, a helpful attitude,
> reminders of "here's the way things must be done in OSM, nobody gets a pass
> on following our guidelines and rules," I have poured countless hours into
> (positively) critiquing the process, substantially improving the wiki that
> was written for it ("BC2020" project), all while attempting to correct a
> national-level (STATSCAN, the federal Canadian Bureau of Statistics)
> "invisible hand" of direction, which piloted this aircraft directly into a
> suicide tailspin.
>
> My main complaint was that OSM was being used as a "dumping ground"
> repository for poor-quality data at the same time OSM's (severely
> resource-constrained) Legal Working Group was tasked with making sense out
> of a mess of non-compliant municipal so-called "open data" licenses from
> cities across Canada.  The problems were fairly massive, yet I wanted to
> offer helping hands, fully aware that I'm not Canadian, rather I'm an OSM
> volunteer who cares deeply about high-quality data entering our map with
> methodologies agreeable to our community, standards and tenets.  OSM has
> those, (some) Canadian OSM users intent on getting massive and problematic
> building data into our map simply didn't respect them:  "import away,
> anyway" was the charge-ahead cry two years ago, and even with
> course-correction and a reboot as wounds are apparently licked, appears to
> be the charge-ahead cry today.  What happened to "correct the data FIRST,
> then import them?"  Nothing, that's what.
>
> A "reboot" of the project started a couple months ago and I largely stayed
> out of the way, hoping that the "lessons learned" (a principal at STATSCAN
> said "sometimes these things simply don't hatch right") would finally
> "stick" and improvements could be expected.
>
> Alas, we have very mixed results, as while Yaro says he is able to
> successfully import many Tasking Manager "squares," we also have Danny
> McD's overlapping buildings that are "unacceptable" and this "has to be
> dealt with."  To wit, an essential admission that the process has broken
> down ("Task Manager instructions...clearly requests to validate and fix all
> building-related errors and warnings") has taken place.  This is a failure
> at the most basic level of an import, one which (many think) should be
> watched like a hawk at national and international levels, given the mess
> that happened during the first attempt to import.  Further, Yaro says both
> that he ALWAYS uses "Replace Geometry" on buildings (with meaningful tags)
> yet executing this for each pair of problematic buildings "would be
> insanely time-consuming."  Yes, Canada is importing millions of buildings:
> what part of "that would be insanely time-consuming" surprises anybody?
>
> Further, Yaro says:
> > Thanks for flagging issues with the import. I'll ask guys to stop
> importing
> > and address the raised concerns and resume only after everything has been
> > dealt with.
>
> Can it be assumed that Yaro is the "import lead technical coordinator"
> (that's loose, but works) for this endeavor?  Is anybody else "in charge"
> of this import?  (John Whelan was and possibly is a "guiding hand" in this
> endeavor, and I cc him as a courtesy, given our past interactions and his
> recent talk-ca posts on this).
>
> This project remains a very messy process and obviously I want it to
> improve to meet it goals, but not at the expense of the concerns outlined
> by Nate:  "truly terrible data quality, the approach seems to be import
> first, validate second (if ever?), the wiki gives no clear description of
> the plan for integrating new data with old, the data (are) being integrated
> in extremely large chunks, (too large to be properly reviewed)," and more.
> So, it appears that much or all that was wrong and true about this massive
> Canadian building Import in 2017 are still wrong and true in 2019.
>
> Really, I don't know how to say this in other ways, other times, or do
> what I haven't already felt I could do to correct this, so I'm
> communicating my frustration at a process which appears wholly broken.  If
> ever there was a test case of OSM "saving itself from an existential
> crisis," getting this project on track (or pulling its plug) is it.  Good
> luck to us, OSM:  I'm simply the messenger at this point.
>
> SteveA
> California
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/imports/attachments/20190117/275339aa/attachment-0001.html>


More information about the Imports mailing list