[Imports] Ongoing Canadian building import needs to be stopped, possibly reverted

Thu Jan 17 16:38:58 UTC 2019

Hi Nate,

I'm one of the three guys actively doing the import as zzptichka_import. I
didn't write the wiki and joined the import once it started but so far it's
been going fairly well IMO.
If you go back a couple of months into this list archive you'll see the
discussion of this import. There has been discussion of this import on
the osm-ca channel as well.
Data, at least in Toronto is of very good quality. Some buildings (only in
some parts of Toronto) have overlapping 3D features that need to be dealt
with and we've discussed that on slack chats in details once we've come
across this in Toronto. I agree that needs to be added to the wiki.

As for your specific concerns,

1) There was a discussion going on in the import list (starting with Ottawa
import a year ago) and in the slack channel. If you have any concerns let's
talk.

2) Overlapping buildings in this particular example is what you get with
aforementioned 3D features. It had to be dealt with, we've discussed that
with DannyMcD. I agree this is not acceptable and he'll have to go back and
re-validate those (as far as I can see there are not many errors like
these). You can go through 300000 buildings imported by me and won't find
anything like that: https://osmcha.mapbox.com/?filters=%7B
"users"%3A%5B%7B"label"%3A"zzptichka_import"%2C"value"%3A"zzptichka_import"%7D%5D%2C"date__gte"%3A%5B%7B"label"%3A""%2C"value"%3A""%7D%5D%7D

3) Task manager doesn't allow to split squares further unfortunately.
However, as someone who has done multiple of those squares over last couple
of weeks I can tell you with JOSM hotkeys and filters it's fairly easy to
do. Typically it takes me no more that 1 hour to validate dense square with
~5000 buildings and fix several dozen other building and highway-related
JOSM warnings (Toronto unfortunately has rather poor OSM quality)

4) Task Manager instructions page (that I've edited) clearly requests to
validate and fix all building-related errors and warnings. Can't say for
DannyMcD(who I know as a very meticulous mapper) and carpbunker but I
follow this process religiously.

5) Again, instructions page request to use "Replace geometry" when
possible. I'll see if I can add this to the wiki page if it's not there. As
for me, I ALWAYS us "Replace geometry" on buildings that have any
meaningful tags. I only delete existing buildings when I come across a
cluster of roughly drawn shapes with the only generic "building=yes" tag.
Executing replace geometry for each pair of those buildings would be
insanely time-consuming. If you know of any better solution please let me
know.

Thanks for flagging issues with the import. I'll ask guys to stop importing
and address the raised concerns and resume only after everything has been
dealt with.

On Thu, Jan 17, 2019 at 10:30 AM Nate Wessel <bike756 at gmail.com> wrote:

> I just learned yesterday of a very large import of buildings across Canada
> that seems to have been posted to this list in November without getting
> much of a response. I remember seeing the email about it at the time, and
> as there was substantial reference to an import in Ottawa, I thought the
> import was limited to that city and failed to read the actual details of
> the proposal. This appears to be an import of building footprints across
> ALL MAJOR CANADIAN CITIES.
>
>
> https://wiki.openstreetmap.org/wiki/WikiProject_Canada/Canada_Stats_Canada_Building_Outlines_Import/Plan
>
> https://lists.openstreetmap.org/pipermail/imports/2018-November/005812.html
>
> This is being imported NOW, very quickly, and the data I've seen in
> Toronto is of dubious quality at best. Given the speed with which this has
> taken place and the lack of clear discussion about what exactly the import
> is, what exactly it covers, I would like to see this import effort halted
> immediately if not reverted.
>
> My concerns are as follows:
>
> 1. I'm a very active mapper living and editing in Toronto and subscribed
> to the relevant lists and slack channels, yet I only just learned of this
> when the buildings encroached on my own neighborhood. I imagine many others
> also missed the notice and have not had any chance to review the plan.
>
> 2. There are countless examples of truly terrible data quality, weird
> footprints, overlapping buildings, wildly unsimplified geometries, etc.
> e.g.
> https://www.openstreetmap.org/way/663824488
> https://www.openstreetmap.org/way/662764169
> etc.
> These issues clearly fail in a simple JOSM validation, which indicated
> that no validation is happening before upload.
>
> 3. The data is being imported in extremely large chunks, (see the tasking
> manager grid for Ontario: http://tasks.osmcanada.ca/project/145), much
> larger I think than any one person could validate or review in a single
> sitting. There are more than ten thousand buildings in some of these grid
> cells. As changesets are limited to 10k objects, including nodes, it is
> extremely difficult to see what data was there before these changes, as
> each task involves multiple essentially simultaneous changesets.
>
> 4. The approach seems to be import first, validate second (if ever?).
>
> 5. The wiki plan gives no clear description of the plan for integrating
> new data with old. In places where there used to be user-contributed
> buildings, I now see buildings from this import covering or replacing those
> buildings with no remaining way history. I don't think I need to say that
> this is completely unacceptable.
>
> What are the options on the table here? There has been a truly massive
> amount of data imported in the last couple of weeks. I'd love to wind back
> time and criticize the living hell out of this plan before it started, but
> that's not where we're at. What can be done?
>
> :-(
> --
> Nate Wessel
> Jack of all trades, Master of Geography, PhD candidate in Urban Planning
> NateWessel.com <http://natewessel.com>
>
> _______________________________________________
> Imports mailing list
> Imports at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/imports
>

-- 
Best Regards,
          Yaro Shkvorets
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/imports/attachments/20190117/5bd95909/attachment-0001.html>