[Talk-ca] 2020 building import wiki comment by Nate Wessel
John Whelan
jwhelan0112 at gmail.com
Fri Jan 18 21:04:53 UTC 2019
James you know I could never resist a dare!
Cheerio John
James wrote on 2019-01-18 4:03 PM:
> dare you to run simplify tool on anything remotely round, it will make
> it look like garbage
>
> On Fri., Jan. 18, 2019, 3:49 p.m. John Whelan <jwhelan0112 at gmail.com
> <mailto:jwhelan0112 at gmail.com> wrote:
>
> The import mailing list was pointed to the correct page of the
> wiki. The initial post was to say this is what we were thinking
> of and there was a comment saying we needed to change the comment
> line.
>
> >There is no mention of this proposed import on the import catalogue
>
>
> The import process was reviewed by the person who set up the
> Ottawa import did we miss that step on the Ottawa import as well?
> Neither was it raised as a concern on the import mailing list. I
> think this is very minor and can be corrected.
>
> We learnt a fair bit on the Ottawa import and my expectation is
> since we are using experienced mappers to do the import conflation
> would be either handled by them or the building not imported. We
> aren't using new mappers in a mapathon here and with experienced
> mappers then I think you have to trust them. The world isn't
> perfect. Think in terms of service level.
>
> >There are 2X more nodes than needed to represent the building
> accurately.
>
> The problem with correcting this is you are introducing
> approximations. This will vary according to the source and this
> can be simplified or corrected once its in OSM. I think this is a
> different issue of a mechanical edit that needs to be considered
> separately.
>
> If we are concerned with database size then I suggest we change
> the instructions to say put the source comment on the change set
> rather than on the building outline.
>
> Cheerio John
>
>
> Nate Wessel wrote on 2019-01-18 3:06 PM:
>>
>> John,
>>
>> You seem to be playing the long game with this data - it sounds
>> like you've been working with this a lot longer than I have, and
>> you've put in the time and effort to help make this
>> actually-quite-incredible dataset available to us. I don't want
>> to stop the import from happening - quite the opposite. I just
>> want to make sure that the time is taken to do this right. OSM
>> deserves that. Your (our) long awaited victory will be the
>> sweeter for our patience now.
>>
>> There are several specific issues I see where the I's are not
>> crossed, nor the t's dotted. I've mentioned several already, so
>> I'll try to be brief (I really need to get back to working on my
>> dissertation).
>>
>> 1) There was extremely limited discussion on the imports mailing
>> list. The initial email did not make clear the scope of the
>> project. I read the email and did not think twice at it, thinking
>> it was entirely about Ottawa. The link in that email was actually
>> to the Ottawa import, and not this one, which seems to have been
>> only in draft at the time.
>> https://lists.openstreetmap.org/pipermail/imports/2018-November/005812.html
>> As such, this project has NOT been reviewed by the imports list,
>> which is a requirement for proceeding with the import.
>>
>> 2) There is no mention of this proposed import on the import
>> catalogue (https://wiki.openstreetmap.org/wiki/Import/Catalogue)
>> which is required in the imports guidelines. I suspect many other
>> guidelines have not been followed.
>>
>> 3) The wiki page describing the import is not adequate to assess
>> the quality of the data or of the proposed import. See for
>> example:
>> https://wiki.openstreetmap.org/wiki/WikiProject_Canada/Canada_Stats_Canada_Building_Outlines_Import/Plan#Risks
>> The import guidelines call for a description of how conflation
>> will be handled. The fact that two of the major importers seem to
>> have a substantial disagreement about how to handle existing data
>> indicates this was not well discussed and I can see that it isn't
>> well documented.
>>
>> 4) The buildings need to be simplified, quite a bit actually.
>> Most buildings have multiple nodes representing straight lines.
>> This bloats the database and makes things harder to edit by hand
>> later. There are probably 2x more nodes than are needed to
>> represent the data accurately, making it harder for editors and
>> data consumers to work with down the road.This is a simple fix
>> that will save countless hours later.
>>
>> ... I could go on, but I think this is plenty sufficient to
>> justify pressing pause on all this.
>>
>> Again, I don't in any way want to disrespect the work that has
>> gone into this effort already. We're all volunteers here and I
>> know how much time this all takes. However. importing all/most of
>> the buildings in Canada is a monstrously large task, which will
>> have to dance around a lot of people's toes. We should expect
>> this to take a really damn long time if we're going to do it
>> right. We need to have the patience to learn from experience,
>> from critique, and from the wisdom of the people who've learned
>> from flawed imports in the past and have devised guidelines and
>> processes so that we can have better experiences with this in the
>> future.
>>
>> Nate Wessel
>> Jack of all trades, Master of Geography, PhD candidate in Urban
>> Planning
>> NateWessel.com <http://natewessel.com>
>>
>> On 1/18/19 2:24 PM, john whelan wrote:
>>> My background is I'm a retired civil servant who has written and
>>> overseen procurement documents and fairly large procurements.
>>> Dotting the is and crossing the Ts are my speciality.
>>>
>>> There are two parts to an import. The first part is the part
>>> played by the import mailing group. They confine themselves to
>>> is the license correct and do you have a reasonable plan. In
>>> this case the license is one of the few that has been confirmed
>>> by the Legal Working Group of OpenStreetMap and as such no
>>> questions were raised about it on the import mailing list. We
>>> have methodology that has been used before successfully with the
>>> Ottawa building outline import. There were major discussions
>>> both on talk-ca and the import mailing group before that import
>>> took place and we took note of the issues raised and addressed
>>> them. The licensing issue goes back about eight years to when I
>>> was talking to Federal Government Treasury Board and explaining
>>> their Open Data license did not align with OSM. That is why
>>> their license is now known as 2.0.
>>>
>>> The second part is the local group makes the decision to import
>>> they are the authority no one else.
>>>
>>> Apparently you were not part of the talk-ca when the discussions
>>> took place which would have been the time and place to raise
>>> concerns.
>>>
>>> When the Ottawa import was done there were one or two places
>>> where the existing buildings and the import overlapped. In the
>>> instructions on the import there are instructions to cover this.
>>> Specifically there is a validation step. I seem to recall the
>>> error rate was of the order of 1% and I expect this latest batch
>>> to be roughly the same.
>>>
>>> If you can identify which municipalities data is of poor quality
>>> then I'm sure we can remove these. For the most part these are
>>> from the foundation plans recorded by the municipality using
>>> professional surveying techniques.
>>>
>>> Would you like to clarify exactly where I failed to dot the Is
>>> and cross the Ts please.
>>>
>>> Many Thanks
>>>
>>> John
>>>
>>>
>>>
>>> On Fri, 18 Jan 2019 at 13:37, Nate Wessel <bike756 at gmail.com
>>> <mailto:bike756 at gmail.com>> wrote:
>>>
>>> Hi John,
>>>
>>> As Steve has said, you seem to be the only one suggesting
>>> that thousands of import committees might need to be formed.
>>> Certainly I'm not suggesting that.
>>>
>>> My understanding of OSM import procedure (and wiki-style
>>> projects more generally) is that imports should operate in
>>> an essentially consensual way where possible. The goal is to
>>> build consent and bring people on board with a project or a
>>> change by addressing their concerns in a meaningful and
>>> respectful way.
>>>
>>> I think that I have made some substantive and troubling
>>> claims about the quality of the data being imported. I've
>>> pointed out that this project has not followed the import
>>> procedures that were produced by a community of mappers
>>> larger than just those in Canada.
>>>
>>> So to respond to your implication, I am in some sense the
>>> one reviewing the project, just as I would welcome you to
>>> find ways that my own contributions could be better. If you
>>> want my credentials for reviewing your work, here they are:
>>>
>>> 1) I am an active contributor to OSM in Toronto, where I
>>> live (and elsewhere)
>>>
>>> 2) I am currently helping to lead a building import in
>>> Hamilton County Ohio that has better addressed some of the
>>> issues I see this import struggling with. I can help you do
>>> the same.
>>>
>>> 3) I've been doing research in GIS for a long time now,
>>> though I don't need that to tell you that the issues I've
>>> described are hardly insurmountable technically or even all
>>> that difficult to fix. It would take maybe one day's hard
>>> work to get the technical side of this right.
>>>
>>> I think Canadian OSMers will agree that we can take a pause
>>> to get things right on such a massive import. If they don't
>>> - if I'm shouted down or better, if my critiques are
>>> adequately addressed, then I will leave you to finish the
>>> project in peace. I might even lend a hand if all goes well,
>>> as I sincerely hope it does :-)
>>>
>>> Best,
>>>
>>> Nate Wessel
>>> Jack of all trades, Master of Geography, PhD candidate in
>>> Urban Planning
>>> NateWessel.com <http://natewessel.com>
>>>
>>> On 1/18/19 1:11 PM, john whelan wrote:
>>>> I know of no other way to contact him but he made an
>>>> interesting comment that the project is on hold in the wiki
>>>> pending review.
>>>>
>>>> Would he care to comment on who is supposed to be reviewing
>>>> the project?
>>>>
>>>> My understanding is that the import was raised in talk-ca
>>>> before it commenced for comment and these were generally
>>>> favourable. I took that as the local mappers to Canada had
>>>> been consulted and they are the "local mappers" authority
>>>> in this case.
>>>>
>>>> I understand he has concerns about local mappers making
>>>> decisions but in Canada we have been importing similar data
>>>> through CANVEC for some time. CANVEC data comes from a
>>>> number of sources including municipal data.
>>>>
>>>> Is he suggesting that each of the 3,700 municipalities in
>>>> Canada should form a group of local mappers who can make
>>>> individual decisions on whether their municipal data should
>>>> be imported and we should end up with 3,700 import plans?
>>>>
>>>> Thanks John
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Talk-ca mailing list
>>>> Talk-ca at openstreetmap.org <mailto:Talk-ca at openstreetmap.org>
>>>> https://lists.openstreetmap.org/listinfo/talk-ca
>>> _______________________________________________
>>> Talk-ca mailing list
>>> Talk-ca at openstreetmap.org <mailto:Talk-ca at openstreetmap.org>
>>> https://lists.openstreetmap.org/listinfo/talk-ca
>>>
>
> --
> Sent from Postbox
> <https://www.postbox-inc.com/?utm_source=email&utm_medium=siglink&utm_campaign=reach>
> _______________________________________________
> Talk-ca mailing list
> Talk-ca at openstreetmap.org <mailto:Talk-ca at openstreetmap.org>
> https://lists.openstreetmap.org/listinfo/talk-ca
--
Sent from Postbox
<https://www.postbox-inc.com/?utm_source=email&utm_medium=siglink&utm_campaign=reach>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/talk-ca/attachments/20190118/d94dd3ac/attachment-0001.html>
More information about the Talk-ca
mailing list