[Talk-nz] Continuing the Street Address Import

[Kyle Hensel]

Hi Eliot, cc the rest of the mailing list

> I see the proposal excludes addresses "deprecated" (?deleted?) by LINZ

Sorry, typo. ‘Deprecated’ should be ‘deleted’. I think you’re right – it does make sense to also delete addresses from OSM if LINZ says they no longer exist, but I think this may be controversial. How does the community feel about this?

> I looked at this last year, with similar results

I don’t know how I missed that email while looking through the archives, sorry! I did see the emails from 2014-2018 on the imports mailing list though.
My thoughts on your question from that email thread:

> Two broad options for the data source:
>   1. Changeset from LINZ
>   2. Full data dump from LINZ, conflation with existing OSM How to process the changes, how to organise the updates (tasking manager?) etc. etc.

As it stands, I’ve used option 2 to process the data, but if we want to include deletions then we’ll also need LINZ’s changeset.

> Looking [through] the raw data reports, a couple of tweaks would make them more useful for manual review and cleanup

Those txt reports will definitely be improved - I was hoping to get feedback on how this data could be best reviewed/processed.
I think those are great ideas, some other thoughts I had were:

  *   Generate osmchange files that can be imported into JOSM or Level0 (but not iD sadly)
  *   Make some code changes to the RapiD editor to support updating or moving nodes, not just adding data.

> Perhaps with "wrong location" we'd still want a way to mark these as reviewed and OK from OSM perspective

Good point. The StreetComplete editor adds an OSM field called check_date= when mappers confirm that post boxes still exist, or road closures are still ongoing. Perhaps we could add the check_date field to addresses that have been reviewed but no change is required?

From: Eliot Blennerhassett<mailto:eliot at blennerhassett.gen.nz>
Sent: Wednesday, 10 March 2021 11:49
To: Kyle Hensel<mailto:K.y.l.e at outlook.co.nz>; talk-nz<mailto:talk-nz at openstreetmap.org>
Subject: Re: [Talk-nz] Continuing the Street Address Import

Hi Kyle,

I'm very glad to see someone else taking a look at this, and interested in contributing.

On 9/03/21 10:08 pm, Kyle Hensel wrote:
> Hi all,
>
> I had a look at how many street addresses are missing from OSM, which was rather simple thanks to the `ref:linz:address_id` tag which exists on most address points.
>
> Here are some stats:
>
> - 1.9 million address points in OSM are still ‘perfect’ since the initial import in 2017.
>
> - about 39,000 addresses in OSM almost perfectly match LINZ's data, but are just missing the `ref:linz:address_id` tag (the majority of these are in Te Anau and Napier where they were merged with buildings and the ref was removed)
>
> - about 17,000 addresses have been modified by LINZ, so the OSM data is now out-of-date (e.g. spelling corrected, suburb changed)
>
> - <300 addresses have been duplicated by mappers, so there are now two nodes with the same `ref:linz:address_id` tag
>
> - And finally, 161,000 new addresses are totally missing from OSM.

I looked at this last year, with similar results.  https://lists.openstreetmap.org/pipermail/talk-nz/2020/000221.html  my starting point was a changeset
from LINZ that contained changes since the import.

> I'm a regular OSM contributor (username kylenz) and I've been thinking about how this dataset could be updated.
>
> Since the original import in 2017, the RapiD editor has emerged, which is a modified version of the iD editor, designed to speed up importing data in a controlled manner.
>
> I've created a detailed wiki page over at https://wiki.openstreetmap.org/wiki/Import/New_Zealand_Street_Addresses_(2021) <https://wiki.openstreetmap.org/wiki/Import/New_Zealand_Street_Addresses_(2021)> which documents a proposal to use a custom version of the RapiD editor to import the new addresses, and to fix incorrect data in addresses. I've also created a quick proof-of-concept which you can try out here: https://linz-addr.kyle.kiwi <https://linz-addr.kyle.kiwi> - *NOTE:* it deliberately does not let you upload changesets (yet).
>
> I'd love to hear thoughts and feedback on this proposal, not least from those involved in the original import.



It would be great to eventually catch up with the LINZ data so that the subsequent flow of changes was not overwhelming.

I see the proposal excludes addresses "deprecated" (?deleted?) by LINZ.  I think this needs to be in scope, as these are often replaced by different addresses, e.g. in
new subdivisions.

The original import did not include updating existing addresses with the ref:linz:id, I agree with adding this to existing matching addresses.

----------------------------
Looking the the raw data reports, a couple of tweaks would make them more useful for manual review and cleanup
* turning them into html so that the links are clickable.
* making locations into references to the OSM  map
* making LINZ ID link to linz address data - I don't see an easy way to make this a link to linz website data itself


E.g. wrong location
2559 https://osm.org/node/5616392035 needs to move 834m to https://www.openstreetmap.org/#map=17/-45.03458/168.67680

With these tweaks some of the errors (duplicate addresses, wrong location) could be knocked off fairly quickly.
Perhaps with "wrong location" we'd still want a way to mark these as reviewed and OK from OSM perspective.  E.g. a house on a large block of land might be far from the nominal location of that address.

--
Eliot

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/talk-nz/attachments/20210309/cf7f1ba3/attachment.htm>