[Imports] New Zealand address import

[Eliot Blennerhassett]

[Resending this to the list, as I just noticed I replied only to Michael]

Hello Michael

On 14/06/17 06:03, Michael Reichert wrote:
> Hi Eliot,
> 
> Am Freitag, den 02.06.2017, 12:59 +1200 schrieb Eliot Blennerhassett:
>> Merging this dataset would also free mappers from the near-impossible
>> task of manual address entry for the whole country.
> 
> Mapping a country's addresses is not impossible, it is just difficult
> and a lengthy task. Urban areas are relatively fast mapped if the
> number of mappers per 1000 inhabitants is similar to some well known
> countries in Central Europe. 

I don't think that assumption (about mapper density) is correct.

At the current rate it could easily be more than 100 years.

There are 1.9 million addresses in the official database.

There are about 46000 addresses currently in OSM.  I have reason to
suspect that about 25000 of these are the result of stealthy importing
of the LINZ data set.  The remaining 20000 manually entered ones
correspond to 0.1% of the total.


> The lengthy part is mapping rural areas
> where the distances between the settlements are large. If you want to
> strengthen your urban communities, you should only import addresses in
> rural areas. (This does not mean that you have to wait until urban
> areas are finished)
> 
> Mapping addresses is no stupid work. Depending on your mapping
> technique, you can also collect other useful data or check if new
> buildings have been built and existing building been knocked down.

Addresses (here in NZ at least) are valid whether or not buildings are
on the site.  Here in Christchurch that has been particularly obvious
over the past 6 years.

> These are just two examples on how to update data.
> 
>> To anticipate some questions.
>> * The source data is CC-BY, and the attribution requirements are well
>> documented.
> 
> The wiki page does not mention that the owner of the data granted a
> permission which lifts the limitations of CC-BY for OpenStreetMap.
> Please use the template provided by License Working Group. (It is not
> only the attribution requirement which makes the CC-BY to be
> incompatible with our licensing).
> https://drive.google.com/file/d/0B3PN5zfbzThqeTdWR1l3SzJVcTg/view

Have you read the parent page where the details are laid out?
https://wiki.openstreetmap.org/wiki/LINZ#Data_License

> Some other thoughts while reading your wiki page:
> 
> - What's the purpose of adding LINZ:address_id=* and linz:change_id=*
> to OSM?

To retain the connection to the source data for ongoing updates.

The change id is in the 'possible' section, but I don't propose adding
it to OSM.

> - You propose to add a tag source:revision=*. Where will it be placed?
> On the changeset or on the imported objects?

As you will have read, it is to go in the changeset tags.

> - Where will you place the source=* tag you mention? On objects or on
> changesets?
> - You write on the wiki page that you are not sure whether to use
> addr:city=*, addr:place=* or addr:hamlet=* for the field "town_city" of
> the LINZ data. Why are you not sure? Is town_city a mix of information
> which should be stored in two distinct fields in an ideal dataset or
> are you unsure about the definitions of the three OSM tags?

By default I think that it is not necessary to have the higher level
keys on every address, because this information is already provided by
e.g. city and suburb or town, village or hamlet boundaries or proximity
to named places.

... as described in https://wiki.openstreetmap.org/wiki/Addresses

But, if they are to be included...

There are two general cases in the source data.

1: town_city=NULL, suburb_locality=<locality or village>
(1930 of these)
The closest key is addr:hamlet, but for many there is no 'hamlet', i.e.
no village as such.  Maybe addr:place is a better match for those.

2: town_city=<town or city name>, suburb_locality=<suburb>
(1182 of these)
This maps fairly naturally to addr:city and addr:suburb

Part of my uncertainty is that place= has more options, e.g.
city,town,village,hamlet,locality,
whereas there are only (commonly) addr:city, addr:hamlet

> - Have you assessed the quality of the data by checking it on the
> ground on a few locations in urban and rural areas?

I have checked against areas I'm familiar with, and found it to be high
quality.

> - The wiki page should be improved by adding more technical details of
> the import, namely how duplicates will be detected and which
> software/workflow will be used to upload the data. Please publish the
> source code of the scripts if you use any (at a sufficient location for
> source code).

Linked from here
https://wiki.openstreetmap.org/wiki/LINZ/Address_Import#Notes

> - Please add a list of user accounts who will upload the data. Their
> user profiles at openstreetmap.org should mention who they belong to.
> One import account should only be used by one natural person.

These accounts are documented at
https://wiki.openstreetmap.org/wiki/LINZ#List_of_Import_Accounts

Yes, it was an error on my part not to use my dedicated import account
in this case.

> Although you write that your import proposal is in an early stage, you
> have already started uploading data. https://www.openstreetmap.org/chan
> geset/49492831 

I first tested on the dev server, but it seems to contain no other data
in NZ (JOSM retrieved empty data until I uploaded points)?
So it is not a good test of how the proposed import interacts with
existing data.

> Please wait at least one week in addition whether
> further concerns are raised. Please read the Import Guideline
> carefully! I doubt that you haven't read it as carefully as necessary
> to do a good import. 

I think you mean you doubt I *have* read it carefully...

> (Btw, it is a very good idea to read most emails
> of the last six months in the archive of this mailing list, too)
> 
> Best regards
> 
> Michael