[Imports] import of data from LINZ

Sun Aug 19 06:21:46 BST 2012

hi paul,
answers inline

On 17 August 2012 15:51, Paul Norman <penorman at mac.com> wrote:
> A few comments
>
> Has the tagging been finalized?

No - We are review all of the existing mapping an quite a few layers
have yet to be tagged. We have gone through an _extensive_ peer
reviewed tagging process on the NZOpenGPS Google Group.

> Could you post a sample .osm of a typical area?

We've already done a "trial" import. about two years ago, we imported
all the layers for the Chatham Islands, several 100 km east of the
mainland. We chose this location, as there was no previous mapping
there, although there are plenty of map-worthy items

check here for more:
http://www.openstreetmap.org/?lat=-43.89&lon=-176.525&zoom=10&layers=M

i.e. we are confident the import application generates decent .osm files

> How were the building_use translations developed? Were only the descriptions
> from LINZ used or were the from LINZ names to OSM verified for each mapping
> by looking at imagery or another suitable means?

A painstaking manual process of reviewing the LINZ tags and finding
the best OSM tags to use. Our conversion scripts allow complex
translation of building uses with conditional logic, etc.

> What verification is the importer expected to do before uploading, and is
> this well-documented? Some multiple-person imports like CanVec have had very
> inconsistent quality with some people just dumping the data in even when
> it's clearly wrong.

They are expected to review the data they have downloaded, download
surrounding OSM data, merge with existing hand edited data (taking
existing data where appropriate, using LINZ data where we know it is
better (coastlines). For example the airport polygons have just been
merged and there was a range of data cleaning performed:

- Existing airport points had things like call signs. These were
merged into the new polygons.

-  Some existing polygons were better than the LINZ ones - we used those.

- Some airport names were updated, alt names used, etc.

Before we let others loose on the tool we will be documenting the
correct processes. We are also going to limit the checkouts to a max
100 features at a time. No one can just import the whole of NZ for an
entire layer. Offshore islands are somewhat unique in that there is
no/little existing OSM data so we can import more wholesale. These
also provide a great testing ground for our imports. Please check the
Chatham Islands for our test bed import that was done 18 months ago.

People are already trying to import the LINZ data off their own back.
Using our tools we do this in a managed way. The process has been well
thought out and we are proceeding with caution. It has taken us well
over two years to get to this point.

> I suggest having a requirement that the changesets are tagged with source=*
> and attribution=*. It's also a good idea for the source to be linked in the
> user page of the import accounts being used.

Data from LINZ will be updated on a regular basis and by November we
will have ID's for all points. We plan to provide tools for 2 way
review as LINZ and others will be able to use OSM data as pointers to
where they should update their data.

> I also suggest instead of attribution=*, source_ref=*, LINZ:source_version=*
> you just use source=LINZ v16

We'd like to keep the source separate from the attribution, collapsing
the two requires data-parsing, if it's ever processed at a later date,
which seems unnecessary

> You cannot count on the attribution tag always being there as a user could
> remove it - LINZ needs to be satisfied with attribution through the wiki
> page and the history showing who imported it.

LINZ are satisfied with our tagging and have actively encouraged it.
We have a very good relationship with the government on open data. A
couple of the local OSM team (that's Glen Barnes and Robert Coup) also
have a lot of engagement with the working group on open data policy in
central government. We are fully supported in this regard.

Robin and the rest of the NZ importers

>> From: Robin Paulson [mailto:robin.paulson at gmail.com]
>> Sent: Thursday, August 16, 2012 7:26 PM
>> To: imports at openstreetmap.org
>> Subject: Re: [Imports] import of data from LINZ
>>
>> further information:
>> a list of the layers and the tags we will use are included here:
>> https://wiki.openstreetmap.org/wiki/LINZ_attribute_matching
>>
>> also, please note: the majority of the work will be done by hand, there
>> is likely to be very little (or none) import done by bot.
>>
>> On 17 August 2012 14:21, Robin Paulson <robin.paulson at gmail.com> wrote:
>> > hi all,
>> > my name's robin, i'm involved with mapping in new zealand. we have
>> > been given access and permission to import data from LINZ (land
>> > information new zealand), who manage land parcel data in nz. the
>> > information we have from them is huge, and is not limited to land
>> > parcels. there are 150+ layers, including roads.
>> >
>> > i am posting here to ensure a proper consultative process is carried
>> > out, as outlined on the imports page. we have already engaged the
>> > community in new zealand, and made assurances that no existing data
>> > will be overwritten/deleted/modified, without discussing with the
>> > community first.
>> >
>> > the community is mainly centred around the nzopengis list on google
>> > groups, with personal relationships between other mappers.
>> >
>> > this is the first mass import many of us in nz have carried out, so
>> > we're not entirely sure of the process from here.
>> >
>> > LINZ have given osm permission to import the data under either
>> > CC-BY-SA 2.0 or ODBL - a number of mappers have been in close contact
>> > with this and other departments, including via koordinates.com (a
>> > local GIS company, who's employees are also members of the osm
>> > community) who are also carrying out the technical side of things.
>> >
>> > koordinates have assigned a body to work on the import project, he is
>> > currently developing a web application through which we can:
>> > a) convert the data to .osm format, including tag translation, the
>> > details of which the community has been involved in
>> > b) import the data
>> > c) allow mappers to allocate a portion of the map to themselves, so it
>> > can be merged
>> >
>> > at all steps, koordinates are keeping the rest of the community up to
>> > date on progress
>> >
>> > so, we assume we have missed something, and i am here to ask for
>> > assistance. is there anything we need to cover?
>> >
>> > progress so far is detailed here:
>> > https://wiki.openstreetmap.org/wiki/LINZ
>> >
>> > --
>> > robin
>> >
>> > http://fu.ac.nz - Auckland's Free University
>>
>>
>>
>> --
>> robin
>>
>> http://fu.ac.nz - Auckland's Free University
>>
>> _______________________________________________
>> Imports mailing list
>> Imports at openstreetmap.org
>> http://lists.openstreetmap.org/listinfo/imports
>

-- 
robin

http://fu.ac.nz - Auckland's Free University