[Imports] import of data from LINZ

[Paul Norman]

> From: Robin Paulson [mailto:robin.paulson at gmail.com]
> Subject: Re: [Imports] import of data from LINZ
> 
> hi paul,
> answers inline
> 
> On 17 August 2012 15:51, Paul Norman <penorman at mac.com> wrote:
> > A few comments
> >
> > Has the tagging been finalized?
> 
> No - We are review all of the existing mapping an quite a few layers
> have yet to be tagged. We have gone through an _extensive_ peer reviewed
> tagging process on the NZOpenGPS Google Group.
> 
> > Could you post a sample .osm of a typical area?
> 
> We've already done a "trial" import. about two years ago, we imported
> all the layers for the Chatham Islands, several 100 km east of the
> mainland. We chose this location, as there was no previous mapping
> there, although there are plenty of map-worthy items
> 
> check here for more:
> http://www.openstreetmap.org/?lat=-43.89&lon=-176.525&zoom=10&layers=M
> 	
> i.e. we are confident the import application generates decent .osm files

The point is not to generate a valid .osm file, but to make it easier to
review the tagging. The area in the DB gives me enough to provide some
comments, which I'll do below.

> > How were the building_use translations developed? Were only the
> > descriptions from LINZ used or were the from LINZ names to OSM
> > verified for each mapping by looking at imagery or another suitable
> means?
> 
> A painstaking manual process of reviewing the LINZ tags and finding the
> best OSM tags to use. Our conversion scripts allow complex translation
> of building uses with conditional logic, etc.

But did you look at just the LINZ documentation or did you verify that the
mappings are correct with imagery or local knowledge?

> Before we let others loose on the tool we will be documenting the
> correct processes. We are also going to limit the checkouts to a max
> 100 features at a time. No one can just import the whole of NZ for an
> entire layer. Offshore islands are somewhat unique in that there is
> no/little existing OSM data so we can import more wholesale. These also
> provide a great testing ground for our imports. Please check the Chatham
> Islands for our test bed import that was done 18 months ago.

It's good to hear that it will be limited and it will be documented

> People are already trying to import the LINZ data off their own back.
> Using our tools we do this in a managed way. The process has been well
> thought out and we are proceeding with caution. It has taken us well
> over two years to get to this point.
> 
> > I suggest having a requirement that the changesets are tagged with
> > source=* and attribution=*. It's also a good idea for the source to be
> > linked in the user page of the import accounts being used.
> 
> Data from LINZ will be updated on a regular basis and by November we
> will have ID's for all points. We plan to provide tools for 2 way review
> as LINZ and others will be able to use OSM data as pointers to where
> they should update their data.
> 
> > I also suggest instead of attribution=*, source_ref=*,
> > LINZ:source_version=* you just use source=LINZ v16
> 
> We'd like to keep the source separate from the attribution, collapsing
> the two requires data-parsing, if it's ever processed at a later date,
> which seems unnecessary

The current thinking on import best practices is to minimize the number of
metadata tags.

> > You cannot count on the attribution tag always being there as a user
> > could remove it - LINZ needs to be satisfied with attribution through
> > the wiki page and the history showing who imported it.
> 
> LINZ are satisfied with our tagging and have actively encouraged it.
> We have a very good relationship with the government on open data. A
> couple of the local OSM team (that's Glen Barnes and Robert Coup) also
> have a lot of engagement with the working group on open data policy in
> central government. We are fully supported in this regard.

The problem is that attribution through a tag is no guarantee that
attribution will be provided as that tag can be removed. OSM offers
attribution through the wiki page, the history and changeset tags. 

Now, for some feedback on the area you linked:

Some objects have no OSM tags, for example
http://www.openstreetmap.org/browse/node/767087010 or
http://www.openstreetmap.org/browse/node/767087005 which have

LINZ:point_size = 6.0
LINZ:size = 4.1
LINZ:text_placement_justification = 1.0
LINZ:x_offset = 1.0
name = Loch Long memorial

It looks like some display information from their database made it in.

http://www.openstreetmap.org/browse/node/767114113 has tags which belong on
the nearby waterway=river

http://maps.paulnorman.ca/imports/review/streamconverge.png is a point where
three waterways converge on one spot. It appears that some waterways are
reversed.

http://www.openstreetmap.org/browse/way/58380136 is a way which has tags
which belong on the multipolygon (e.g. natural=sand)

Should most of the natural=sand be natural=beach surface=sand? That's what
it looks like from the imagery, but I'd defer to someone with local
knowledge.

The display information on some nodes is the most serious issue, but it
should be easy to fix.

Overall it looks not bad, but I look forward to seeing an updated .osm file
with these issues fixed.

Paul Norman