[Imports] [nzopengis] LINZ Upload Process - Documentation

Tue Apr 6 06:56:03 BST 2010

Hi Glen,
Its great to read your progress :-)

Im not the ninja when it comes to shp2osm, but im am the 'go-getter' for it.
This has proven to have faults, but also have benifits.
As Canadians we are naturally 'passive' so its fine. We also have the
Ibycus topo which saves the day :-)

If you have questions, please cc the talk-ca list, and let us help as
others are happy to comment.
Both for the technical & social impact of having imports happen.

Hopefully, your methods can be a model for the 'Ordnance Survay'
(head-ache) mind boggling amount of data :-)
....that i dont want to be the one to sort through.

Cheers,
Sam

p.s. You might want to look at FME and OpenJUMP as these prove to be helpfull.

On 4/5/10, Glen Barnes <barnaclebarnes at gmail.com> wrote:
> In reference to the earlier threads on importing LINZ data:
>
> -
> http://groups.google.com/group/nzopengis/browse_thread/thread/b3ba3b3a4b55ce18
> -
> http://groups.google.com/group/nzopengis/browse_thread/thread/782b883d8fd96989
> -
> http://groups.google.com/group/nzopengis/browse_thread/thread/6e1f43eec69fe6b7
>
> I thought it nest to start a new thread with the full details. I've
> been away the last few days so haven't had a chance to update
> everybody on the status and Rob Coup is also away for the week.
>
> Background
> -----------------
>
> There have been several attempts over the last few months to import
> LINZ data that has stalled due to license issues or various data
> issues. To get the ball rolling I suggested a hackfest/meetup at my
> place to see if we could at least get one piece of data imported and
> uploaded to OSM. Ed Corkery suggested we use the Chatham Islands as a
> test bed for the import as it had no user generated OSM data and would
> be of value to several organisations such as DoC.
>
> In attendance at the hackfest were Robin Paulson, Rob Coup and myself.
> The hackfest was a success - We managed to get the roads imported into
> OSM with most of the conversion done correctly. The next steps were to
> take our learnings from the hackfest and work out how we could apply
> this to the other LINZ layers.
>
> Issues
> ---------
>
> There were a few issues that we found with the process.
>
> 1) Not everyone is technical. Robin has ninja skills with the OSM wiki
> and Rob Coup can manipulate geo data better than anyone I know. We
> needed a way that all of us could work together remotely on the
> problems and get a better feedback loop.
> 2) The existing mapping tools were pretty weak when it came to
> converting LINZ tags to OSM tags. There was no logic operators ("if
> surface=gravel and highway_number not null then highway=road")
> 3) getting the LINZ data in a format that we could use and easily
> distribute was going to reduce the number of people who could
> contribute
>
> As a passing comment Rob said he would make a tool which we could use
> to export the LINZ data to OSM and do the field mapping.
>
> linz2osm
> -------------
>
> The linz2osm tool (http://linz2osm.coup.net.nz/) tries to get around
> these issues. The runways and airports layers are the first ones we
> have tried to map using this tool. It does require a few changes based
> on the feedback from Robin regarding the multi polygons but nothing to
> serious. The process for using the tool is:
>
> - Choose a layer to edit (http://linz2osm.coup.net.nz/data_dict/
> layer/). A full list of layers is here -
> http://wiki.openstreetmap.org/w/index.php?title=List_of_LINZ_layers_and_notes
> - Enter the osm tag name in the tag field. For example if you wanted
> to set the 'name' field enter 'name'. The tool embeds the LINZ web
> page with the field names on the right had side so it is easy to see
> what fields you have avaliable.
> - In the code field enter any javascript code to set the value
> variable. For the airports the code was: value = fields.name (fields
> is a variable that holds all of the LINZ fields). You can use any
> javascript in here to provide any logic you may need.
> - Repeat for additional tags you want to add. For the airports we also
> wanted to set aeroway=aerodrome so we put value = "aerodrome" in the
> next code field.
> - Add any notes about the mapping and choices you made. I've found it
> good to link to any wiki pages describing the tags you should use for
> certain features.
> - Click save (or save and continue editing to stay on the same page)
> - Now export the data - There are links on the top right of the page
> with NZ Mainland and Chathams layers. Click on one of those
> - Enter any bounds for the map. This feature will be updated to to
> allow easier selection of areas of NZ. Since we are only doing the
> Chatham Islands right now just press export and it will export all of
> the Chatham data for that layer.
> - Open it up in your favourite text editor. I personally use Tidy on
> it to format the XML to be nicely indented.
> - Eyeball it and see if it looks right.
> - Open it up in JOSM along with the base Chatham Island outline so you
> can see what it looks like.
> - If you are happy then upload it using the "LINZ Data Upload" user
> that we set up (I have the password for those that want it).
>
> So that is the basic process from start to finish but I think based on
> what we have seen with the runways and airports we should probably do
> parts of the process first and then have another hackfest to iron out
> the exporting issues before uploading any more data. So what I suggest
> is:
>
> - Go through each of the LINZ layers and have a first pass at creating
> the mappings
> - Document in the comments the assumptions you have made and any
> manual fixups you think may be needed.
> - Post any questions (to the group?) along with what mappings you made
> and what extra things you think are needed.
>
>
> Default tags (http://linz2osm.coup.net.nz/data_dict/tag/) are added to
> the layer by default. The current default tags are
> LINZ:source_version, attribution and source_ref.
>
> Email me (glen [at] open [dot] org [dot] nz) if you want a login to
> the tool to help out with the mapping.
>
> The source code for the tool is available on GitHub here:
> http://github.com/rcoup/linz2osm. if you are that way inclined feel
> free to download and fix bugs.
>
>
> Next Hackfest
> ---------------------
>
> It will be good to have another hackfest at some point in the next
> month to resolve any of the issues that are still outstanding. After
> that we should be able to move forward quite quickly on importing the
> actual data for the rest of the country. More details to come once we
> get closer to the time.
>
> Questions/Comments
> -------------------------------
>
> I'm happy to hear any constructive feedback on the approach. I think
> we've come up with something that is somewhere between fully automated
> and totally manual but we are open to suggestions for improvements.
>
> --
> You received this message because you are subscribed to the Google Groups
> "nzopengis" group.
> To post to this group, send email to nzopengis at googlegroups.com.
> To unsubscribe from this group, send email to
> nzopengis+unsubscribe at googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/nzopengis?hl=en.
>
>

-- 
Twitter: @Acrosscanada
Blogs: http://acrosscanadatrails.posterous.com/
http://Acrosscanadatrails.blogspot.com
Facebook: http://www.facebook.com/sam.vekemans
Skype: samvekemans
OpenStreetMap IRC: http://irc.openstreetmap.org
@Acrosscanadatrails