[OHM] Building Infrastructure for OHM RPA Historic Wetland Mapping

Sat Oct 31 18:21:14 UTC 2015

Thanks for the reply, Tim – and for building out a tool like mapwarper.net.

I was thinking more about batch import of maps, rather than control points.

I've had a chance to play around with it a bit and was getting some of the distortion you mentioned, which is likely unavoidable. I'm going to test the same map in ArcGIS and in MapWarper – I have a feeling I will get a similar amount of distortion in each. Some of it may come down to adding points to adjust the map by eye.

Sanjay Seth | Research Analyst
(917) 546-4327 | www.rpa.org
Regional Plan Association
________________________________________
From: Tim Waters [chippy2005 at gmail.com]
Sent: Saturday, October 31, 2015 6:44 AM
To: Sanjay Seth
Cc: Albin Larsson; Historic at openstreetmap.org
Subject: Re: [OHM] Building Infrastructure for OHM  RPA Historic Wetland Mapping

Hello,

I run, maintain and develop mapwarper.net<http://mapwarper.net>

Currently there is no facility to batch import control points. However I am developing this functionality for a related project and so I might get around to updating mapwarper.net<http://mapwarper.net> with these and other changes if I have a spare few weeks, but there is no timetable to do this currently. The software is relatively straightforward to install on your own server and bespoke features can also be developed, it's open source software.

You might have success using the API to add points by a script.

There is documentation on github https://github.com/timwaters/mapwarper/blob/master/README_API.md by adding a control point at a time.

I'm not aware of any people having used this though.

One thing with entering in the coordinates from images or maps is that (depending on a number of factors) there may be a little bit of distortion or stretching of the base material, distortion during the scanning process or camera lens. What this means is that there may be a need to manually add points to correct for the distortions. Newer, better surveyed and better digitized images should have less of an issue of course. You'd just need to give it a go and see :)

Tim

On 30 October 2015 at 14:54, Sanjay Seth <sanjay at rpa.org<mailto:sanjay at rpa.org>> wrote:
It looks like I would have to get an Admin account to do batch imports?

Between manual control points and batch imports, I think we’re set. This is a great tool.

—

Sanjay Seth | Research Analyst

Regional Plan Association

(917) 546-4327<tel:%28917%29%20546-4327> | rpa.org<http://rpa.org>

From: Albin Larsson <albin.post at gmail.com<mailto:albin.post at gmail.com>>
Date: Fri, 30 Oct 2015 10:49:05 -0400
To: Sanjay Seth <sanjay at rpa.org<mailto:sanjay at rpa.org>>
Cc: SK53 <sk53.osm at gmail.com<mailto:sk53.osm at gmail.com>>, "Historic at openstreetmap.org<mailto:Historic at openstreetmap.org>" <historic at openstreetmap.org<mailto:historic at openstreetmap.org>>
Subject: Re: [OHM] Building Infrastructure for OHM  RPA Historic Wetland Mapping

Maps Warper support batch imports too so I guess it would be a great match for this project.

//
Albin

On Oct 30, 2015 15:45, "Sanjay Seth" <sanjay at rpa.org<mailto:sanjay at rpa.org>> wrote:
Awesome! Sorry, a bit new to MapWarper. But thanks for pointing that out. Didn’t see the control panel. That’s perfect!

—

Sanjay Seth | Research Analyst

Regional Plan Association

(917) 546-4327<tel:%28917%29%20546-4327> | rpa.org<http://rpa.org>

From: SK53 <sk53.osm at gmail.com<mailto:sk53.osm at gmail.com>>
Date: Fri, 30 Oct 2015 10:35:34 -0400
To: Sanjay Seth <sanjay at rpa.org<mailto:sanjay at rpa.org>>
Cc: Albin Larsson <albin.post at gmail.com<mailto:albin.post at gmail.com>>, "Historic at openstreetmap.org<mailto:Historic at openstreetmap.org>" <historic at openstreetmap.org<mailto:historic at openstreetmap.org>>
Subject: Re: [OHM] Building Infrastructure for OHM  RPA Historic Wetland Mapping

You can manually add control points in MapWarper: I've done it as a starting point for a few GSGS 3906 maps by finding the lat/lon for the corners (IIRC using nearby.co.uk<http://nearby.co.uk>). All you do is mark the relevant point on the target, mark anywhere in the rectification layer and then adjust the lat lon values in the control panel.

Jerry

On 30 October 2015 at 14:34, Sanjay Seth <sanjay at rpa.org<mailto:sanjay at rpa.org>> wrote:
Very cool! Thanks for doing that, Albin. I’ll start testing it out locally and will let you know how it goes!

I would love to use Map Warper. It just doesn’t seem to work for these maps. The NOAA maps only map out natural features – so, there aren’t really any strong points to use for rectification. It’s from the 1830’s, so you can’t use the coastline or riverbeds – and there aren’t any roads on these maps.

The only accurate way to do the rectification for these maps is to use these graticules, translate into decimal degrees, and create a rectification point from that:

It doesn’t seem as though map warper allows you to make custom rectification points based off the decimal degrees of the point – but is rather a more visual interface that requires the historic maps to have a feature represented on the current OSM map.

—

Sanjay Seth | Research Analyst

Regional Plan Association

(917) 546-4327<tel:%28917%29%20546-4327> | rpa.org<http://rpa.org>

From: Albin Larsson <albin.post at gmail.com<mailto:albin.post at gmail.com>>
Date: Thu, 29 Oct 2015 19:52:42 -0400
To: Sanjay Seth <sanjay at rpa.org<mailto:sanjay at rpa.org>>
Cc: SK53 <sk53.osm at gmail.com<mailto:sk53.osm at gmail.com>>, "Historic at openstreetmap.org<mailto:Historic at openstreetmap.org>" <historic at openstreetmap.org<mailto:historic at openstreetmap.org>>
Subject: Re: [OHM] Building Infrastructure for OHM – RPA Historic Wetland Mapping

Maps Warpers mask feature can deal with those map borders ;-)

Here is the OHM instance of TM:
https://github.com/Abbe98/osm-tasking-manager2/tree/RPA-wetlands

Everything should be working except, making queries directly to OHM using the Overpass Turbo link, it's possible using some data setting in the URL but I haven't used it recently. Some string could say OpenStreetMap in some places to but most of them should be right.

For OAuth I created an app "RPA - Wetlands" it's "Main Application URL" is set to the Github repo but as far as I remember this won't be an issue at least not when running it locally.

//
Albin

2015-10-29 21:17 GMT+01:00 Sanjay Seth <sanjay at rpa.org<mailto:sanjay at rpa.org>>:
That was quick! Looking forward to test driving the TM tomorrow. Thanks for jumping in on this, Albin.

The good thing about the maps we have chosen is that, based on the 1830 to 1850 time period, there isn't significant overlap of map content. However, there are overlapping borders that do need to be cropped out.

I am hesitant to use Map Warper for these historic maps, as the features on the map are mostly natural. But I could be convinced. Here's our current methodology for georeferencing: https://www.dropbox.com/s/g2fz8ti9npmbh30/Tutorial_and_Methodology_for_Mapping_Historic_Wetlands.pdf?dl=0

These are all of the maps we are going to georeference from the NY metro region: https://www.dropbox.com/sh/q73vhjrnj0e1lbf/AADTjbuSbYM7n2JgtWLG-vOya?dl=0

This is a map RPA created in 2003 that looks at many of the historic wetlands in the NY/NJ Harbor, from a single British survey map: https://www.dropbox.com/s/pc0p9i6nlvfs1tm/RPA_Historic_TIdelands_NYNJ_Harbor_Estuary.jpg?dl=0

And here are some examples of georeferenced maps (Tif): https://www.dropbox.com/sh/xe0naanoabdtavt/AACiPDiUyGJA6LbrOjIOpqO2a?dl=0

In order to create a draft TMS, I uploaded the individually referenced maps as GeoTIFFs into TileMill, combined into a single map, and was able to export a single map in a TMS format.

However, ideally to make this scalable beyond the New York metro region, we would want to use something like MapWarper to make it easy for others to georeference other geographies or times series.

Thoughts?

Thanks! -s

Sanjay Seth | Research Analyst
(917) 546-4327<tel:%28917%29%20546-4327> | www.rpa.org<http://www.rpa.org>
Regional Plan Association
________________________________________
From: Albin Larsson [albin.post at gmail.com<mailto:albin.post at gmail.com>]
Sent: Thursday, October 29, 2015 3:51 PM
To: SK53
Cc: Historic at openstreetmap.org<mailto:Historic at openstreetmap.org>
Subject: Re: [OHM]      Building Infrastructure for OHM – RPA Historic Wetland Mapping

But that wouldn't work if the maps overlap? Such as maps over the same area but for different time periods.

I got an instance of TM up and running against OHM, will probably find some issues but is published my code later tonight.

//
Albin

You need to look at what Donal Diamond (IrlJdel) does for the GSGS3906 maps which are individually warped on map warper, but pulled together in a single TMS layer.

Example individual map TMS: http://mapwarper.net/maps/tile/11180/{zoom}/{x}/{y}.png<http://mapwarper.net/maps/tile/11180/%7Bzoom%7D/%7Bx%7D/%7By%7D.png><http://mapwarper.net/maps/tile/11180/%7Bzoom%7D/%7Bx%7D/%7By%7D.png>
Combined TMS: tms[18]:http://mapwarper.net/layers/tile/101/{zoom}/{x}/{y}.png<http://mapwarper.net/layers/tile/101/%7Bzoom%7D/%7Bx%7D/%7By%7D.png><http://mapwarper.net/layers/tile/101/%7Bzoom%7D/%7Bx%7D/%7By%7D.png>

I don't know what magic is involved, but it works.

Jerry

On 29 October 2015 at 19:20, Albin Larsson <albin.post at gmail.com<mailto:albin.post at gmail.com><mailto:albin.post at gmail.com<mailto:albin.post at gmail.com>>> wrote:
3000 maps is a bit of a issue, iD API for custom TMS layer only support one at a time, and I guess that those 3000 maps is overlapping too? For example the HOT TM has one TMS layer for each project, something that wouldn't work in this case.

I see two possible solutions for this, one is to add the link to each relevant TMS layer in the task description, this would be a lot of manual work. Another and probably better solution is to add all maps to iDs imagery index, currently there is less then 200 maps there, I'm not sure about how much 3000 maps would affect performance... This solution requires that all maps has bounding boxes. Maybe I would need to setup a custom iD instance too...

//
Albin

2015-10-29 20:03 GMT+01:00 Steven Johnson <sejohnson8 at gmail.com<mailto:sejohnson8 at gmail.com><mailto:sejohnson8 at gmail.com<mailto:sejohnson8 at gmail.com>>>:
Hi Albin,
I think that's essentially the task, but instead of invoking openstreetmap, the TM would instead point to OHM. Awesome if you can stand up a new instance!
Thanks,

-- SEJ
-- twitter: @geomantic
-- skype: sejohnson8

There are two types of people in the world. Those that can extrapolate from incomplete data.

On Thu, Oct 29, 2015 at 2:46 PM, Albin Larsson <albin.post at gmail.com<mailto:albin.post at gmail.com><mailto:albin.post at gmail.com<mailto:albin.post at gmail.com>>> wrote:

I'm looking into it, shouldn't take much time at all to setup. Just setting up a new TM instance connected to OpenHistoricalMap?

//
Albin

On Oct 29, 2015 19:32, "Sanjay Seth" <sanjay at rpa.org<mailto:sanjay at rpa.org><mailto:sanjay at rpa.org<mailto:sanjay at rpa.org>>> wrote:
Hi folks –

I had sent around an email about a month ago about a project at RPA (www.rpa.org<http://www.rpa.org><http://www.rpa.org>) to map the historic wetlands of the New York metropolitan region from the 1830’s - 1850’s, as part of our research on climate adaptation. (See email chain below for details).

The project has started to take off. We have had a great response from the community so far – and have partnered with Steven Johnson at TeachOSM and others to shepherd this mapping effort. Through this project, we hope to build out some of the infrastructure that would help the OHM community continue to grow.

For this project, we will need a Tasking Manager that is connected to OHM. We could build off the TeachOSM TM by overriding the default iD link for each job – or set up an entirely separate TM for OHM. I think the latter may be a better option. Is there other infrastructure OHM needs to move forward that could fit in the scope of this project? (I have a modest amount of funding that I could use to defray some of the cost of development time. Can anyone on this list point me in the direction of someone who would be willing to get an OHM-connected TM up and running?)

I look forward to hearing any comments, questions, ideas, suggestions from you about this project as it moves forward – and to gauge your interest in the project as a whole. If you would like to be involved in this project, please feel free to contact either myself (sanjay at rpa.org<mailto:sanjay at rpa.org><mailto:sanjay at rpa.org<mailto:sanjay at rpa.org>>) or Steven (sejohnson8 at gmail.com<mailto:sejohnson8 at gmail.com><mailto:sejohnson8 at gmail.com<mailto:sejohnson8 at gmail.com>>).

Thanks!
Sanjay

Sanjay Seth | Research Analyst
(917) 546-4327<tel:%28917%29%20546-4327><tel:%28917%29%20546-4327> | www.rpa.org<http://www.rpa.org><http://www.rpa.org>
Regional Plan Association
_______________________________________________
Historic mailing list
Historic at openstreetmap.org<mailto:Historic at openstreetmap.org><mailto:Historic at openstreetmap.org<mailto:Historic at openstreetmap.org>>
https://lists.openstreetmap.org/listinfo/historic

_______________________________________________
Historic mailing list
Historic at openstreetmap.org<mailto:Historic at openstreetmap.org><mailto:Historic at openstreetmap.org<mailto:Historic at openstreetmap.org>>
https://lists.openstreetmap.org/listinfo/historic

_______________________________________________
Historic mailing list
Historic at openstreetmap.org<mailto:Historic at openstreetmap.org><mailto:Historic at openstreetmap.org<mailto:Historic at openstreetmap.org>>
https://lists.openstreetmap.org/listinfo/historic

________________________________

This email has been scanned for spam and viruses by Proofpoint Essentials cloud email security - click here<https://us1.proofpointessentials.com/index01.php?mod_id=11&mod_option=logitem&mid=762151066&rid=48209320&report=1> to report this email as spam.

________________________________

This email has been scanned for spam and viruses by Proofpoint Essentials cloud email security - click here<https://us1.proofpointessentials.com/index01.php?mod_id=11&mod_option=logitem&mid=762696488&rid=48209320&report=1> to report this email as spam.

________________________________

This email has been scanned for spam and viruses by Proofpoint Essentials cloud email security - click here<https://us1.proofpointessentials.com/index01.php?mod_id=11&mod_option=logitem&mid=764042427&rid=48209320&report=1> to report this email as spam.

________________________________

This email has been scanned for spam and viruses by Proofpoint Essentials cloud email security - click here<https://us1.proofpointessentials.com/index01.php?mod_id=11&mod_option=logitem&mid=764094492&rid=48209320&report=1> to report this email as spam.

_______________________________________________
Historic mailing list
Historic at openstreetmap.org<mailto:Historic at openstreetmap.org>
https://lists.openstreetmap.org/listinfo/historic

________________________________

This email has been scanned for spam and viruses by Proofpoint Essentials cloud email security - click here<https://us1.proofpointessentials.com/index01.php?mod_id=11&mod_option=logitem&mid=766016402&rid=48209320&report=1> to report this email as spam.

[OHM] Building Infrastructure for OHM ­­ RPA Historic Wetland Mapping

[OHM] Building Infrastructure for OHM RPA Historic Wetland Mapping