[Talk-us] (Second attempt) Potential data source: Adirondack Park Freshwater Wetlands

Kevin Kenny kkenny2 at nycap.rr.com
Sat Feb 27 05:10:04 UTC 2016


My apologies if this message turns out to be a duplicate. I mistakenly 
sent it from a mailbox that isn't subscribed to the lists.

Potential data source: Adirondack Park Freshwater Wetlands

This message is a 'trial balloon' for a potential import of (a subset
of) the data in the series of data sets: "Adirondack Park Freshwater
Wetlands," found at
http://apa.ny.gov/gis/shared/htmlpages/data.html#wetl
This data set, in addition to indicating marshes, bogs and fens, has
extensive information about open water, with detailed information
about lakes and ponds, permanent, intermittent and ephemeral streams,
falls and rapids. In the area of coverage (the Adirondack Park of New
York State), it is considerably more comprehensive and detailed than,
say, the National Hydrographic Database. My experience, from at least
a few hundred miles of hiking and mapping in the area in question, is
that it is quite close to what one finds with boots, literally, on the
ground - or, more likely, half-sunk in beaver swamp.

I'm aware of the controversial nature of imports, and I'm suggesting
this with some trepidation. Please try to be gentle when you flame me.
I'm willing to take 'absolutely not!' for an answer.

To give a rough visual impression of the extent of the coverage
difference that could be achieved, compare what is OpenStreetMap for a
typical spot today:
http://www.openstreetmap.org/#map=14/43.4894/-74.5106&layers=C
with a hiking map that I have rendered from OSM and other layers
https://kbk.is-a-geek.net/catskills/test3.html?la=43.4895&lo=-74.5104&z=14
(Note that some of the layers I use cannot be shared - or cannot be
shared yet -  under ODBL. The hydrography here, however, is all
derived from either NHD or the Adirondack Park Agency data set. The
use of the two data sets is only part of what gives certain of the
shorelines a 'cubist' look. More significant is that the APA data set
gives typical seasonal high and low water levels, plus extreme
inundation limits if known. This causes a great many watercourses to
have two rendered shorelines, plus perhaps a dashed surround showing
the flood stage. I think that the map at kbk.is-a-geek.net shows how
much this import might help avoid having the Adirondack Park simply be
a blank green area on a regional map.

Now, let me explain how far I am along with following the import
guidelines. (The answer is: not very far yet: it's important to reach
out to the community early.)

I. PREREQUISITES

1. Familiarity with the basics of OpenStreetMap: I've been mapping off
and on for several years, with my chief interest being sharing data
about hiking trails, parks, nature preserves, and associated
amenities. I believe that I at least know my way around. My biggest
single project was getting the complete centerline of the
Northville-Placid Trail entered and organized into a route relation
http://www.openstreetmap.org/relation/4286650  . This task involved
conflating several other attempts to enter (or import, badly, from
TIGER) portions of the trail, together with walking all 138 miles (222
km) with a GPS unit to get the tracks that were missing.  I've also
done the boundaries, trail systems and watercourses of a number of my
local nature preserves and forest tracks, such as
http://www.openstreetmap.org/way/190452078,
http://www.openstreetmap.org/way/339443446,
http://www.openstreetmap.org/way/345643852,
http://www.openstreetmap.org/way/270499380,
http://www.openstreetmap.org/relation/4836454  and
http://www.openstreetmap.org/way/147772635  . I've learnt in the course
of this some of the basics of conflating data and of managing things
like multipolygon and multilinestring relations. The map at
kbk.is-a-geek.net is also largely my doing. I started from Lars
Ahlzen's 'TopOSM,' but I've taken it in a very different direction and
added a couple of dozen layers that he didn't need to work with for
his project. I think that it demonstrates at least basic competency in
Osmosis, Mapnik and PostGIS.

2. Being aware of what can go wrong with imports: I'm acutely aware of
it! I deal with TIGER's hallucinations all the time and I intend to go
to any length to avoid a mess like that.

3. Identiy data I'd like to import: The eighteen files enumerated in
the table athttp://apa.ny.gov/gis/shared/htmlpages/data.html#wetl

II. COMMUNITY BUY-IN

1. Contact the community.  Hi, there!
2. Discuss plan on imports, talk-us, and the appropriate regional
mailing list: Hi there, again!
3. Be prepared to answer questions: I will surely give it a go! I
don't have all the answers.
4. Review with the assistance of more technically-oriented and
experienced OSM volunteers. I hope this message will put me in touch
with a few. I'm fairly technically oriented myself (I'm a PhD in
computer science), but I surely don't have tremendous depth of
experience with the specifics of OSM's data management.
5. Not import the data without local buy-in. This goes without saying.

III. LICENSE APPROVAL

I've taken only a few preliminary steps here, but I anticipate little
trouble. The file metadata already indicate that the license is
permissive:

Access_Constraints:       None
Use_Constraints:         These data may not be used for legal
determinations. Please credit use of this data set to the New York
State Adirondack Park Agency, Ray Brook, New York 12977. Please send a
copy of any reports or papers in which these data were used or
referenced to the above address, Attention: Susan VanWormer,
Librarian.

The first part is merely a statement to dispel confusion, since the
agency that releases this data set is also the executive agency that
makes the initial legal determination of what is and is not a wetland,
is or is not within a certain number of feet of a watercourse, and so
on. OSM surely does not represent itself as an authoritative data
source for making legal determinations regarding those matters, and
importing these data surely will make it no more authoritative. My
understanding in an informal discussion of the 'please credit use of
the data' and 'please send a copy of any reports or papers' is that
they are polite requests, rather than inflexible terms and conditions.
I shall, of course, obtain whatever written confirmation is required
from the data custodians before proceeding. I'm aware of the sample
letters athttp://wiki.openstreetmap.org/wiki/Import/GettingPermission
- does the LWG require signed hard copies? Are there further
formalities required (such as witnessing of signatures)? Is there
someone official that this should be bumped up to, rather than having
Kevin Random Volunteer do it?

IV. DOCUMENTATION

I don't want to put in a lot of work on doing the formal registration
and plan until I have agreement in principle here on the mailing
lists, and the necessary legalities taken care of. I already
understand the data quite well and (obviously) have Mapnik scripts
that can deal with it. I do want input on appropriate tagging. This
data set is much more detailed than anything that I've seen so far in
OSM - certainly, more detailed than NHD - and it's not immediately
clear to me what to call everything. I'll be glad to discuss my early
ideas about the technical aspects separately as we move forward -
assuming that the whole plan doesn't get nipped in the bud.

I expect that handling conflation should be fairly straightforward,
since there are few hydrographic features in the region of coverage.
Most of the features that are there, in turn, were created in an
earlier import of lake and pond outlines done mostly by Russ Nelson.
In most cases that I've checked (I haven't yet automated the process),
a simple collision check in PostGIS identifies the lakes and ponds. My
inclination at this point is that the default conflict resolution
would be to accept the tagging in OSM above that in the import, but to
give the geometry of the import priority, since it was digitized
originally from a smaller-scale map.

There will also be a step required that I haven't done yet of
combining features that cross USGS 7.5 minute quadrangle boundaries.
https://kbk.is-a-geek.net/catskills/test3.html?la=43.5004&lo=-74.3749&z=14
demonstrates the issue that I'm talking about: the polygons are all
splot at latitude 43°30' N and longitude 74°22'30" W, because those
lines were the boundaries of map sheets in the original digitization
effort. I'm confident that I'm enough of a GIS programmer to be able
to identify these straight-line boundaries - which all will follow 7.5
minute lines of latitude and longitude, and coalesce like features on
either side.

I also need to review satellite orthophotos and redigitize the area
surrounding Duck Hole. This was a substantial lake that was dewatered
when the dam that retained it failed in Hurricane Irene. (The wreckage
is clearly evident inhttps://goo.gl/maps/2QTTa4WWZJy) Most maps,
including NHD, Google Maps, and the USA TOPO series, still show it.
OSM has it deleted, but the watercourses now are disconnected. We can
surely do better, and flag it to avoid having someone else import or
edit over top of it.

Of course, moving forward, I can present samples of the data to be
imported in OSM XML. I'm not yet technically that far along.

ANTICIPATION OF OBJECTIONS

It is somewhat debatable whether it's appropriate to include
hydrography in OSM at all. In producing my own maps, I typically do
not render any hydrographic features except for coastlines from OSM,
preferring instead to use NHD, the USFWS National Wetlands Inventory,
the NYSDEC wetlands inventory, and the APA freshwater wetlands data
set. The reason is simply that all of these data sets provide
redundant data, and OSM's coverage and data quality, at least in my
area, are among the poorest of what I have available. Given the
difficulties inherent in getting changes made by local mappers working
independently (the data are a bit difficult to verify in the field),
it's arguable that we should always use third-party sources to make
our maps and have it be Someone Else's Problem. That said, we
unquestionably do have hydrography in OSM, and it doesn't in fact
require a lot of updating - these natural features are quite stable,
particularly in a remote area such as I'm considering here. I feel
that it would be more appropriate to discuss this problem as it
relates to the map as a whole, and not to burden a single import with
proposing a resolution.

I can also foresee someone raising an objection that the area in
question is too small to deserve special treatment in this sort of
import. To this I reply that the Adirondack Park is no ordinary state
park. It encompasses a land area larger than the Commonwealth of
Massachusetts, and would fit any five of the US National Parks within
its borders. It is a private and public partnership totalling 5.6
million acres (8700 square miles, 22600 km²) - a land area larger than
the Commonwealth of Massachusetts. Any five of the US National Parks
would fit within its borders. Roughly 43% of the land is owned by the
State of New York, and well over half the rest is subject to
conservation easements that severely restrict development.

I also anticipate that an objection will be raised that I don't yet
provide enough technical detail. Rest assured that I won't proceed
until the technical details are worked out and a plan approved.

Right now, I'm doing this solo, since the project is little more than
a gleam in my eye. If anyone objects to the fact that I'm a single
point of failure, I'd welcome the help! This is really a bigger
project than I ought to be taking on, but if nobody ever took on a
too-big project, what would ever get done?

Thanks for reading this too-long message!


-- 
73 de ke9tv/2, Kevin




More information about the Talk-us mailing list