[Talk-us] [Talk-us-newyork] Interested in importing address points in New York State

[Skyler Hawthorne]

Well, it turned it to be a lot easier than I was thinking it would be! I 
reached out to the contact listed on the Clearing House web site, using the 
template in the wiki page, and he replied confirming that we have 
permission to use the data. This is the text of the email exchange, and 
I've also attached the raw .eml file.

From: Winters, Frank (ITS) Frank.Winters at its.ny.gov
Date: July 17, 2020 22:30:24
Subject: RE: Interested in importing the Address Point data from the 
Clearing House into OpenStreetMap
To: Skyler Hawthorne osm at dead10ck.com
CC: Coryell, Rodger (ITS) Rodger.Coryell at its.ny.gov, Fargione, Craig (ITS) 
Craig.Fargione at its.ny.gov

Hi Skyler, nice to hear form you. We would very much like the SAM address 
points to be included in Open Street Map. The permitted use of the points 
is quite simple. You may use the points for any lawful purpose. While we do 
our best to maintain a comprehensive and accurate set of address points 
with our limited resources we know it has shortcomings. See the metadata 
for the liability disclaimer.

We generally post quarterly updates to the data set.


Frank Winters
Geographic Information Officer

Office of Information Technology Services
W. Averell Harriman State Office Campus
Bldg. 5 - Floor 1
Albany, NY 12226
518.242.5036 | 518.281.9140 m | frank.winters at its.ny.gov


-----Original Message-----
From: Skyler Hawthorne <osm at dead10ck.com> Sent: Friday, July 17, 2020 6:30 PM
To: Winters, Frank (ITS) <Frank.Winters at its.ny.gov>
Subject: Interested in importing the Address Point data from the Clearing 
House into OpenStreetMap

ATTENTION: This email came from an external source. Do not open attachments 
or click on links from unknown senders or unexpected emails.

Hello Mr Winters,

Thank you for your part in making the GIS data for New York State available 
to the public through the Clearing House project!

I am a contributor to the OpenStreetMap project [1], a collaborative open 
project to create a global geodata set freely usable by anyone [2].

We respect the IP rights of others and I write to ask if we can use this 
data. There does not appear to be any explicit information about the 
license under which the data sets in the Cleaning House web site are 
distributed. It's unclear what the terms are for its use, and specifically 
whether or not it is public domain, and if it is permitted to import into 
the OpenStreetMap project and redistribute to the world under an open license.

At the most simple, I would seek a statement like this:

"The New York State GIS Program Office [or the relevant NYS department(s)] 
has no objections to geodata derived in part from the GIS Clearing House 
data sets being incorporated into the OpenStreetMap project geodata 
database and released under a free and open license" [1]

I also ask that whatever statement you are prepared to make can be made 
public for information purposes.

Below is a fact sheet. If you would like any more information, I will do my 
best to help or can ask our project's License Working Group to get in touch 
with you.

Regards,
Skyler Hawthorne

Fact Sheet

[1] The OpenStreetMap project currently has over 750,000 registered 
contributors worldwide. Our main website is https://www.openstreetmap.org

[2] We are mandated to make our geodata available in perpetuity under a 
free and open licence. We are not allowed to use a commercial license, but 
commercial organisations are allowed to use our data under similar terms.

[3] Our data is currently published under the Open Database License 1.0, 
https://protect2.fireeye.com/v1/url?k=76761582-2a4eb33f-7674ecb7-000babd9fa3f-f71edf933744da0d&q=1&e=391ef603-5912-439f-b6e4-b8ac749598bd&u=http%3A%2F%2Fwww.opendatacommons.org%2Flicenses%2Fodbl%2F

[4] Most of our geodata is contributed by individuals. However, we are very 
grateful when able to incorporate or derive from other geo-data datasets 
where license terms are compatible.

[5] We formally attribute all such sources at 
https://wiki.openstreetmap.org/wiki/Attribution, using any specific wording 
if you request. We also try to provide a link to this page with any extract 
of data from our database. However, for reasons of practicality, we do not 
require end-users to repeat such attribution since it runs into hundreds.

[6] We also keep a public track of third party data use at 
https://wiki.openstreetmap.org/wiki/Import/Catalogue and usually have a 
project page for each dataset, describing how we use it and whether there 
are any license restrictions to be aware of.

[7] If you have any specific legal questions, the OpenStreetMap 
Foundation's License Working Group can be reached at 
legal at osmfoundation.org and will be glad to help.

On July 16, 2020 12:16:19 Kevin Kenny <kevin.b.kenny at gmail.com> wrote:

> (By the way, hi, Skyler, and welcome!  You've stepped into a difficult
> area - most programmers don't realize just how difficult until they've
> waded in.
>
> The legal situation in New York is _very_ complicated, because the key
> court case that governs GIS data settled out of court before the
> underlying issues were resolved. You simply will not ever get a clear
> statement about the legalities, because, alas, there was a court case
> that left a door open a tiny crack - and then never reached a judgment
> because the parties settled on undisclosed terms.  The only people who
> could actually cut through the weeds at this point would be the Court
> of Appeals for the Second Circuit.  Copyright litigation is ruinously
> expensive, and neither the agencies who would have a purported
> interest in the data nor any user has the means to pursue a copyright
> case that far.
>
> I've been asked this question enough that it deserves a more
> comprehensive answer.  I probably will write at greater length over
> the weekend and put the results in an OSM diary entry and blog post,
> and come back here to link to them.
>
> My personal belief, with which OSM's legal team may or may not agree
> (with that said, there's never any certainty in the law!) is that New
> York's open government law renders the use of the data pretty much
> fair game from the legal perspective.
>
> This data set has a  complex history.  It was produced because there
> were grants offered to prepare a geocoding system for E911 use. The
> address points used to support this effort, for the most part, did not
> originate in the state's GIS office, but instead were provided by the
> counties. The role of NYSGIS was to coordinate the efffort, to pass
> grant money through to local governments, and to normalize and adapt
> the data, in some cases to anonymize, and to conform the data to
> national standards (https://nena.org/). When I've referred on the list
> (or in changeset comments, etc.) to county E911 data in New York, I've
> ordinarily been talking about exactly this data set.
>
> I use it routinely, but only as a source of address data when I'm
> tracing buildings; in my town. It's usable for that when taken with a
> grain of salt and backed up with field survey for items that look
> questionable (e.g., the addresses of all buildings in a subdivision or
> apartment complex sharing a single point).  I do field surveys only
> afoot, so I'm somewhat limited in how far from home I've been ready to
> map, but you can see the results in the general area of
> https://www.openstreetmap.org/#map=16/42.8213/-73.8833. I'm confident
> that my use steers clear of copyright issues in any case. I'm making
> my own selection of the data with significant revisions, so the
> "selection, sequence and arrangement" of the data are not even similar
> to what is in the data set.  Essentially, all that I extract are bald,
> individual facts. To assert that such extraction is a violation of
> copyright is to advance a 'mental contamination' theory, and would
> disqualify most mappers from ever mapping anything, since it would
> effectively mean that if you'd ever seen anything in a commercial data
> set, you couldn't map that object even if you verified it
> independently. (There are copyright maximalists who have attempted to
> advance just that sort of claim.  The US pretty much shot them down in
> Feist Publications, Inc., v. Rural Telephone Service Co., 499 U.S. 340
> (1991) 
> <URL:https://en.wikipedia.org/wiki/Feist_Publications,_Inc.,_v._Rural_Telephone_Service_Co.>
> The solution in the UK and Australia is murkier, and does verge on the
> "mental contamination" theory.) Moreover, the purpose of the database
> is to enable both governments and NGO's to have access to the
> geocoding for emergency response. Asserting copyright against OSM for
> use of E911 data would almost certainly be held to go against sound
> public policy.
>
> I'm less sanguine than Skyler is about the data quality.  I suspect
> s/he (the given name doesn't clearly identify a preferred pronoun) has
> been looking at urban or suburban areas in counties whose GIS
> departments have relatively stable funding. In those situations, yes,
> the data are fairly good.  There is still a serious conflation issue
> that isn't addressed, with respect to buildings whose footprints are
> already mapped but do not bear addresses, where the address point may
> or may not be in the building footprint.  Many address points, too,
> get clustered at the entrance of a private or shared driveway, rather
> than being on the indivdual dwellings. I seem to recall that at least
> one or two of the apartment and townhouse complexes in the general
> area of https://www.openstreetmap.org/#map=18/42.83211/-73.89931 had
> to have their house numbers collected on foot, because the E911 data
> showed all the address points in a single cluster.
>
> In the rural areas, particularly in the counties with tiny
> populations, the situation is grimmer. I'm not certain that Schuyler
> or Wyoming Counties even would _have_ dedicated GIS departments!
> Until relatively recently, when grant money was available to have this
> information in GIS systems for E911 use, they mostly were still using
> paper maps, often referenced to an unknown datum.  (The first job in
> dealing with any scanned tax plat is figuring out what coordinate
> frame it's using - around here, NAD27 differs from NAD83 by a few tens
> of metres.) The address points may be parcel centroids, or building
> centroids, or the point where the driveway meets the road, or even
> just something that was digitized from a pencil sketch made by an
> assessor.  Import of this sort of data could well prove to be a
> short-term gain but impose a heavy long-term burden; consider the
> love-hate relationship that we all have with TIGER. (The import means
> that we've got a nearly-filled-in map, a lot of which is of
> halfway-decent quality, and we don't have the mappers to have done it
> nearly as quickly any other way. Nevertheless, for some years we've
> been paying the price in bad data and worse conflation.)
>
> In particular, when I look at the Ulster County data, I find that the
> address points are aligned to the building footprints in the county's
> database. Wiping my fevered brow, I recall that we had an import of
> Ulster County building footprints, and that it did not conform very
> well at all to reality in the field.
> https://worstofosm.tumblr.com/post/148016716836/in-ulster-county-ny-people-live-in-small-square
> Eventually, the entire import was reverted; there was some
> consternation, since there were various isolated spots where mappers
> had corrected the data, and the initial attempt at reversion did not
> go cleanly.  (It got fixed.)  Mateusz, do I recall that you took part
> in the reversion?
>
> While, as Dave suggests, I've been around the block a few times in New
> York, I've never attempted an import on the scale of the address
> points, and do not plan to unless there's a large team to pursue
> conflation and correction.  The largest imports of state data I've
> done have been public land boundaries - and each of those took a
> couple of months of evening work to curate. Even there, the quality of
> the job doesn't really satisfy me, but it was enough at least to
> convince me that doing nothing would have been worse. (I then wind up
> having at least a week of evenings annually dedicated to sporadically
> _updating_ the import, and any big import really needs to come with an
> update plan!)
>
> So, my advice for both legal and technical reasons would be to use
> caution, and recognize that mechanical import is likely to be a
> disaster - the data will need to be eyeballed by human beings and
> corrected. From the legal standpoint, it would be best to proceed only
> with those counties that have granted fairly broad authority to use
> their cadastral data. Those include the five boroughs of New York City
> (that is, Bronx, Kings, New York, RIchmond and Queens Counties), and
> the counties of Cayuga, Chautauqua, Cortland, Erie, Genesee, Greene,
> Lewis, Ontario, Orange, Rensselaer, Sullivan, Tioga, Tompkins, Ulster,
> Warren and Westchester.  In New York City, the job is  essentially
> done, because there have been massive (and relatively well curated)
> imports of the public data from the city's GIS department.  I'd
> recommend avoiding the Long Island counties of Nassau and Suffolk,
> because they've been litigious in the past about their data.
>
> In any case, it's a huge job, and probably needs to be done a block, a
> census tract, or at the very most a township at a time. Even to do an
> entire county is a relatively daunting task unless you can find the
> staff for it. (I'm willing to do my part, but it remains a huge job.)
>
> I've spent far too much time getting this far, but I do want to write
> a piece that summarizes how we got where we are with respect to NYS
> GIS data. The technology press got rather hysterical about the one
> court decision at the time, which in fact said much less than the
> press read into it.  I'll write that one up separately, as I said.
>
> On Thu, Jul 16, 2020 at 10:28 AM Adam Franco <adamfranco at gmail.com> wrote:
>>
>> Skyler, I was able to reach out to Vermont's State GIS office and received 
>> an explicit usage grant that is now recorded in the OSM wiki. While former 
>> similar efforts in NY may have failed in years past, a new request might 
>> succeed in getting a more blanket approval that could cover this and other 
>> OSM projects. In case it help you or others as a starting point, here is 
>> the message I used for Vermont:
>>
>>> ---------- Forwarded message ---------
>>> From: Adam Franco <adamfranco at gmail.com>
>>> Date: Wed, Sep 20, 2017 at 4:59 PM
>>> Subject: Vermont/VCGI data usage in OpenStreetMap
>>> To: <leslie.pelch at vermont.gov>
>>>
>>> Dear Leslie,
>>>
>>> I contacted you about this topic several years ago (2013!) but then shifted 
>>> my focus to other projects and never followed up again.
>>>
>>> I am a contributor to the OpenStreetMap project [1], a collaborative open 
>>> project to create a global geodata set freely usable by anyone [2].
>>>
>>> We respect the IP rights of others and I write to ask if we can use data 
>>> published by VGCI, in particular [but not necessarily limited to] the 
>>> following data-sets:
>>>
>>> VTrans::vt-road-centerline
>>> vt-boundaries-all-lines
>>> VT Hydrography Dataset
>>> VT E911 Site Locations
>>>
>>> I've read through the VCGI Warranty/Copyright document and while it seems 
>>> to my lay reading that importing data from these data-sets into the 
>>> OpenStreetMap would constitute a "value-add" and therefore allow such use, 
>>> it would be helpful to the community to have a clear statement allowing our 
>>> use of the data.
>>>
>>> At the most simple, I would seek a statement like this:
>>>
>>> "The State of Vermont and its Vermont Center for Geographic Information has 
>>> no objections to geodata derived in part from the 
>>> VTrans::vt-road-centerline, vt-boundaries-all-lines, VT Hydrography 
>>> Dataset, and VT E911 Site Locations data sets being incorporated into the 
>>> OpenStreetMap project geodata database and released under a free and open 
>>> license" [1]
>>>
>>> A broader, less itemized grant might be even better, but I'm not sure if 
>>> your agency might have concerns about over-broad usage grants. Maybe 
>>> something like this:
>>>
>>> "The State of Vermont and its Vermont Center for Geographic Information has 
>>> no objections to geodata derived in part from data-sets published by the 
>>> VCGI being incorporated into the OpenStreetMap project geodata database and 
>>> released under a free and open license"
>>>
>>> I also ask that whatever statement you are prepared to make can be made 
>>> public for information purposes, posting it to the project's data-source 
>>> documentation for others to reference.
>>>
>>> Below is a fact sheet. If you would like any more information, I will do my 
>>> best to help or can ask our project's License Working Group to get in touch 
>>> with you.
>>>
>>> Regards, Adam Franco
>>>
>>>
>>> Fact Sheet
>>>
>>> [1] The OpenStreetMap project currently has over 750,000 registered 
>>> contributors worldwide. Our main website is http://www.openstreetmap.org
>>>
>>> [2] We are mandated to make our geodata available in perpetuity under a 
>>> free and open licence. We are not allowed to use a commercial license, but 
>>> commercial organisations are allowed to use our data under similar terms.
>>>
>>> [3] Our data is currently published under the Open Database License 1.0, 
>>> http://www.opendatacommons.org/licenses/odbl/
>>>
>>> [4] Most of our geodata is contributed by individuals. However, we are very 
>>> grateful when able to incorporate or derive from other geo-data datasets 
>>> where license terms are compatible.
>>>
>>> [5] We formally attribute all such sources at 
>>> http://wiki.openstreetmap.org/wiki/Attribution, using any specific wording 
>>> if you request. We also try to provide a link to this page with any extract 
>>> of data from our database. However, for reasons of practicality, we do not 
>>> require end-users to repeat such attribution since it runs into hundreds.
>>>
>>> [6] We also keep a public track of third party data use at 
>>> http://wiki.openstreetmap.org/wiki/Import/Catalogue and usually have a 
>>> project page for each dataset, describing how we use it and whether there 
>>> are any license restrictions to be aware of.
>>>
>>> [7] If you have any specific legal questions, the OpenStreetMap 
>>> Foundation's License Working Group can be reached at 
>>> legal at osmfoundation.org and will be glad to help.
>>
>>
>> And this is what I got back:
>>>
>>> ---------- Forwarded message ---------
>>> From: Adams, John E. <John.E.Adams at vermont.gov>
>>> Date: Thu, Sep 21, 2017 at 10:34 AM
>>> Subject: RE: Vermont/VCGI data usage in OpenStreetMap
>>> To: adamfranco at gmail.com <adamfranco at gmail.com>
>>> Cc: Pelch, Leslie <Leslie.Pelch at vermont.gov>, Brown, Ivan 
>>> <Ivan.Brown at vermont.gov>
>>>
>>>
>>> Hi Adam,
>>>
>>> The State of Vermont Center for Geographic Information has no objections to 
>>> geodata derived in part from data-sets published by the Vermont Center for 
>>> Geographic Information in the Vermont Open Geodata Portal being 
>>> incorporated into the OpenStreetMap project geodata database and released 
>>> under a free and open license.
>>>
>>> Regards,
>>>
>>> John
>>>
>>>
>>>
>>> John E. Adams AICP
>>>
>>> Director | Vermont Center for Geographic Information
>>>
>>> State of Vermont
>>> 1 National Life Dr, Deane C. Davis Bldg, 6th Floor | Montpelier, VT 05620-0501
>>> 802-522-0172
>>> vcgi.vermont.gov
>>
>> _______________________________________________
>> Talk-us mailing list
>> Talk-us at openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/talk-us
>
>
>
> --
> 73 de ke9tv/2, Kevin
>
> _______________________________________________
> Talk-us-newyork mailing list
> Talk-us-newyork at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-us-newyork

--
Skyler
-------------- next part --------------
An embedded message was scrubbed...
From: unknown sender
Subject: no subject
Date: no date
Size: 24074
URL: <http://lists.openstreetmap.org/pipermail/talk-us/attachments/20200717/42ed3d50/attachment-0001.eml>