[Imports] Address import from city government source

Thu Aug 4 04:21:06 UTC 2022

I beg your pardon, I see that a CC-BY-SA license is NOT sufficient. So please disregard that part; I will seek explicit approval from the county.

From: Joe Rhodes
Sent: Wednesday, August 3, 2022 10:17 PM
To: Mike Thompson <miketho16 at gmail.com>; Greg Troxel <gdt at lexort.com>; Clifford Snow <clifford at snowandsnow.us>
Cc: imports at openstreetmap.org
Subject: RE: [Imports] Address import from city government source

Thanks Mike, Greg, and Clifford. I’m working my way through all of the information in Greg’s email, but I can attempt to start to address the licensing question and a couple of others. I am indeed new to OSM (except for a HOT mapathon several years ago), so I greatly appreciate the generous information and guidance.

The motive for this task is a practical one: Castle Pines is expected to more than double in population over the next several years, and we wish to establish processes to keep addresses up to date for the benefit of our residents (ensuring package delivery, navigation in consumer GIS apps, etc.). Once we get addresses sorted, we wish to evaluate the appropriateness and feasibility of updating other OSM features in Castle Pines, including parks, land use/land cover, and other features.

In my estimation, using address points straight from Douglas County will probably be the best path. Their data is marked with a Creative Commons/CC-BY-SA 4.0 license, but it sounds like it’s preferable or necessary to get explicit permission from the county, which I can start on. (Regarding Mike’s comment on disclaimer vs. license on the city’s open data site, he is correct - that’s a quirk of ArcGIS Open Data; it feeds the hosted item’s “Terms of Use” to the license field. I’m no legal expert either, but I don’t think ‘terms of use’ == ‘license’.)

Regarding the boundary: If acceptable, we can also take this directly from the county (layer: municipality<https://gis-dougco.opendata.arcgis.com/datasets/municipality/explore?location=39.458592%2C-104.866197%2C13.30>), so there’s no need to dissolve parcels. To Greg’s comments: The half that is present is correct, but nothing west of I-25 shows as part of the Castle Pines administrative boundary. I don’t yet know enough about OSM to say whether there may be a difference between what’s rendered vs the database, but it certainly appears that half is missing. So I don’t wish to correct any existing geometry, or posit the accuracy of any dataset’s features over any other, simply to get the western half of the city’s boundary added.

Further to the above, and perhaps because of the above, all of the addresses I’ve spot-checked so far west of I-25 show as being in unincorporated Douglas County. So once a conflation process is sorted, I’ll also want to add the correct town name to the many address points that do exist yet lack the correct town designation.

Lastly, to the question of conflation: It seems to me that scripting a comparison of addresses proposed to be imported against the existing database would not be an extraordinarily heavy lift. We’re fortunate that our street names are relatively unique (no 1st, Main, Lincoln, etc.), so I’m hopeful that I won’t find many matches outside the immediate area.

Thanks again for the insight and guidance. To my understanding, the next steps are:

  1.  Get explicit written permission from Douglas County. If the Creative Commons license is sufficient, I’ll skip this, but I certainly don’t mind reaching out to the county if necessary or preferable.
  2.  Reach out to local mappers for input
  3.  Familiarize myself with OSM tools and retrieve an OSM address list for comparison
  4.  Document a conflation process and write a script to compare county addresses to the existing OSM list, then manually review matches outside the immediate area, if any

Let me know if I’m missing or misunderstanding anything. City of Castle Pines is a member of DRCOG, so let me know how that might impact the process.

Thanks again,

Joe Rhodes

From: Mike Thompson <miketho16 at gmail.com<mailto:miketho16 at gmail.com>>
Sent: Wednesday, August 3, 2022 6:05 PM
To: Greg Troxel <gdt at lexort.com<mailto:gdt at lexort.com>>
Cc: Joe Rhodes <joe.rhodes at castlepinesco.gov<mailto:joe.rhodes at castlepinesco.gov>>; imports at openstreetmap.org<mailto:imports at openstreetmap.org>
Subject: Re: [Imports] Address import from city government source

Hello again Joe,

I went to your city's website and found the address data I believe you are interested in having in OSM[0].  Probably the first thing is to get a clear official statement as to the license,  Currently the data has this "license":
GIS information is not adequate for legal boundary definition. The information depicted on GIS maps is for general planning purposes only and should not be used for legal boundary definition, regulatory interpretation, or property conveyance purposes.

Data and information is provided by the GIS system with the understanding that it is not guaranteed to be correct or complete. All data is subject to change with periodic updates. The City of Castle Pines makes no claims, representations or warranties, express or implied, concerning the validity, reliability or the accuracy of the GIS data and GIS data products furnished by the City, specifically including the implied or expressed validity of any uses of such data.

I am not a legal expert, but this seems to be more of a disclaimer rather than a license.  This page on the OSM wiki shows which common licenses are compatible with ODBL[1].  Also, it seems the data was taken from Douglas County, and if they are the true data owner, then we probably should engage them in this discussion.   Alternatively, a letter signed by an authorized person in the city (or county if they are the data owner) explicitly giving permission for the data to be imported into OSM could work.  Suggested templates are here[2]. Is your city a member of DRCOG?  DRCOG has worked with OSM before.

I was unable to find a dataset on your website for your city's boundary, although I could probably just dissolve all of the parcel data.  Once that data is available with an appropriate license making the necessary boundary edits in OSM should be relatively easy.  OSM uses "relations" for boundaries such that the boundary between two cities (e.g.) is only represented once.  This prevents any "no man's land" in the data, but it is a concept that may be foreign to someone coming from a GIS/OGC simple features background.

Mike

[0] https://cp-public-info-castlepines.hub.arcgis.com/datasets/castlepines::address-points-with-use-code/explore?location=39.471247%2C-104.883243%2C14.89
[1] https://wiki.openstreetmap.org/wiki/Import/ODbL_Compatibility
[2] https://wiki.openstreetmap.org/wiki/Import/Getting_permission

On Tue, Aug 2, 2022 at 5:14 PM Mike Thompson <miketho16 at gmail.com<mailto:miketho16 at gmail.com>> wrote:

On Tue, Aug 2, 2022 at 5:02 PM Greg Troxel <gdt at lexort.com<mailto:gdt at lexort.com>> wrote:

Joe Rhodes <joe.rhodes at castlepinesco.gov<mailto:joe.rhodes at castlepinesco.gov>> writes:

> I'm the GIS manager for the newest municipality in the State of
> Colorado, City of Castle Pines.

Welcome to OSM.
Yes, welcome to OSM!

I am in Colorado and an experienced mapper.  I can help, but you will need to deal with the license issue, and some of the other things Greg mentions (good advice from Greg all around, which I second).

Mike

I don't know if you have experience editing OSM (not doing an import,
just normal mapper activity).  Generally, I and I think most others see
that as a prereq.

Have you read:

  https://wiki.openstreetmap.org/wiki/Import/Guidelines

You'll have to prepare documentation.  One of the harder parts will be
conflation with existing data.

The easy part, to get one of the showstoppers out of the way:

  Please provide a link to download the data (no login, that anybody can
  do).

  Please provide a link to a public statement of the license for the
  data, so that we can see if it meets the OSM requirements.

  Are you in touch with the mappers that have edited in your area?  Have
  you discussed this with them?  What do they think?  If not, please
  find them.

> Many addresses in our city are missing
> from OSM, and the ones that are there do not show Castle Pines as the
> city (they appear to just be in unincorporated Douglas County). I am
> seeking approval to do an import of the full set of address points for
> our city in order to correct these problems.

"import the full set" and "many are missing" do not go together.  You'll
need a conflation process -- which I think should be one with published,
open-source code, so others in the community can replicate it -- that
looks at your address data and matches it with OSM data, more or less
sorting it into "address match, close geographically", "address match,
far off", and "address not in OSM".  The middle category needs manual
review (that doesn't solidly presume either db is right) and the third
is what ends up proceeding in import.  Then there's the question of what
object it is attached too, or just a bare point with address tags.

> In addition, the city boundary is incorrect in OSM. The city has a
> multipart boundary (there are two non-contiguous sections on each side
> of an interstate highway), and only the eastern portion shows on
> OSM. Because it is non-contiguous/multipart, I am unable to edit it to
> reflect the correct boundary, so I'm seeking approval to import the
> correct boundary as well.

I don't understand how you can't edit it.  Have you asked the local
mappers for help?

It sounds like it should be a multipolygon.  You say "shows on OSM" but
we should be talking about the objects in the database, not the output
of any render.  Let's assume there is a single closed way which is one
of the parts and it has the admin/name tags.  The first thing to do is
look at the history (^H in JOSM, and if you haven't learned JOSM I would
recommend that before importing, because if you import anything you have
bit off the repsonsibility to fix any problems) and contact the people
that edited it before and talk about the situation.

If what you want to do is take a polygon for the missing half -- after
verifying it really is missing -- and convert that to osm and turn the
existing closed way and the new one into a multipolygon, then that's not
that big a deal, but given that you seem to be new I think the steps of
community consultation, public data, license, transform mechanism should
be written down.

You didn't talk about the half that is there being wrong, so I'm
assuming you don't wnat to change that.  In general "my data is better,
and so I want to delete and add mine" is not ok; OSM has a strong
doctrine of deference to hand mapping, even if any particular thing that
is actually wrong can be fixed.  Sometimes OSM people have good data
about boundaries and it can be better than the official data.  As an
example I've measured my town's boundary stones with RTK, and the
official data is an 1890 survey in the New England datum reckoned
forward to NAD27 and then NAD83.  The amazing thing is that we agree at
the 0.5m level, and I may this year get to the point where I can
reasonably confidently say my values are better.  But most OSM data is
not like this.

Finally, and this is not a big deal, but I'm guessing your data is in
NAD83(2011) epoch 2010.0 and OSM is nominally in WGS84.  That's fuzzy,
but you should transform to the latest realization, WGS84(G2159), or
something like ITRF2014 as a close proxy.

Greg (osm user gdt)
_______________________________________________
Imports mailing list
Imports at openstreetmap.org<mailto:Imports at openstreetmap.org>
https://lists.openstreetmap.org/listinfo/imports
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/imports/attachments/20220804/dcc93087/attachment-0001.htm>