[Imports] [Imports-us] Address Data Import for Fulton County, Georgia

Wed Jan 8 03:44:25 UTC 2014

Regarding the public domain licensing, the PDF file provided with the
data states "Freely use, copy and distribute as long as credit is
given". On the metadata information on the download page, it states that
there are no access constraints, but the use constraint just states that
there may be errors in the data and Fulton County is not responsible for
such errors (exact wording is "DATA Disclaimer: Fulton County makes
known to user, and user acknowledges notice by Fulton County that this
Data contains known errors and inconsistencies. Fulton County in no way
ensures, represents or warrants the accuracy and/or reliabilibity for
any purpose.").

My custom application source code is up at
https://github.com/saiarcot895/osm-fulton-address, which is linked to on
the wiki page. Currently, the application takes in a BBox and uses the
Overpass API to download address info and streets, but I'm considering
just getting the full extract for that area. In addition, the
application takes into account all existing /valid/ addresses (has an
addr:street and the addr:street matches the street name) and skips
importing those addresses, expands abbreviations in the street name, and
corrects the casing to match those in OSM. At the moment, the position
of the address data point isn't checked for reasonableness, but I'm
planning on including that in the application itself.

I wasn't aware that ogr2osm could filter the tags at that stage itself.
In this case, I'll create a translations file to filter the tags and
create a new converted OSM file.

Regarding updates, I initially didn't plan on having this be updated
with newer versions of the data, since at that time, I wasn't aware that
there was a usable ID. Now, if there is an ID tag in the final OSM data,
address updates /may/ be possible depending on whether a new ID is
created for any address change (Carl?). What may be easier is just to
import any new addresses that are not already in OSM.

Regarding the street name mismatches, this data is from 2005 (although I
suspect there may be some updates from 2008, since some of the dates in
the data are from 2008), and if I read correctly, the TIGER data here is
from 2006. Therefore, it is more likely that the street names in OSM are
correct. That being said, there is, I believe, at least one street in
existing OSM data that is not fully expanded. For this road, it might be
easier to fix this outside of the import.

Regarding the conflation issue, the main issue I see is address nodes
being added on top of existing buildings. For this, I could add in code
to determine if the node is inside a building and, if so, merge the
address data, although it looks like the addressmerge tool already does
this. If an existing node, way (building), or relation already has an
address that is being imported, then the address will be skipped from
the import.

For the upload process itself, I'm considering using either the Task
Manager or Google spreadsheet. I intend to have only the addition of
addresses as part of the import process and won't be fixing anything else.

Saikrishna Arcot

On 01/07/2014 09:03 PM, Jason Remillard wrote:
> Hi Saikrishna,
>
> I agree with Serge, we need to be 100% sure the license situation is
> straight. Often the data providers say the data is in the "public
> domain", but in fact, isn't.
>
> Some workflow suggestions
>  - In general, you should try to script the workflow from start to
> finish. You will be running it many, many times while debugging the
> import. The splitting, conversion to OSM, and possibly conflation to
> OSM. Many of us on the list would like to look at the code to help
> with the QA.
>  - ogr2osm allows you to specify an python file on the command line
> that can be used to translate the shape file columns to osm tags. I
> would look to get things as close as possible with it first. If
> needed, then do the rest with the QT program.
>  - You should take a look at the addressmerge,
> https://github.com/pnorman/addressmerge . Paul's code works without
> buildings, it could be a good match.
>  - Subscribe to the ny city address import on git hub, review the many
> issues they needed to work through. I would also pick a tile from the
> ny city import and run the import yourself. You don't need to do it
> the same way, but I think it would be useful to be familiar with what
> they are doing.
> - If you have code to detect differences between the address street
> name and the OSM name, you should pull them out and resolve the
> differences by hand. Rather than just skipping them. It might be an
> opportunity to improve OSM street names.
>
> Things that need to be done that are missing from your wiiki
>  - how the uploading is going to happen. If multiple people are going
> to do it, they need to be coordinated. Task manager, google spread
> sheet, wiki, etc.
>  - QA on the data (proper removal of street abbreviations, positional
> accuracy).
>  - Are you going to fix other things when uploading.
>  - You should try to contact any local mappers and ask for help.
> http://resultmaps.neis-one.org/oooc
>
> Anyway, I suggest converting the data, double check the license, and
> see if you can get some local help. When you have some 100% complete,
> ready to upload OSM files, post them onto the list again and we can
> see how things look.
>
> Thanks
> Jason.
>
>
>
>
>
>
> On Tue, Jan 7, 2014 at 4:53 PM, Saikrishna Arcot <saiarcot895 at gmail.com> wrote:
>> Hi,
>>
>> I'm planning on importing the address data for Fulton County, Georgia from
>> the Georgia GIS Clearinghouse. See the wiki page here for review. I've
>> included details about the source data, the final tags, and the conditions
>> for each address.
>>
>> I will be contacting other users who have edited many addresses recently in
>> the county regarding the import over the next few days.
>>
>> --
>> Saikrishna Arcot
>>
>>
>> _______________________________________________
>> Imports-us mailing list
>> Imports-us at openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/imports-us
>>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/imports/attachments/20140107/e1beaf0c/attachment-0001.html>