[Imports] [Imports-us] [Talk-us-newyork] NYS GIS SAM Address Point Import - Status Update

[Skyler Hawthorne]

Hey everyone, I'm pleased to announce that I've finished all counties outside of NYC! The tally of addresses now stands at: 4,370,743

For NYC, it will require a code update, so it will be a bit before I can generate the data, but once I get there, since NYC already has its own previous imports, I'll be scrutinizing the data a bit more closely to see if there are any special cases to consider with interactions with the existing NYC data. If it looks fine, then I'll go ahead and import it. If it's too messy or troublesome, then I will leave it out now and in the future.

For those who did not see this in Slack, it turns out there was an issue with the importer that happened at part of the migration to osm2pgsql. I was using the command line option named `hstore-all` to create a column with all the element's tags. It is described with this help message:

-j|--hstore-all  Add all tags to an additional hstore (key/value) column.

Only, it turns out this is a lie. It actually removes the `type` tag from relations, which had the effect of removing `type=multipolygon` from all multipolygons that were modified by the import. I fixed this with JOSM already, and a code fix will be implemented before proceeding with NYC or performing updates with new data.

After NYC is done, my next steps are to write up a section on the wiki page that provides some guidelines for cleaning up the data. There have been a few common questions about how to handle questionable data, so I think it will help to have this there in the coming months as people encounter these addresses.

After that, I'll be implementing updates on the data. I have so far been doing the entire import with data from October 2020, and there is already new data to work with for implementing updates.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/imports/attachments/20210606/41571c0e/attachment.htm>