[Talk-us] [Imports] Boston, MA, USA addr:housenumber Import

[Roman Yepishev]

Hi Greg, all.

I'd like to provide some information on the import I did not share
initially to make my intentions clear (so far I may be seen as dumping
the data into OSM and running away).

TL;DR
=====

* I believe in an iterative approach to any project.
* I want to make as many useful things as possible in the limited time
I have. I want a strict deadline to mark the end of my active
involvement and enable others to pick up without creating conflicts.
* I want to improve the routing and location information for me and for
others.
* I have never wanted to introduce buildings with fixme in them into
OSM, I will not import notes.
* I will import data, fix buildings with issues, regenerate the data,
rinse, repeat.
* I will not overwrite the existing building numbers.
* All the scripts I'm using are in public domain.

Long version
============

My background
-------------

First of all, I'm a development/operations guy with mostly web service
background, and iterative approach to anything is basically the only
way to do things there. Additionally I took part in a number of
proprietary and open-source projects where the lack of completion
deadlines forced the project to be ready when it is ready. Sometimes
never.

Motivation
----------

  As I was walking around Boston using OsmAnd, I would often fail to
locate the venues just because there were no building numbers on the
street I'm supposed to go to. Now, I can still go along the street, but
a number of times I got confused because the block no longer looked
like it used to with the redevelopment going on. The only way is to
bruteforce, expecting that the buildings will be in order. Building
numbers "14 1/2" or "38R" are hard to locate unless you've been there
already.
  As a novice driver I find lack of numbers and details also daunting,
leaving me with no choice but to use Google Maps/Here maps, since I
need to scan building numbers (sometimes located under a rock) as I am
going.

Rushing with a deadline
-----------------------

  Again, based on my involvement with various open-source projects, it
is sometimes impossible to get anything vetted unless motivation is
strong and a sense of urgency is introduced. I have to apologize for
that here, as I should have explained it better in my first message.

Importing artifacts
-------------------

  My plan has always been to:

  1. Import unique buildings.
  2. Regenerate a set of .osc, .osn files, post them to wiki for review
(if somebody wants to look at them)
  3. Upload ONLY unique buildings into OSM (no fixmes, no notes, etc.).
  4. Modify the rest of buildings so that they become unique (if
possible, if not - skip them)
  5. Add missing buildings, verify they are unique.
  6. Repeat from 2.

I will add this information to the wiki.

Iterations
----------

The import will have to go through multiple iterations. At first I will
import ONLY buildings that are already good to go and require no manual
interaction. The quality of building polygons vary, and it may take
much more time to reach 100% coverage (we may not even be able to reach
it at all due to a changing nature of the cities), but marking 90%
buildings will make locating the missing ones much easier.

As I am trying to use OsmAnd more on my phone I find that presence of a
lot of "height estimated" fixmes in Downtown Boston (you can enable OSM
map helper) makes the map unreadable. Again, the -fixmes.osc files that
you see in the wiki page are just for finding issues with existing
buildings:
 - more than one addresses for a single building.
 - address is already provided, but differs from what the city expects
it to be (e.g. 90 South St, Boston, MA - "Farnsworth House" has to be
#100 according to two city sources, but the sign says #90, so it is #90
in OSM - https://www.openstreetmap.org/way/29939006 - I am planning to
resurvey the 90-86 building above that.) I don't want to overwrite
other people's hand-entered data, but I want to be able to see where
the conflicts are.

The notes must and will not be imported. They will be periodically
regenerated so that it is possible to see how the OSM updates go
(missing buildings added, existing buildings split).

Tools
-----

All the python/SQL scripts I am using to create the datasets are
already available on bitbucket. You can perform the steps mentioned in
README and you will end up with the same results as mine. I am yet to
add the license text there, they are in a public domain.

Please note the scripts originated from two all-nighters and I am
refining them as I go.

Closing statement
-----------------

What I hope to accomplish is to make the routing work better for me,
and as a great side effect, these changes will make OSRM pick the right
buildings (or approximate locations), the search for the building in
Nominatim will yield results instead of finding everything but the
thing I searched for, and I am sure this will benefit the community as
well.

Before involvement with OSM I treated the routing directions and
address lookup services as magic. Now that I know how much work needs
to be put into it to make it usable I have deep respect for all the
mappers who work hard to improve the map for themselves and for others.
And having been directed to do a u-turn on a trunk highway (Jamaicaway,
part of MA-203) to go to a side street through a "NO THRU STREET" way I
know how important it is to provide correct information (added
restriction relation there, so it is fixed now).

One more thing. I am not a US citizen, and it is currently not clear
when I will be required to leave the US, so I decided to get as much
stuff done as possible in the limited time I have.

On Thu, 2016-03-17 at 11:27 -0400, Greg Troxel wrote:
> I should point out that I'm not opposed to this import or address
> imports in general; generally, I am supportive.  But, I think the
> doing
> an import right is vastly harder than someone who hasn't been through
> one thinks, and that it's good to iterate on approach and data
> quality,
> and not rush or have a deadline/planned completion time.
> 
> These comments are partly based on the Mass buildings import, which I
> think was hugely successful.  The data was spot checked by lots of
> people and found to be very good, it was added in a way which avoided
> changing any hand-mapped data, it's led to mapping missing
> subdivisions,
> and we've had basically zero reports of problems from it.
> _______________________________________________
> Talk-us mailing list
> Talk-us at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/talk-us
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: This is a digitally signed message part
URL: <http://lists.openstreetmap.org/pipermail/talk-us/attachments/20160317/856eadac/attachment.sig>