[Imports] [MappingDC] Re: Questions about DC import plan

Mikel Maron mikel.maron at gmail.com
Wed Aug 13 22:41:42 UTC 2014


Hi

> So I suppose the reasoning was that it was worth sacrificing having a complete DC address database in order to avoid re-importing a 
> bunch of footprints that already exist.

 
Really, the reason was to make the first import process with OCTO straight forward. We can now think towards what it would take to do merging. Would require a similar compilation of OCTO data for remaining area, and we can ask OCTO for help with that. But from then, what would be the process? I'm thinking that analysis of differences and maybe production of a tile set highlighting merge cases to focus on would be helpful.

> I think better documentation on the wiki and even github would have been in order, since much of these decisions appeared to have been 
> been made offline.

We can always document better. Anyone here could help update the wiki with key points from this thread.

> This may be what someone else was concerned about when they mentioned reservations about an import being conducted primarily by 
> a local government.  It's great that OCTO have taken the initiative, but their priorities and concerns would naturally not exactly be in 
> alignment with those of the OSM community.


That's not really the case. Certainly OCTO initiated this effort, and commendation is appropriate. But from there, several of us in the OSM community talked with OCTO about what will work, and we did discuss in several forums. But we can always do better. And in that vein, glad we're getting into these details on this thread.

That all being said, what do we do next Mapping DC?

Mikel
* Mikel Maron * +14152835207 @mikel s:mikelmaron


On Friday, August 8, 2014 10:39 AM, Brandon Knight <bknight430 at gmail.com> wrote:
 

>
>
>Thanks, David and Mikel.   After digging through the github repo, I was able to find the information I was looking for in the makefile and corresponding commit comment[1].  Apparently, the decision was made not to use the full DC buildings dataset but a partial dataset produced by OCTO.  So I suppose the reasoning was that it was worth sacrificing having a complete DC address database in order to avoid re-importing a bunch of footprints that already exist.
>
>
>I'm fairly new to community.  I think better documentation on the wiki and even github would have been in order, since much of these decisions appeared to have been been made offline.  Anyone jumping in now should be able to easily follow the details of the plan, including why choices were made, without digging through the code or commit messages.
>
>
>This may be what someone else was concerned about when they mentioned reservations about an import being conducted primarily by a local government.  It's great that OCTO have taken the initiative, but their priorities and concerns would naturally not exactly be in alignment with those of the OSM community.
>
>
>With that being said, I applaud all of the hard work that has been done to make this happen.
>
>
>-Brandon
>
>
>[1] https://github.com/osmlab/dcbuildings/commit/117b1a7636eb04d70e7876eac88eabf6f9b60fc2
>
>
>
>On Thu, Aug 7, 2014 at 9:22 AM, Mikel Maron <mikel.maron at gmail.com> wrote:
>
>> "Will this be taken care of in a separate process that I am unaware of?"   No plans for that at this time.  
>>
>>
>>But we (DC OSM community) can start thinking about it. Was decided that this after this first "easy" import pass, would look at this harder conflation import later.
>> 
>>* Mikel Maron * +14152835207 @mikel s:mikelmaron
>>
>>
>>
>>On Thursday, August 7, 2014 8:19 AM, David Jackson (OCTO) <davidy.jackson at dc.gov> wrote:
>> 
>>
>>>
>>>
>>>Brandon,
>>>
>>>
>>>To answer your solid questions.
>>>
>>>
>>>1) " Are these chunks empty because there was found to be no difference from what already existed in OSM from previous imports? "
>>>
>>>
>>>No. There are many differences between existing building / address data in OSM and what is being loaded. Examples included have buildings tagged with addresses,  geometry of data among others.
>>>
>>>
>>>2) "Why was the decision made to use the tasking manager to conduct a partial import as opposed to conflating the entire dataset, given that the new import process is adding not only building footprints but footprints+addresses?  "
>>>
>>>
>>>The goal of the project was to 'complete' or fill in (almost 100%) the buildings (with address where they exist) for the rest of DC.  This means over 50,000 buildings.   The decision was made because to do a partial import because it would be much less time intensive.
>>>
>>>
>>>"Will this be taken care of in a separate process that I am unaware of?"   No plans for that at this time.  
>>>
>>>
>>>- David
>>>
>>>
>>>
>>>
>>>David Jackson   |  GIS Analyst  |  Office of the Chief technology Officer
>>>
>>>(W) 202.724.5135  |  200 I ST SE,  5TH FL,  Washington, DC 
>>>davidy.jackson at dc.gov |   Blog:  dcaddresscoordinates.blogspot.com
>>>
>>>Telework Day: Fri
>>>
>>>
>>>
>>>On Wed, Aug 6, 2014 at 4:44 PM, Brandon Knight <bknight430 at gmail.com> wrote:
>>>
>>>Posting this to the OSM Imports and the MappingDC lists:
>>>>
>>>>I've been reviewing the work by OCTO on DC building/address imports currently being conducted via the tasking manager[1] and I've noticed a few things that raised some questions about this plan.  Excuse me if I don't understand this or if there are obvious answers to these questions.  I'm new here.
>>>>
>>>>
>>>>From my understanding, the current plan is to import all DC buildings and addresses in areas that were not imported in 2008-10.  Tags were converted, building shapes simplified, and then the tags from MAR points[2] were joined to building footprints[3] when a building was found to contain only one address point, and the point deleted.  Data was then split up into chunks by block group and converted to .osm files. Scripts found in the dcbuildings repo[4]
>>>>
>>>>
>>>>After loading some of block group chunks using the tasking manager I've found that many are completely empty (0 nodes).  The explanation I was given by OCTO was:
>>>>
>>>>
>>>>" Many of the chunks created by mapbox did not contain any data, this is due to the fact that these areas already had complete or near complete building / addresses in OSM for that area."
>>>>
>>>>
>>>>
>>>>
>>>>My questions are:
>>>>
>>>>
>>>>1) Are these chunks empty because there was found to be no difference from what already existed in OSM from previous imports? 
>>>>
>>>>
>>>>I originally thought that the entire DC dataset would be represented in these chunks. If chunks containing only "diffs" was the plan, I couldn't find a clear description of this or representation in the code.
>>>>
>>>>
>>>>2) Why was the decision made to use the tasking manager to conduct a partial import as opposed to conflating the entire dataset, given that the new import process is adding not only building footprints but footprints+addresses?  
>>>>
>>>>
>>>>I found examples of large areas of block groups with previously imported buildings lacking addresses that corresponded to empty chunk files.  I looked at the raw data and these should contain buildings with joined address tags that could be copied to the existing buildings.  Currently there are large swaths of buildings with no addresses.  Will this be taken care of in a separate process that I am unaware of?
>>>>
>>>>
>>>>I also found previously imported buildings that does not exist in the current dataset (or in any satellite imagery that I could find).  Since this import is semi-automated, wouldn't a comparison of the full dataset allowed for someone to spot this and remove it?
>>>>
>>>>
>>>>Thanks,
>>>>
>>>>
>>>>-Brandon
>>>>@geobrando
>>>>
>>>>
>>>>
>>>>[1]http://tasks.openstreetmap.us/job/6
>>>>[2]http://data.dc.gov/Metadata.aspx?id=190
>>>>[3]http://data.dc.gov/Metadata.aspx?id=59
>>>>[4]https://github.com/osmlab/dcbuildings
>>>>
>>>>
>>>>_______________________________________________
>>>>Imports mailing list
>>>>Imports at openstreetmap.org
>>>>https://lists.openstreetmap.org/listinfo/imports
>>>>
>>>>
>>>
>>>
>>>_______________________________________________
>>>Imports mailing list
>>>Imports at openstreetmap.org
>>>https://lists.openstreetmap.org/listinfo/imports
>>>
>>>
>>> -- 
>>You received this message because you are subscribed to the Google Groups "mappingdc" group.
>>To unsubscribe from this group and stop receiving emails from it, send an email to mappingdc+unsubscribe at googlegroups.com.
>>To post to this group, send email to mappingdc at googlegroups.com.
>>Visit this group at http://groups.google.com/group/mappingdc.
>>For more options, visit https://groups.google.com/d/optout.
>>
>
>
>_______________________________________________
>Imports mailing list
>Imports at openstreetmap.org
>https://lists.openstreetmap.org/listinfo/imports
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/imports/attachments/20140813/bd7beea0/attachment-0001.html>


More information about the Imports mailing list