[Talk-us] Available Building Footprints

Nathan Mixter nmixter at gmail.com
Tue Mar 28 17:46:34 UTC 2017


Steve, I'm not sure how or why you are jumping to the conclusion that
because a wiki page was created that somehow means the import has already
occurred. Your impulsive reaction and rants are unwarranted and
unappreciated. No one has said anything about importing other than raising
the possibility.

The page was created by another user who was listing the data sets
available. I simply added another section on the page to break apart the
California areas into cities so they can be reviewed further. The page
clearly states that the import procedures need to be followed and I don't
have any plans to import the data now and I don't think anyone else does
either.

Denis was right on with his response, and those are the type or responses
that we need if ... and I do say if ... this project is to move forward.
There are several hurdles in using this data, one being the size and scope.
That is why the wiki page was created was to hash out the best way to
proceed from people who have successfully done large scale imports. The
data can't be reviewed effectively as one big file. The project is still
young, and before we can even post on the imports list we need to have a
procedure in place.

Let's continue to pool resources and add ideas and suggestions to the wiki
page as we look into the possibility of importing this data. Looking
forward to hearing what others have to say and the ideas we can come up
with together.

Nathan

On Tue, Mar 28, 2017 at 9:23 AM, OSM Volunteer stevea <
steveaOSM at softworkers.com> wrote:

> I couldn't agree more, Denis.  The only thing that this (poorly
> named/indexed in OSM's wiki) "Available Building Footprints" page mentions
> about importing is "Any import of these building footprints must strictly
> follow the import guidelines."  Well, then, please do so!  I'm not saying
> that this built-in-the-last-week wiki isn't informative, for what it is, it
> is.  However, it is not an import project (yet), and while Nathan implies
> these data (data are plural) SHOULD be imported, he has not taken the time
> nor effort to correctly turn these into a real import project (posting to
> the imports list, following our "five steps...").
>
> Nathan just emailed me to say that he couldn't open the Bay Area shapefile
> either, saying he split it apart with QGIS.  He continues "Even though I
> split the file into different areas that file still seems to be the same
> size as the original. It's almost like the other data is still there even
> though it shows only the Bay Area shapes. Maybe some one else has a better
> way to split up the file. The Bay Area data all runs together so it is hard
> to see where natural splits occur. Maybe you (stevea) or some one else will
> have better luck trying to split it."  This does not bode well for someone
> who wants to lead or contribute to an import.  (Nathan, Nathan, Nathan...).
>
> Nathan and I have become friends, we met eight years ago in OSM, go on
> many hikes and camping trips together, he comes to my house parties and we
> further collaborate via email.  I have worked with Nathan on numerous
> import tasks, noted in our Santa Cruz County wiki page, including the major
> (now in version 3) import of Santa Cruz County landuse areas and the
> Monterey County California Farmland import.  The first one was nearly a
> disaster:  it took me four years to manually untangle Nathan's mess/data
> upload crashes and finally supersede with v3.  The Monterey County import
> was a constant struggle of "throttling down" Nathan's constant instinct to
> "spill buckets of import paint quickly and with little regard for data
> quality" until I hardcore task-managed the project between the two of us
> over months of careful project husbandry so it eventually became a sane and
> high-quality import of which we can both be proud.  Nathan is also
> notorious for, let's be candid, "making a mess of California's Central
> Valley" which, even to this day, I am not sure he has fully cleaned up.
>
> Nathan, I do not say these things lightly in a public forum like talk-us,
> but it appears that you are yet again taking a cavalier and hair-trigger
> approach to doing a major (MAJOR!) import in California.  If you wish to do
> so, please learn from your past that this is a tremendous effort, bigger by
> an order of magnitude or more than anything you have attempted to import
> before, listen to friends of yours like me and Denis (below) and bite off
> only as much as you can chew, with the technical, social and OSM community
> skills needed that it takes to complete such an endeavor.  We are asking
> you to please do it right this time, if indeed you feel that you can and
> will.  There are many, many tasks ahead if you wish to see these data in
> OSM and not even the first tasks of what would encourage me to say I see a
> high quality data import ahead have happened yet, save for posting the
> data.  THAT is often the first step of a hasty, poor (and ultimately
> redacted) data import.  You have had opportunity after opportunity to do
> data imports into OSM and should be able to learn that there is a correct
> way to do it:  the only way to do it.
>
> The next place I hope to read anything about this is on the imports list.
>
> SteveA
> California
>
>
> > On Mar 28, 2017, at 8:48 AM, Denis Carriere <carriere.denis at gmail.com>
> wrote:
> >
> > Instead of having tons of different people trying to attempt loading all
> of these 8 Million buildings, we shoul collectively start an import
> proposal (OSM wiki, draft a plan, set up tasking managers, pre-process
> data, host entire dataset, etc...).
> >
> > The best/easiest solution we (OSM Ottawa) did for importing 1M+
> buildings was to convert the data into GeoJSON and then convert them into
> VectorTiles using Tippecanoe [0] and host them using our own custom server
> (Micro Data Service [1]) which hosts those vector tiles into OSM & GeoJSON.
> After all the "hard work" is done.. you can simply add those small chunks
> of data with JOSM using any Tasking Manager by adding the URL [2] in Extra
> Instructions. An example of a final OSM tile would look like this [3] which
> would be ready to import (semi-manually).
> >
> > There's also integration with QA-Tiles [4] to prevent loading any
> duplicate data (this feature requires continuously loading the most current
> QA-Tile during the import process).
> >
> > Summary: Before anyone attempts to import this data, we need to create a
> plan first. I'm more than willing to help out, but this would be a large
> task and would need to be done collectively as a group.
> >
> > [0] https://github.com/mapbox/tippecanoe
> > [1] https://github.com/osmottawa/micro-data-service
> > [2] http://localhost:8111/import?new_layer=true&url=https://
> data.osmcanada.ca/{z}/{x}/{y}/ottawa-buildings.osm
> > [3] https://data.osmcanada.com/15/9478/21019/ottawa-buildings.osm
> > [4] https://osmlab.github.io/osm-qa-tiles/
> >
> > ~~~~~~
> > Denis Carriere
> > GIS Software & Systems Specialist
> >
> > On Tue, Mar 28, 2017 at 11:00 AM, OSM Volunteer stevea <
> steveaOSM at softworkers.com> wrote:
> > Hi Nathan:
> >
> > I've got pretty beefy hardware, but the "Bay Area" shapefile pointed to
> by your recent post chokes my JOSM to a gasping strangle:  >3.7 million
> objects?!  These need to be broken up further to smaller files, to either
> the county level or even smaller to a sub-county level, in a sane way.  You
> may as well save them as .osm files (and host them on some other place
> besides a Microsoft cloud), as shapefiles still remain a "foreign" (though
> importable) format within OSM.
> >
> > SteveA
> > California
> >
> >
> > > On Mar 28, 2017, at 5:00 AM, talk-us-request at openstreetmap.org wrote:
> > >
> > > https://wiki.openstreetmap.org/wiki/Available_Building_Footprints
> >
> >
> > _______________________________________________
> > Talk-us mailing list
> > Talk-us at openstreetmap.org
> > https://lists.openstreetmap.org/listinfo/talk-us
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/talk-us/attachments/20170328/2fb5eaa1/attachment-0001.html>


More information about the Talk-us mailing list