[Imports] Sardinia Regional Dataset Building Import: second proposal

Tue Mar 25 01:46:43 UTC 2014

Hi Leonardo,

- The import wiki page should include the information from your email,
describing the process. I think this is a requirement.
- The final OSM files should be made available to the import list before
any uploads happen.

Some suggestions (I don't think are required, but might make things better)

- Consider automating the preparation work with postGIS. It is likely new
problems will be found when you start uploading the data. Fixing the
problems going forward will require that the data be re-processed again. If
it is automated, you can keep finding/fixing issues quickly.
- Check out the the NY city address import. They are using a tasking
manager to dole out OSM chucks to upload/review.
- Breaking up the source data by town might not make sense. Ideally, each
OSM chuck will be small enough where somebody can do a good job checking it
over and uploading in in less than an hour or so. You also want to keep
each upload < 5,000 nodes/ways. An entire town might be too large.
- Selecting chunks of data in JOSM is tricky. Consider the case of having a
multi-polygon building laying on the border of a town.
- If you have address data, you should think about importing them too (even
though it is 10x more work than just the buildings).

Thanks
Jason

On Mon, Mar 24, 2014 at 7:57 PM, Leonardo Frassetto
<kinetocore86 at gmail.com>wrote:

> Hi,
>
> i'm writing here to request some feedback on an updated import proposal
> for Sardinia. The original proposal was here:
>
> https://lists.openstreetmap.org/pipermail/imports/2013-November/002370.html
>
> This database contains a lot of useful information that can be used to
> improve the map of Sardinia, like streets, landuses, housenumbers and
> almost all the buildings on the island. The data goes between 1994 and 2009
> and in the first phase we will import the buildings (if all goes well we
> could go further with the rest of the data but this proposal is ONLY for
> buildings).
>
> The data is licensed under IODL 2.0 that is ODBL compatible (
> http://www.sardegnageoportale.it/disclaimer.html).
>
> Data is here:
> http://www.sardegnageoportale.it/index.php?xsl=1598&s=140641&v=2&c=8831&t=1
>
> We discussed a lot on ML about a semi-mechanic import, using a translator
> as shp-to-osm and a rule files but after many tries and tweaking, still the
> converted data was too much error filled and quite confused for tagging.
> Also we discovered that many of the features covered in the data table on
> the official pdf manual aren't present at all on the shp data, making us
> wasting time searching for an appropriate tag or thinking for a possible
> new tag scheme. It was decided to scrap completely the idea of a mechanical
> import and goes for a complete manual check/cut/tag workflow that i'll
> describe now. The workflow will be this:
>
> On the main file (that is a HUGE shp file, 300+ MB, converted to WGS84),
> every polygon has some attibutes as numbers that is translated on the pdf
> manual table with the corresponding feature ex. 0010 on a polygon means a
> school building (so building=school) and 00101 on an area means the
> corresponding area of the school (so amenity=school). Using QGIS and the
> filter option we select for example all the polygon with one number and
> save that selection as a separate file. Then we open that file on JOSM +
> opendata plugin and we do a manual assigned tagging and run the validator
> to fix possible errors like duplicated nodes and so on. This "clean" file
> will be saved as .osm. We will do this process to all the possible number
> selection, so we create a clean file for every single type of buildings. We
> will merge back all the data, doing a validator check every layer merged
> until we have back the original SHP file converted into an .osm file that
> has 0 error and all correctly tagged. This will be the first step.
>
> The second step is cut this big file into smaller chucks of data, that can
> be easily checked by importers and is not so big that can be too heavy to
> work with on JOSM. We will use the an OSM boundaries service (
> http://osm.wno-edv-service.de:8080/boundaries/) to download the
> boundaries of a single city/town and use it as an "area selection tool " on
> the original big .osm, selecting all the buildings inside and saving it as
> TOWNNAME.osm that will be sent to the importer for the final check before
> the upload.
>
> The imported will do the final check that will be check for existing data,
> integrate it with the import data keeping in mind that he must contact the
> original mapper for discussing every possible modify/delete on his data, do
> a final validation check and then upload it with the appropriate source tag
> on the upload window on JOSM.
>
> Data is already well aligned against the italian PCN2012 aerial image,
> that is proved many times as a good standard for alignment, much better
> than Bing images. Again the final importer will check the data to see if
> there are any missing building and consider to add them on the import data.
> Same thing will be for demolished building that will be deleted before the
> import. Using such smaller .osm files helps the importer on the task of
> visual check between photo and data.
>
> Here is the data imported:
> -Houses
> -Industrial buildings
> -Public building (townshall, police, schools, hospitals, libraries), area
> + building associated to it.
>  -Churches + cementery buildings and area.
> -Smaller monuments
> -Sport centers + pitch + sport if specified on the original shp file.
>
> Luckly all of these type of building have a specific, widespread used,
> possible tag to assign them (using the presets on JOSM we can cover all of
> them, without the need to create new ones).
>
> We have already a wiki import page (
> http://wiki.openstreetmap.org/wiki/Sardegna/Import/Edificato) were we
> will keep track of the imported data with all the changesets. Please ignore
> the first part with the old rules test, it will be deleted. We are writing
> the manual rules and we will add them to the page very soon. We will start
> with the first 60 town heavily hitted by the cyclone (first table) then we
> will move to the others.
>
> We think that with smaller steps instead of a big import, the quality of
> the import can be much more consistent.
>
> Any concern?
>
> Regards
>
> Leonardo (OSM user: DarkSwan_Import)
>
> _______________________________________________
> Imports mailing list
> Imports at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/imports
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/imports/attachments/20140324/f9b32711/attachment.html>