[Imports] Province of Biella open data import

Andrea Musuruane musuruan at gmail.com
Wed Nov 23 21:38:34 UTC 2016


Hi Frederik,

On Wed, Nov 23, 2016 at 7:20 PM, Frederik Ramm <frederik at remote.org> wrote:
> Andrea,
>
>    your Wiki page does not give an example of the data to be imported.
> Which properties are present in the source data, and how are you going
> to convert this to OSM tags?

Probably you haven't found them.

The 3rd line in "about" section states:
There are specific pages about importing: address, building, toponym
and road data.

address, building, toponym and road are links to specific pages.

> This is a crucial step where lots of things can go wrong. For example,
> you say you want to import "topoynms" - of course we want to know what
> kind of topoynms these are and how you intend to tag them in OSM. Or you
> say you ewant to import roads - but of course the source data isn't
> going to use OSM road classification, so what are your plans here? And
> so on.

Clicking on the toponym link you will open the following page:
https://wiki.openstreetmap.org/wiki/Import/Catalogue/Provincia_di_Biella/Toponyms

There is a section about data transformation. There is a link to the
ogr2osm translation script, inside the script you can find how the
actual transformation works. You can also find a link to the actual
OSM XML result file.

Road data is a bit more complex:
https://wiki.openstreetmap.org/wiki/Import/Catalogue/Provincia_di_Biella/Roads

We won't import road data for every town but just for the few that are
poorly mapped. We will use cygnus tool to conflate the data.

All ogr2osm translations scripts can be found at my github repository
which is linked in the every specific import page in the wiki:
https://github.com/musuruan/osm_imports/tree/master/prov_bi

> Also, there are two very imporant things in your Wiki page that I
> implore you to take seriosly:

I'm sorry but I find your tone not really appropriate.

You know, I spent a lot of time in the last months with these import procedures.

I talked with Martin and Gabriela from Telenav to analyse if it was
possible to use the cygnus tool to conflate OSM and road data.

I wrote the ogr2osm translation scripts and I made many revisions to
correct transformation errors.

I talked with the upstream data owner to fix errors in the open data
they provide.

The upstream owner is also very sensible and released updated data
sets for our import effort.

Moreover I spent another bunch of time trying to document everything
in the wiki. It may not be perfect and there may be errors, but I
think it is much above the standard quality of import procedures I
find listed in the import catalogue.

> 1. "The dataset will be loaded in JOSM and it will be merged with
> existing OpenStreetMap data manually and prior to the upload."
>
> Do not underestimate this process. Sadly, often enough an import claims
> to be merging things with existing data, and in practice, people don't
> have the patience and just delete existing data and replace it. Please
> make sure this doesn't happen.

I already made an import in the past, regarding the address data for
the City of Biella:
https://wiki.openstreetmap.org/wiki/Import/Catalogue/Address_import_for_Biella

I know how many patience you must have and I know that most of the
time will be spent in QA.

About existing data. What I aim is quality. There are things in OSM in
this area which are made very very well. For example I find road data
particularly good. Other data, such as buildings, are very poorly
drown. The mapper often roughly draws the outline of a group of
buildings using aerial imagery, not even caring to orthogonalize the
shape if necessary.

Every time we will have to choose the best data and I assure you we
will be very careful about it.

> 2. "A local mapper MUST choose the towns he knows best."
>
> You currently have two collaborators listed, and it is unlikely that
> between the two of you, you know every place best. This logically means
> that some parts of the import will have to wait until someone is found
> who knows the area well. Many importers start out with big plans for a
> community import, and then end up importing everything themselves when
> they can't find other local community members to help quickly enough.
> Please don't succumb to this; if your import takes longer then it takes
> longer, but don't run off and import all the towns yourself just because
> you want it finished.

We are two for now, and hopefully there many be some more coming.

Please note we are talking about a very small province in Italy -
about 900 km² and less than 180k inhabitants - where a good part of
its land is covered by the Alps. A province I know very well and that
I already extensively mapped as my OSM history easily reveals.

The other collaborator listed is working for the data owner - and
actually worked in collecting the data.

In contrast I see the Los Angeles (about 1300 km² and 4M inhabitants)
buildings import page lists only 7 people:
https://wiki.openstreetmap.org/wiki/Los_Angeles,_California/Buildings_Import

I have no rush to end this import quickly and I already spend much of
my time in OSM doing Quality Assurance.

Bye,

Andrea



More information about the Imports mailing list