[Imports] Fwd: Re: Spanish cadastre

Wed Mar 28 07:44:08 UTC 2012

Hello,
 My short summary of my experiences from mass-imports:

 - Never mass-upload data to OSM. It usually creates a lot of too detailed and non-maintained information in the database. It is not maintained for multiple reasons: (1) raw quantity (there is too much of it), so loading of complex data (e.g. land usage relations) data to editors is slower, simple editors like potlach2 which have no tag filters would from it. (2) even more important - there is no owner of every piece/region, importers cannot keep their eye on nation-wide datasets. (3) new users will see that "there is ready map" and they do not bother to check/update it. Maybe it does not really kill the community, but it can really discourage it. It is extremely important to understand that the community must come first, not the map.

 - Proper importing would be manual copy-pasting in individual object level, but the big .osm files what you probably get as result of cat2osm are technically hard to use: slow to download and open/manipulate in JOSM.

My suggestions:

  - Instead of static .osm files create a service which provides small pre-tagged filtered extracts from the external data by theme/tag and bounding box. Use OSM (X)API as API for it, and there are several opensource xapi server solutions.

  - Extend JOSM so it is easier to use multiple map services: main DB and your local data sources. Currently you can do it kind of manually:
1) change API URL in settings, then download data from your imports server based on bbox,
2) then change API URL to api.osm.org, 
3) then download real data to second layer;
4) then you merge them: copy-paste, merge nodes/relations etc, 
5) then upload to api.osm.org

 The problem is that 1-2 here is hard, (maybe) even JOSM restart is needed; so you need to save data temporarily in local .OSM file, otherwise JOSM looses loaded data with restart. Also you cannot add extra query parameters there to filter tags/theme which you are interested in. But at least BBOX creation is user-friendly.

 I have myself used File>Open URL in JSOM, but manually creating proper XAPI URL with BBOX and other filters there is way too much; I could use it only because I have created the server-side part myself.

 I also have dream about MetaMapping service where external datasets do not need to be copied to OSM at all, and instead links would be used [1], but I do not see own time to make it to reality. Any volunteers?

[1] http://wiki.openstreetmap.org/wiki/OpenMetaMap

Jaak

On 28.03.2012, at 9:59, Carlos Dávila wrote:

> Some days ago I sent the mail below to the list, but by an error it went only to Paul Norman and the Spanish mailing list. I would like to get feedback from you about the sample data from Spanish Cadastre to be imported [1]. A new sample can be downloaded from [2].
> P.S. FYI, we have written some draft information in English about the import at [3]
> [2] http://mapas.alternativaslibres.es/Don_Benito.zip
> [3] http://wiki.openstreetmap.org/wiki/Spanish_Cadastre_import
> 
> -------- Mensaje original --------
> Asunto:	Re: [Imports] Spanish cadastre
> Fecha:	Fri, 23 Mar 2012 22:47:17 +0100
> De:	Carlos Dávila <cdavilam at orangecorreo.es>
> Para:	Paul Norman <penorman at mac.com>
> CC:	Discusión en Español de OpenStreetMap <talk-es at openstreetmap.org>
> 
> El 22/03/12 20:38, Paul Norman escribió:
> >> From: Carlos Dávila [mailto:cdavilam at orangecorreo.es]
> >> Sent: Thursday, March 22, 2012 5:28 AM
> >> To: imports at openstreetmap.org
> >> Subject: [Imports] Spanish cadastre
> >>
> >> This mail is to announce the intention of the Spanish OSM community to
> >> import data from Spanish Cadastre and get feedback from you.
> >> The idea of this import arises with the publication of the license and
> >> access criteria to massive download of data from Spanish Cadastre [1] on
> >> march 2011, which states: "Public use of cadastrial data is permitted,
> >> including commercial use, if such data have been previously transformed
> >> in any way...". Original data is supplied as shapefiles with different
> >> coordinate reference systems. Reprojecting those shapefiles to EPSG:4326
> >> and converting to *.osm format is enough transformation to allow
> >> mentioned public use, according with the reference Spanish "Law of
> >> intellectual property" [2], as established in its article 21, which also
> >> says that copyrights of the transformed product belongs to the
> >> transformer, in this case the (Spanish) OSM community.
> >> Data from Spanish Cadastre is known to have a high accuracy and include
> >> a lot of information of interest for OSM project, such as buildings
> >> (shape, use, height, number of floors, etc.), rural and urban parcels
> >> land use, most of streets and roads of the country, power stations,
> >> tracks and paths, trees... Many of this information is currently in OSM,
> >> mainly ways and roads, but a great part is still missing, specially in
> >> low populated areas where a ground survey is less likely to be made.
> >> This import has been largely discussed on Spanish mailing list, with a
> >> general consensus on the usefulness of the import and a "we must do it"
> >> agreement.
> >> Specific software (cat2osm) is being developed [3] to import data, which
> >> source code is available at Github [4]. It is currently being tested by
> >> some members of the community to get a stable status. Some images of the
> >> preliminary results can be seen at [5]. Transformed data is thought to
> >> be always manually checked in JOSM by osmers knowing the area and only
> >> after that uploaded. The way original data is translated into osm format
> >> by cat2osm tool is documented in [6] and has been established with the
> >> participation of several members of the community.
> >>      
> > What is your proposed tagging of these objects?
> >
> > Cadastre generally have a lot of detailed information, not all of which is
> > suitable for importing into OSM. Which datasets do you intend to import and
> > which do you intend to leave out?
> >
> >    
> >> Specific accounts have been created for the import, one for each Spanish
> >> province [7]. There's a member of the Spanish community responsible of
> >> each province import.
> >>      
> > What software do you plan to use to conflate with existing OSM data?
> > How do you intend to upload the data?
> > Approximately how large will the changesets be?
> >
> >    
> >> Looking forward for your comments
> >> Carlos Dávila
> >>      
> > I suggest breaking your import proposal into sub-proposals for each type of
> > data set. E.g. have a different proposal for roads than for buildings.
> >
> >
> >
> >    
> cat2osm can process data of a given municipality as a whole or in 
> separate steps, processing different kind of data. That result in 
> separate *.osm files, containing buildings, highway=*, parcels, 
> subparcels, etc. Also urban and rural areas are in separate input 
> shapefiles. What I want to show you is that importer (remember it's not 
> planned as an automated import) can control what to import and the size 
> of the dataset to upload. I have uploaded some sample data to [1]. For a 
> given municipality, zip contains raw data provided by cat2osm for 
> highway=*, current data in OSM database and merged data for the urban 
> area processed with JOSM. IMHO these data could be uploaded to OSM 
> (using JOSM) without any risk to face those problems have been mentioned 
> in this thread.
> 
> [1] http://mapas.alternativaslibres.es/Navalvillar.zip
> _______________________________________________
> Imports mailing list
> Imports at openstreetmap.org
> http://lists.openstreetmap.org/listinfo/imports

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/imports/attachments/20120328/f9731b14/attachment-0001.html>