[OSM-talk] Administrative boundaries export

César Martínez Izquierdo cesar.izq at gmail.com
Thu Oct 3 13:31:20 UTC 2013


That sounds interesting. I am not familiar with Nominatim, but I have
correctly understood, the result is a Postgres/postgis database with all
those polygons and hierarchies. This could be an interesting approach as
the post-processing could be directly done there using PostGIS predicates.

However, I am not sure about the generated hierarchies, as they don't keep
all OSM admin_levels into account (at least in the nomenclature: country,
state, county) and I see clear errors for Spain using the link [2] that you
provided. It would be interesting to know how these hierarchies are
generated (just OSM tags and geometric relations, external database, etc).

In any case, it seems a good starting point for my project.

Cheers,

Cesar


2013/10/3 Sarah Hoffmann <lonvia at denofr.de>

> On Thu, Oct 03, 2013 at 12:32:36PM +0200, César Martínez Izquierdo wrote:
> > Hi Frederik, regarding software, I am already familiar with Mapit scripts
> > code, which are able to extract admin boundary polygons for each level
> (it
> > is not creating relationships though). How do you see Nominatim or
> > Osmium/osmjs better for the purpose?
>
> Nominatim already does a lot of the stuff you are planning to do. It
> creates geometries for admin boundaries from OSM data and puts them
> in a proper hierarchy. It is able to process updates and in the
> process makes sure that boundaries do not just disappear if somebody
> breaks the relation. If you only process the data that interests you
> (boundary=administrative and place nodes) it is not even that
> resource-hungry.
>
> It does have support for listing broken boundaries [1] and during the
> last hack day Brian has written a proof-of-concept for browsing admin
> hierarchies[2]. There is even a script to dump objects within a certain
> boundary[3] which could be easily extended to dump entire hierarchies.
> All these functions should currently be used with care though. There are
> known bugs and the output needs to be improved to make them really
> usable.
>
> Basically, most of the work to do would be on the output side, the
> heavy lifting of processing the data is already done. Well, apart
> from checking the OSM boundaries against reality. But I think wiki data
> is a good starting point for that.
>
> > I will probably try the first option, but I expect that any of them would
> > be costly to maintain if there are frequent changes of the tagging scheme
> > for each country. (But nobody said it would be an easy task :-)
>
> Making boundary tagging more visible will hopefully help to stablize
> and unify the tagging schemas. The less country-specific exceptions
> the better.
>
> Sarah
>
> [1] http://nominatim.osm.org/polygons
> [2] http://nominatim.openstreetmap.org/hierarchy.php?place_id=97944985
>     (This is for demonstration only, please do not scrape. It will not
>                 always give you the expected results anyways because there
> is a
>                 known bug with parenting which still lingers in the DB.)
> [3]
> https://github.com/lonvia/Nominatim/blob/export-script/utils/export.php
>
> > 2013/10/2 Frederik Ramm <frederik at remote.org>
> >
> > > Hi,
> > >
> > > On 02.10.2013 18:23, César Martínez Izquierdo wrote:
> > > > I plan to create and make easily available a world-wide
> administrative
> > > > layer based on OSM data, ideally including existing administrative
> codes
> > > > (ISO, NUTS in Europe, etc) for each level and producing regular
> updates
> > > > (for instance once a year).
> > >
> > > This is something I have been thinking about for a long time (but never
> > > written any usable code).
> > >
> > > Nominatim is probably a good starting point - a better one than MapIt I
> > > should think.
> > >
> > > If you're only after extracting certain relation polygons then you
> could
> > > as well use osmjs (part of Osmium) and have it generate shape files for
> > > you, or adapt the shapefile/ogr export samples in Osmium; this will not
> > > yet give you a hierarchy, only individual boundaries, and you have to
> > > find out the hierarchy yourself.
> > >
> > > Finding out the hierarchy is going to be tricky. Nominatim does go to
> > > some lengths to do that already. It sounds easy ("find all polygons
> with
> > > an admin level smaller than X where this polygon I'm looking at lies
> > > in"). But in reality you will encounter at least:
> > >
> > > * missing polygons on all levels - sometimes simply not mapped,
> > > sometimes missing by design, e.g. Germany has some areas where admin
> > > levels 8, 6, and 4 coincide, these are mapped as admin_level 4, so draw
> > > a map of all admin_level 8 areas in Germany and you have lots of holes
> > > in them
> > >
> > > * broken polygons on all levels; brokenness changes by the day, i.e.
> > > what is working today may be broken tomorrow and vice versa
> > >
> > > * occasionally (e.g. Japan) linear regional boundaries that simply go
> > > from coast to coast without including the coastline
> > >
> > > * occasionally (e.g. Chile) a regional boundary that is not a
> > > multipolygon relation but instead a grouping of smaller regional
> entities
> > >
> > > * sometimes small geometric inaccuracies mean that e.g. a state
> boundary
> > > fails the "is-in" test for the country boundary because there's just a
> > > little square metre somewhere that is mapped as belonging to the state
> > > but not the country
> > >
> > > * overlapping admin polygons of the same admin level
> > >
> > > I think that ate the very least you need to run the evaluation
> regularly
> > > and compare: Do I have new polygons this week - have others vanished,
> > > and if so, is that because they were explicitly deleted/replaced, or
> > > were they just accidentally broken and I should continue to use last
> > > week's?
> > >
> > > What we would really need though, is something much bigger: A separate
> > > database of admin hierarchies, where people could - in a crowdsourced
> > > manner - record things like:
> > >
> > > "There is an adminlevel 2 entities called Germany"
> > > "It is divided into 16 exclusive adminlevel 4 entities with the
> > > following names: ..."
> > > "These 16 entities cover the area of Germany completely (no holes or
> sea
> > > areas that would be outside of one of the entities)"
> > > "The adminlevel 4 entity named 'Brandenburg' is divided in X adminlevel
> > > 6 entities..."
> > >
> > > and so on. A tree of arbitrary size where people can add and edit at
> will.
> > >
> > > Now you will say "but this tree could be generated from OpenStreetMap",
> > > and I grant that one could attempt to build such a tree but it will
> > > always be faulty and reflect the current brokenness of geometries in
> > > OSM. One could *start* with an OSM-generated tree, but after that, the
> > > tree must be kept separate. People should be able to add stuff to the
> > > tree even when it is not in OpenStreetMap - "there should be an
> > > adminlevel 8 boundary called so-and-so". A regularly-running process
> > > would then compare the tree to OpenStreetMap, and generate error
> reports
> > > that can be presented visually:
> > >
> > > "The tree says that there should be a region called X in Germany but
> OSM
> > > doesn't have one."
> > >
> > > "There is an area here that is not covered by any adminlevel 4 area but
> > > the tree says that taken together the adminlevel 4 areas must cover all
> > > of the country."
> > >
> > > "The tree claims there should be a region called X but in OSM there's
> > > only a region with the similar name Y, which one is correct?"
> > >
> > > and so on. - I would expect the tree to be much more stable than the
> > > data in OSM. Most of all, the tree could be worked on independently,
> > > even by people unfamiliar with OSM. Of course the tree could link to
> OSM
> > > objects but these links would regularly be checked and perhaps even
> > > changed by the automated comparison system.
> > >
> > > As I said, a simple export is easy but it will have too many weaknesses
> > > - you couldn't even say to what level a country is "complete". Some
> > > people have started to link regional boundaries in OSM together with
> > > concepts like "subarea" but I don't think this can replace an external
> > > country structure tree because the tree could describe what is expected
> > > to be there whereas in OSM you never know if some thing is missing
> > > because it doesn't exist (cf. my example about the "missing" admin6/8
> > > boundaries in Germany) or if it just hasn't been mapped yet, or is
> broken.
> > >
> > > It would be great to see someone drive this important topic but it
> > > certainly isn't something you can set up in a week or two ;)
> > >
> > > Bye
> > > Frederik
> > >
> > > --
> > > Frederik Ramm  ##  eMail frederik at remote.org  ##  N49°00'09"
> E008°23'33"
> > >
> > > _______________________________________________
> > > talk mailing list
> > > talk at openstreetmap.org
> > > https://lists.openstreetmap.org/listinfo/talk
> > >
> >
> >
> >
> > --
> > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> >    César Martínez Izquierdo
> >    GIS developer
> >    -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
> >    ETC-SIA: http://sia.eionet.europa.eu/
> >    Universitat Autònoma de Barcelona (SPAIN)
> > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>
> > _______________________________________________
> > talk mailing list
> > talk at openstreetmap.org
> > https://lists.openstreetmap.org/listinfo/talk
>
>


-- 
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
   César Martínez Izquierdo
   GIS developer
   -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -
   ETC-SIA: http://sia.eionet.europa.eu/
   Universitat Autònoma de Barcelona (SPAIN)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/talk/attachments/20131003/dd0c84cd/attachment-0001.html>


More information about the talk mailing list