[OSM-dev] osm2pgsql - some polygons not ending up in osm_polygon

Jon Burgess jburgess777 at gmail.com
Thu Dec 20 17:31:35 GMT 2012


On Thu, 2012-12-20 at 15:23 +0000, Svavar Kjarrval wrote:
> On 20/12/12 14:47, Jon Burgess wrote:
> > On Thu, 2012-12-20 at 13:58 +0000, Svavar Kjarrval wrote:
> > The most likely reason they would not make it into the polygon table is
> > if the data does not form a complete ring. In this case they are sent to
> > the line table instead. If you know the ID of the relation then you can
> > find the data in the line or polygon tables with osm_id = -relation_id.
> I imported the data myself and made sure they formed a complete ring
> before submitting.
> 
> Checked the data in the main OSM database and for some reason Garðabær
> didn't form a complete ring. However, Hafnarfjörður and Mosfellsbær do.
> Reykjavík forms two rings since it's two separate areas.

I agree that the data for those looks OK. Otherwise they wouldn't have
appeared for me. This suggests that something else that you are doing is
breaking things.


> >> The top 3 municipalities in the table are in the capital area in
> >> Iceland. The ones that should be there too are Hafnarfjörður,
> >> Garðabær, Reykjavík and Mosfellsbær.
> > I just repeated this locally with current iceland.osm.pbf from Geofabrik
> > and get:
> >
> > gis=# SELECT name, admin_level,boundary FROM iceland_polygon WHERE
> > admin_level = '6' order by name;
> >        name       | admin_level |    boundary    
> > ------------------+-------------+----------------
> >  Álftanes         | 6           | administrative
> >  Álftanes         | 6           | administrative
> >  Garðabær         | 6           | administrative
> >  Hafnarfjörður    | 6           | administrative
> >  Hrísey           | 6           | administrative
> >  Hríseyjarhreppur | 6           | administrative
> >  Kópavogur        | 6           | administrative
> >  Mosfellsbær      | 6           | administrative
> >  Reykjavík        | 6           | administrative
> >  Reykjavík        | 6           | administrative
> >  Seltjarnarnes    | 6           | administrative
> > (11 rows)
> >
> > Looks like all the ones you listed appear now.
> Because of the error, I got another copy of iceland.osm.bz2 today (dated
> the 19th). The state.txt file was backtracked to the 18th and an update
> executed. The data is from after I updated, which was this noon. I
> wonder if there's a difference between the osm.bz2 version and the
> osm.pbf one. The boundary data is from some weeks ago anyway.

I repeated this test again using the BZ2 data from 19th. The data is
older but I still get 11 rows returned by the query above.

This suggests that your data broke when doing the updates. Can try a
similar test yourself to confirm that things work OK with the data
before the update?

What commands do you use to do the updates?

> I corrected the ring for Garðabær and waited for the next update, it
> didn't appear in the osm_polygon table. I assume entries are moved or
> recreated in the other table if osm2pgsql detects that they should be
> elsewhere.

This is what is meant to occur but it has been known for things to go
wrong in some edge cases.

> > Yours:
> > Node stats: total(1147968), max(2076135241) in 18s
> > Way stats: total(101787), max(197363513) in 17s
> > Relation stats: total(2381), max(2649305) in 12s
> >
> > Mine:
> > Node stats: total(1222634), max(2074844232) in 5s
> > Way stats: total(105203), max(197203875) in 5s
> > Relation stats: total(2429), max(2647535) in 5s
> >
> > Maybe try downloading the data again?
> >
> > I'm using this file:
> >
> > $ ls -l iceland.osm.pbf
> > -rw-rw-r--. 1 jburgess jburgess 10832119 Dec 20 02:38 iceland.osm.pbf
> I downloaded the data this noon and these are the results. The osm.bz2
> file is dated the 19th and the osm.pbf is dated today. These seem to be
> two very different versions because I don't think so many changes have
> been made for Iceland in that short amount of time.

When I repeat the test with the BZ2 file I see slightly less nodes than
the PBF. The difference is probably because the data is slightly older. 

Node stats: total(1220229), max(2072717631) in 10s
Way stats: total(105049), max(197028951) in 9s
Relation stats: total(2428), max(2645812) in 5s

Even so, the total number of nodes in the BZ2 (1220229) is still higher
than the figure you reported (1147968) which makes we think that some
data is getting lost in your processing before you import it. Do you try
to apply any bounding box filter when applying the diffs?

   Jon






More information about the dev mailing list