[OSM-dev] osm2pgsql fails on pbf file

[Jon Burgess]

On Sat, 2011-07-23 at 13:23 +0200, Stephan Knauss wrote:
> On 23.07.2011 12:43, marqqs at gmx.eu wrote:
> >> That really sounds like a problem inside the pbf. Did you try to read
> >> another pbf-file or read asia.osm.pbf with osmosis?
> 
> yesterday on irc someone had a similar problem.
> 
> I tried today with a different file. Same error.
> 
> I used this:
> http://download.geofabrik.de/osm/europe/albania.osm.pbf
> 
> osm2pgsql is build from svn and is revision 26385.
> # ./osm2pgsql --create --slim --cache 800 --hstore --database testdb 
> --username osm --style default.style ../albania.osm.pbf
> osm2pgsql SVN version 0.70.5
> 
...
> Reading in file: ../albania.osm.pbf
> error parsing member id of DenseNodes
> error parsing member dense of PrimitiveGroup
> error parsing member stringtable of PrimitiveBlock
> Error unpacking PrimitiveBlock message
> Error occurred, cleaning up
> 
> 
> osmosis can work with the file.
> "osmosis/bin/osmosis --read-pbf file=albania.osm.pbf --report-entity" 
> worked fine.
> 
> This sounds a bit like an issue inside osm2pgsql.

If I run a test which is practically identical to the one you listed
above then it works for me. I'm using Fedora 15 x86_64 with:

protobuf-compiler-2.3.0-7.fc15.x86_64
protobuf-2.3.0-7.fc15.x86_64
protobuf-devel-2.3.0-7.fc15.x86_64
protobuf-java-2.3.0-7.fc15.x86_64

protobuf-c-0.14.tar.gz compiled from source

$ ls -l albania.osm.pbf 
-rw-rw-r--. 1 jburgess jburgess 3502954 Jul 23 02:57 albania.osm.pbf

$ ./osm2pgsql --create --slim --cache 800 --hstore --database testdb
--style default.style albania.osm.pbf
osm2pgsql SVN version 0.70.5

Using projection SRS 900913 (Spherical Mercator)
Setting up table: planet_osm_point
NOTICE:  table "planet_osm_point" does not exist, skipping
NOTICE:  table "planet_osm_point_tmp" does not exist, skipping
Setting up table: planet_osm_line
NOTICE:  table "planet_osm_line" does not exist, skipping
NOTICE:  table "planet_osm_line_tmp" does not exist, skipping
Setting up table: planet_osm_polygon
NOTICE:  table "planet_osm_polygon" does not exist, skipping
NOTICE:  table "planet_osm_polygon_tmp" does not exist, skipping
Setting up table: planet_osm_roads
NOTICE:  table "planet_osm_roads_tmp" does not exist, skipping
Mid: pgsql, scale=100, cache=800MB, maxblocks=102401*8192
Setting up table: planet_osm_nodes
NOTICE:  table "planet_osm_nodes" does not exist, skipping
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index
"planet_osm_nodes_pkey" for table "planet_osm_nodes"
Setting up table: planet_osm_ways
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index
"planet_osm_ways_pkey" for table "planet_osm_ways"
Setting up table: planet_osm_rels
NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index
"planet_osm_rels_pkey" for table "planet_osm_rels"

Reading in file: albania.osm.pbf
Processing: Node(390k) Way(27k) Relation(450)  parse time: 8s

Node stats: total(390168), max(1369916430)
Way stats: total(27680), max(122653485)
Relation stats: total(456), max(1674699)

Going over pending ways
processing way (4k)

Going over pending relations

node cache: stored: 390168(100.00%), storage efficiency: 8.17%, hit
rate: 100.00%
Committing transaction for planet_osm_roads
Stopping table: planet_osm_nodes
Stopping table: planet_osm_ways
Stopping table: planet_osm_rels
Sorting data and creating indexes for planet_osm_roads
Building index on table: planet_osm_rels
Building index on table: planet_osm_ways
Committing transaction for planet_osm_polygon
Sorting data and creating indexes for planet_osm_polygon
Committing transaction for planet_osm_line
Sorting data and creating indexes for planet_osm_line
Stopped table: planet_osm_nodes
Committing transaction for planet_osm_point
Sorting data and creating indexes for planet_osm_point
Stopped table: planet_osm_rels
Completed planet_osm_roads
Completed planet_osm_polygon
Completed planet_osm_point
Stopped table: planet_osm_ways
Completed planet_osm_line


   Jon