[Tile-serving] [osm2pgsql] [bug] Critical perfomance regression in 0.91 (10.03.2016) (#557)

[Stranger]

```
osm2pgsql -c -d gis  --tablespace-main-data gis_data \
  --tablespace-slim-data gis_data \
  --tablespace-main-index gis_index \
  --tablespace-slim-index gis_index \
  --slim -S osm2pgsql-topo.style -G \
  -C 6144M \
  -U gis \
  --number-processes 7 \
  --unlogged \
  source.osm.pbf 
```

After 5 days osm2pgsql still in parsing stage:

```
osm2pgsql version 0.91.0-dev (64 bit id space)

Using built-in tag processing pipeline
Using projection SRS 3857 (Spherical Mercator)
Setting up table: planet_osm_point
Setting up table: planet_osm_line
Setting up table: planet_osm_polygon
Setting up table: planet_osm_roads
Allocating memory for dense node cache
Allocating dense node cache in one big chunk
Allocating memory for sparse node cache
Sharing dense sparse
Node-cache: cache=6144MB, maxblocks=98304*65536, allocation method=11
Mid: pgsql, scale=100 cache=6144
Setting up table: planet_osm_nodes
Setting up table: planet_osm_ways
Setting up table: planet_osm_rels

Reading in file: source.osm.pbf
Using PBF parser.

Processing: Node(309134k 332.4k/s) Way(29605k 69.01k/s) Relation(55950 0.33/s)
^C
```

I'm back to 0.89.20160205 snapshot:

osm2pgsql SVN version 0.89.0-dev (64 bit id space)

```
Using built-in tag processing pipeline
Using projection SRS 900913 (Spherical Mercator)
Setting up table: planet_osm_point
Setting up table: planet_osm_line
Setting up table: planet_osm_polygon
Setting up table: planet_osm_roads
Allocating memory for dense node cache
Allocating dense node cache in one big chunk
Allocating memory for sparse node cache
Sharing dense sparse
Node-cache: cache=6144MB, maxblocks=98304*65536, allocation method=11
Mid: pgsql, scale=100 cache=6144
Setting up table: planet_osm_nodes
Setting up table: planet_osm_ways
Setting up table: planet_osm_rels

Reading in file: source.osm.pbf
Using PBF parser.

Processing: Node(309134k 333.1k/s) Way(29605k 67.59k/s) Relation(699070 102.93/s)  parse time: 8159s
Node stats: total(309134147), max(4044947576) in 928s
Way stats: total(29605035), max(402061908) in 438s
Relation stats: total(699079), max(6031832) in 6792s

```


---
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/issues/557
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20160312/2a424b9f/attachment.html>