[Tile-serving] [osm2pgsql] Hang on unsimplified diffs (#112)
Paul Norman
notifications at github.com
Mon Dec 23 15:47:11 UTC 2013
I'm getting it still with `--simc` but later on, with the second set of diffs
```
Using 8 helper-processes
Mid: loading persistent node cache from flat.nodes
Maximum node in persistent node cache: 2447521791
Mid: loading persistent node cache from flat.nodes
Maximum node in persistent node cache: 2447521791
Mid: loading persistent node cache from flat.nodes
Maximum node in persistent node cache: 2447521791
Mid: loading persistent node cache from flat.nodes
Maximum node in persistent node cache: 2447521791
Mid: loading persistent node cache from flat.nodes
Maximum node in persistent node cache: 2447521791
Mid: loading persistent node cache from flat.nodes
Maximum node in persistent node cache: 2447521791
Mid: loading persistent node cache from flat.nodes
Maximum node in persistent node cache: 2447521791
Mid: loading persistent node cache from flat.nodes
Maximum node in persistent node cache: 2447521791
processing relation (2818) at 162.82/s (done 0 of 8)^C
```
Command was `clear && sudo ./free_mem.sh && date && time (for n in `seq 556 655`; do osmosis/bin/osmosis --rxc "diffs/008/$n.osc.gz" --simc --wxc - | osm2pgsql/osm2pgsql -d gis -a -S osm2pgsql/default.style --flat-nodes flat.nodes -C 2000 --slim --number-processes 8 - ; done) && date && sync`
```
gis=# select * from pg_stat_activity where pid=10747;
-[ RECORD 1 ]----+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
datid | 16385
datname | gis
pid | 10747
usesysid | 16384
usename | pnorman
application_name |
client_addr |
client_hostname |
client_port | -1
backend_start | 2013-12-23 11:09:36.611401+01
xact_start | 2013-12-23 11:09:48.037928+01
query_start | 2013-12-23 11:09:48.037928+01
state_change | 2013-12-23 11:09:48.037932+01
waiting | f
state | active
query | COPY planet_osm_polygon (osm_id,"access","addr:housename","addr:housenumber","addr:interpolation","admin_level","aerialway","aeroway","amenity","area","barrier","bicycle","brand","bridge","boundary","building","construction","covered","culvert","cutting","denomination","disused","embankment","foot","generator:source","harbour","highway","historic","horse","intermittent","junction","landuse","layer","leisure","lock","man_made","military","motorcar","name","natural","office","oneway","operator","place","population","power","power_source","public_transport","railway","ref","religion","route","service","shop","sport","surface","toll","tourism","tower:type","tracktype","tunnel","water","waterway","wetland","width","wood","z_order","way_area",way) FROM STDIN
```
No disk IO either.
In case it helps, here are the other queries that persisted after terminating osm2pgsql
```
-[ RECORD 2 ]-+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
pid | 10630
backend_start | 2013-12-23 11:05:51.671354+01
xact_start | 2013-12-23 11:09:48.173081+01
query_start | 2013-12-23 11:09:48.173081+01
state_change | 2013-12-23 11:09:48.173085+01
waiting | f
state | active
query | COPY planet_osm_polygon (osm_id,"access","addr:housename","addr:housenumber","addr:interpolation","admin_level","aerialway","aeroway","amenity","area","barrier","bicycle","brand","bridge","boundary","building","construction","covered","culvert","cutting","denomination","disused","embankment","foot","generator:source","harbour","highway","historic","horse","intermittent","junction","landuse","layer","leisure","lock","man_made","military","motorcar","name","natural","office","oneway","operator","place","population","power","power_source","public_transport","railway","ref","religion","route","service","shop","sport","surface","toll","tourism","tower:type","tracktype","tunnel","water","waterway","wetland","width","wood","z_order","way_area",way) FROM STDIN
-[ RECORD 3 ]-+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
pid | 10708
backend_start | 2013-12-23 11:09:36.594764+01
xact_start | 2013-12-23 11:09:52.673359+01
query_start | 2013-12-23 11:09:52.673359+01
state_change | 2013-12-23 11:09:52.673362+01
waiting | f
state | active
query | COPY planet_osm_polygon (osm_id,"access","addr:housename","addr:housenumber","addr:interpolation","admin_level","aerialway","aeroway","amenity","area","barrier","bicycle","brand","bridge","boundary","building","construction","covered","culvert","cutting","denomination","disused","embankment","foot","generator:source","harbour","highway","historic","horse","intermittent","junction","landuse","layer","leisure","lock","man_made","military","motorcar","name","natural","office","oneway","operator","place","population","power","power_source","public_transport","railway","ref","religion","route","service","shop","sport","surface","toll","tourism","tower:type","tracktype","tunnel","water","waterway","wetland","width","wood","z_order","way_area",way) FROM STDIN
-[ RECORD 4 ]-+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
pid | 10710
backend_start | 2013-12-23 11:09:36.595498+01
xact_start | 2013-12-23 11:09:49.361676+01
query_start | 2013-12-23 11:09:49.361676+01
state_change | 2013-12-23 11:09:49.361678+01
waiting | f
state | active
query | COPY planet_osm_polygon (osm_id,"access","addr:housename","addr:housenumber","addr:interpolation","admin_level","aerialway","aeroway","amenity","area","barrier","bicycle","brand","bridge","boundary","building","construction","covered","culvert","cutting","denomination","disused","embankment","foot","generator:source","harbour","highway","historic","horse","intermittent","junction","landuse","layer","leisure","lock","man_made","military","motorcar","name","natural","office","oneway","operator","place","population","power","power_source","public_transport","railway","ref","religion","route","service","shop","sport","surface","toll","tourism","tower:type","tracktype","tunnel","water","waterway","wetland","width","wood","z_order","way_area",way) FROM STDIN
-[ RECORD 5 ]-+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
pid | 10714
backend_start | 2013-12-23 11:09:36.59706+01
xact_start | 2013-12-23 11:09:48.943313+01
query_start | 2013-12-23 11:09:48.943313+01
state_change | 2013-12-23 11:09:48.943316+01
waiting | f
state | active
query | COPY planet_osm_polygon (osm_id,"access","addr:housename","addr:housenumber","addr:interpolation","admin_level","aerialway","aeroway","amenity","area","barrier","bicycle","brand","bridge","boundary","building","construction","covered","culvert","cutting","denomination","disused","embankment","foot","generator:source","harbour","highway","historic","horse","intermittent","junction","landuse","layer","leisure","lock","man_made","military","motorcar","name","natural","office","oneway","operator","place","population","power","power_source","public_transport","railway","ref","religion","route","service","shop","sport","surface","toll","tourism","tower:type","tracktype","tunnel","water","waterway","wetland","width","wood","z_order","way_area",way) FROM STDIN
-[ RECORD 6 ]-+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
pid | 10722
backend_start | 2013-12-23 11:09:36.60013+01
xact_start | 2013-12-23 11:09:56.89275+01
query_start | 2013-12-23 11:09:56.89275+01
state_change | 2013-12-23 11:09:56.892753+01
waiting | f
state | active
query | COPY planet_osm_polygon (osm_id,"access","addr:housename","addr:housenumber","addr:interpolation","admin_level","aerialway","aeroway","amenity","area","barrier","bicycle","brand","bridge","boundary","building","construction","covered","culvert","cutting","denomination","disused","embankment","foot","generator:source","harbour","highway","historic","horse","intermittent","junction","landuse","layer","leisure","lock","man_made","military","motorcar","name","natural","office","oneway","operator","place","population","power","power_source","public_transport","railway","ref","religion","route","service","shop","sport","surface","toll","tourism","tower:type","tracktype","tunnel","water","waterway","wetland","width","wood","z_order","way_area",way) FROM STDIN
-[ RECORD 7 ]-+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
pid | 10731
backend_start | 2013-12-23 11:09:36.603504+01
xact_start | 2013-12-23 11:09:52.780831+01
query_start | 2013-12-23 11:09:52.780831+01
state_change | 2013-12-23 11:09:52.780833+01
waiting | f
state | active
query | COPY planet_osm_polygon (osm_id,"access","addr:housename","addr:housenumber","addr:interpolation","admin_level","aerialway","aeroway","amenity","area","barrier","bicycle","brand","bridge","boundary","building","construction","covered","culvert","cutting","denomination","disused","embankment","foot","generator:source","harbour","highway","historic","horse","intermittent","junction","landuse","layer","leisure","lock","man_made","military","motorcar","name","natural","office","oneway","operator","place","population","power","power_source","public_transport","railway","ref","religion","route","service","shop","sport","surface","toll","tourism","tower:type","tracktype","tunnel","water","waterway","wetland","width","wood","z_order","way_area",way) FROM STDIN
-[ RECORD 8 ]-+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
pid | 10740
backend_start | 2013-12-23 11:09:36.605992+01
xact_start | 2013-12-23 11:10:02.582783+01
query_start | 2013-12-23 11:10:02.582783+01
state_change | 2013-12-23 11:10:02.582786+01
waiting | f
state | active
query | COPY planet_osm_polygon (osm_id,"access","addr:housename","addr:housenumber","addr:interpolation","admin_level","aerialway","aeroway","amenity","area","barrier","bicycle","brand","bridge","boundary","building","construction","covered","culvert","cutting","denomination","disused","embankment","foot","generator:source","harbour","highway","historic","horse","intermittent","junction","landuse","layer","leisure","lock","man_made","military","motorcar","name","natural","office","oneway","operator","place","population","power","power_source","public_transport","railway","ref","religion","route","service","shop","sport","surface","toll","tourism","tower:type","tracktype","tunnel","water","waterway","wetland","width","wood","z_order","way_area",way) FROM STDIN
-[ RECORD 9 ]-+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
pid | 10747
backend_start | 2013-12-23 11:09:36.611401+01
xact_start | 2013-12-23 11:09:48.037928+01
query_start | 2013-12-23 11:09:48.037928+01
state_change | 2013-12-23 11:09:48.037932+01
waiting | f
state | active
query | COPY planet_osm_polygon (osm_id,"access","addr:housename","addr:housenumber","addr:interpolation","admin_level","aerialway","aeroway","amenity","area","barrier","bicycle","brand","bridge","boundary","building","construction","covered","culvert","cutting","denomination","disused","embankment","foot","generator:source","harbour","highway","historic","horse","intermittent","junction","landuse","layer","leisure","lock","man_made","military","motorcar","name","natural","office","oneway","operator","place","population","power","power_source","public_transport","railway","ref","religion","route","service","shop","sport","surface","toll","tourism","tower:type","tracktype","tunnel","water","waterway","wetland","width","wood","z_order","way_area",way) FROM STDIN
```
---
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/issues/112#issuecomment-31125787
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20131223/a0cfcf1b/attachment-0001.html>
More information about the Tile-serving
mailing list