[Tile-serving] [openstreetmap/osm2pgsql] Long import of Europe data file (#676)

[Koles500]

Alright, it has finished and first generated tiles look good ! Below is full log. Took almost 15 days ;0 :)

```
PS C:\OSM\rendering> osm2pgsql -U postgres -m -d osm -p osm -E 3857 -s -S C:\OSM\osm2pgsql\default.style C:\OSM\Data\eur
ope-latest.osm.pbf
osm2pgsql version 0.92.0 (64 bit id space)

Using built-in tag processing pipeline
Using projection SRS 3857 (Spherical Mercator)
Setting up table: osm_point
Setting up table: osm_line
Setting up table: osm_polygon
Setting up table: osm_roads
Allocating memory for sparse node cache
Node-cache: cache=800MB, maxblocks=12800*65536, allocation method=9
Mid: pgsql, scale=100 cache=800
Setting up table: osm_nodes
Setting up table: osm_ways
Setting up table: osm_rels

Reading in file: C:\OSM\Data\europe-latest.osm.pbf
Using PBF parser.
Processing: Node(1827652k 131.7k/s) Way(223763k 0.78k/s) Relation(3148140 15.84/s)
Standard exception processing relation id=6540291: TopologyException: side location conflict at -85245.029999999999 6666
499.7699999996
Processing: Node(1827652k 131.7k/s) Way(223763k 0.78k/s) Relation(3254340 15.98/s)
Standard exception processing relation id=6752256: TopologyException: side location conflict at 538047.93000000005 56717
07.1900000004
Processing: Node(1827652k 131.7k/s) Way(223763k 0.78k/s) Relation(3316870 16.07/s)  parse time: 507324s
Node stats: total(1827652739), max(4604487172) in 13878s
Way stats: total(223763746), max(465494091) in 287105s
Relation stats: total(3316871), max(6875737) in 206341s
Committing transaction for osm_point
Committing transaction for osm_line
Committing transaction for osm_polygon
Committing transaction for osm_roads
Setting up table: osm_nodes
Setting up table: osm_ways
Setting up table: osm_rels
Using built-in tag processing pipeline
Setting up table: osm_nodes
Setting up table: osm_ways
Setting up table: osm_rels
Using built-in tag processing pipeline
Setting up table: osm_nodes
Setting up table: osm_ways
Setting up table: osm_rels
Using built-in tag processing pipeline
Setting up table: osm_nodes
Setting up table: osm_ways
Setting up table: osm_rels
Using built-in tag processing pipeline
Setting up table: osm_nodes
Setting up table: osm_ways
Setting up table: osm_rels
Using built-in tag processing pipeline
Setting up table: osm_nodes
Setting up table: osm_ways
Setting up table: osm_rels
Using built-in tag processing pipeline
Setting up table: osm_nodes
Setting up table: osm_ways
Setting up table: osm_rels
Using built-in tag processing pipeline
Setting up table: osm_nodes
Setting up table: osm_ways
Setting up table: osm_rels
Using built-in tag processing pipeline

Going over pending ways...
        165944299 ways are pending

Using 8 helper-processes
Finished processing 165944299 ways in 200997 s

165944299 Pending ways took 200997s at a rate of 825.61/s
Committing transaction for osm_point
Committing transaction for osm_line
Committing transaction for osm_polygon
Committing transaction for osm_roads
Committing transaction for osm_point
Committing transaction for osm_line
Committing transaction for osm_polygon
Committing transaction for osm_roads
Committing transaction for osm_point
Committing transaction for osm_line
Committing transaction for osm_polygon
Committing transaction for osm_roads
Committing transaction for osm_point
Committing transaction for osm_line
Committing transaction for osm_polygon
Committing transaction for osm_roads
Committing transaction for osm_point
Committing transaction for osm_line
Committing transaction for osm_polygon
Committing transaction for osm_roads
Committing transaction for osm_point
Committing transaction for osm_line
Committing transaction for osm_polygon
Committing transaction for osm_roads
Committing transaction for osm_point
Committing transaction for osm_line
Committing transaction for osm_polygon
Committing transaction for osm_roads
Committing transaction for osm_point
Committing transaction for osm_line
Committing transaction for osm_polygon
Committing transaction for osm_roads

Going over pending relations...
        0 relations are pending

Using 8 helper-processes
Finished processing 0 relations in 0 s

Committing transaction for osm_point
WARNING:  there is no transaction in progress
Committing transaction for osm_line
WARNING:  there is no transaction in progress
Committing transaction for osm_polygon
WARNING:  there is no transaction in progress
Committing transaction for osm_roads
WARNING:  there is no transaction in progress
Committing transaction for osm_point
WARNING:  there is no transaction in progress
Committing transaction for osm_line
WARNING:  there is no transaction in progress
Committing transaction for osm_polygon
WARNING:  there is no transaction in progress
Committing transaction for osm_roads
WARNING:  there is no transaction in progress
Committing transaction for osm_point
WARNING:  there is no transaction in progress
Committing transaction for osm_line
WARNING:  there is no transaction in progress
Committing transaction for osm_polygon
WARNING:  there is no transaction in progress
Committing transaction for osm_roads
WARNING:  there is no transaction in progress
Committing transaction for osm_point
WARNING:  there is no transaction in progress
Committing transaction for osm_line
WARNING:  there is no transaction in progress
Committing transaction for osm_polygon
WARNING:  there is no transaction in progress
Committing transaction for osm_roads
WARNING:  there is no transaction in progress
Committing transaction for osm_point
WARNING:  there is no transaction in progress
Committing transaction for osm_line
WARNING:  there is no transaction in progress
Committing transaction for osm_polygon
WARNING:  there is no transaction in progress
Committing transaction for osm_roads
WARNING:  there is no transaction in progress
Committing transaction for osm_point
WARNING:  there is no transaction in progress
Committing transaction for osm_line
WARNING:  there is no transaction in progress
Committing transaction for osm_polygon
WARNING:  there is no transaction in progress
Committing transaction for osm_roads
WARNING:  there is no transaction in progress
Committing transaction for osm_point
WARNING:  there is no transaction in progress
Committing transaction for osm_line
WARNING:  there is no transaction in progress
Committing transaction for osm_polygon
WARNING:  there is no transaction in progress
Committing transaction for osm_roads
WARNING:  there is no transaction in progress
Committing transaction for osm_point
WARNING:  there is no transaction in progress
Committing transaction for osm_line
WARNING:  there is no transaction in progress
Committing transaction for osm_polygon
WARNING:  there is no transaction in progress
Committing transaction for osm_roads
WARNING:  there is no transaction in progress
Sorting data and creating indexes for osm_point
Sorting data and creating indexes for osm_roads
Sorting data and creating indexes for osm_polygon
Sorting data and creating indexes for osm_line
Stopping table: osm_nodes
Stopped table: osm_nodes in 0s
Stopping table: osm_ways
Building index on table: osm_ways
Stopping table: osm_rels
Building index on table: osm_rels
Copying osm_roads to cluster by geometry finished
Creating geometry index on osm_roads
Creating osm_id index on osm_roads
Creating indexes on osm_roads finished
All indexes on osm_roads created in 13205s
Completed osm_roads
Copying osm_point to cluster by geometry finished
Creating geometry index on osm_point
Creating osm_id index on osm_point
Creating indexes on osm_point finished
All indexes on osm_point created in 30703s
Completed osm_point
Stopped table: osm_rels in 51844s
Copying osm_line to cluster by geometry finished
Creating geometry index on osm_line
Creating osm_id index on osm_line
Creating indexes on osm_line finished
All indexes on osm_line created in 103561s
Completed osm_line
Copying osm_polygon to cluster by geometry finished
Creating geometry index on osm_polygon
Creating osm_id index on osm_polygon
Creating indexes on osm_polygon finished
All indexes on osm_polygon created in 324884s
Completed osm_polygon
Stopped table: osm_ways in 562079s
node cache: stored: 52428801(2.87%), storage efficiency: 50.00% (dense blocks: 0, sparse nodes: 52428801), hit rate: -6.
78%

Osm2pgsql took 1270744s overall
```

-- 
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/issues/676#issuecomment-277763521
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20170206/c7fbecc6/attachment-0001.html>