[Tile-serving] [openstreetmap/osm2pgsql] COPY_END for planet_osm_polygon failed: ERROR: out of memory (#846)

[Hartmut Holzgraefe]

After upgrading my server to Ubuntu 18.04, and PostgreSQL/PostGIS from 9.6/2.3 to 10/2.4 with the help of  pg_upgrade and PostGIS specific instructions found in 

http://www.bostongis.com/blog/index.php?/archives/268-Using-pg_upgrade-to-upgrade-PostGIS-without-installing-an-older-version-of-PostGIS.html

I see one osm2pgsql running for a very long time and eventually postgres process eventually running out of memory even when applying a single minutely diff file only.

Originally I got:

```
[...]
Using XML parser.
[13.2K blob data]
Osm2pgsql failed due to ERROR: result COPY_END for planet_osm_polygon failed: ERROR:  out of memory
DETAIL:  Failed on request of size 40.
CONTEXT:  COPY planet_osm_hstore_polygon, line 1
```

I found some reports where this was caused by invalid polygons in the database, so I tried:

`DELETE FROM planet_osm_hstore_polygon WHERE NOT st_isvalid(way);`

This only made things fail with the same message in a later stage now:

```
Going over pending ways...
	67 ways are pending

Using 6 helper-processes
node cache: stored: 911(100.00%), storage efficiency: 0.10% (dense blocks: 117, sparse nodes: 0), hit rate: 5.36%
Osm2pgsql failed due to ERROR: result COPY_END for planet_osm_polygon failed: ERROR:  out of memory
DETAIL:  Failed on request of size 16.
CONTEXT:  COPY planet_osm_polygon, line 1
```

osm2pgsql was built from latest git source, most recent changeset being 48a45c6e605a6e3477f951aa2c06589d4fd638bb


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/issues/846
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20180430/7097dc73/attachment.html>