[Tile-serving] [openstreetmap/osm2pgsql] Planet load fails with ERROR: relation "osm_point" does not exist (#855)

[mboeringa]

> I am doing a planet import on an EC2 instance with 16G of RAM. The machine is not doing any other work. I'm using --cache 8000 and these PG settings (mostly taken from pgtune):

8000 is not even near enough RAM to store all nodes for the planet. Based on some tests with Europe I recently did, I estimate you need at least 48GB of RAM to store all nodes for planet in memory.

Unless you cannot afford SSD storage, I would thus strongly recommend to use the **--flat-nodes** option instead which stores the nodes on disk instead of in RAM, and drop the -C cache option of the osm2pgsql command line, and let osm2pgsql itself figure out its optimal processing settings. This is best done with SSD storage.

I managed to load Europe, which is about half planet, on a 12GB RAM Virtualbox instance, and I am currently re-running the same import with an even lower 8GB RAM setting for the VM. Since it already passed the node load stage (about 2 billion of them for Europe), I am pretty sure it will finish the import even with this reduced amount of RAM. As a consequence, I am also deeply convinced the same **--flat-nodes** and 8GB RAM setup will import planet fine, although I haven't tested this yet. However, if it can do Europe with such low RAM settings, I see no reason why it wouldn't be able to load planet, as it is just about 2-2.5 times bigger.

I actually see osm2pgsql defaulting to only 800MB cache when I start the command with --flat-nodes.

Of course, if you only have HDD storage and no SSD, trading the cost for more RAM (so probably the 48GB as I estimate) is probably the only way to get acceptable performance on your instance, as using --flat-nodes and storing all nodes on disk on HDD is likely very slow.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/issues/855#issuecomment-405849443
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20180718/ea0d66a3/attachment.html>