[Tile-serving] [openstreetmap/osm2pgsql] OOM killed while building index on table planet_osm_rels (#834)

[Josip Rodin]

Hm, good catch, I missed that sentence. It should be separated into its own paragraph, and put into better context if people are to actually pay attention.

For example, is the import file size meant to match PBF or the other format? The difference in case of europe-latest is 20 GB vs 32 GB, so it's a significant difference.

Then, I notice there's a note about the parameter being "optional but strongly recommended for planet imports". This kind of a prescriptive statement is essentially useless, it would be better if it was descriptive instead. For example, it should simply say that the use of -C is meant to improve the speed of import, but if it's set too high, it can prevent the process from completing because the machine has to support both the normal operation of osm2pgsql, postgresql, whatever other processes, and some amount of contingency.

When mentioning that 2/3 rule of thumb, mention how swap can factor in. (Indeed, after I had added swap, my -C value became almost exactly 2/3. D'oh!)

And, some explanation of how exactly things change if one strays from the rule of thumb would be helpful. If you have a 128 GB RAM machine, does it make sense to leave 42 GB free at all times; if you have a 16 GB RAM machine, does it make sense to even try importing a 16 GB file or will it take forever?

Furthermore, the Hardware requirements section has similar problems. It mentions a minimum amount of RAM, which is probably a useless factoid because it's not even correlated to a dataset size. (You can probably have Luxembourg at 2 GB, but not Germany, right?)

It then mentions a full planet import in a slightly mangled sentence - word order should be fixed - more importantly there's no explanation for what's inbetween.

Later, there are some more examples, but again focusing only on the full import, no elaboration on how it scales. Even just a link to the osm2pgsql benchmarks wiki page would be useful, because there one can glean the numbers of nodes, ways and relations in various examples, and at least to try to extrapolate.


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/issues/834#issuecomment-376703588
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20180327/2668b9d2/attachment.html>