[Tile-serving] [osm2pgsql-dev/osm2pgsql] Some research on middle performance (Issue #2110)

[rouen-sk]

I believe using postgres as node cache was bad choice in the first place, and minor optimizations wont "save us" here. 
Postgres has way too much overhead for this purpose, in terms of time, storage space and IOPS. Simple high-performance key-value store library, such as LevelDB, would outperform it significantly.

For example, Imposm is using LevelDB for node cache, and on my machine (cloud VM, so poor IOPS) building node cache takes about 1 hour, and 160 GB of disk space. osm2pgsql slim mode (with 50GB RAM cache) takes several hours, and I honestly don't know how much disk space, because I tried the latest planet yesterday, and it crashed on low disk space, with about 550 GB free at the start. 

-- 
Reply to this email directly or view it on GitHub:
https://github.com/osm2pgsql-dev/osm2pgsql/issues/2110#issuecomment-1845068275
You are receiving this because you are subscribed to this thread.

Message ID: <osm2pgsql-dev/osm2pgsql/issues/2110/1845068275 at github.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20231207/9e4d9d34/attachment.htm>