[Tile-serving] [openstreetmap/osm2pgsql] Fix: Make sure index is used for type/id indexes in flex output (PR #2007)

mboeringa notifications at github.com
Thu Jul 20 20:28:08 UTC 2023


Did this issue only affect --slim import, not **non-slim** ones? 

I am asking because I noticed on [some recent tests](https://wiki.openstreetmap.org/wiki/Osm2pgsql/benchmarks#Planet_(75.0GB_PBF,_2023-07-10)_on_HP_Z840_workstation,_512GB_DDR4_2133,_2x_Xeon_E5-2699v4_22C,_5x_Samsung_970_EVO_Plus_2TB_NVMe_RAID_0,_4x_Intel_OPTANE_P1600X_110GB_NVMe_RAID_0,_Ubuntu_22.04_~8h) with a full Planet import, that the _relation_ stage loading is about 2-3 faster in **non-slim** mode (5k/s, versus 2k/s with --slim), where everything is stored in RAM.

Now the fact that everything is stored in RAM may be the main cause of the speed up in relation processing, but I would in that case at least also have expected some speed up of the way processing. However, in reality, I see a consistently slightly slower (by about 10%-15%) way loading stage (+/- 83k/s to 70k/s) in **non-slim** mode, which slightly surprises me as unexpected with all data in RAM. 

Note that this is a comparison with --slim mode import using also a --flat-node file, so maybe the specific structure of --flat-node is in fact more efficient than pulling nodes from RAM for way building on high performance NVMe backed hardware??

-- 
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/pull/2007#issuecomment-1644557665
You are receiving this because you are subscribed to this thread.

Message ID: <openstreetmap/osm2pgsql/pull/2007/c1644557665 at github.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20230720/4fed0626/attachment.htm>


More information about the Tile-serving mailing list