[Tile-serving] [openstreetmap/osm2pgsql] Parallel index creation, to minimize required time (and I/O) (#1562)

[Christian Quest]

It looks like index creation on one table are currently done sequentially:
- geometry
- osm_id (if slim mode which I suppose is a very common case)
- hstore (if asked for)

https://github.com/openstreetmap/osm2pgsql/blob/b1a634a0ed3c0f15fce4d5c18e1eb46eab413e6c/src/table.cpp#L231

Creating index on a table means reading the whole table data sequentially, so creating several index on the same table benefits a lot of doing them in parallel, as data has high chance to still be in some cache and requires no additional read I/O.
When creating them sequentially on large tables do not benefit from read cache when it is smaller than the data to read (which is usually the case).

I'm usually creating additionnal index in parallel and it takes not much more than the longest one in most cases.

I have no idea if this is easy to implement or not...

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/discussions/1562
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20210831/05d3e920/attachment.htm>