[Tile-serving] [openstreetmap/osm2pgsql] --number-processes value not used in create or append? (Issue #1973)

wgds60 notifications at github.com
Tue Jun 6 09:05:27 UTC 2023


<!-- Please don't use screenshots. Copy and paste the *text* output here if that's needed for context. -->

## What version of osm2pgsql are you using?

<!-- Paste output of `osm2pgsql --version` here. Please use the [latest
release](https://osm2pgsql.org/releases/) of osm2pgsql if at all
possible. -->

```
2023-06-06 09:00:48  osm2pgsql version 1.8.1
Build: RelWithDebInfo
Compiled using the following library versions:
Libosmium 2.19.0
Proj [API 6] 8.2.1
Lua 5.1.4 (LuaJIT 2.1.0-beta3)

```
## What operating system and PostgreSQL/PostGIS version are you using?

<!-- Also what Linux distribution if applicable, OS version? -->

Ubuntu 22.04.
Postgresql 15.3
Postgis 3.3.

## Tell us something about your system

<!-- How much RAM do you have, how many CPUs, bare metal or cloud setup? -->

16 threads CPU (bare metal)
256 GB RAM
~ 2TB NVMe space on Raid 1 for postgresql and flat-node file.

## What did you do exactly?

<!-- Please provide the command(s) you used including all options etc. Include
links to input files. -->

`osm2pgsql --slim -d <dbname> --number-processes=10 --hstore --flat-nodes=<flatnode-file>  --style=<osm-carto-style-file>--tag-transform-script=<osm-carto-lua-file> -C 64000 <planet-file>`

## What did you expect to happen?

<!-- Describe in detail what you expected the above would do. -->

As per https://osm2pgsql.org/doc/manual.html#preparing-the-database (3.7. Number of Connections):

"reduce the number of threads with the --number-processes=THREADS command line option of osm2pgsql."

from the manual I would expect osm2pgsql to use 10 threads for the command line option `--number-processes=10` used. 

## What did happen instead?

<!-- Please describe what happened and why you think this is wrong. Please include
(or link to, if it is too verbose) the log output. -->

But instead osm2pgsql uses 1 thread per logical core, in this case 16 threads.

## What did you do to try analyzing the problem?

<!-- Describe what steps you already did to try analyzing the problem before
reporting. -->

This does not seem to be a problem at create operation (the command above) but may be a reason for the --append operation to be really slow (see problem discussed here: https://github.com/openstreetmap/osm2pgsql/discussions/1971 ).



-- 
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/osm2pgsql/issues/1973
You are receiving this because you are subscribed to this thread.

Message ID: <openstreetmap/osm2pgsql/issues/1973 at github.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/tile-serving/attachments/20230606/85975cc9/attachment.htm>


More information about the Tile-serving mailing list