[OSM-dev] Slow osmosis import

[Marco Boeringa]

Hi,

Building ways and relations requires fast random access, not sequential 
read / write speed. I think it likely your HDD raid is the culprit, as 
the 96 RAM won't allow you to process everything in RAM. All of the 
recent osm2pgsql benchmarks with high throughput for building ways and 
relations I have seen assume SSDs, and preferably even NVMe if you can 
afford it.

It is likely even a basic USB 3 connected 4TB SATA SSD will give you 
better results.

Marco

Op 5-10-2019 om 00:47 schreef merspieler:
> I've wanted to use osm2pgsql but the schema is a different one.
> The software [1] I'm going to use the db with only supports the osmosis one.
>
> As for the hardware:
> 2x Xeon E5 8 cores/16 threads
> 96GB ram
> 5x 4TB HDD in a RAIDZ2
>
> I've done some benchmarking of the raid and osmosis doesn't even reach
> 5% of what was possible with the benchmark.
>
> I haven't tried Imposm yet... does it work with the osmosis schema?
>
> [1] https://gitlab.com/fg-radi/osm2city
>
> Frederik Ramm:
>> Hi,
>>
>> first question: are you absolutely sure you need an Osmosis import -
>> does your use case not work with an osm2pgsql import?
>>
>> Best
>> Frederik
>>
> _______________________________________________
> dev mailing list
> dev at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/dev