[Geocoding] Nominatim script log output - how to tell progress?

[Sarah Hoffmann]

Hi,

On Tue, Jun 16, 2015 at 01:59:50PM +0100, Simon Nuttall wrote:
> We've got a dedicated server with 96GB RAM, and 4 cpus, and so I've
> updated our Nominatim installation script for Ubuntu 14.04 LTS:
> 
>  https://github.com/cyclestreets/nominatim-install
> 
> I'm running it for a full planet and current output from the script looks like:
> 
>   Done 1014482 in 8666 @ 117.064621 per second - Rank 26 ETA
> (seconds): 431857.781250
>   Done 1014572 in 8667 @ 117.061501 per second - Rank 26 ETA
> (seconds): 431868.531250
>   Done 1014572 in 8667 @ 117.061501 per second - Rank 26 ETA
> (seconds): 431868.531250
>   Slow place_id 134595635
>   Slow place_id 134623932
>   Done 1014586 in 8668 @ 117.049606 per second - Rank 26 ETA
> (seconds): 431912.281250
>   Slow place_id 134622856
>   Done 1014599 in 8668 @ 117.051109 per second - Rank 26 ETA
> (seconds): 431906.656250
>   Done 1014599 in 8669 @ 117.037605 per second - Rank 26 ETA
> (seconds): 431956.468750
>   Done 1014605 in 8669 @ 117.038300 per second - Rank 26 ETA
> (seconds): 431953.843750
> 
> I think this means it is in the osm2pgsql stage updating the downloaded planet?

This is the indexing stage, osm2pgsql has already finished.

> How can I tell how well it is doing on this?

For the indexing stage you need to know that it goes through
rank 0-30 and that only rank 26 and 30 take significant time,
each about 50%. So use the ETA and done time (the one in front
of the @) to compute the total time for rank 26, double that
and you have the time the indexing is going to take in total.
I your case about 240h or 10 days.

> How do I get a sense of what is going on and how well it is performing?

Difficult because it always depends on the hardware. If you do
the import on spinning disks, then performance is good. If you have
SSDs, then you could do better by a factor of 3 to 4.

Sarah