[Geocoding] Nominatim script log output - how to tell progress?

[Simon Nuttall]

On 17 June 2015 at 22:45, Sarah Hoffmann <lonvia at denofr.de> wrote:
> Hi,
>
> On Tue, Jun 16, 2015 at 01:59:50PM +0100, Simon Nuttall wrote:
>> We've got a dedicated server with 96GB RAM, and 4 cpus, and so I've
>> updated our Nominatim installation script for Ubuntu 14.04 LTS:
>>
>>  https://github.com/cyclestreets/nominatim-install
>>
>> I'm running it for a full planet and current output from the script looks like:
>>
>>   Done 1014482 in 8666 @ 117.064621 per second - Rank 26 ETA
>> (seconds): 431857.781250
>>   Done 1014572 in 8667 @ 117.061501 per second - Rank 26 ETA
>> (seconds): 431868.531250
>>   Done 1014572 in 8667 @ 117.061501 per second - Rank 26 ETA
>> (seconds): 431868.531250
>>   Slow place_id 134595635
>>   Slow place_id 134623932
>>   Done 1014586 in 8668 @ 117.049606 per second - Rank 26 ETA
>> (seconds): 431912.281250
>>   Slow place_id 134622856
>>   Done 1014599 in 8668 @ 117.051109 per second - Rank 26 ETA
>> (seconds): 431906.656250
>>   Done 1014599 in 8669 @ 117.037605 per second - Rank 26 ETA
>> (seconds): 431956.468750
>>   Done 1014605 in 8669 @ 117.038300 per second - Rank 26 ETA
>> (seconds): 431953.843750
>>
>> I think this means it is in the osm2pgsql stage updating the downloaded planet?
>
> This is the indexing stage, osm2pgsql has already finished.
>
>> How can I tell how well it is doing on this?
>
> For the indexing stage you need to know that it goes through
> rank 0-30 and that only rank 26 and 30 take significant time,
> each about 50%. So use the ETA and done time (the one in front
> of the @) to compute the total time for rank 26, double that
> and you have the time the indexing is going to take in total.
> I your case about 240h or 10 days.
>
>> How do I get a sense of what is going on and how well it is performing?
>
> Difficult because it always depends on the hardware. If you do
> the import on spinning disks, then performance is good. If you have
> SSDs, then you could do better by a factor of 3 to 4.
>
> Sarah


The Nominatim build I started  on 13 June has is now showing:

  Done 73137484 in 821606 @ 89.017715 per second - ETA (seconds): -8.357887
  Done 73138021 in 822413 @ 88.931015 per second - ETA (seconds): -14.404424
  Done 73138021 in 822413 @ 88.931015 per second - FINISHED

Search indices
CREATE INDEX


Any idea how much more there is to go?


-- 
Simon Nuttall

Route Master, CycleStreets.net