[Taginfo-dev] Problem updating to latest taginfo

Tue Jan 5 08:46:06 UTC 2021

Hi!

okay, there were several problems in my code:
1. The script should have failed and stopped completely after the first
   error.
2. The script should not fail in the first place accessing the
   non-existing taginfo-chronology.db

I have fixed those two problems. Please try again with newest master.

Please also have a look at the example config provided in the repository
and compare it with your config to make sure you have everything set up
correctly. There were some changes in the config.

Jochen

On Tue, Jan 05, 2021 at 12:06:14AM +0000, Shaun McDonald via Taginfo-dev wrote:
> Date: Tue, 5 Jan 2021 00:06:14 +0000
> From: Shaun McDonald via Taginfo-dev <taginfo-dev at openstreetmap.org>
> To: Shaun McDonald via Taginfo-dev <taginfo-dev at openstreetmap.org>
> Subject: Re: [Taginfo-dev] Problem updating to latest taginfo
> 
> Hi Jochen,
> 
> The tail end of the log shows this:
> 
> 2020-12-31T08:33:30 | 5 | all | compress_source_databases | Done.
> 2020-12-31T08:33:30 | 5 | all | create_extra_indexes | Creating extra indexes...
> 2020-12-31T08:33:30 | 5 | all | create_extra_indexes | Running SQL script '/home/shaunmcdonald/taginfo-gb/taginfo/sources/db/add_extra_indexes.sql' on database '/home/shaunmcdonald/taginfo-gb/build/db/taginfo-db.db'...
> 2020-12-31T08:33:54 | 5 | all | create_extra_indexes | Running SQL script '/home/shaunmcdonald/taginfo-gb/taginfo/sources/db/add_ftsearch.sql' on database '/home/shaunmcdonald/taginfo-gb/build/db/taginfo-db.db'...
> 2020-12-31T08:34:20 | 6 | all | create_extra_indexes | Done.
> 2020-12-31T08:34:20 | 6 | all | update_master | Running master/update.sh...
> 2020-12-31T08:34:20 | 6 | master | main | Start master...
> 2020-12-31T08:34:20 | 6 | master | create_master_database | Running SQL script '/home/shaunmcdonald/taginfo-gb/taginfo/sources/master/languages.sql' on database '/home/shaunmcdonald/taginfo-gb/build/taginfo-master.db'...
> 2020-12-31T08:34:20 | 6 | master | create_master_database | Running SQL script '/home/shaunmcdonald/taginfo-gb/taginfo/sources/master/master.sql' on database '/home/shaunmcdonald/taginfo-gb/build/taginfo-master.db'...
> 2020-12-31T08:34:20 | 6 | master | create_master_database |   with parameter: DIR=/home/shaunmcdonald/taginfo-gb/build
> Error: near line 13: unable to open database: /home/shaunmcdonald/taginfo-gb/build/chronology/taginfo-chronology.db
> 2020-12-31T08:34:20 | 6 | master | create_selection_database | Running SQL script '/home/shaunmcdonald/taginfo-gb/taginfo/sources/master/selection.sql' on database '/home/shaunmcdonald/taginfo-gb/build/selection.db'...
> 2020-12-31T08:34:20 | 6 | master | create_selection_database |   with parameter: DIR=/home/shaunmcdonald/taginfo-gb/build
> 2020-12-31T08:34:20 | 6 | master | create_selection_database |   with parameter: MIN_COUNT_FOR_MAP=1000
> 2020-12-31T08:34:20 | 6 | master | create_selection_database |   with parameter: MIN_COUNT_TAGS=10000
> 2020-12-31T08:34:20 | 6 | master | create_selection_database |   with parameter: MIN_COUNT_RELATIONS_PER_TYPE=100
> 2020-12-31T08:34:20 | 6 | master | create_selection_database | Selection database contents:
>   interesting_tags count .........: 572
>   frequent_tags count ............: 2375
>   interesting_relation_types count: 20
> 2020-12-31T08:34:20 | 6 | master | update_history_database | Running SQL script '/home/shaunmcdonald/taginfo-gb/taginfo/sources/master/history_update.sql' on database '/home/shaunmcdonald/taginfo-gb/build/taginfo-history.db'...
> 2020-12-31T08:34:20 | 6 | master | update_history_database |   with parameter: DIR=/home/shaunmcdonald/taginfo-gb/build
> Error: near line 11: no such table: master.master_stats
> 2020-12-31T08:34:20 | 6 | master | main | Done master.
> 2020-12-31T08:34:20 | 6 | all | update_master | Done.
> 2020-12-31T08:34:20 | 6 | all | compress_extra_databases | Compressing all extra databases...
> 2020-12-31T08:34:20 | 6 | all | compress_file | Compressing 'taginfo-master' to 'master' using 'bzip2'
> 2020-12-31T08:34:20 | 6 | all | compress_file | Compressing 'taginfo-history' to 'history' using 'bzip2'
> 2020-12-31T08:34:21 | 6 | all | compress_extra_databases | Done.
> 2020-12-31T08:34:21 | 6 | all | main | Done update_all.
> 
> Note the 2 Error lines. I’m wondering if they would be relevant?
> 
> I’m using the Geofabrik Great Britain PBF file. The other option is there’s some further changes to the config that I need to do… I’m not using the newer history files.
> 
> Shaun
> 
> > On 31 Dec 2020, at 15:27, Jochen Topf <jochen at remote.org> wrote:
> > 
> > Hi Shaun,
> > 
> > "sources" is a table in the master database. If that isn't there it
> > means your update scripts didn't run through correctly. So you should
> > look into the logs for those to see what happened.
> > 
> > Jochen
> > 
> > On Wed, Dec 30, 2020 at 05:19:07PM +0000, Shaun McDonald via Taginfo-dev wrote:
> >> Date: Wed, 30 Dec 2020 17:19:07 +0000
> >> From: Shaun McDonald via Taginfo-dev <taginfo-dev at openstreetmap.org <mailto:taginfo-dev at openstreetmap.org>>
> >> To: Jochen Topf via Taginfo-dev <taginfo-dev at openstreetmap.org <mailto:taginfo-dev at openstreetmap.org>>
> >> Subject: [Taginfo-dev] Problem updating to latest taginfo
> >> 
> >> Hi Jochen,
> >> 
> >> I’m updating the GB taginfo to the latest OS (from Ubuntu LTS 18.04 to 20.04) and taginfo code. 
> >> 
> >> I’m using nginx and passenger and getting the following error:
> >> 
> >> [ E 2020-12-30 16:56:06.4398 39087/T8 age/Cor/Con/CheckoutSession.cpp:274 ]: [Client 2-145] Cannot checkout session because a spawning error occurred. The identifier of the error is ecfe71c7. Please see earlier logs for details about the error.
> >> App 80895 output: Error: The application encountered the following error: no such table: sources (SQLite3::SQLException)
> >> App 80895 output:     /var/lib/gems/2.7.0/gems/sqlite3-1.4.2/lib/sqlite3/database.rb:147:in `initialize'
> >> App 80895 output:     /var/lib/gems/2.7.0/gems/sqlite3-1.4.2/lib/sqlite3/database.rb:147:in `new'
> >> App 80895 output:     /var/lib/gems/2.7.0/gems/sqlite3-1.4.2/lib/sqlite3/database.rb:147:in `prepare'
> >> App 80895 output:     /var/lib/gems/2.7.0/gems/sqlite3-1.4.2/lib/sqlite3/database.rb:193:in `execute'
> >> App 80895 output:     /home/shaunmcdonald/taginfo-gb/taginfo/web/lib/sql.rb:79:in `block in execute'
> >> App 80895 output:     /home/shaunmcdonald/taginfo-gb/taginfo/web/lib/sql.rb:61:in `wrap_query'
> >> App 80895 output:     /home/shaunmcdonald/taginfo-gb/taginfo/web/lib/sql.rb:78:in `execute'
> >> App 80895 output:     /home/shaunmcdonald/taginfo-gb/taginfo/web/lib/sql.rb:223:in `execute'
> >> App 80895 output:     /home/shaunmcdonald/taginfo-gb/taginfo/web/lib/sql.rb:15:in `init'
> >> App 80895 output:     /home/shaunmcdonald/taginfo-gb/taginfo/web/taginfo.rb:68:in `<top (required)>'
> >> App 80895 output:     /usr/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require'
> >> App 80895 output:     /usr/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:92:in `require'
> >> App 80895 output:     config.ru:8:in `block in <main>'
> >> App 80895 output:     /var/lib/gems/2.7.0/gems/rack-2.2.3/lib/rack/builder.rb:125:in `instance_eval'
> >> App 80895 output:     /var/lib/gems/2.7.0/gems/rack-2.2.3/lib/rack/builder.rb:125:in `initialize'
> >> App 80895 output:     config.ru:1:in `new'
> >> App 80895 output:     config.ru:1:in `<main>'
> >> App 80895 output:     /usr/share/passenger/helper-scripts/rack-preloader.rb:101:in `eval'
> >> App 80895 output:     /usr/share/passenger/helper-scripts/rack-preloader.rb:101:in `preload_app'
> >> App 80895 output:     /usr/share/passenger/helper-scripts/rack-preloader.rb:189:in `block in <module:App>'
> >> App 80895 output:     /usr/lib/ruby/vendor_ruby/phusion_passenger/loader_shared_helpers.rb:382:in `run_block_and_record_step_progress'
> >> App 80895 output:     /usr/share/passenger/helper-scripts/rack-preloader.rb:188:in `<module:App>'
> >> App 80895 output:     /usr/share/passenger/helper-scripts/rack-preloader.rb:30:in `<module:PhusionPassenger>'
> >> App 80895 output:     /usr/share/passenger/helper-scripts/rack-preloader.rb:29:in `<main>'
> >> [ E 2020-12-30 16:56:27.7325 39087/T13w age/Cor/App/Implementation.cpp:221 ]: Could not spawn process for application /home/shaunmcdonald/taginfo-gb/taginfo/web: The application encountered the following error: no such table: sources (SQLite3::SQLException)
> >>  Error ID: d0b53fe5
> >>  Error details saved to: /tmp/passenger-error-88c2fN.html
> >> 
> >> I’m wonder if this is something you’ve come across before and what it could be? What’s forced me to get around to updating the code is the updating stopped mid November, potentially due to an upstream change in the download data.
> >> 
> >> It’s the first update of the code in probably a year or so. I’ve not enabled the new chronology things.
> >> 
> >> The hill shape on https://taginfo.openstreetmap.org/tags/highway=road#chronology <https://taginfo.openstreetmap.org/tags/highway=road#chronology> <https://taginfo.openstreetmap.org/tags/highway=road#chronology <https://taginfo.openstreetmap.org/tags/highway=road#chronology>> makes it rather interesting as shows the rise and fall of the usage of the tag, so will look into it at some point. Will need to find out how to keep a limited country OSM history file up to date.
> >> 
> >> Thanks
> >> Shaun
> > 
> >> _______________________________________________
> >> Taginfo-dev mailing list
> >> Taginfo-dev at openstreetmap.org <mailto:Taginfo-dev at openstreetmap.org>
> >> https://lists.openstreetmap.org/listinfo/taginfo-dev <https://lists.openstreetmap.org/listinfo/taginfo-dev>
> > 
> > 
> > -- 
> > Jochen Topf  jochen at remote.org <mailto:jochen at remote.org>  https://www.jochentopf.com/ <https://www.jochentopf.com/>  +49-351-31778688
> > 
> > _______________________________________________
> > Taginfo-dev mailing list
> > Taginfo-dev at openstreetmap.org <mailto:Taginfo-dev at openstreetmap.org>
> > https://lists.openstreetmap.org/listinfo/taginfo-dev <https://lists.openstreetmap.org/listinfo/taginfo-dev>

> _______________________________________________
> Taginfo-dev mailing list
> Taginfo-dev at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/taginfo-dev

-- 
Jochen Topf  jochen at remote.org  https://www.jochentopf.com/  +49-351-31778688