[Taginfo-dev] Problem running taginfo GB since 2023-10-04

Fri Nov 17 08:25:23 UTC 2023

Hi!

it complaines about the FROM in line 83:
UPDATE project_unique_keys SET in_wiki = lang_count FROM wiki.wikipages_keys w WHERE project_unique_keys.key = w.key;

There were some changes to this area of the code around the date you
mentioned, so that fits.

Looks like your version of Sqlite doesn't support UPDATE FROM which was
introduced in version 3.33.0 more than three years ago. Which version
are use using? (The versions in Debian 11 and 12, and Ubuntu 22.04
should be okay, but Ubuntu 20.04 is no longer supported.)

I will update the README to document the new version needed.

Jochen

On Fri, Nov 17, 2023 at 08:03:41AM +0000, Shaun McDonald wrote:
> Date: Fri, 17 Nov 2023 08:03:41 +0000
> From: Shaun McDonald <shaun at shaunmcdonald.me.uk>
> To: Shaun McDonald via Taginfo-dev <taginfo-dev at openstreetmap.org>
> Subject: [Taginfo-dev] Problem running taginfo GB since 2023-10-04
> 
> Hi,
> 
> I’ve been alerted that there is an issue with tag info GB which has been running quietly in the background for a year or so. Since 2023-10-04 https://taginfo.openstreetmap.org.uk/sources/db the build has been failing. I’ve tried updating the code and all minor OS updates, however I’m still getting the following error:
> 
> [ 5:02] Writing results to database...
> [ 5:11]   writing results to database took 9 seconds (about 0 minutes)
> [ 5:11]
> [ 5:11] Estimated memory usage:
> [ 5:11]   tags_stat: ...............     3471 kB [size=14099 buckets=16383 sizeof(entry)=217]
> [ 5:11]   key_value_stats: .........       64 kB [size=738 buckets=1023 sizeof(entry)=65]
> [ 5:11]   key_value_geodistribution:      195 kB [size=3319 buckets=4095 sizeof(entry)=49]
> [ 5:11]   relation_type_stats: .....        2 kB [size=22 sizeof(value_type)=104]
> [ 5:11]   values: ..................   156559 kB [(sizeof(hash key)(8) + sizeof(hash value)(12) + 2.5 bit overhead) * bucket_count(7868309), size=5080007]
> [ 5:11]   key_combos: ..............    15279 kB [(sizeof(hash key)(8) + sizeof(hash value)(12) + 2.5 bit overhead) * bucket_count(767911), size=477878]
> [ 5:11]   users: ...................      115 kB [(sizeof(hash key)(4) + sizeof(hash value)(4) + 2.5 bit overhead) * bucket_count(14099), size=14099]
> [ 5:11]   string_store: ............    92160 kB [chunk_size=10240kB chunks=9]
> [ 5:11]   location_index: ..........  2888673 kB [size=184875094]
> [ 5:11]   ======================================
> [ 5:11]   total: ...................  3156522 kB
> [ 5:11]
> [ 5:11] Actual memory usage:
> [ 5:11]   current: 4686MB
> [ 5:11]   peak:    6541MB
> [ 5:11] ------------------------------------------------------------------------------
> 2023-11-17T07:53:25 | 6 | db | run_similarity | Running similarity...
> 2023-11-17T07:53:25 | 6 | db | run_similarity | Running '/home/shaunmcdonald/taginfo-tools/build/src/taginfo-similarity /home/shaunmcdonald/taginfo-gb/build/db/taginfo-db.db'...
> 2023-11-17T07:53:50 | 6 | db | run_similarity | Running SQL script '/home/shaunmcdonald/taginfo-gb/taginfo/sources/db/post_similar_keys.sql' on database '/home/shaunmcdonald/taginfo-gb/build/db/taginfo-db.db'...
> 2023-11-17T07:53:50 | 6 | db | update_characters | Running update_characters...
> 2023-11-17T07:53:50 | 6 | db | update_characters | Running 'env - ruby -E utf-8 -w -I /home/shaunmcdonald/taginfo-gb/taginfo/sources/db/lib /home/shaunmcdonald/taginfo-gb/taginfo/sources/db/update_characters.rb /home/shaunmcdonald/taginfo-gb/build/db'...
> 2023-11-17T07:53:51 | 6 | db | main | Running SQL script '/home/shaunmcdonald/taginfo-gb/taginfo/sources/db/post_grades.sql' on database '/home/shaunmcdonald/taginfo-gb/build/db/taginfo-db.db'...
> 2023-11-17T07:53:51 | 6 | db | main | Running SQL script '/home/shaunmcdonald/taginfo-gb/taginfo/sources/db/post_indexes.sql' on database '/home/shaunmcdonald/taginfo-gb/build/db/taginfo-db.db'...
> 2023-11-17T07:53:59 | 7 | db | finalize_database | Running SQL script '/home/shaunmcdonald/taginfo-gb/taginfo/sources/db/post.sql' on database '/home/shaunmcdonald/taginfo-gb/build/db/taginfo-db.db'...
> 2023-11-17T07:54:05 | 7 | db | main | Done db.
> 2023-11-17T07:54:05 | 7 | all | update_source | Done.
> 2023-11-17T07:54:05 | 7 | all | compress_source_databases | Compressing all source databases...
> 2023-11-17T07:54:05 | 7 | all | compress_file | Compressing 'db/taginfo-db' to 'db' using 'bzip2'
> 2023-11-17T07:54:56 | 8 | all | compress_source_databases | Done.
> 2023-11-17T07:54:56 | 8 | all | create_extra_indexes | Creating extra indexes...
> 2023-11-17T07:54:56 | 8 | all | create_extra_indexes | Running SQL script '/home/shaunmcdonald/taginfo-gb/taginfo/sources/db/add_extra_indexes.sql' on database '/home/shaunmcdonald/taginfo-gb/build/db/taginfo-db.db'...
> 2023-11-17T07:55:32 | 8 | all | create_extra_indexes | Running SQL script '/home/shaunmcdonald/taginfo-gb/taginfo/sources/db/add_ftsearch.sql' on database '/home/shaunmcdonald/taginfo-gb/build/db/taginfo-db.db'...
> 2023-11-17T07:56:02 | 9 | all | create_extra_indexes | Done.
> 2023-11-17T07:56:02 | 9 | all | update_master | Running master/update.sh...
> 2023-11-17T07:56:02 | 9 | master | main | Start master...
> 2023-11-17T07:56:02 | 9 | master | create_master_database | Running SQL script '/home/shaunmcdonald/taginfo-gb/taginfo/sources/master/languages.sql' on database '/home/shaunmcdonald/taginfo-gb/build/taginfo-master.db'...
> 2023-11-17T07:56:02 | 9 | master | create_master_database | Running SQL script '/home/shaunmcdonald/taginfo-gb/taginfo/sources/master/master.sql' on database '/home/shaunmcdonald/taginfo-gb/build/taginfo-master.db'...
> 2023-11-17T07:56:02 | 9 | master | create_master_database |   with parameter: DIR=/home/shaunmcdonald/taginfo-gb/build
> 2023-11-17T07:56:02 | 9 | master | create_master_database |   with parameter: MIN_COUNT_POPULAR=1000
> 2023-11-17T07:56:02 | 9 | master | create_master_database |   with parameter: MIN_COUNT_SUGGESTION=10000
> Error: near line 83: near "FROM": syntax error
> 
> I’m wondering what could have changed?
> Is it possible that the memory required has increased but it’s not mentioning that as the error?
> Was there some other code or data change that I need to consider?
> 
> Thanks,
> Shaun

> _______________________________________________
> Taginfo-dev mailing list
> Taginfo-dev at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/taginfo-dev

-- 
Jochen Topf  jochen at remote.org  https://www.jochentopf.com/  +49-351-31778688