[Taginfo-dev] Taginfo GB std::bad_alloc error

[Shaun McDonald]

Hi,

I’ve git a bit of a snag with Taginfo GB, I got it moved to a new VM and updated to the latest VM on 24th March, and it’s been updating daily until 7th April when I first got a std:bad_alloc error in the log. Then on the 8 th, 9th, and 10th it updated as expected. However since then it’s been having the error, and I can’t understand why.

e.g. here’s the log from the 11th (all are similar):
2019-04-11T09:46:33 | 0 | all | main | Start update_all...
2019-04-11T09:46:33 | 0 | all | download_source | Downloading and uncompressing languages...
2019-04-11T09:46:33 | 0 | all | download_source | Running 'curl --silent --fail --output /home/shaunmcdonald/taginfo-gb/build/download/taginfo-languages.db.bz2 --time-cond /home/shaunmcdonald/taginfo-gb/build/download/taginfo-languages.db.bz2 https://taginfo.openstreetmap.org/download/taginfo-languages.db.bz2'...
2019-04-11T09:46:33 | 0 | all | download_source | Running 'bzip2 -d -c /home/shaunmcdonald/taginfo-gb/build/download/taginfo-languages.db.bz2'...
2019-04-11T09:46:33 | 0 | all | download_source |   Writing output to '/home/shaunmcdonald/taginfo-gb/build/languages/taginfo-languages.db'...
2019-04-11T09:46:33 | 0 | all | download_source | Done.
2019-04-11T09:46:33 | 0 | all | download_source | Downloading and uncompressing projects...
2019-04-11T09:46:33 | 0 | all | download_source | Running 'curl --silent --fail --output /home/shaunmcdonald/taginfo-gb/build/download/taginfo-projects.db.bz2 --time-cond /home/shaunmcdonald/taginfo-gb/build/download/taginfo-projects.db.bz2 https://taginfo.openstreetmap.org/download/taginfo-projects.db.bz2'...
2019-04-11T09:46:34 | 0 | all | download_source | Running 'bzip2 -d -c /home/shaunmcdonald/taginfo-gb/build/download/taginfo-projects.db.bz2'...
2019-04-11T09:46:34 | 0 | all | download_source |   Writing output to '/home/shaunmcdonald/taginfo-gb/build/projects/taginfo-projects.db'...
2019-04-11T09:46:35 | 0 | all | download_source | Done.
2019-04-11T09:46:35 | 0 | all | download_source | Downloading and uncompressing wiki...
2019-04-11T09:46:35 | 0 | all | download_source | Running 'curl --silent --fail --output /home/shaunmcdonald/taginfo-gb/build/download/taginfo-wiki.db.bz2 --time-cond /home/shaunmcdonald/taginfo-gb/build/download/taginfo-wiki.db.bz2 https://taginfo.openstreetmap.org/download/taginfo-wiki.db.bz2'...
2019-04-11T09:46:36 | 0 | all | download_source | Running 'bzip2 -d -c /home/shaunmcdonald/taginfo-gb/build/download/taginfo-wiki.db.bz2'...
2019-04-11T09:46:36 | 0 | all | download_source |   Writing output to '/home/shaunmcdonald/taginfo-gb/build/wiki/taginfo-wiki.db'...
2019-04-11T09:46:38 | 0 | all | download_source | Done.
2019-04-11T09:46:38 | 0 | all | update_source | Running db/update.sh...
2019-04-11T09:46:38 | 0 | db | main | Start db...
2019-04-11T09:46:38 | 0 | db | initialize_database | Running SQL script '/home/shaunmcdonald/taginfo-gb/taginfo/sources/db/../init.sql' on database '/home/shaunmcdonald/taginfo-gb/build/db/taginfo-db.db'...
2019-04-11T09:46:38 | 0 | db | initialize_database | Running SQL script '/home/shaunmcdonald/taginfo-gb/taginfo/sources/db/pre.sql' on database '/home/shaunmcdonald/taginfo-gb/build/db/taginfo-db.db'...
2019-04-11T09:46:39 | 0 | db | run_tagstats | Reading selection database '/home/shaunmcdonald/taginfo-gb/build/db/../selection.db'
2019-04-11T09:46:39 | 0 | db | run_tagstats | Selection database contents:
  interesting_tags count .........: 496
  frequent_tags count ............: 1997
  interesting_relation_types count: 19
2019-04-11T09:46:39 | 0 | db | run_tagstats | Running tagstats...
2019-04-11T09:46:39 | 0 | db | run_tagstats | Running '/home/shaunmcdonald/taginfo-gb/taginfo/tagstats/tagstats --selection-db=/home/shaunmcdonald/taginfo-gb/build/db/../selection.db --index=SparseMemArray --min-tag-combination-count=1000 --left=-180 --bottom=-90 --top=90 --right=180 --width=360 --height=180 /home/shaunmcdonald/taginfo-gb/osm_planets/great-britain-latest.osm.pbf /home/shaunmcdonald/taginfo-gb/build/db/taginfo-db.db'...
[ 0:00] Starting tagstats...
[ 0:00] ------------------------------------------------------------------------------
[ 0:00] Processing nodes...
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
/home/shaunmcdonald/taginfo-gb/taginfo/sources/db/../util.sh: line 71: 11043 Aborted                 (core dumped) env - $@


I’ve tried updating the map to be gb instead of the world, recompiling tagstats and even running tagstats under gdb (after adding the -g flag), however I’ve been unable to figure out what’s going on. The code is on the latest version.
Watching top, it hits about 25% RAM before crashing.
I’ve tried older files listed on http://download.geofabrik.de/europe/great-britain.html# <http://download.geofabrik.de/europe/great-britain.html#> which seem to have the same issue.


(gdb) set args --index=SparseMemArray --min-tag-combination-count=1000 --left=-11 --bottom=49.8 --top=60.8 --right=2.5 --width=110 --height=135 /home/shaunmcdonald/taginfo-gb/osm_planets/great-britain-latest.osm.pbf /home/shaunmcdonald/taginfo-gb/build/db/taginfo-db.db
(gdb) run
Starting program: /home/shaunmcdonald/taginfo-gb/taginfo/tagstats/tagstats --index=SparseMemArray --min-tag-combination-count=1000 --left=-11 --bottom=49.8 --top=60.8 --right=2.5 --width=110 --height=135 /home/shaunmcdonald/taginfo-gb/osm_planets/great-britain-latest.osm.pbf /home/shaunmcdonald/taginfo-gb/build/db/taginfo-db.db
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[ 0:00] Starting tagstats...
[ 0:00] ------------------------------------------------------------------------------
[ 0:00] Processing nodes...
[New Thread 0x7ffff3471700 (LWP 24039)]
[New Thread 0x7ffff2c70700 (LWP 24040)]
[New Thread 0x7ffff246f700 (LWP 24041)]
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc

Thread 1 "tagstats" received signal SIGABRT, Aborted.
__GI_raise (sig=sig at entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51	../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb)

Any ideas of what’s happening?

Shaun
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/taginfo-dev/attachments/20190414/fc8870da/attachment.html>