[OSRM-talk] planet prepare problems
Grant Heffernan
grant at mapzen.com
Fri Dec 27 23:46:00 UTC 2013
Hi all. Was wondering if anyone has successfully run osrm-prepare on the planet recently. I’ve successfully set up north-america and other extracts, but the planet load fails every time, and in a very odd way, which is making tracking the problem down an issue. More specifically, the host it’s being run on becomes completely unresponsive (I’m running on EC2) and I’m unable to recover the instance to track additional details down. I can terminate but not reboot the instance, which makes me suspect it’s getting starved of resources and the hypervisor is unable to get it to respond. I’m presently testing on the following setup:
hi1.4xlarge (ssd instance store, 60gb ram) with a 250gb swap partition configured. I’m running the osrm-prepare as a non root user, although I’ve upped limits (file handles, memory) for that user.
Immediately before the system becomes unresponsive, I don’t see anything that looks terribly dire. I’m using all available ram and about 50gb of swap. The prepare gets to the following point, shortly after which I run into the aforementioned system issues. Any ideas greatly appreciated.
[info] generating edge-expanded edges
. 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100%
[info] Generated 923822461 edge based nodes
[info] Node-based graph contains 923822461 edges
[info] Edge-expanded graph ...
[info] contains 1124240856 edges
[info] skips 277973 turns, defined by 221581 restrictions
[info] writing node map ...
[info] building r-tree ...
[info] constructing r-tree of 923822461 elements
Last reported system level stats:
11:25:06 PM kbmemfree kbmemused %memused kbbuffers kbcached kbcommit %commit kbactive kbinact
11:25:07 PM 282484 61881104 99.55 816 572684 97878112 30.18 56481372 4227560
11:25:08 PM 277008 61886580 99.55 816 535896 97878112 30.18 56484656 4229868
11:25:09 PM 280336 61883252 99.55 816 489340 97878112 30.18 56488076 4222880
11:25:10 PM 278192 61885396 99.55 824 448888 97878112 30.18 56491680 4220584
11:25:11 PM 278120 61885468 99.55 836 412472 97878112 30.18 56495220 4219444
11:25:12 PM 275620 61887968 99.56 836 371140 97878112 30.18 56498616 4216208
11:25:13 PM 279580 61884008 99.55 836 330600 97878112 30.18 56502572 4208796
11:25:14 PM 278364 61885224 99.55 836 295072 97878112 30.18 56506756 4205680
11:25:15 PM 276504 61887084 99.56 856 266924 97878112 30.18 56510380 4204908
11:25:16 PM 269716 61893872 99.57 856 243168 97878112 30.18 56514980 4209116
From top:
Tasks: 187 total, 4 running, 183 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.8%us, 5.3%sy, 0.0%ni, 90.2%id, 3.6%wa, 0.0%hi, 0.0%si, 0.1%st
Mem: 62163588k total, 61896352k used, 267236k free, 856k buffers
Swap: 262143996k total, 56792628k used, 205351368k free, 245440k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
8262 osrm 20 0 93.0g 56g 0 R 49 95.1 66:02.14 osrm-prepare
I see the following open handles:
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
osrm-prep 8262 osrm cwd DIR 252,2 4096 91488287 /opt/osrm/osrm/build
osrm-prep 8262 osrm rtd DIR 202,1 4096 2 /
osrm-prep 8262 osrm txt REG 252,2 569458 91488764 /opt/osrm/osrm/build/osrm-prepare
osrm-prep 8262 osrm mem REG 202,1 18282384 53981 /usr/lib/libicudata.so.48.1.1
osrm-prep 8262 osrm mem REG 202,1 31752 53598 /lib/x86_64-linux-gnu/librt-2.15.so
osrm-prep 8262 osrm mem REG 202,1 183824 4631 /usr/lib/x86_64-linux-gnu/liblua5.1.so.0.0.0
osrm-prep 8262 osrm mem REG 202,1 14768 53606 /lib/x86_64-linux-gnu/libdl-2.15.so
osrm-prep 8262 osrm mem REG 202,1 1866528 53983 /usr/lib/libicui18n.so.48.1.1
osrm-prep 8262 osrm mem REG 202,1 1465096 53984 /usr/lib/libicuuc.so.48.1.1
osrm-prep 8262 osrm mem REG 202,1 1815224 53591 /lib/x86_64-linux-gnu/libc-2.15.so
osrm-prep 8262 osrm mem REG 202,1 88384 2197 /lib/x86_64-linux-gnu/libgcc_s.so.1
osrm-prep 8262 osrm mem REG 202,1 55888 2087 /usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0
osrm-prep 8262 osrm mem REG 202,1 1030512 53602 /lib/x86_64-linux-gnu/libm-2.15.so
osrm-prep 8262 osrm mem REG 202,1 962656 4735 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16
osrm-prep 8262 osrm mem REG 202,1 122392 21027 /usr/lib/libluabind.so.0.9.1
osrm-prep 8262 osrm mem REG 202,1 416272 13016 /usr/lib/x86_64-linux-gnu/libluajit-5.1.so.2.0.0
osrm-prep 8262 osrm mem REG 202,1 135366 53600 /lib/x86_64-linux-gnu/libpthread-2.15.so
osrm-prep 8262 osrm mem REG 202,1 101304 21680 /usr/lib/libboost_thread.so.1.46.1
osrm-prep 8262 osrm mem REG 202,1 14568 13058 /usr/lib/libboost_system.so.1.46.1
osrm-prep 8262 osrm mem REG 202,1 1054456 13138 /usr/lib/libboost_regex.so.1.46.1
osrm-prep 8262 osrm mem REG 202,1 414280 16257 /usr/lib/libboost_program_options.so.1.46.1
osrm-prep 8262 osrm mem REG 202,1 122280 13064 /usr/lib/libboost_filesystem.so.1.46.1
osrm-prep 8262 osrm mem REG 202,1 149280 53603 /lib/x86_64-linux-gnu/ld-2.15.so
osrm-prep 8262 osrm mem REG 202,1 1607664 53207 /usr/lib/locale/locale-archive
osrm-prep 8262 osrm mem REG 202,1 26258 53523 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache
osrm-prep 8262 osrm 0w CHR 1,3 0t0 3076 /dev/null
osrm-prep 8262 osrm 1w REG 252,2 1218 12 /opt/osrm/osrm_prepare.out
osrm-prep 8262 osrm 2w REG 252,2 1218 12 /opt/osrm/osrm_prepare.out
osrm-prep 8262 osrm 3w REG 252,2 7022111544 22544395 /opt/osrm/osrm-data/planet/car/planet-latest.osrm.fileIndex
--
Grant
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/osrm-talk/attachments/20131227/b8c08e17/attachment.html>
More information about the OSRM-talk
mailing list