[OSRM-talk] planet prepare problems

Grant Heffernan grant at mapzen.com
Fri Dec 27 23:46:00 UTC 2013


Hi all. Was wondering if anyone has successfully run osrm-prepare on the planet recently. I’ve successfully set up north-america and other extracts, but the planet load fails every time, and in a very odd way, which is making tracking the problem down an issue. More specifically, the host it’s being run on becomes completely unresponsive (I’m running on EC2) and I’m unable to recover the instance to track additional details down. I can terminate but not reboot the instance, which makes me suspect it’s getting starved of resources and the hypervisor is unable to get it to respond. I’m presently testing on the following setup:

hi1.4xlarge (ssd instance store, 60gb ram) with a 250gb swap partition configured. I’m running the osrm-prepare as a non root user, although I’ve upped limits (file handles, memory) for that user.

Immediately before the system becomes unresponsive, I don’t see anything that looks terribly dire. I’m using all available ram and about 50gb of swap. The prepare gets to the following point, shortly after which I run into the aforementioned system issues. Any ideas greatly appreciated.

[info] generating edge-expanded edges
. 10% . 20% . 30% . 40% . 50% . 60% . 70% . 80% . 90% . 100%
[info] Generated 923822461 edge based nodes
[info] Node-based graph contains 923822461 edges
[info] Edge-expanded graph ...
[info]   contains 1124240856 edges
[info]   skips 277973 turns, defined by 221581 restrictions
[info] writing node map ...
[info] building r-tree ...
[info] constructing r-tree of 923822461 elements


Last reported system level stats:

11:25:06 PM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit  kbactive   kbinact
11:25:07 PM    282484  61881104     99.55       816    572684  97878112     30.18  56481372   4227560
11:25:08 PM    277008  61886580     99.55       816    535896  97878112     30.18  56484656   4229868
11:25:09 PM    280336  61883252     99.55       816    489340  97878112     30.18  56488076   4222880
11:25:10 PM    278192  61885396     99.55       824    448888  97878112     30.18  56491680   4220584
11:25:11 PM    278120  61885468     99.55       836    412472  97878112     30.18  56495220   4219444
11:25:12 PM    275620  61887968     99.56       836    371140  97878112     30.18  56498616   4216208
11:25:13 PM    279580  61884008     99.55       836    330600  97878112     30.18  56502572   4208796
11:25:14 PM    278364  61885224     99.55       836    295072  97878112     30.18  56506756   4205680
11:25:15 PM    276504  61887084     99.56       856    266924  97878112     30.18  56510380   4204908
11:25:16 PM    269716  61893872     99.57       856    243168  97878112     30.18  56514980   4209116

From top:

Tasks: 187 total,   4 running, 183 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.8%us,  5.3%sy,  0.0%ni, 90.2%id,  3.6%wa,  0.0%hi,  0.0%si,  0.1%st
Mem:  62163588k total, 61896352k used,   267236k free,      856k buffers
Swap: 262143996k total, 56792628k used, 205351368k free,   245440k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 8262 osrm      20   0 93.0g  56g    0 R   49 95.1  66:02.14 osrm-prepare



I see the following open handles:

COMMAND    PID USER   FD   TYPE DEVICE   SIZE/OFF     NODE NAME
osrm-prep 8262 osrm  cwd    DIR  252,2       4096 91488287 /opt/osrm/osrm/build
osrm-prep 8262 osrm  rtd    DIR  202,1       4096        2 /
osrm-prep 8262 osrm  txt    REG  252,2     569458 91488764 /opt/osrm/osrm/build/osrm-prepare
osrm-prep 8262 osrm  mem    REG  202,1   18282384    53981 /usr/lib/libicudata.so.48.1.1
osrm-prep 8262 osrm  mem    REG  202,1      31752    53598 /lib/x86_64-linux-gnu/librt-2.15.so
osrm-prep 8262 osrm  mem    REG  202,1     183824     4631 /usr/lib/x86_64-linux-gnu/liblua5.1.so.0.0.0
osrm-prep 8262 osrm  mem    REG  202,1      14768    53606 /lib/x86_64-linux-gnu/libdl-2.15.so
osrm-prep 8262 osrm  mem    REG  202,1    1866528    53983 /usr/lib/libicui18n.so.48.1.1
osrm-prep 8262 osrm  mem    REG  202,1    1465096    53984 /usr/lib/libicuuc.so.48.1.1
osrm-prep 8262 osrm  mem    REG  202,1    1815224    53591 /lib/x86_64-linux-gnu/libc-2.15.so
osrm-prep 8262 osrm  mem    REG  202,1      88384     2197 /lib/x86_64-linux-gnu/libgcc_s.so.1
osrm-prep 8262 osrm  mem    REG  202,1      55888     2087 /usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0
osrm-prep 8262 osrm  mem    REG  202,1    1030512    53602 /lib/x86_64-linux-gnu/libm-2.15.so
osrm-prep 8262 osrm  mem    REG  202,1     962656     4735 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16
osrm-prep 8262 osrm  mem    REG  202,1     122392    21027 /usr/lib/libluabind.so.0.9.1
osrm-prep 8262 osrm  mem    REG  202,1     416272    13016 /usr/lib/x86_64-linux-gnu/libluajit-5.1.so.2.0.0
osrm-prep 8262 osrm  mem    REG  202,1     135366    53600 /lib/x86_64-linux-gnu/libpthread-2.15.so
osrm-prep 8262 osrm  mem    REG  202,1     101304    21680 /usr/lib/libboost_thread.so.1.46.1
osrm-prep 8262 osrm  mem    REG  202,1      14568    13058 /usr/lib/libboost_system.so.1.46.1
osrm-prep 8262 osrm  mem    REG  202,1    1054456    13138 /usr/lib/libboost_regex.so.1.46.1
osrm-prep 8262 osrm  mem    REG  202,1     414280    16257 /usr/lib/libboost_program_options.so.1.46.1
osrm-prep 8262 osrm  mem    REG  202,1     122280    13064 /usr/lib/libboost_filesystem.so.1.46.1
osrm-prep 8262 osrm  mem    REG  202,1     149280    53603 /lib/x86_64-linux-gnu/ld-2.15.so
osrm-prep 8262 osrm  mem    REG  202,1    1607664    53207 /usr/lib/locale/locale-archive
osrm-prep 8262 osrm  mem    REG  202,1      26258    53523 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache
osrm-prep 8262 osrm    0w   CHR    1,3        0t0     3076 /dev/null
osrm-prep 8262 osrm    1w   REG  252,2       1218       12 /opt/osrm/osrm_prepare.out
osrm-prep 8262 osrm    2w   REG  252,2       1218       12 /opt/osrm/osrm_prepare.out
osrm-prep 8262 osrm    3w   REG  252,2 7022111544 22544395 /opt/osrm/osrm-data/planet/car/planet-latest.osrm.fileIndex


--  
Grant

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/osrm-talk/attachments/20131227/b8c08e17/attachment.html>


More information about the OSRM-talk mailing list