<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none"><!--P{margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<p>Hi<font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"> Kieran</font><font face="Calibri, sans-serif">,</font></p>
<p><br>
<font face="Calibri, sans-serif"></font></p>
<p><font face="Calibri, sans-serif">we needed around 88G of RAM, 100G of SWAP and 250G of stxxl.</font></p>
<p>I guess SWAP and stxxl is a little bit oversized. </p>
<p><br>
</p>
<p>BR </p>
<p>Björn<br>
</p>
<p><br>
</p>
<p><br>
</p>
<div style="color: rgb(33, 33, 33);">
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>Von:</b> Kieran Caplice <kieran.caplice@temetra.com><br>
<b>Gesendet:</b> Dienstag, 8. März 2016 10:02<br>
<b>An:</b> Mailing list to discuss Project OSRM<br>
<b>Betreff:</b> [OSRM-talk] Required RAM for planet</font>
<div> </div>
</div>
<div>Hi all,<br>
<br>
Just enquiring if anyone has an up-to-date value for the required RAM to extract/prepare the planet PBF on OSRM v4.9.1? Below is the output from extraction on a 64 GB RAM machine:<br>
<br>
<tt>[info] Input file: planet-latest.osm.pbf</tt><tt><br>
</tt><tt>[info] Profile: profile.lua</tt><tt><br>
</tt><tt>[info] Threads: 12</tt><tt><br>
</tt><tt>[info] Using script profile.lua</tt><tt><br>
</tt><tt>[STXXL-MSG] STXXL v1.3.1 (release)</tt><tt><br>
</tt><tt>[STXXL-MSG] 1 disks are allocated, total space: 250000 MiB</tt><tt><br>
</tt><tt>[info] Parsing in progress..</tt><tt><br>
</tt><tt>[info] input file generated by planet-dump-ng 1.1.3</tt><tt><br>
</tt><tt>[info] timestamp: 2016-02-29T01:59:57Z</tt><tt><br>
</tt><tt>[info] Using turn restrictions</tt><tt><br>
</tt><tt>[info] Found 3 exceptions to turn restrictions:</tt><tt><br>
</tt><tt>[info] motorcar</tt><tt><br>
</tt><tt>[info] motor_vehicle</tt><tt><br>
</tt><tt>[info] vehicle</tt><tt><br>
</tt><tt>[info] Parsing finished after 3630.62 seconds</tt><tt><br>
</tt><tt>[info] Raw input contains 3240515916 nodes, 333181812 ways, and 4046532 relations, and 0 unknown entities</tt><tt><br>
</tt><tt>[extractor] Sorting used nodes ... ok, after 301.697s</tt><tt><br>
</tt><tt>[extractor] Erasing duplicate nodes ... ok, after 262.325s</tt><tt><br>
</tt><tt>[extractor] Sorting all nodes ... ok, after 3419.92s</tt><tt><br>
</tt><tt>[extractor] Building node id map ... ok, after 1674.42s</tt><tt><br>
</tt><tt>[extractor] setting number of nodes ... ok</tt><tt><br>
</tt><tt>[extractor] Confirming/Writing used nodes ... ok, after 837.176s</tt><tt><br>
</tt><tt>[info] Processed 578533637 nodes</tt><tt><br>
</tt><tt>[extractor] Sorting edges by start ... ok, after 1919.47s</tt><tt><br>
</tt><tt>[extractor] Setting start coords ... ok, after 2657.74s</tt><tt><br>
</tt><tt>[extractor] Sorting edges by target ... ok, after 1894.56s</tt><tt><br>
</tt><tt>[extractor] Computing edge weights ... ok, after 2911.29s</tt><tt><br>
</tt><tt>[extractor] Sorting edges by renumbered start ... ok, after 1864.9s</tt><tt><br>
</tt><tt>[extractor] Writing used edges ... ok, after 557.274s</tt><tt><br>
</tt><tt>[extractor] setting number of edges ... ok</tt><tt><br>
</tt><tt>[info] Processed 610970822 edges</tt><tt><br>
</tt><tt>[extractor] Sorting used ways ... ok, after 89.8886s</tt><tt><br>
</tt><tt>[extractor] Sorting 491077 restriction. by from... ok, after 0.906043s</tt><tt><br>
</tt><tt>[extractor] Fixing restriction starts ... ok, after 39.6105s</tt><tt><br>
</tt><tt>[extractor] Sorting restrictions. by to ... ok, after 0.734124s</tt><tt><br>
</tt><tt>[extractor] Fixing restriction ends ... ok, after 40.6053s</tt><tt><br>
</tt><tt>[info] usable restrictions: 459264</tt><tt><br>
</tt><tt>[extractor] writing street name index ... ok, after 3.3452s</tt><tt><br>
</tt><tt>[info] extraction finished after 23181.9s</tt><tt><br>
</tt><tt>[info] Generating edge-expanded graph representation</tt><tt><br>
</tt><tt>[info] - 459264 restrictions.</tt><tt><br>
</tt><tt>[info] Importing n = 578533637 nodes </tt><tt><br>
</tt><tt>[info] - 157152 bollard nodes, 792260 traffic lights</tt><tt><br>
</tt><tt>[info] and 610970822 edges </tt><tt><br>
</tt><tt>[info] Graph loaded ok and has 610970822 edges</tt><tt><br>
</tt><font color="#990000"><tt>[warn] std::bad_alloc</tt></font><br>
<div class="moz-signature"><br>
>From my reading, this is caused by running out of RAM. The only files created were:<br>
<br>
<tt>-rw-r--r-- 1 root root 25G Mar 7 18:57 planet-latest.osrm</tt><tt><br>
</tt><tt>-rw-r--r-- 1 root root 118M Mar 7 19:00 planet-latest.osrm.names</tt><tt><br>
</tt><tt>-rw-r--r-- 1 root root 15M Mar 7 19:00 planet-latest.osrm.restrictions</tt><tt><br>
</tt><tt>-rw-r--r-- 1 root root 20 Mar 7 12:33 planet-latest.osrm.timestamp</tt><br>
<br>
Which obviously means preparing gives the following:<br>
<br>
<tt>[info] Input file: planet-latest.osrm</tt><tt><br>
</tt><tt>[info] Profile: profile.lua</tt><tt><br>
</tt><tt>[info] Threads: 12</tt><tt><br>
</tt><tt>[info] Loading edge-expanded graph representation</tt><tt><br>
</tt><tt>[info] Opening planet-latest.osrm.ebg</tt><tt><br>
</tt><font color="#990000"><tt>[warn] [exception] osrm input file misses magic number. Check or reprocess the file</tt></font><br>
<br>
Thanks.<br>
<br>
Kind regards,<br>
Kieran Caplice<br>
<br>
</div>
</div>
</div>
</body>
</html>