<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hi Daniel,</p>
<div class="moz-signature">Can you clarify if you use Docker at
Mapbox? What kind of server do you guys have (if not the one
described on the wiki page)?<br>
<br>
Over night, the process has gone to just 75% complete from 65%
yesterday - 41 hours in total now, and back to maxing CPU again.<br>
<br>
Kind regards,<br>
Kieran Caplice<br>
<br>
</div>
<div class="moz-cite-prefix">On 21/09/17 17:17, Daniel Patterson
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CANvddSLRjKgje165N6B88WDn5q44o164xW7f_UvEPg1oiFqgmQ@mail.gmail.com">
<div dir="ltr">OSRM supports *two* core routing algorithms - CH
and MLD. The `osrm-contract` tool generates the CH dataset, but
you can use the MLD pipeline instead with:
<div><br>
</div>
<div>osrm-extract -p profiles/bicycle.lua yourmap.osm.pbf</div>
<div>osrm-partition yourmap.osrm</div>
<div>osrm-customize yourmap.osrm</div>
<div>osrm-routed -a MLD yourmap.osrm</div>
<div><br>
</div>
<div>This sequence of tools should be significantly quicker than
osrm-contract - the price you pay is that routing requests are
about 5x slower (still pretty fast though!). The reason that
MLD exists is for the `osrm-customize` step - it allows you to
import traffic data very quickly and update the routing graph
(~1 minute for North America).</div>
<div><br>
</div>
<div>It's hard to say exactly what's going wrong with
osrm-contract here - here at Mapbox, we daily run
`osrm-contract` over the latest planet with the bicycle
profile without a problem, however, Alex and others have
reported issues with what seem like hangs on much smaller
datasets that we've been unable to reproduce so far.</div>
<div><br>
</div>
<div>The runtime of osrm-contract is affected by how much
hierarchy exists in the data - the more similar the edge
speeds (like in foot) and the more edges there are, the slower
it gets, often in a non-linear fashion. The car profile has a
very hierarchical structure (many different road speeds), so
it fits well into the CH, the construction algorithm doesn't
need to compare as many options.</div>
<div><br>
</div>
<div>daniel</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu, Sep 21, 2017 at 9:03 AM, Kieran
Caplice <span dir="ltr"><<a
href="mailto:kieran.caplice@temetra.com" target="_blank"
moz-do-not-send="true">kieran.caplice@temetra.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<p>We're actually looking for the best of both car and
foot, so in my head, bicycle would be the happy medium
(though I could be completely wrong on this).<br>
</p>
<div class="m_6646586754541093830moz-signature"> Kind
regards,<br>
Kieran Caplice<br>
<br>
</div>
<div>
<div class="h5">
<div class="m_6646586754541093830moz-cite-prefix">On
21/09/17 16:53, Alex Farioletti wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">i've run into the same issues, and
now i just use metroextracts of the areas that i
need for the bike stuff i do and it reduces the
time significantly</div>
<div class="gmail_extra"><br clear="all">
<div>
<div
class="m_6646586754541093830gmail_signature"
data-smartmail="gmail_signature">
<div dir="ltr"><b>Alex Farioletti</b><br>
<div><b><a value="+14153121674"
moz-do-not-send="true">415.312.1674</a></b>
<div><i style="font-weight:bold"><a
href="http://tcbcourier.com"
target="_blank"
moz-do-not-send="true">tcbcourier.com</a> </i></div>
</div>
</div>
</div>
</div>
<br>
<div class="gmail_quote">On Thu, Sep 21, 2017 at
8:49 AM, Kieran Caplice <span dir="ltr"><<a
href="mailto:kieran.caplice@temetra.com"
target="_blank" moz-do-not-send="true">kieran.caplice@temetra.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0
0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<p>Thanks Daniel.</p>
<p>I'm using the bicycle profile, so I would
expect based on what you've said that
somewhere up to 36 hours would be likely.
However, this is the current output, after
25h40m:</p>
<div
class="m_6646586754541093830m_-3407091792784772007moz-signature"><tt>[info]
Input file:
/data/1505492056/planet-latest<wbr>.osrm</tt><tt><br>
</tt><tt>[info] Threads: 12</tt><tt><br>
</tt><tt>[info] Reading node weights.</tt><tt><br>
</tt><tt>[info] Done reading node weights.</tt><tt><br>
</tt><tt>[info] Loading edge-expanded
graph representation</tt><tt><br>
</tt><tt>[info] merged 2379332 edges out
of 1777752432</tt><tt><br>
</tt><tt>[info] initializing node
priorities... ok.</tt><tt><br>
</tt><tt>[info] preprocessing 389797971
(90%) nodes...</tt><tt><br>
</tt><tt>[info] . 10% . 20% . 30% . 40% .
50% . 60% </tt><br>
<br>
It hasn't advanced past 60% in the last
2-3 hours. It is however maxing CPU and
using approximately the same amount of RAM
since it started.<br>
<br>
Kind regards,<br>
Kieran Caplice<br>
<br>
</div>
<div>
<div class="m_6646586754541093830h5">
<div
class="m_6646586754541093830m_-3407091792784772007moz-cite-prefix">On
21/09/17 16:39, Daniel Patterson
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi Kieran,
<div><br>
</div>
<div> The contraction time will be
slow - many, many hours for the
whole planet. *Typically* for the
car profile it's about 12 hours,
but if you use bike or foot, or
your own profile, it can get a lot
bigger.</div>
<div><br>
</div>
<div> If you've messed with the
travel speeds, that can have a big
effect too. 24 hours is not
unheard of, but whether it's legit
will depend a lot on the details.</div>
<div><br>
</div>
<div>daniel</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu, Sep
21, 2017 at 7:00 AM, Kieran
Caplice <span dir="ltr"><<a
href="mailto:kieran.caplice@temetra.com"
target="_blank"
moz-do-not-send="true">kieran.caplice@temetra.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">Hi all,<br>
<br>
Could anyone give an approx
estimate for the time required
to run the osrm-contract on
planet data on a 12 thread, 256
GB RAM, SSD machine?<br>
<br>
The osrm-extract process
finished in 232 minutes, but the
contract has now been running
solid for 24 hours, and appears
to be stuck at 60% on
"preprocessing nodes". All 12
cores are generally maxed out,
and the process is using nearly
90 GB of RAM.<br>
<br>
This is the second time I've run
the contract process, as my SSH
connection to the server dropped
the first time and the process
wasn't running in a screen etc,
so I assumed after the 40-odd
hours it was running for, the
connection drop caused it to
hang, but now I'm not so sure.
Were there any files I should
maybe have cleared before trying
to run it again?<br>
<br>
I'm using the docker image to
run the command (using
osrm/osrm-backend:latest): time
docker run -t -v
/opt/osrm/data:/data
osrm/osrm-backend osrm-contract
/data/1505492056/planet-latest<wbr>.osrm<br>
<br>
Kind regards,<br>
Kieran Caplice<br>
<br>
<br>
______________________________<wbr>_________________<br>
OSRM-talk mailing list<br>
<a
href="mailto:OSRM-talk@openstreetmap.org"
target="_blank"
moz-do-not-send="true">OSRM-talk@openstreetmap.org</a><br>
<a
href="https://lists.openstreetmap.org/listinfo/osrm-talk"
rel="noreferrer"
target="_blank"
moz-do-not-send="true">https://lists.openstreetmap.or<wbr>g/listinfo/osrm-talk</a><br>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset
class="m_6646586754541093830m_-3407091792784772007mimeAttachmentHeader"></fieldset>
<br>
<pre>______________________________<wbr>_________________
OSRM-talk mailing list
<a class="m_6646586754541093830m_-3407091792784772007moz-txt-link-abbreviated" href="mailto:OSRM-talk@openstreetmap.org" target="_blank" moz-do-not-send="true">OSRM-talk@openstreetmap.org</a>
<a class="m_6646586754541093830m_-3407091792784772007moz-txt-link-freetext" href="https://lists.openstreetmap.org/listinfo/osrm-talk" target="_blank" moz-do-not-send="true">https://lists.openstreetmap.or<wbr>g/listinfo/osrm-talk</a>
</pre>
</blockquote>
<br>
</div>
</div>
</div>
<br>
______________________________<wbr>_________________<br>
OSRM-talk mailing list<br>
<a href="mailto:OSRM-talk@openstreetmap.org"
target="_blank" moz-do-not-send="true">OSRM-talk@openstreetmap.org</a><br>
<a
href="https://lists.openstreetmap.org/listinfo/osrm-talk"
rel="noreferrer" target="_blank"
moz-do-not-send="true">https://lists.openstreetmap.or<wbr>g/listinfo/osrm-talk</a><br>
<br>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset
class="m_6646586754541093830mimeAttachmentHeader"></fieldset>
<br>
<pre>______________________________<wbr>_________________
OSRM-talk mailing list
<a class="m_6646586754541093830moz-txt-link-abbreviated" href="mailto:OSRM-talk@openstreetmap.org" target="_blank" moz-do-not-send="true">OSRM-talk@openstreetmap.org</a>
<a class="m_6646586754541093830moz-txt-link-freetext" href="https://lists.openstreetmap.org/listinfo/osrm-talk" target="_blank" moz-do-not-send="true">https://lists.openstreetmap.<wbr>org/listinfo/osrm-talk</a>
</pre>
</blockquote>
<br>
</div>
</div>
</div>
<br>
______________________________<wbr>_________________<br>
OSRM-talk mailing list<br>
<a href="mailto:OSRM-talk@openstreetmap.org"
moz-do-not-send="true">OSRM-talk@openstreetmap.org</a><br>
<a href="https://lists.openstreetmap.org/listinfo/osrm-talk"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://lists.openstreetmap.<wbr>org/listinfo/osrm-talk</a><br>
<br>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
OSRM-talk mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OSRM-talk@openstreetmap.org">OSRM-talk@openstreetmap.org</a>
<a class="moz-txt-link-freetext" href="https://lists.openstreetmap.org/listinfo/osrm-talk">https://lists.openstreetmap.org/listinfo/osrm-talk</a>
</pre>
</blockquote>
<br>
</body>
</html>