[OSRM-talk] Expecting time for osrm-contract for planet

Daniel Patterson daniel at mapbox.com
Fri Sep 22 14:22:04 UTC 2017


OSRM pre-processing is kind of worst-case for disk swapping - it does a lot
of random access, so if you run out of RAM, pages will get continuously
swapped in/out.  The slowdown between RAM-only and swap-on-SSD can be 50x
or more.  The difference between swap on a spinning disk and an SSD is a
good 10x on top of that.

If you can, avoid letting OSRM pre-processing touch swap at all costs -
it's worth renting a big machine on AWS/Google/Azure for a short time to
pre-process the data.

daniel

On Fri, Sep 22, 2017 at 7:14 AM, Sayer, Bryan <BSayer at s-3.com> wrote:

> Just as a point of reference, you mention that avoiding swapping and just
> using RAM will be much faster. If the swap space is an SSD drive, how much
> does using only RAM speed up contract? That is, what is the difference
> between using say a 7200 rpm SATA drive versus an SSD drive make?
> ------------------------------
> *From:* Daniel Patterson <daniel at mapbox.com>
> *Sent:* Friday, September 22, 2017 8:05:56 AM
> *To:* Mailing list to discuss Project OSRM
> *Subject:* Re: [OSRM-talk] Expecting time for osrm-contract for planet
>
> Hi Kieran,
>
>   We don't, but I've done some profiling in the past, Docker itself should
> have negligible impact.
>
>   The two things that might be slowing things down:
>
>     1) The docker images don't bundle jemalloc (http://jemalloc.net/),
> which can speed things up by 10-15%
>     2) osrm-contract is very CPU intensive - the more threads you give it,
> the faster it will run.  Give it more if you can.
>
>  256GB of RAM should be more than enough to avoid swapping, but you might
> want to disable swap to be sure - if it's being used, it will have a huge
> impact on speed.
>
> daniel
>
> On Fri, Sep 22, 2017 at 2:30 AM, Kieran Caplice <
> kieran.caplice at temetra.com> wrote:
>
>> Hi Daniel,
>> Can you clarify if you use Docker at Mapbox? What kind of server do you
>> guys have (if not the one described on the wiki page)?
>>
>> Over night, the process has gone to just 75% complete from 65% yesterday
>> - 41 hours in total now, and back to maxing CPU again.
>>
>> Kind regards,
>> Kieran Caplice
>>
>> On 21/09/17 17:17, Daniel Patterson wrote:
>>
>> OSRM supports *two* core routing algorithms - CH and MLD.  The
>> `osrm-contract` tool generates the CH dataset, but you can use the MLD
>> pipeline instead with:
>>
>> osrm-extract -p profiles/bicycle.lua yourmap.osm.pbf
>> osrm-partition yourmap.osrm
>> osrm-customize yourmap.osrm
>> osrm-routed -a MLD yourmap.osrm
>>
>> This sequence of tools should be significantly quicker than osrm-contract
>> - the price you pay is that routing requests are about 5x slower (still
>> pretty fast though!).  The reason that MLD exists is for the
>> `osrm-customize` step - it allows you to import traffic data very quickly
>> and update the routing graph (~1 minute for North America).
>>
>> It's hard to say exactly what's going wrong with osrm-contract here -
>> here at Mapbox, we daily run `osrm-contract` over the latest planet with
>> the bicycle profile without a problem, however, Alex and others have
>> reported issues with what seem like hangs on much smaller datasets that
>> we've been unable to reproduce so far.
>>
>> The runtime of osrm-contract is affected by how much hierarchy exists in
>> the data - the more similar the edge speeds (like in foot) and the more
>> edges there are, the slower it gets, often in a non-linear fashion.  The
>> car profile has a very hierarchical structure (many different road speeds),
>> so it fits well into the CH, the construction algorithm  doesn't need to
>> compare as many options.
>>
>> daniel
>>
>> On Thu, Sep 21, 2017 at 9:03 AM, Kieran Caplice <
>> kieran.caplice at temetra.com> wrote:
>>
>>> We're actually looking for the best of both car and foot, so in my head,
>>> bicycle would be the happy medium (though I could be completely wrong on
>>> this).
>>> Kind regards,
>>> Kieran Caplice
>>>
>>> On 21/09/17 16:53, Alex Farioletti wrote:
>>>
>>> i've run into the same issues, and now i just use metroextracts of the
>>> areas that i need for the bike stuff i do and it reduces the time
>>> significantly
>>>
>>> *Alex Farioletti*
>>> *415.312.1674*
>>> *tcbcourier.com <http://tcbcourier.com> *
>>>
>>> On Thu, Sep 21, 2017 at 8:49 AM, Kieran Caplice <
>>> kieran.caplice at temetra.com> wrote:
>>>
>>>> Thanks Daniel.
>>>>
>>>> I'm using the bicycle profile, so I would expect based on what you've
>>>> said that somewhere up to 36 hours would be likely. However, this is the
>>>> current output, after 25h40m:
>>>> [info] Input file: /data/1505492056/planet-latest.osrm
>>>> [info] Threads: 12
>>>> [info] Reading node weights.
>>>> [info] Done reading node weights.
>>>> [info] Loading edge-expanded graph representation
>>>> [info] merged 2379332 edges out of 1777752432
>>>> [info] initializing node priorities... ok.
>>>> [info] preprocessing 389797971 (90%) nodes...
>>>> [info] . 10% . 20% . 30% . 40% . 50% . 60%
>>>>
>>>> It hasn't advanced past 60% in the last 2-3 hours. It is however maxing
>>>> CPU and using approximately the same amount of RAM since it started.
>>>>
>>>> Kind regards,
>>>> Kieran Caplice
>>>>
>>>> On 21/09/17 16:39, Daniel Patterson wrote:
>>>>
>>>> Hi Kieran,
>>>>
>>>>   The contraction time will be slow - many, many hours for the whole
>>>> planet.  *Typically* for the car profile it's about 12 hours, but if you
>>>> use bike or foot, or your own profile, it can get a lot bigger.
>>>>
>>>>   If you've messed with the travel speeds, that can have a big effect
>>>> too.  24 hours is not unheard of, but whether it's legit will depend a lot
>>>> on the details.
>>>>
>>>> daniel
>>>>
>>>> On Thu, Sep 21, 2017 at 7:00 AM, Kieran Caplice <
>>>> kieran.caplice at temetra.com> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> Could anyone give an approx estimate for the time required to run the
>>>>> osrm-contract on planet data on a 12 thread, 256 GB RAM, SSD machine?
>>>>>
>>>>> The osrm-extract process finished in 232 minutes, but the contract has
>>>>> now been running solid for 24 hours, and appears to be stuck at 60% on
>>>>> "preprocessing nodes". All 12 cores are generally maxed out, and the
>>>>> process is using nearly 90 GB of RAM.
>>>>>
>>>>> This is the second time I've run the contract process, as my SSH
>>>>> connection to the server dropped the first time and the process wasn't
>>>>> running in a screen etc, so I assumed after the 40-odd hours it was running
>>>>> for, the connection drop caused it to hang, but now I'm not so sure. Were
>>>>> there any files I should maybe have cleared before trying to run it again?
>>>>>
>>>>> I'm using the docker image to run the command (using
>>>>> osrm/osrm-backend:latest): time docker run -t -v /opt/osrm/data:/data
>>>>> osrm/osrm-backend osrm-contract /data/1505492056/planet-latest.osrm
>>>>>
>>>>> Kind regards,
>>>>> Kieran Caplice
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> OSRM-talk mailing list
>>>>> OSRM-talk at openstreetmap.org
>>>>> https://lists.openstreetmap.org/listinfo/osrm-talk
>>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> OSRM-talk mailing listOSRM-talk at openstreetmap.orghttps://lists.openstreetmap.org/listinfo/osrm-talk
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> OSRM-talk mailing list
>>>> OSRM-talk at openstreetmap.org
>>>> https://lists.openstreetmap.org/listinfo/osrm-talk
>>>>
>>>>
>>>
>>>
>>> _______________________________________________
>>> OSRM-talk mailing listOSRM-talk at openstreetmap.orghttps://lists.openstreetmap.org/listinfo/osrm-talk
>>>
>>>
>>>
>>> _______________________________________________
>>> OSRM-talk mailing list
>>> OSRM-talk at openstreetmap.org
>>> https://lists.openstreetmap.org/listinfo/osrm-talk
>>>
>>>
>>
>>
>> _______________________________________________
>> OSRM-talk mailing listOSRM-talk at openstreetmap.orghttps://lists.openstreetmap.org/listinfo/osrm-talk
>>
>>
>>
>> _______________________________________________
>> OSRM-talk mailing list
>> OSRM-talk at openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/osrm-talk
>>
>>
>
> ****WARNING**** This information may be confidential. It is intended only
> for the addressee(s) identified above. If you are not the addressee(s), or
> an employee or agent of the addressee(s), please note that any
> dissemination, distribution, or copying of this communication is strictly
> prohibited. If you have received this information in error, please destroy
> the information and notify the sender of the error. Thank you.
>
> _______________________________________________
> OSRM-talk mailing list
> OSRM-talk at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/osrm-talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/osrm-talk/attachments/20170922/15bc9484/attachment.html>


More information about the OSRM-talk mailing list