[OSRM-talk] Computing resources

[Andrew King]

Thank you very much Michael. This is most helpful. I shall look into this when I return to work next week.

Please can you just confirm that these memory requirements that you have specified are all without STXXL (which I believe is no longer supported)?

Kind regards
Andrew


________________________________________
From: michael spreng [mailinglists at m.spreng.ch]
Sent: 29 May 2021 08:51
To: osrm-talk at openstreetmap.org
Subject: Re: [OSRM-talk] Computing resources

Hi

I run routing.openstreetmap.de (which is also behind the osrm demo)

Preprocessing has a huge memory demand in the first stage, which forces
me to split the planet into three regions (Americas, Eurasia, and
Africa+Oceania) on a 256GB machine. It might be possible to process it
on 512GB, but it would be already tight, and OSM data is growing.

bike and foot need significantly more space, as there are a lot of ways
which are inaccessible to cars.

130G of ram is needed for the car profile, and 165G for the foot or bike
profile for the server.

When splitting up the data, for each region a separate osrm-routed
instance is needed. I use a python script to dispatch the queries to
each region, see
https://github.com/fossgis-routing-server/request-by-coordinate/

Hope that helps

Michael

On 28.05.21 13:48, Andrew King via OSRM-talk wrote:
> Hi Roland,
>
> Thank you so much for this useful link and advice. This is really helpful.
>
> I have a couple of questions though:
>
> 1. The page refers to 280 GB of STXXL disk space, however, in August
> 2020, STXXL support was removed [commit 2d0b2ca9...21f96221]. Does this
> affect those requirements at all?
>
> 2. You mentioned splitting up the data. Would you then need to run two
> separate services (in your example) for each profile, or can they be
> recombined into one service after processing?
>
> Many thanks
> Andrew
>
>
> ------------------------------------------------------------------------
> *From:* Roland Swingler via OSRM-talk [osrm-talk at openstreetmap.org]
> *Sent:* 28 May 2021 12:14
> *To:* Mailing list to discuss Project OSRM
> *Cc:* Roland Swingler
> *Subject:* Re: [OSRM-talk] Computing resources
>
> This may be useful -
> https://github.com/Project-OSRM/osrm-backend/wiki/Disk-and-Memory-Requirements
> <https://github.com/Project-OSRM/osrm-backend/wiki/Disk-and-Memory-Requirements>
> although I would expect the memory requirements to be a bit higher since
> this page was written.
>
> As far as I know the OSRM processes use multiple cores.
>
> You may want to split up the planet file before processing if you think
> the memory requirements are too high. i.e. you're probably not going to
> have a route from Australia to Europe for example, so they could be run
> in separate instances.
>
> Thanks,
> Roland
>
> On Fri, 28 May 2021 at 12:08, Andrew King <andrew.king at satmap.com
> <mailto:andrew.king at satmap.com>> wrote:
>
>     Hello,
>
>     We have recently taken on responsibility to create osrm instances
>     for foot + bicycle + car routing with planet data, and we are trying
>     to determine what compute specs are recommended in order to be able
>     to process these.
>
>     Do you have this information available, or can you recommend a link
>     where I can find out this information?
>
>     - How much memory is required/recommended to process planet data?
>     - Can the osrm processes (ie osrm-extract, osrm-partition, etc) make
>     use of multiple cores?
>     - How long do each of the processes typically take to run on planet
>     data?
>
>     Kind regards
>     Andrew
>
>     _______________________________________________
>     OSRM-talk mailing list
>     OSRM-talk at openstreetmap.org <mailto:OSRM-talk at openstreetmap.org>
>     https://lists.openstreetmap.org/listinfo/osrm-talk
>     <https://lists.openstreetmap.org/listinfo/osrm-talk>
>
>
> _______________________________________________
> OSRM-talk mailing list
> OSRM-talk at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/osrm-talk
>

_______________________________________________
OSRM-talk mailing list
OSRM-talk at openstreetmap.org
https://lists.openstreetmap.org/listinfo/osrm-talk