[OSRM-talk] Computing resources

[michael spreng]

Yes this is with the current version which does not have stxxl.  Disk
usage is about 300G per profile

On 29.05.21 09:58, Andrew King wrote:
> Thank you very much Michael. This is most helpful. I shall look into this when I return to work next week.
> 
> Please can you just confirm that these memory requirements that you have specified are all without STXXL (which I believe is no longer supported)?
> 
> Kind regards
> Andrew
> 
> 
> ________________________________________
> From: michael spreng [mailinglists at m.spreng.ch]
> Sent: 29 May 2021 08:51
> To: osrm-talk at openstreetmap.org
> Subject: Re: [OSRM-talk] Computing resources
> 
> Hi
> 
> I run routing.openstreetmap.de (which is also behind the osrm demo)
> 
> Preprocessing has a huge memory demand in the first stage, which forces
> me to split the planet into three regions (Americas, Eurasia, and
> Africa+Oceania) on a 256GB machine. It might be possible to process it
> on 512GB, but it would be already tight, and OSM data is growing.
> 
> bike and foot need significantly more space, as there are a lot of ways
> which are inaccessible to cars.
> 
> 130G of ram is needed for the car profile, and 165G for the foot or bike
> profile for the server.
> 
> When splitting up the data, for each region a separate osrm-routed
> instance is needed. I use a python script to dispatch the queries to
> each region, see
> https://github.com/fossgis-routing-server/request-by-coordinate/
> 
> Hope that helps
> 
> Michael
> 
> On 28.05.21 13:48, Andrew King via OSRM-talk wrote:
>> Hi Roland,
>>
>> Thank you so much for this useful link and advice. This is really helpful.
>>
>> I have a couple of questions though:
>>
>> 1. The page refers to 280 GB of STXXL disk space, however, in August
>> 2020, STXXL support was removed [commit 2d0b2ca9...21f96221]. Does this
>> affect those requirements at all?
>>
>> 2. You mentioned splitting up the data. Would you then need to run two
>> separate services (in your example) for each profile, or can they be
>> recombined into one service after processing?
>>
>> Many thanks
>> Andrew
>>
>>
>> ------------------------------------------------------------------------
>> *From:* Roland Swingler via OSRM-talk [osrm-talk at openstreetmap.org]
>> *Sent:* 28 May 2021 12:14
>> *To:* Mailing list to discuss Project OSRM
>> *Cc:* Roland Swingler
>> *Subject:* Re: [OSRM-talk] Computing resources
>>
>> This may be useful -
>> https://github.com/Project-OSRM/osrm-backend/wiki/Disk-and-Memory-Requirements
>> <https://github.com/Project-OSRM/osrm-backend/wiki/Disk-and-Memory-Requirements>
>> although I would expect the memory requirements to be a bit higher since
>> this page was written.
>>
>> As far as I know the OSRM processes use multiple cores.
>>
>> You may want to split up the planet file before processing if you think
>> the memory requirements are too high. i.e. you're probably not going to
>> have a route from Australia to Europe for example, so they could be run
>> in separate instances.
>>
>> Thanks,
>> Roland
>>
>> On Fri, 28 May 2021 at 12:08, Andrew King <andrew.king at satmap.com
>> <mailto:andrew.king at satmap.com>> wrote:
>>
>>     Hello,
>>
>>     We have recently taken on responsibility to create osrm instances
>>     for foot + bicycle + car routing with planet data, and we are trying
>>     to determine what compute specs are recommended in order to be able
>>     to process these.
>>
>>     Do you have this information available, or can you recommend a link
>>     where I can find out this information?
>>
>>     - How much memory is required/recommended to process planet data?
>>     - Can the osrm processes (ie osrm-extract, osrm-partition, etc) make
>>     use of multiple cores?
>>     - How long do each of the processes typically take to run on planet
>>     data?
>>
>>     Kind regards
>>     Andrew
>>
>>     _______________________________________________
>>     OSRM-talk mailing list
>>     OSRM-talk at openstreetmap.org <mailto:OSRM-talk at openstreetmap.org>
>>     https://lists.openstreetmap.org/listinfo/osrm-talk
>>     <https://lists.openstreetmap.org/listinfo/osrm-talk>
>>
>>
>> _______________________________________________
>> OSRM-talk mailing list
>> OSRM-talk at openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/osrm-talk
>>
> 
> _______________________________________________
> OSRM-talk mailing list
> OSRM-talk at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/osrm-talk
> 
> _______________________________________________
> OSRM-talk mailing list
> OSRM-talk at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/osrm-talk
>