[OSRM-talk] map matching service for a huge number of requests

[Daniel Patterson]

Patrick,

  There are about a million possible paths you could take here, a lot of it
will depend on what skills you have available.  Off the top of my head:

    1) Speed things up by avoiding HTTP overhead and calling the libosrm.a
functions directly instead of hitting `osrm-routed` over HTTP
    2) Modify the OSRM C++ source code and strip out the parts of the
map-matching response you don't need
    3) Simplify your trace geometries to speed up map-matching
    4) Break your trace list into sets and run these on multiple machines
in parallel (make copies of the OSRM data onto multiple machines)
    5) Just wait 10 hours, and get a good nights sleep

  libosrm.a is thread-safe, so if you're calling functions from threads you
can do many at once.

  osrm-routed is multi-threaded, so you can run many queries in parallel -
how many will depend on how many CPUs your machine has.  Profiling
multi-threaded server performance is kind of beyond the scope of OSRM
itself, there is lots of literature on it.

daniel

On Fri, May 11, 2018 at 10:57 AM, Patrick Agin <agin.patrick at gmail.com>
wrote:

> Sorry for the newbie question but what's the difference between
> osrm-routed and libosrm? Is it mandatory to use the latter for a parallel
> usage? And do you have an example of code that does the calls in parallel
> threads? Thanks Sasha for your help.
> Patrick
>
> 2018-05-11 13:50 GMT-04:00 Sasha Khapyorsky <sashakh at gmail.com>:
>
>> Hi Patrick,
>>
>> If you are using libosrm (which reported to be thread safe:
>> https://github.com/Project-OSRM/osrm-backend/issues/4966) you can just
>> split your list and run its parts in different parallel threads.
>>
>> Sasha
>>
>> On Fri, May 11, 2018 at 8:14 PM, Patrick Agin <agin.patrick at gmail.com>
>> wrote:
>> > Hi,
>> > I want to call the map matching service for nearly a million of routes.
>> I
>> > calculated that the task could take 10 hours on my laptop to complete.
>> I was
>> > wondering if there's someone who faced the same problem and who
>> addressed it
>> > in parallel, maybe with Spark? Another tool or way?
>> > Any hint is greatly appreciated!
>> > Patrick
>> >
>> > _______________________________________________
>> > OSRM-talk mailing list
>> > OSRM-talk at openstreetmap.org
>> > https://lists.openstreetmap.org/listinfo/osrm-talk
>> >
>>
>> _______________________________________________
>> OSRM-talk mailing list
>> OSRM-talk at openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/osrm-talk
>>
>
>
> _______________________________________________
> OSRM-talk mailing list
> OSRM-talk at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/osrm-talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/osrm-talk/attachments/20180511/26cfae23/attachment-0001.html>