[OSRM-talk] map matching service for a huge number of requests

[Patrick Agin]

Thanks Daniel. My first idea was to break my list into sets and running the
job on different machines as you said in 4).
But out of curiosity, can I make calls to libosrm.a functions directly from
Python?
And are you aware of some python code that would do the calls to
osrm-routed in parallel threads?
Regards,
Patrick

2018-05-11 14:07 GMT-04:00 Daniel Patterson <daniel at mapbox.com>:

> Patrick,
>
>   There are about a million possible paths you could take here, a lot of
> it will depend on what skills you have available.  Off the top of my head:
>
>     1) Speed things up by avoiding HTTP overhead and calling the libosrm.a
> functions directly instead of hitting `osrm-routed` over HTTP
>     2) Modify the OSRM C++ source code and strip out the parts of the
> map-matching response you don't need
>     3) Simplify your trace geometries to speed up map-matching
>     4) Break your trace list into sets and run these on multiple machines
> in parallel (make copies of the OSRM data onto multiple machines)
>     5) Just wait 10 hours, and get a good nights sleep
>
>   libosrm.a is thread-safe, so if you're calling functions from threads
> you can do many at once.
>
>   osrm-routed is multi-threaded, so you can run many queries in parallel -
> how many will depend on how many CPUs your machine has.  Profiling
> multi-threaded server performance is kind of beyond the scope of OSRM
> itself, there is lots of literature on it.
>
> daniel
>
> On Fri, May 11, 2018 at 10:57 AM, Patrick Agin <agin.patrick at gmail.com>
> wrote:
>
>> Sorry for the newbie question but what's the difference between
>> osrm-routed and libosrm? Is it mandatory to use the latter for a parallel
>> usage? And do you have an example of code that does the calls in parallel
>> threads? Thanks Sasha for your help.
>> Patrick
>>
>> 2018-05-11 13:50 GMT-04:00 Sasha Khapyorsky <sashakh at gmail.com>:
>>
>>> Hi Patrick,
>>>
>>> If you are using libosrm (which reported to be thread safe:
>>> https://github.com/Project-OSRM/osrm-backend/issues/4966) you can just
>>> split your list and run its parts in different parallel threads.
>>>
>>> Sasha
>>>
>>> On Fri, May 11, 2018 at 8:14 PM, Patrick Agin <agin.patrick at gmail.com>
>>> wrote:
>>> > Hi,
>>> > I want to call the map matching service for nearly a million of
>>> routes. I
>>> > calculated that the task could take 10 hours on my laptop to complete.
>>> I was
>>> > wondering if there's someone who faced the same problem and who
>>> addressed it
>>> > in parallel, maybe with Spark? Another tool or way?
>>> > Any hint is greatly appreciated!
>>> > Patrick
>>> >
>>> > _______________________________________________
>>> > OSRM-talk mailing list
>>> > OSRM-talk at openstreetmap.org
>>> > https://lists.openstreetmap.org/listinfo/osrm-talk
>>> >
>>>
>>> _______________________________________________
>>> OSRM-talk mailing list
>>> OSRM-talk at openstreetmap.org
>>> https://lists.openstreetmap.org/listinfo/osrm-talk
>>>
>>
>>
>> _______________________________________________
>> OSRM-talk mailing list
>> OSRM-talk at openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/osrm-talk
>>
>>
>
> _______________________________________________
> OSRM-talk mailing list
> OSRM-talk at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/osrm-talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/osrm-talk/attachments/20180511/2f30c2ef/attachment.html>