[OSRM-talk] map matching service for a huge number of requests

[Sasha Khapyorsky]

Hi again, Patrick,

On Fri, May 11, 2018 at 9:22 PM, Patrick Agin <agin.patrick at gmail.com> wrote:
>
> And are you aware of some python code that would do the calls to osrm-routed
> in parallel threads?

There are lot of examples of how to make python things in parallel
threads. For example:
https://stackoverflow.com/questions/2846653/how-to-use-threading-in-python

Sasha

> Regards,
> Patrick
>
> 2018-05-11 14:07 GMT-04:00 Daniel Patterson <daniel at mapbox.com>:
>>
>> Patrick,
>>
>>   There are about a million possible paths you could take here, a lot of
>> it will depend on what skills you have available.  Off the top of my head:
>>
>>     1) Speed things up by avoiding HTTP overhead and calling the libosrm.a
>> functions directly instead of hitting `osrm-routed` over HTTP
>>     2) Modify the OSRM C++ source code and strip out the parts of the
>> map-matching response you don't need
>>     3) Simplify your trace geometries to speed up map-matching
>>     4) Break your trace list into sets and run these on multiple machines
>> in parallel (make copies of the OSRM data onto multiple machines)
>>     5) Just wait 10 hours, and get a good nights sleep
>>
>>   libosrm.a is thread-safe, so if you're calling functions from threads
>> you can do many at once.
>>
>>   osrm-routed is multi-threaded, so you can run many queries in parallel -
>> how many will depend on how many CPUs your machine has.  Profiling
>> multi-threaded server performance is kind of beyond the scope of OSRM
>> itself, there is lots of literature on it.
>>
>> daniel
>>
>> On Fri, May 11, 2018 at 10:57 AM, Patrick Agin <agin.patrick at gmail.com>
>> wrote:
>>>
>>> Sorry for the newbie question but what's the difference between
>>> osrm-routed and libosrm? Is it mandatory to use the latter for a parallel
>>> usage? And do you have an example of code that does the calls in parallel
>>> threads? Thanks Sasha for your help.
>>> Patrick
>>>
>>> 2018-05-11 13:50 GMT-04:00 Sasha Khapyorsky <sashakh at gmail.com>:
>>>>
>>>> Hi Patrick,
>>>>
>>>> If you are using libosrm (which reported to be thread safe:
>>>> https://github.com/Project-OSRM/osrm-backend/issues/4966) you can just
>>>> split your list and run its parts in different parallel threads.
>>>>
>>>> Sasha
>>>>
>>>> On Fri, May 11, 2018 at 8:14 PM, Patrick Agin <agin.patrick at gmail.com>
>>>> wrote:
>>>> > Hi,
>>>> > I want to call the map matching service for nearly a million of
>>>> > routes. I
>>>> > calculated that the task could take 10 hours on my laptop to complete.
>>>> > I was
>>>> > wondering if there's someone who faced the same problem and who
>>>> > addressed it
>>>> > in parallel, maybe with Spark? Another tool or way?
>>>> > Any hint is greatly appreciated!
>>>> > Patrick
>>>> >
>>>> > _______________________________________________
>>>> > OSRM-talk mailing list
>>>> > OSRM-talk at openstreetmap.org
>>>> > https://lists.openstreetmap.org/listinfo/osrm-talk
>>>> >
>>>>
>>>> _______________________________________________
>>>> OSRM-talk mailing list
>>>> OSRM-talk at openstreetmap.org
>>>> https://lists.openstreetmap.org/listinfo/osrm-talk
>>>
>>>
>>>
>>> _______________________________________________
>>> OSRM-talk mailing list
>>> OSRM-talk at openstreetmap.org
>>> https://lists.openstreetmap.org/listinfo/osrm-talk
>>>
>>
>>
>> _______________________________________________
>> OSRM-talk mailing list
>> OSRM-talk at openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/osrm-talk
>>
>
>
> _______________________________________________
> OSRM-talk mailing list
> OSRM-talk at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/osrm-talk
>