<div dir="ltr">thanks Sasha. And do you know if it's possible to do calls to libosrm functions directly from Python?</div><div class="gmail_extra"><br><div class="gmail_quote">2018-05-11 14:33 GMT-04:00 Sasha Khapyorsky <span dir="ltr"><<a href="mailto:sashakh@gmail.com" target="_blank">sashakh@gmail.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi again, Patrick,<br>
<span class=""><br>
On Fri, May 11, 2018 at 9:22 PM, Patrick Agin <<a href="mailto:agin.patrick@gmail.com">agin.patrick@gmail.com</a>> wrote:<br>
><br>
> And are you aware of some python code that would do the calls to osrm-routed<br>
> in parallel threads?<br>
<br>
</span>There are lot of examples of how to make python things in parallel<br>
threads. For example:<br>
<a href="https://stackoverflow.com/questions/2846653/how-to-use-threading-in-python" rel="noreferrer" target="_blank">https://stackoverflow.com/<wbr>questions/2846653/how-to-use-<wbr>threading-in-python</a><br>
<span class="HOEnZb"><font color="#888888"><br>
Sasha<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
> Regards,<br>
> Patrick<br>
><br>
> 2018-05-11 14:07 GMT-04:00 Daniel Patterson <<a href="mailto:daniel@mapbox.com">daniel@mapbox.com</a>>:<br>
>><br>
>> Patrick,<br>
>><br>
>> There are about a million possible paths you could take here, a lot of<br>
>> it will depend on what skills you have available. Off the top of my head:<br>
>><br>
>> 1) Speed things up by avoiding HTTP overhead and calling the libosrm.a<br>
>> functions directly instead of hitting `osrm-routed` over HTTP<br>
>> 2) Modify the OSRM C++ source code and strip out the parts of the<br>
>> map-matching response you don't need<br>
>> 3) Simplify your trace geometries to speed up map-matching<br>
>> 4) Break your trace list into sets and run these on multiple machines<br>
>> in parallel (make copies of the OSRM data onto multiple machines)<br>
>> 5) Just wait 10 hours, and get a good nights sleep<br>
>><br>
>> libosrm.a is thread-safe, so if you're calling functions from threads<br>
>> you can do many at once.<br>
>><br>
>> osrm-routed is multi-threaded, so you can run many queries in parallel -<br>
>> how many will depend on how many CPUs your machine has. Profiling<br>
>> multi-threaded server performance is kind of beyond the scope of OSRM<br>
>> itself, there is lots of literature on it.<br>
>><br>
>> daniel<br>
>><br>
>> On Fri, May 11, 2018 at 10:57 AM, Patrick Agin <<a href="mailto:agin.patrick@gmail.com">agin.patrick@gmail.com</a>><br>
>> wrote:<br>
>>><br>
>>> Sorry for the newbie question but what's the difference between<br>
>>> osrm-routed and libosrm? Is it mandatory to use the latter for a parallel<br>
>>> usage? And do you have an example of code that does the calls in parallel<br>
>>> threads? Thanks Sasha for your help.<br>
>>> Patrick<br>
>>><br>
>>> 2018-05-11 13:50 GMT-04:00 Sasha Khapyorsky <<a href="mailto:sashakh@gmail.com">sashakh@gmail.com</a>>:<br>
>>>><br>
>>>> Hi Patrick,<br>
>>>><br>
>>>> If you are using libosrm (which reported to be thread safe:<br>
>>>> <a href="https://github.com/Project-OSRM/osrm-backend/issues/4966" rel="noreferrer" target="_blank">https://github.com/Project-<wbr>OSRM/osrm-backend/issues/4966</a>) you can just<br>
>>>> split your list and run its parts in different parallel threads.<br>
>>>><br>
>>>> Sasha<br>
>>>><br>
>>>> On Fri, May 11, 2018 at 8:14 PM, Patrick Agin <<a href="mailto:agin.patrick@gmail.com">agin.patrick@gmail.com</a>><br>
>>>> wrote:<br>
>>>> > Hi,<br>
>>>> > I want to call the map matching service for nearly a million of<br>
>>>> > routes. I<br>
>>>> > calculated that the task could take 10 hours on my laptop to complete.<br>
>>>> > I was<br>
>>>> > wondering if there's someone who faced the same problem and who<br>
>>>> > addressed it<br>
>>>> > in parallel, maybe with Spark? Another tool or way?<br>
>>>> > Any hint is greatly appreciated!<br>
>>>> > Patrick<br>
>>>> ><br>
>>>> > ______________________________<wbr>_________________<br>
>>>> > OSRM-talk mailing list<br>
>>>> > <a href="mailto:OSRM-talk@openstreetmap.org">OSRM-talk@openstreetmap.org</a><br>
>>>> > <a href="https://lists.openstreetmap.org/listinfo/osrm-talk" rel="noreferrer" target="_blank">https://lists.openstreetmap.<wbr>org/listinfo/osrm-talk</a><br>
>>>> ><br>
>>>><br>
>>>> ______________________________<wbr>_________________<br>
>>>> OSRM-talk mailing list<br>
>>>> <a href="mailto:OSRM-talk@openstreetmap.org">OSRM-talk@openstreetmap.org</a><br>
>>>> <a href="https://lists.openstreetmap.org/listinfo/osrm-talk" rel="noreferrer" target="_blank">https://lists.openstreetmap.<wbr>org/listinfo/osrm-talk</a><br>
>>><br>
>>><br>
>>><br>
>>> ______________________________<wbr>_________________<br>
>>> OSRM-talk mailing list<br>
>>> <a href="mailto:OSRM-talk@openstreetmap.org">OSRM-talk@openstreetmap.org</a><br>
>>> <a href="https://lists.openstreetmap.org/listinfo/osrm-talk" rel="noreferrer" target="_blank">https://lists.openstreetmap.<wbr>org/listinfo/osrm-talk</a><br>
>>><br>
>><br>
>><br>
>> ______________________________<wbr>_________________<br>
>> OSRM-talk mailing list<br>
>> <a href="mailto:OSRM-talk@openstreetmap.org">OSRM-talk@openstreetmap.org</a><br>
>> <a href="https://lists.openstreetmap.org/listinfo/osrm-talk" rel="noreferrer" target="_blank">https://lists.openstreetmap.<wbr>org/listinfo/osrm-talk</a><br>
>><br>
><br>
><br>
> ______________________________<wbr>_________________<br>
> OSRM-talk mailing list<br>
> <a href="mailto:OSRM-talk@openstreetmap.org">OSRM-talk@openstreetmap.org</a><br>
> <a href="https://lists.openstreetmap.org/listinfo/osrm-talk" rel="noreferrer" target="_blank">https://lists.openstreetmap.<wbr>org/listinfo/osrm-talk</a><br>
><br>
<br>
______________________________<wbr>_________________<br>
OSRM-talk mailing list<br>
<a href="mailto:OSRM-talk@openstreetmap.org">OSRM-talk@openstreetmap.org</a><br>
<a href="https://lists.openstreetmap.org/listinfo/osrm-talk" rel="noreferrer" target="_blank">https://lists.openstreetmap.<wbr>org/listinfo/osrm-talk</a><br>
</div></div></blockquote></div><br></div>