[OSRM-talk] map matching service for a huge number of requests

[Patrick Agin]

thanks Sasha. And do you know if it's possible to do calls to libosrm
functions directly from Python?

2018-05-11 14:33 GMT-04:00 Sasha Khapyorsky <sashakh at gmail.com>:

> Hi again, Patrick,
>
> On Fri, May 11, 2018 at 9:22 PM, Patrick Agin <agin.patrick at gmail.com>
> wrote:
> >
> > And are you aware of some python code that would do the calls to
> osrm-routed
> > in parallel threads?
>
> There are lot of examples of how to make python things in parallel
> threads. For example:
> https://stackoverflow.com/questions/2846653/how-to-use-threading-in-python
>
> Sasha
>
> > Regards,
> > Patrick
> >
> > 2018-05-11 14:07 GMT-04:00 Daniel Patterson <daniel at mapbox.com>:
> >>
> >> Patrick,
> >>
> >>   There are about a million possible paths you could take here, a lot of
> >> it will depend on what skills you have available.  Off the top of my
> head:
> >>
> >>     1) Speed things up by avoiding HTTP overhead and calling the
> libosrm.a
> >> functions directly instead of hitting `osrm-routed` over HTTP
> >>     2) Modify the OSRM C++ source code and strip out the parts of the
> >> map-matching response you don't need
> >>     3) Simplify your trace geometries to speed up map-matching
> >>     4) Break your trace list into sets and run these on multiple
> machines
> >> in parallel (make copies of the OSRM data onto multiple machines)
> >>     5) Just wait 10 hours, and get a good nights sleep
> >>
> >>   libosrm.a is thread-safe, so if you're calling functions from threads
> >> you can do many at once.
> >>
> >>   osrm-routed is multi-threaded, so you can run many queries in
> parallel -
> >> how many will depend on how many CPUs your machine has.  Profiling
> >> multi-threaded server performance is kind of beyond the scope of OSRM
> >> itself, there is lots of literature on it.
> >>
> >> daniel
> >>
> >> On Fri, May 11, 2018 at 10:57 AM, Patrick Agin <agin.patrick at gmail.com>
> >> wrote:
> >>>
> >>> Sorry for the newbie question but what's the difference between
> >>> osrm-routed and libosrm? Is it mandatory to use the latter for a
> parallel
> >>> usage? And do you have an example of code that does the calls in
> parallel
> >>> threads? Thanks Sasha for your help.
> >>> Patrick
> >>>
> >>> 2018-05-11 13:50 GMT-04:00 Sasha Khapyorsky <sashakh at gmail.com>:
> >>>>
> >>>> Hi Patrick,
> >>>>
> >>>> If you are using libosrm (which reported to be thread safe:
> >>>> https://github.com/Project-OSRM/osrm-backend/issues/4966) you can
> just
> >>>> split your list and run its parts in different parallel threads.
> >>>>
> >>>> Sasha
> >>>>
> >>>> On Fri, May 11, 2018 at 8:14 PM, Patrick Agin <agin.patrick at gmail.com
> >
> >>>> wrote:
> >>>> > Hi,
> >>>> > I want to call the map matching service for nearly a million of
> >>>> > routes. I
> >>>> > calculated that the task could take 10 hours on my laptop to
> complete.
> >>>> > I was
> >>>> > wondering if there's someone who faced the same problem and who
> >>>> > addressed it
> >>>> > in parallel, maybe with Spark? Another tool or way?
> >>>> > Any hint is greatly appreciated!
> >>>> > Patrick
> >>>> >
> >>>> > _______________________________________________
> >>>> > OSRM-talk mailing list
> >>>> > OSRM-talk at openstreetmap.org
> >>>> > https://lists.openstreetmap.org/listinfo/osrm-talk
> >>>> >
> >>>>
> >>>> _______________________________________________
> >>>> OSRM-talk mailing list
> >>>> OSRM-talk at openstreetmap.org
> >>>> https://lists.openstreetmap.org/listinfo/osrm-talk
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> OSRM-talk mailing list
> >>> OSRM-talk at openstreetmap.org
> >>> https://lists.openstreetmap.org/listinfo/osrm-talk
> >>>
> >>
> >>
> >> _______________________________________________
> >> OSRM-talk mailing list
> >> OSRM-talk at openstreetmap.org
> >> https://lists.openstreetmap.org/listinfo/osrm-talk
> >>
> >
> >
> > _______________________________________________
> > OSRM-talk mailing list
> > OSRM-talk at openstreetmap.org
> > https://lists.openstreetmap.org/listinfo/osrm-talk
> >
>
> _______________________________________________
> OSRM-talk mailing list
> OSRM-talk at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/osrm-talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/osrm-talk/attachments/20180511/f075e020/attachment.html>