[OSRM-talk] Road speeds and profile restrictions

[Daniel Hofmann]

If osrm-prepare crashed, just add more swap. Also make sure to read this
ticket:
> https://github.com/Project-OSRM/osrm-backend/issues/1698

I don't think we have documentation for the profiles. I know they are
somewhat ugly and huge. I would recommend taking a look at the profile's
general structure first, so that you roughly understand e.g. that there are
callbacks for OSM nodes and ways, and what they do. After that I would
simply trace the globals through the script. That is e.g. search for all
places where a certain whitelist is used and build a mental model based on
that. Pen and paper may help :)

> https://github.com/Project-OSRM/osrm-backend/blob/develop/profiles/car.lua

You can find the C++ side of things in the extractor directory:
> https://github.com/Project-OSRM/osrm-backend/tree/develop/extractor

For example, the scripting environment uses luabind to make a few functions
visible, e.g. the get_value_by_key function, that then can be used in the
profiles. The other way, that is using lua functions in C++ is mostly done
in the extractor implementation: I would recommend just searching for the
symbols you want to know more about:

>
https://github.com/Project-OSRM/osrm-backend/search?utf8=%E2%9C%93&q=node_function&type=Code

Cheers,
Daniel

On Mon, Oct 12, 2015 at 4:00 AM, Richard Marsden <winwaed at gmail.com> wrote:

> It is possible I've just managed to demonstrate this to myself.
> osrm-prepare crashed out on the Europe-wide dataset  at the 90+%
> complete level with 30GB swap (16GB real). For much of the run it
> didn't go above about 15GB swap. Very slow of course - but that was to
> be expected.
>
> This continues to be a side / evaluation project, but I'm continuing
> to investigate. Looks like the PC in question should be able to handle
> most countries, but will balk at most continents, and the very largest
> countries (e.g. the US). High-RAM hardware is relatively pricey, but
> it is a possibility in the new year.
>
> Looking at the individual pieces, next I'm going to look at the
> profile.lua file. I've never used LUA before, but that is easy enough
> to remedy with online tutorials and the various books that are
> available. But what about the script that is used by OSRM? Is there
> any documentation as to the variables and/or functions that are
> expected?  And the tags used in OSM that the script is expected to
> map?
>
>
> Cheers,
>
> Richard Marsden
>
> On Thu, Sep 17, 2015 at 4:37 PM, Patrick Niklaus
> <patrick.niklaus at student.kit.edu> wrote:
> > W.r.t. the pre-preprocessing you are correct.
> >
> >> What is that extra power used for?
> >
> > Including all sorts of external data sources. Also the logic in the
> > lua profiles is not just replaceable by simple key-value pairs, OSM
> > requires you to handle a lot of special cases.
> >
> >> Presumably I could do the same for world preparation & routing? Have,
> perhaps a 100GB+ swap file, ideally on an SSD.
> >
> > This will fall apart when you have some actual load pressure on the
> > system. We need random access to memory, which will create a lot of
> > page faults (== slow). Even an SSD is not even close to memory speed.
> >
> > You have two options:
> > - split the datasets
> > - get a bigger server
> >
> > Cheers,
> > Patrick
> >
> >
> > On Thu, Sep 17, 2015 at 10:06 PM, Richard Marsden <winwaed at gmail.com>
> wrote:
> >> I've been evaluating OSRM, using it primarily as a library from C++.
> >>
> >> I believe I've determined the answer to most of the questions, but I'm
> >> also looking for confirmation.
> >> (I understand the reason for these constraints - the trade-off of
> >> speed vs flexibility)
> >>
> >> First, road speeds are set with 'profile.lua' at the osrm-extract
> >> stage. This filters out unnecessary roads (eg. foot paths for car
> >> routing), but also applies the road speeds.
> >> If I wish to change the speed profile, I need to regenerate the road
> >> network with osrm-extract and osrm-routed.
> >> Correct?
> >>
> >> If I wanted different speeds for the final distance/time calculations,
> >> I could use the returned route, and apply my own speed table according
> >> to the road type of each road segment. This would not, of course,
> >> change the route geometry is calculated.
> >>
> >> If I want a shortest route (distance optimized) instead of a quickest
> >> route (time optimized), I need to set all the road speeds to the same
> >> speed and regenerate the network. I.e. osrm does not directly support
> >> the concept of a "shortest route".
> >>
> >> The profile is provided with a LUA file. I had to look this one up :-)
> >> Looks a useful scripting language, but why is this profile a script
> >> file, and not a simple configuration file of constants (eg. key-value
> >> pairs)?
> >> Seems like an unnecessary complexity - I'd like to understand the
> >> perceived advantages. What is that extra power used for?
> >>
> >> Finally, the memory usage... I saw a reference to the server requiring
> >> 40GB of memory for pan-European routing. Presumably that could be
> >> offset with a large swap file(?)
> >> A large swap file has worked well when I was testing the US-South
> >> region on an 8GB machine.
> >> Presumably I could do the same for world preparation & routing? Have,
> >> perhaps a 100GB+ swap file, ideally on an SSD.
> >>
> >>
> >> Cheers,
> >>
> >> Richard Marsden
> >>
> >> _______________________________________________
> >> OSRM-talk mailing list
> >> OSRM-talk at openstreetmap.org
> >> https://lists.openstreetmap.org/listinfo/osrm-talk
> >
> > _______________________________________________
> > OSRM-talk mailing list
> > OSRM-talk at openstreetmap.org
> > https://lists.openstreetmap.org/listinfo/osrm-talk
>
> _______________________________________________
> OSRM-talk mailing list
> OSRM-talk at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/osrm-talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/osrm-talk/attachments/20151014/cc19b38d/attachment.html>