[OSRM-talk] Road speeds and profile restrictions

Richard Marsden winwaed at gmail.com
Thu Oct 15 13:43:25 UTC 2015


Thanks - I'll try to work through the profiles - hopefully they'll
make some sense as I work through them.

osrm-prepare also took two weeks to crash.Perhaps it did have a sudden
spike in memory usage before crashing - it was overnight, so I don't
know for sure. Anyway I'm not in a rush to try another two week run.

Cheers,
Richard

On Wed, Oct 14, 2015 at 4:20 AM, Daniel Hofmann <hofmann at mapbox.com> wrote:
> If osrm-prepare crashed, just add more swap. Also make sure to read this
> ticket:
>> https://github.com/Project-OSRM/osrm-backend/issues/1698
>
> I don't think we have documentation for the profiles. I know they are
> somewhat ugly and huge. I would recommend taking a look at the profile's
> general structure first, so that you roughly understand e.g. that there are
> callbacks for OSM nodes and ways, and what they do. After that I would
> simply trace the globals through the script. That is e.g. search for all
> places where a certain whitelist is used and build a mental model based on
> that. Pen and paper may help :)
>
>> https://github.com/Project-OSRM/osrm-backend/blob/develop/profiles/car.lua
>
> You can find the C++ side of things in the extractor directory:
>> https://github.com/Project-OSRM/osrm-backend/tree/develop/extractor
>
> For example, the scripting environment uses luabind to make a few functions
> visible, e.g. the get_value_by_key function, that then can be used in the
> profiles. The other way, that is using lua functions in C++ is mostly done
> in the extractor implementation: I would recommend just searching for the
> symbols you want to know more about:
>
>>
>> https://github.com/Project-OSRM/osrm-backend/search?utf8=%E2%9C%93&q=node_function&type=Code
>
> Cheers,
> Daniel
>
> On Mon, Oct 12, 2015 at 4:00 AM, Richard Marsden <winwaed at gmail.com> wrote:
>>
>> It is possible I've just managed to demonstrate this to myself.
>> osrm-prepare crashed out on the Europe-wide dataset  at the 90+%
>> complete level with 30GB swap (16GB real). For much of the run it
>> didn't go above about 15GB swap. Very slow of course - but that was to
>> be expected.
>>
>> This continues to be a side / evaluation project, but I'm continuing
>> to investigate. Looks like the PC in question should be able to handle
>> most countries, but will balk at most continents, and the very largest
>> countries (e.g. the US). High-RAM hardware is relatively pricey, but
>> it is a possibility in the new year.
>>
>> Looking at the individual pieces, next I'm going to look at the
>> profile.lua file. I've never used LUA before, but that is easy enough
>> to remedy with online tutorials and the various books that are
>> available. But what about the script that is used by OSRM? Is there
>> any documentation as to the variables and/or functions that are
>> expected?  And the tags used in OSM that the script is expected to
>> map?
>>
>>
>> Cheers,
>>
>> Richard Marsden
>>
>> On Thu, Sep 17, 2015 at 4:37 PM, Patrick Niklaus
>> <patrick.niklaus at student.kit.edu> wrote:
>> > W.r.t. the pre-preprocessing you are correct.
>> >
>> >> What is that extra power used for?
>> >
>> > Including all sorts of external data sources. Also the logic in the
>> > lua profiles is not just replaceable by simple key-value pairs, OSM
>> > requires you to handle a lot of special cases.
>> >
>> >> Presumably I could do the same for world preparation & routing? Have,
>> >> perhaps a 100GB+ swap file, ideally on an SSD.
>> >
>> > This will fall apart when you have some actual load pressure on the
>> > system. We need random access to memory, which will create a lot of
>> > page faults (== slow). Even an SSD is not even close to memory speed.
>> >
>> > You have two options:
>> > - split the datasets
>> > - get a bigger server
>> >
>> > Cheers,
>> > Patrick
>> >
>> >
>> > On Thu, Sep 17, 2015 at 10:06 PM, Richard Marsden <winwaed at gmail.com>
>> > wrote:
>> >> I've been evaluating OSRM, using it primarily as a library from C++.
>> >>
>> >> I believe I've determined the answer to most of the questions, but I'm
>> >> also looking for confirmation.
>> >> (I understand the reason for these constraints - the trade-off of
>> >> speed vs flexibility)
>> >>
>> >> First, road speeds are set with 'profile.lua' at the osrm-extract
>> >> stage. This filters out unnecessary roads (eg. foot paths for car
>> >> routing), but also applies the road speeds.
>> >> If I wish to change the speed profile, I need to regenerate the road
>> >> network with osrm-extract and osrm-routed.
>> >> Correct?
>> >>
>> >> If I wanted different speeds for the final distance/time calculations,
>> >> I could use the returned route, and apply my own speed table according
>> >> to the road type of each road segment. This would not, of course,
>> >> change the route geometry is calculated.
>> >>
>> >> If I want a shortest route (distance optimized) instead of a quickest
>> >> route (time optimized), I need to set all the road speeds to the same
>> >> speed and regenerate the network. I.e. osrm does not directly support
>> >> the concept of a "shortest route".
>> >>
>> >> The profile is provided with a LUA file. I had to look this one up :-)
>> >> Looks a useful scripting language, but why is this profile a script
>> >> file, and not a simple configuration file of constants (eg. key-value
>> >> pairs)?
>> >> Seems like an unnecessary complexity - I'd like to understand the
>> >> perceived advantages. What is that extra power used for?
>> >>
>> >> Finally, the memory usage... I saw a reference to the server requiring
>> >> 40GB of memory for pan-European routing. Presumably that could be
>> >> offset with a large swap file(?)
>> >> A large swap file has worked well when I was testing the US-South
>> >> region on an 8GB machine.
>> >> Presumably I could do the same for world preparation & routing? Have,
>> >> perhaps a 100GB+ swap file, ideally on an SSD.
>> >>
>> >>
>> >> Cheers,
>> >>
>> >> Richard Marsden
>> >>
>> >> _______________________________________________
>> >> OSRM-talk mailing list
>> >> OSRM-talk at openstreetmap.org
>> >> https://lists.openstreetmap.org/listinfo/osrm-talk
>> >
>> > _______________________________________________
>> > OSRM-talk mailing list
>> > OSRM-talk at openstreetmap.org
>> > https://lists.openstreetmap.org/listinfo/osrm-talk
>>
>> _______________________________________________
>> OSRM-talk mailing list
>> OSRM-talk at openstreetmap.org
>> https://lists.openstreetmap.org/listinfo/osrm-talk
>
>
>
> _______________________________________________
> OSRM-talk mailing list
> OSRM-talk at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/osrm-talk
>



More information about the OSRM-talk mailing list