<div dir="ltr">Hi Daniel<div><br></div><div>Great Answer, thanks so much for your time. I guess I have to learn Java and figure out HashTables now :(, your suggestion is worth putting some effort into.</div><div><br></div>
<div>Do you, or anyone else have any thoughts about actually creating the graph using a Postgres database instead of an OSM xml File? </div><div><br></div><div>The main reason for looking at this is that I could possibly substitute the OSMID with a new "Segment ID" or combination of both values (Allocating a unique Segment Id would be easy after the ways are split at intersecting nodes in Pg_Routing) to improve granularity</div>
<div><br></div><div>The second reasons is that it will be easier to keep the mapping db, pg_routing db and the grapahhoper graph in sync with the minutely diff updates. For smaller areas (like disasters) we coudl make new graphs on a regular basis, hourly, daily, or whatever is needed</div>
<div><br></div><div>In a disaster, the maps in a disaster area changes dramatically with the people form HOT and locals updating form Sat Images and we would need to be able to pick these up without having to run osmosis on a pbf file.</div>
<div><br></div><div>I guess we could extract the database using osmosis, then run the graphhopper build graph routines on the extracted xml but it does seem like one extra unnecessary step</div><div><br></div><div>Cheers</div>
<div><br></div><div>Mark</div><div><br></div><div><br></div><div><br></div></div><div class="gmail_extra"><br clear="all"><div><div dir="ltr"><div><br></div><div>Regards</div><div><br></div><div>Mark Cupitt</div><div><br>
</div><div><div>"If we change the world, let it bear the mark of our intelligence"</div><div><br></div><div><a href="https://www.openstreetmap.org/user/Mark_Cupitt" target="_blank">See me on Open Street Map</a><br>
</div><div><a></a><div><br></div></div><div><a href="http://ph.linkedin.com/in/markcupitt" target="_blank">See me on LinkedIn</a><br><img src="http://s.c.lnkd.licdn.com/scds/common/u/img/webpromo/btn_myprofile_160x33.png"><br>
</div><span style="font-family:arial black,sans-serif"><div><span style="font-family:arial black,sans-serif"><br></span></div><b><a href="http://gis.stackexchange.com/users/17846/mark-c" target="_blank">See me on StackExchange</a><br>
</b></span><img src="http://gis.stackexchange.com/users/flair/17846.png"><br></div><div>===============================================================================================</div><div>The contents of this email are intended only for the individual(s) to whom it is addressed and may contain</div>
<div>confidential or privileged information. If you are not the intended recipient, you must not disclose, copy, distribute,</div><div>or use the contents of this email. If you have received this email in error, please notify the sender immediately and</div>
<div>delete the email and any attachments.<br></div><div>
===============================================================================================
</div></div></div>
<br><br><div class="gmail_quote">On Sat, Jul 5, 2014 at 1:48 AM, Ciobota <span dir="ltr"><<a href="mailto:ciobota@gmail.com" target="_blank">ciobota@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Hi Mark,<div><br></div><div> Happy to help. To attempt to answer:</div><div><br></div><div> I think the newest version of GraphHopper allows the storage of way id's with each edge. Peter probably knows best, I can only guess from the messages posted in the forum. I haven't updated my version of GraphHopper since early January, when I made the customization changes.</div>
<div><br></div><div> But you are correct, you will need the way id stored with every edge. </div><div> And also yes, the way will be (as far as I know) that ways will be split into multiple edges if they share a common node with another way (like at crossings). That's what I mentioned as granularity issues. So, if you want to only partially block a way, you will most likely do one of three ways:</div>
<div><br></div><div> - a blocking node. In this case in your weighting algorithm will pull all the nodes from the edge as a line (don't need to get a way id in this case, but it helps, see further down), then do a line/point intersection to see if the blocking node is on the line. If it is, then return infinity, since you would consider that entire edge blocked. This would still retain a good bit of speed if the access cost for blocking nodes is kept low. You could keep that cost low if you store the blocking node location along with the way id (that way you filter the returned blocking nodes to only those relevant for the way). I don't know much about Postgress sorry, we're an Oracle shop, but if your database doesn't do caching, you can implement your own cache in memory with a hashmap and update it from the database every so often.</div>
<div><br></div><div> - a line. This is for blocking spans along a way. The reason why this may be needed is for blockages that may span over more than one edge or even more than one way (say, construction). In this case you won't care about way id's at all, but for each edge you would have to calculate the intersection of the edge geometry with every blocking span. There are ways to speed this up, but it will be slower, maybe significantly so if there are lots of spans.</div>
<div><br></div><div> - a polygon. This would be a valid option for blockages that are areas rather than specific spans along a way. Same thing as above, you would do an intersection of the edge geometry and all the blocking polygons. This would probably be the slowest of the three.</div>
<div><br></div><div> Btw, I would not do the indexed file at all. The best mechanism imho is to create an internal memory structure (like I mentioned, maybe a hashmap), and have it refreshed from the database periodically. At program startup it should query the database and build the initial cache.</div>
<div><br></div><div> Btw, aside from the part about assigning way id's to edges, which may already be present in the current coed, the weighing function would not need to be part of the released code. Peter provides a very elegant and simple mechanism to tie in your own weighting algorithm with the GraphHopper solver (I think he has an example in the released code). As we all have more or less unique needs for the cost function, implementing your own weighting class is really the best way to go.</div>
<div><br></div><div> Hope this helps.</div><div><br></div><div>Daniel</div><div><br></div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Jul 3, 2014 at 8:01 PM, Markware Software Services <span dir="ltr"><<a href="mailto:markwaresoftware@gmail.com" target="_blank">markwaresoftware@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Daniel<div><br></div><div>First, many thanks for the detailed reply. This is definitely worth investigating</div>
<div><br></div><div>A couple of questions if I may. I am NOT a Java person at all but may have to invest some time learning it for this project if it pans out. A few questions if I may to make sure I understand the process correctly ..</div>
<div><br></div><div>1. The solution would mean a custom version of Graphhopper which would need to be updated every time a new version of Graphhopper is released? IS that correct? I was hoping to avoid this, but may not be able to.</div>
<div>2. The approach would be to make sure the way Id's were in the edges when the graph was built, correct?</div><div>3. Graphhopper would need to be modified to do the following, Change the OSM reader to read the cost form an external table. I would presume that this could be a Postgres Database, but it may have some performance limitations on the look-up. IN this case, we would need to write some kind of indexed file that would contain the cost data</div>
<div>4. If not the database, this cost file would need to be written out on a periodic basis. I would guess that writing it a temp file then renaming it to the correct file name may minimize the liklihood the file will be unavailable to graphhopper, however some logic may be needed to get graphhopper to lop a few times to allow for the file rename period before it bombs out.</div>
<div>5. PG_Routing segments all ways to intersection points so it should be enough to simply add a high cost to one or more segments (say a bridge that is out, road section flooded, etc)</div><div>6. The algorithm change would basically just need to add the retrieved cost from the file which would most likely be 0 if way is usable or a high number if it is not</div>
<div>7. I would guess that the Reverse Cost should also be considered in the same manner</div><div><br></div><div>One issue that came t mind whilst I was typing this was the fact that pg_routing splits all the ways into small segments for intersection points. I presume that Graphhopper does the same, that means that multiple segments will have the same OSMid. This would only be a problem if it was a long road, for example a freeway. Flooding on part of a freeway should cause a detour, not take make the whole road unavailable.</div>
<div><br></div><div>How realistic would it be to have a database lookup to obtain this information on each routing pass? Would kill the speed? I am unsure what sort of table should then be used if Postgres is not an option</div>
<div><br></div><div>Could/Should this be made part of the standard Graphopper release?</div><div><br></div><div>Again, many thanks for you reponse and ideas. Any observations or suggestions are very welcome.</div><div><br>
</div><div>Kind Regards</div><div><br></div><div><br></div><div class="gmail_extra"><div><div dir="ltr"><div>Mark Cupitt</div><div><br></div><div><div>"If we change the world, let it bear the mark of our intelligence"</div>
<div><br></div><div><a href="https://www.openstreetmap.org/user/Mark_Cupitt" target="_blank">See me on Open Street Map</a><br></div><div><a></a><div><br></div></div><div><a href="http://ph.linkedin.com/in/markcupitt" target="_blank">See me on LinkedIn</a><br>
<img><br></div><span style="font-family:arial black,sans-serif"><div><span style="font-family:arial black,sans-serif"><br></span></div><b><a href="http://gis.stackexchange.com/users/17846/mark-c" target="_blank">See me on StackExchange</a><br>
</b></span><img><br></div><div>===============================================================================================</div><div>The contents of this email are intended only for the individual(s) to whom it is addressed and may contain</div>
<div>confidential or privileged information. If you are not the intended recipient, you must not disclose, copy, distribute,</div><div>or use the contents of this email. If you have received this email in error, please notify the sender immediately and</div>
<div>delete the email and any attachments.<br></div><div>
===============================================================================================
</div></div></div>
<br><br><div class="gmail_quote">On Fri, Jul 4, 2014 at 6:11 AM, Ciobota <span dir="ltr"><<a href="mailto:ciobota@gmail.com" target="_blank">ciobota@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Hi all,<div><br></div><div> By way of introducing myself I will attempt to offer maybe a suggestion for a solution to this. First off, I work for a railroad company, and we have decided to migrate our routing code from a well known commercial vendor to Graphhopper, which has turned out really well. Many thanks to Peter and all who have contributed. So, as you can see, we don't route on normal roads. ;-)</div>
<div><br></div><div> Anyway, I think the version we use is from sometime in January, so it's probably before there was a change to allow to store way id's as part of the graph edges. So we implemented our own by overriding the osm reader and some minor changes to graphhopper (mostly exposing some private variables). We wanted to do this so we could do some custom routing, which included considering some ways as barriers. We created our own weighting algorithm that pulled the way id from the edge, then looked up the needed attributes from the way to determine whether the edge could be crossed or not.</div>
<div><br></div><div> Which brings me to my suggestion. I think the new version of Graphhopper now natively allows the storage of way id's for each edge. Peter, correct me if I'm wrong. So, Mark, you could create a table that would be populated with way id's and a flag associated with each way that would say it blocks or not. Then your custom weighting algorithm can query that table with the way id obtained from each edge passed in, then return infinity (I think) if it should be a barrier. </div>
<div><br></div><div> If that is not granular enough (say, only part of a way should be blocked) you may have to also store the blocking node location(s) for each way that has blockage in your table, then do some geometry intersects to see if the edge contains a blocking node. This will not be as fast as the "regular" routing solution, but if you maintain those dynamic blocking ways/nodes in memory somewhere (say a hashmap?) then it should still retain good speed.</div>
<div><br></div><div> I could be totally off on this of course.</div><div><br></div><div> Hope it helps.</div><div><br></div><div>Daniel</div><div><br></div></div><div><div><div class="gmail_extra">
<br><br><div class="gmail_quote">On Thu, Jul 3, 2014 at 9:44 AM, Markware Software Services <span dir="ltr"><<a href="mailto:markwaresoftware@gmail.com" target="_blank">markwaresoftware@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Guys, to be a little more specific, (I cannot type much on my android), we imported the planet file into Postgres/PostGis using osm2pgsql. We then use Osmosis to get minutely diff updates from osm and apply to the database. So far, it has worked very well.<div>
<br></div><div>Our intention is to use osmosis to extract from the planet database a subset for a particular disaster area, on a regular basis during a disaster (eg: haiyan in the philippies) We would like to create a routing graph based on user input gathered from mobile devices and satellite imagery via HOT OSM, eg: floods, downed power lines, collapsed bridges, etc and make that latest map available to the mobile devices along with a new routing graph</div>
<div><br></div><div>The advantage of pg_routng is it is quite simple to increase the weight of a segment for routing. It is also easy to add a NEW node into the database weighted so that it makes that way impassable. NO need to deal with OSM XML Files.</div>
<div><br></div><div>So far, I have been unable to find a way to make graphhopper do this, but the speed of graphhoper is so impressive we really want to use it on mobile devices and web based applications!.</div><div><br>
</div><div>So my challenge is to somehow to dump an extract of the postgres database form Pg_routing into a graphopper file, without the use of an osm file and supply that updated graph to users in the field, or wherever. It will be of immense value to planners and decision makers in disasters, etc. I can see the update needing to be done several times per day, but creating a pbf or xml file and then processing it via graphhoper is not really an option, especially if the size of the area is significant.</div>
<div><br></div><div>I am totally open to suggestions, discussion on this and I am sure that a number of other uses will manifest as a result of discussion and need</div><div><br></div><div>In my simple mind, taking an extract of a pg_routng database and creating a graphhopper file is probably the simplest apprroach, but perhaps I am being too simplistic</div>
<div><br></div><div>Cheers</div><div><br></div><div>Mark</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div></div><div class="gmail_extra"><br clear="all"><div><div dir="ltr"><div>
<br></div><div>Regards</div><div><br></div><div>Mark Cupitt</div><div><br></div><div><div>"If we change the world, let it bear the mark of our intelligence"</div><div><br></div><div><a href="https://www.openstreetmap.org/user/Mark_Cupitt" target="_blank">See me on Open Street Map</a><br>
</div><div><a></a><div><br></div></div><div><a href="http://ph.linkedin.com/in/markcupitt" target="_blank">See me on LinkedIn</a><br><img><br>
</div><span style="font-family:arial black,sans-serif"><div><span style="font-family:arial black,sans-serif"><br></span></div><b><a href="http://gis.stackexchange.com/users/17846/mark-c" target="_blank">See me on StackExchange</a><br>
</b></span><img><br></div><div>===============================================================================================</div><div>The contents of this email are intended only for the individual(s) to whom it is addressed and may contain</div>
<div>confidential or privileged information. If you are not the intended recipient, you must not disclose, copy, distribute,</div><div>or use the contents of this email. If you have received this email in error, please notify the sender immediately and</div>
<div>delete the email and any attachments.<br></div><div>
===============================================================================================
</div></div></div>
<br><br><div class="gmail_quote">On Thu, Jul 3, 2014 at 10:27 PM, Markware Software Services <span dir="ltr"><<a href="mailto:markwaresoftware@gmail.com" target="_blank">markwaresoftware@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr">Vivek. We put it the osm data postgres db using osm2pgsql but use osmosis for 5min diff updates <br>
Cheers mark </p>
<p dir="ltr">Sent from my android. Please excuse typos.</p><div><div>
<div class="gmail_quote">On Jul 3, 2014 9:45 PM, "Vivek Nallur" <<a href="mailto:vivek.nallur@scss.tcd.ie" target="_blank">vivek.nallur@scss.tcd.ie</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
On a related note (I know this isn't the right mailing list), but does<br>
anyone have any experience using Osmosis to update existing tags in OSM<br>
maps? I've been trying for a while now, but without success. <br>
regs<br>
Vivek<br>
<br>
On Thu, Jul 03, 2014 at 02:51:25PM +0100, Vivek Nallur wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi guys,<br>
<br>
I have almost the exact requirements as Mark! However, what I'm trying<br>
to do is integrate sensor data (like pollution) onto the map, and then<br>
allow the user the ability to either take it into account or go with the<br>
default (car).<br>
<br>
The problem is that sensor data might update the map a few times a day.<br>
<br>
My current idea is as follows (Peter, you may have read this already!):<br>
<br>
1) Grab the sensor data and push it into Redis (or any datastore. I<br>
don't really care)<br>
2) Using a cron job, convert the sensor data into weights, and<br>
create/update a tag inside the OSM file<br>
3) Re-start Graphhopper so that it can re-read the updated OSM file<br>
<br>
Mark, this might work for you, if you have a flooding sensor in place<br>
(or similar). Peter, is this a sane plan, in the first place?<br>
<br>
So, my big question is: would it be easy/feasible to create a postgres/neo4j bridge,<br>
that graphhopper can read from? It'll save me doing<br>
cron....update.....<u></u>restartGraphhopper process. Also, it might make it<br>
easier to combine preferences like 'cycling route' + 'least polluted'<br>
<br>
regs<br>
Vivek<br>
<br>
<br>
On Thu, Jul 03, 2014 at 02:06:02PM +0200, Peter wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi,<br>
<br>
There was work done where neo4j was used as storage and so you could do<br>
with pg routing (although I think it would be even less suited as pg is<br>
not a graph database). But you'll need solid Java skills:<br>
<a href="https://github.com/graphhopper/graphhopper-experiments/blob/master/src/main/java/com/graphhopper/compare/neo4j/Neo4JGraphImpl.java" target="_blank">https://github.com/<u></u>graphhopper/graphhopper-<u></u>experiments/blob/master/src/<u></u>main/java/com/graphhopper/<u></u>compare/neo4j/Neo4JGraphImpl.<u></u>java</a><br>
<br>
Besides this it is probably easier to change the edge costs directly in<br>
GraphHopper but I'm with you that a postgres or neo4j bridge would be a<br>
very interesting thing.<br>
<br>
Regards,<br>
Peter.<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi All<br>
<br>
I have been looking at Graphhopper for our routing needs and also<br>
looking at Pg_Routing and both packages are very impressive.<br>
<br>
In a Nutshell, if you want to compare the two, Graphhopper's best<br>
feature is it speed, PG_Routing's best feature is flexibility (speed<br>
is not it .. that is for sure)<br>
<br>
It struck me last night that there may be a way to get the best form<br>
both environments. I have been looking at how I can use Graphhopper in<br>
a Dynamic and Changing environment for our Disaster Network as well as<br>
a few other possibilities, but I need the ability to be able to<br>
quickly update road unavailability due to flooding, etc so that<br>
routing will not be used on those areas.<br>
<br>
PG_Routing makes this quite easy by simply adding a high cost to that<br>
bridge or segment of road Whereas Graphhopper requires the rebuilding<br>
of the entire graph. (at the moment anyway)<br>
<br>
My thought was, why not use both!! Use PG_Routing to maintain the<br>
graph design and costs but make a program that builds the graphhopper<br>
graph based on the pg_routing database and not the osm file ..<br>
<br>
My gut feel tells me that this might be quite feasible from a timing<br>
standpoint, but I am not sure if the structure of the routing database<br>
is suited to sequential scanning and building a graphhopper graph. If<br>
it was, this would be a relative painless process that could be done<br>
daily (or less) or on demand for disaster prone applications<br>
<br>
Unfortunately my Java and C skills are virtually non existent (I am a<br>
VB and Windows guy) and pulling the code apart to look at this is<br>
going to be extremely difficult for me<br>
<br>
I would be interested in your thoughts and if you see any immediate<br>
ShowStoppers in this or any reason that it should not be done.<br>
<br>
<br>
Regards<br>
<br>
Mark Cupitt<br>
<br>
"If we change the world, let it bear the mark of our intelligence"<br>
<br>
See me on Open Street Map <<a href="https://www.openstreetmap.org/user/Mark_Cupitt" target="_blank">https://www.openstreetmap.<u></u>org/user/Mark_Cupitt</a>><br>
<br>
See me on LinkedIn <<a href="http://ph.linkedin.com/in/markcupitt" target="_blank">http://ph.linkedin.com/in/<u></u>markcupitt</a>><br>
<br>
<br>
*See me on StackExchange <<a href="http://gis.stackexchange.com/users/17846/mark-c" target="_blank">http://gis.stackexchange.com/<u></u>users/17846/mark-c</a>><br>
*<br>
==============================<u></u>==============================<u></u>==============================<u></u>=====<br>
The contents of this email are intended only for the individual(s) to<br>
whom it is addressed and may contain<br>
confidential or privileged information. If you are not the intended<br>
recipient, you must not disclose, copy, distribute,<br>
or use the contents of this email. If you have received this email in<br>
error, please notify the sender immediately and<br>
delete the email and any attachments.<br>
==============================<u></u>==============================<u></u>==============================<u></u>=====<br>
<br>
<br>
<br>
______________________________<u></u>_________________<br>
GraphHopper mailing list<br>
<a href="mailto:GraphHopper@openstreetmap.org" target="_blank">GraphHopper@openstreetmap.org</a><br>
<a href="https://lists.openstreetmap.org/listinfo/graphhopper" target="_blank">https://lists.openstreetmap.<u></u>org/listinfo/graphhopper</a><br>
</blockquote>
<br>
</blockquote>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
______________________________<u></u>_________________<br>
GraphHopper mailing list<br>
<a href="mailto:GraphHopper@openstreetmap.org" target="_blank">GraphHopper@openstreetmap.org</a><br>
<a href="https://lists.openstreetmap.org/listinfo/graphhopper" target="_blank">https://lists.openstreetmap.<u></u>org/listinfo/graphhopper</a><br>
</blockquote>
<br>
<br>
______________________________<u></u>_________________<br>
GraphHopper mailing list<br>
<a href="mailto:GraphHopper@openstreetmap.org" target="_blank">GraphHopper@openstreetmap.org</a><br>
<a href="https://lists.openstreetmap.org/listinfo/graphhopper" target="_blank">https://lists.openstreetmap.<u></u>org/listinfo/graphhopper</a><br>
</blockquote>
<br>
______________________________<u></u>_________________<br>
GraphHopper mailing list<br>
<a href="mailto:GraphHopper@openstreetmap.org" target="_blank">GraphHopper@openstreetmap.org</a><br>
<a href="https://lists.openstreetmap.org/listinfo/graphhopper" target="_blank">https://lists.openstreetmap.<u></u>org/listinfo/graphhopper</a><br>
</blockquote></div>
</div></div></blockquote></div><br></div>
<br>_______________________________________________<br>
GraphHopper mailing list<br>
<a href="mailto:GraphHopper@openstreetmap.org" target="_blank">GraphHopper@openstreetmap.org</a><br>
<a href="https://lists.openstreetmap.org/listinfo/graphhopper" target="_blank">https://lists.openstreetmap.org/listinfo/graphhopper</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
GraphHopper mailing list<br>
<a href="mailto:GraphHopper@openstreetmap.org" target="_blank">GraphHopper@openstreetmap.org</a><br>
<a href="https://lists.openstreetmap.org/listinfo/graphhopper" target="_blank">https://lists.openstreetmap.org/listinfo/graphhopper</a><br>
<br></blockquote></div><br></div></div>
<br>_______________________________________________<br>
GraphHopper mailing list<br>
<a href="mailto:GraphHopper@openstreetmap.org" target="_blank">GraphHopper@openstreetmap.org</a><br>
<a href="https://lists.openstreetmap.org/listinfo/graphhopper" target="_blank">https://lists.openstreetmap.org/listinfo/graphhopper</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
GraphHopper mailing list<br>
<a href="mailto:GraphHopper@openstreetmap.org">GraphHopper@openstreetmap.org</a><br>
<a href="https://lists.openstreetmap.org/listinfo/graphhopper" target="_blank">https://lists.openstreetmap.org/listinfo/graphhopper</a><br>
<br></blockquote></div><br></div>