[GraphHopper] Ordnance Survey Loader
Peter
graphhopper at gmx.de
Mon Nov 24 15:31:16 UTC 2014
Over 30seconds is **very** unusual for this length. Going through entire
Europe should be that range for turn restrictions. E.g. going from Kiel
to Freiburg is roughly the same length and takes 1.5 second without turn
instructions. So with turn instructions should be like 3seconds for
Germany, for your use case even faster as the UK is 'width-limited' :)
Did you try the unmodified GraphHopper master? Also make sure you assign
enough RAM to the JVM, that memory usage is the most problematic
downside of none-CH algorithms if you ask me. Additionally we are
working on none-CH algorithms where we'll get faster calculation and
less RAM in use.
Regards,
Peter
On 24.11.2014 16:19, Stuart Adam wrote:
> Hi Peter
>
> Speedwise with contraction hierarchies turned on the response is
> almost instant. With it turned off in order to enable turnCosts the
> service reaches the point where it doesn't reply quick enough and the
> web demo times out, when routing from Scotland to Hampshire.
>
>
> An example timing (probably not the Scotland example just the most
> recent in my terminal window)
>
> time:229min, points:2895, took:31.092913, debug -
> idLookup[0]:0.068101354s, [1] idLookup:0.12297253s,
> algoInit:3.17604E-4s, dijkstrabi-routing:30.853657s, extract
> time:0.00102621, simplify (6040->2895), , fastest, CAR
>
>
> Sincerely
> Stuart Adam
>
> ------------------------------------------------------------------------
> Date: Mon, 24 Nov 2014 15:49:27 +0100
> From: graphhopper at gmx.de
> To: graphhopper at openstreetmap.org
> Subject: Re: [GraphHopper] Ordnance Survey Loader
>
> Hi Stuart, hi Philip,
>
> thanks for the insights! Also thanks Philip for the suggestion!
>
> > Since once the import is done the graph is the same as one imported
> from osm the query time code is unchanged nd so should still be Java 6
> compatible.
>
> I see. Then we should probably just make the module separation and put
> all import stuff into another module or tools. E.g. the web module is
> also java 7
> Also I'm not 100% sure if java 7 is that problematic as e.g. Android
> development should be possible with java7 too I think.
>
>
> > As commented by Philip Welch on here the simplest one's to provide
> at runtime is either the hsqldb (remember to add the hsql-jdbc driver
> lib as well)
> > or the wkt based one.
>
> Do you have a pom snippet at hand :) ?
>
>
> > (OSITN = too much block caps for my liking)
>
> me too but no problem - just wanted to mention it
>
>
> > The sample data actually requires a license agreement so people
> should be directed to the website rather than trying to automate via
> wget.
> > If it were not for that stage I would have placed the sample data in
> git repo. I will try and negotiate for permission to include it which
> would simplify that stage.
>
> We can also do this license agreement on the (bash) shell.
>
>
> > At the moment execution time without contraction hierarchies (in
> order to support turn restrictions)
> > is very slow and so we would be interested in what work is going on
> to combine the two and if there is any way we can help
>
> What do you mean with 'very slow'? With turn restrictions it should be
> like 2 times slower than without.
>
> Combining the both is no magic but requires some time to work out the
> bits (although all the necessary steps are already defined -> #270).
> And of course this is important for us too but no starting date
> defined yet.
>
> Regards,
> Peter
>
>
> On 24.11.2014 14:52, Stuart Adam wrote:
>
> HI Peter
>
> The logging is so heavy as we are still learning our way around
> the api. Absolutely this should be reduced as time goes on and
> there is already a number of test cases specifically to do with os
> data. Unfortunately due to a late change in coordinate convertor
> (ITN uses British National Grid) some of these are currently
> broken. The tests are almost certainly not as clean as they could
> be. Only recently found the GraphInfo class that might help some
> with that :-)
>
> I used reflection as that would enable us to more dynamically
> extend the available readers but if it causes problems with the
> iOS port then absolutely happy to change that. iOS and android
> are important for us. The import code at the moment takes
> advantage of String based switches so is Java 7 specific. We can
> work on removing that but for now have been relying on using
> desktop class machines to pre import and then making the graph
> folder available to client machines. Since once the import is
> done the graph is the same as one imported from osm the query time
> code is unchanged nd so should still be Java 6 compatible.
>
> Yes we will standardise the naming scheme. Relatively new to git
> so was leaving such refactorings for now but was already unhappy
> with my initial naming (OSITN = too much block caps for my liking)
>
> The geotools supplies multiple different implementations of the
> CRS lookup. As commented by Philip Welch on here the simplest
> one's to provide at runtime is either the hsqldb (remember to add
> the hsql-jdbc driver lib as well) or the wkt based one. Only
> provide a single implementation at runtime. Given the preferences
> of deployment teams the actual implementation used wasn't
> hardwired in to the build.
>
> The sample data actually requires a license agreement so people
> should be directed to the website rather than trying to automate
> via wget. If it were not for that stage I would have placed the
> sample data in git repo. I will try and negotiate for permission
> to include it which would simplify that stage.
>
> At the moment execution time without contraction hierarchies (in
> order to support turn restrictions) is very slow and so we would
> be interested in what work is going on to combine the two and if
> there is any way we can help.
>
> Sincerely
> Stuart Adam
> ------------------------------------------------------------------------
> Date: Mon, 24 Nov 2014 12:26:31 +0100
> From: graphhopper at gmx.de <mailto:graphhopper at gmx.de>
> To: graphhopper at openstreetmap.org
> <mailto:graphhopper at openstreetmap.org>
> Subject: Re: [GraphHopper] Ordnance Survey Loader
>
> Hi Stuart,
>
> this is nice - thanks!
>
> I took a closer look at it. There seems to be an old
> graphhopper.sh script? Also I had a problem with the import when
> using the sample data you mentioned**
>
> What I did I described in the 'current instructions' of the issue
> #227 and let us move the discussion to it:
> https://github.com/graphhopper/graphhopper/issues/277
>
> Let me know when you think that this is stable enough, then we can
> work together on improving the abstraction and refactoring
> mentioned in the notes, if you like.
>
> Kind Regards,
> Peter
>
> **
> org.opengis.referencing.NoSuchAuthorityCodeException: No code
> "EPSG:27700" from authority "EPSG" found for object of type
> "EngineeringCRS".
> at
> org.geotools.referencing.factory.epsg.CartesianAuthorityFactory.noSuchAuthorityException(CartesianAuthorityFactory.java:136)
> at
> org.geotools.referencing.factory.epsg.CartesianAuthorityFactory.createEngineeringCRS(CartesianAuthorityFactory.java:130)
> at
> org.geotools.referencing.factory.epsg.CartesianAuthorityFactory.createCoordinateReferenceSystem(CartesianAuthorityFactory.java:121)
> at
> org.geotools.referencing.factory.AuthorityFactoryAdapter.createCoordinateReferenceSystem(AuthorityFactoryAdapter.java:801)
> at
> org.geotools.referencing.factory.ThreadedAuthorityFactory.createCoordinateReferenceSystem(ThreadedAuthorityFactory.java:731)
> at
> org.geotools.referencing.DefaultAuthorityFactory.createCoordinateReferenceSystem(DefaultAuthorityFactory.java:179)
> at org.geotools.referencing.CRS.decode(CRS.java:519)
> at org.geotools.referencing.CRS.decode(CRS.java:447)
> at
> uk.co.ordnancesurvey.api.srs.OpenCoordConverter.<clinit>(OpenCoordConverter.java:18)
> at
> com.graphhopper.reader.osgb.OSITNNode.parseCoordinateString(OSITNNode.java:143)
> at
> com.graphhopper.reader.osgb.OSITNNode.parseCoords(OSITNNode.java:131)
> at
> com.graphhopper.reader.osgb.OSITNElement.handleCoordinates(OSITNElement.java:289)
> at
> com.graphhopper.reader.osgb.OSITNElement.readTags(OSITNElement.java:89)
> at com.graphhopper.reader.osgb.OSITNNode.create(OSITNNode.java:55)
> at
> com.graphhopper.reader.osgb.OsItnInputFile.getNextXML(OsItnInputFile.java:209)
> at
> com.graphhopper.reader.osgb.OsItnInputFile.getNext(OsItnInputFile.java:174)
> at
> com.graphhopper.reader.osgb.OsItnReader.preProcessSingleFile(OsItnReader.java:294)
> at
> com.graphhopper.reader.osgb.OsItnReader.preProcessSingleFile(OsItnReader.java:284)
> at
> com.graphhopper.reader.osgb.OsItnReader.preProcessDirOrFile(OsItnReader.java:275)
> at
> com.graphhopper.reader.osgb.OsItnReader.preProcess(OsItnReader.java:260)
> at
> com.graphhopper.reader.osgb.OsItnReader.readGraph(OsItnReader.java:244)
> at com.graphhopper.GraphHopper.importData(GraphHopper.java:634)
> at com.graphhopper.GraphHopper.process(GraphHopper.java:603)
> at com.graphhopper.GraphHopper.importOrLoad(GraphHopper.java:576)
> at com.graphhopper.tools.Import.main(Import.java:16)
> Exception in thread "main" java.lang.RuntimeException: Problem
> while parsing file
> at
> com.graphhopper.reader.osgb.OsItnReader.preProcess(OsItnReader.java:262)
> at
> com.graphhopper.reader.osgb.OsItnReader.readGraph(OsItnReader.java:244)
> at com.graphhopper.GraphHopper.importData(GraphHopper.java:634)
> at com.graphhopper.GraphHopper.process(GraphHopper.java:603)
> at com.graphhopper.GraphHopper.importOrLoad(GraphHopper.java:576)
> at com.graphhopper.tools.Import.main(Import.java:16)
> Caused by: java.lang.NullPointerException
> at org.geotools.referencing.CRS.findMathTransform(CRS.java:1201)
> at
> uk.co.ordnancesurvey.api.srs.OpenCoordConverter.toWGS84(OpenCoordConverter.java:30)
> at
> com.graphhopper.reader.osgb.OSITNNode.parseCoordinateString(OSITNNode.java:143)
> at
> com.graphhopper.reader.osgb.OSITNNode.parseCoords(OSITNNode.java:131)
> at
> com.graphhopper.reader.osgb.OSITNElement.handleCoordinates(OSITNElement.java:289)
> at
> com.graphhopper.reader.osgb.OSITNElement.readTags(OSITNElement.java:89)
> at com.graphhopper.reader.osgb.OSITNNode.create(OSITNNode.java:55)
> at
> com.graphhopper.reader.osgb.OsItnInputFile.getNextXML(OsItnInputFile.java:209)
> at
> com.graphhopper.reader.osgb.OsItnInputFile.getNext(OsItnInputFile.java:174)
> at
> com.graphhopper.reader.osgb.OsItnReader.preProcessSingleFile(OsItnReader.java:294)
> at
> com.graphhopper.reader.osgb.OsItnReader.preProcessSingleFile(OsItnReader.java:284)
> at
> com.graphhopper.reader.osgb.OsItnReader.preProcessDirOrFile(OsItnReader.java:275)
> at
> com.graphhopper.reader.osgb.OsItnReader.preProcess(OsItnReader.java:260)
> ... 5 more
>
>
> On 19.11.2014 00:08, Stuart Adam wrote:
>
> Hi All
>
> The dependency on internal code has been removed from the OS
> ITN data loader by using geotools.
> This means you should be able to try it out now. Some of the
> tests currently fail but the basics work. To build you will
> want to skip the acceptancetesting module, since that is an
> internal test which will probably be moved out of this
> repository as it is data dependant. For now skip tests whilst
> I figure out what has broken the unit tests, hopefully tomorrow.
>
> To run graphhopper loading itn files supply the following
> configuration options
>
>
> osmreader.osm=<path to data file as normal but this should be
> either a directory or itn xml file rather than osm>
>
> reader.implementation=com.graphhopper.reader.osgb.OsItnReader
>
>
> the reader.implementation setting changes which DataReader
> implementation GraphHopper uses with
> com.graphhopper.reader.osgb.OsItnReader and
> com.graphhopper.reader.OSMReader now implementing an interface
> DataReader. if reader.implementation is not set
> com.graphhopper.reader.OSMReader is used by default.
>
> osmreader.osm setting should probably change to datareader.src
> with this change but for now I have left that alone for
> compatibility reasons.
>
> Once the data is imported it can be used in a standard
> GraphHopper server without having to switch reader
> implementation which means ingested data should work in a
> GraphHopper instance from the main repo without consuming the
> engaric/graphhopper branch though I believe at the moment I
> need to resync with graphhopper/graphhopper repo as I suspect
> the graph version has been changing rapidly recently.
>
> Sincerely
> Stuart Adam
>
>
>
> _______________________________________________ GraphHopper mailing
> list GraphHopper at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/graphhopper
>
>
> _______________________________________________
> GraphHopper mailing list
> GraphHopper at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/graphhopper
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/graphhopper/attachments/20141124/b91ae172/attachment.html>
More information about the GraphHopper
mailing list