[GraphHopper] Ordnance Survey Loader
Peter
graphhopper at gmx.de
Tue Nov 25 13:26:42 UTC 2014
Hi Stuart,
with your latest changes, which already include all dependencies as you
wrote I still get the exception. Using a clean repo and following my
instructions in the issue.
Do you have an idea how to fix this? I looked into the created jar which
the graphhopper.sh script is using (tools/target/"gh-with-deps") and the
content looks fine: with hsqldb etc
Regards,
Peter
On 24.11.2014 16:04, Stuart Adam wrote:
> Hi Guys
>
> I was using the Import class to pre prep the graph. See itngen.sh for
> a command line. The pom in my fork of core should already contain the
> following dependency info which would ensure the correct jars are
> downloaded to your local mvn repo. gotools.version is 12.1 and you
> may need to add the repo
>
> <repository>
> <id>osgeo</id>
> <name>Open Source Geospatial Foundation Repository</name>
> <url>http://download.osgeo.org/webdav/geotools/</url>
> </repository>
>
>
> Dependencies as follows (you might be able to use gt-epsg-wkt in
> place of gt-epsg-hsql and thereby remove the jdbc dependency as well,
> this worked for me before but wasn't working when I was readying to
> commit hence the more heavyweight dependency)
>
> <dependency>
> <groupId>org.geotools</groupId>
> <artifactId>gt-metadata</artifactId>
> <version>${geotools.version}</version>
> </dependency>
> <dependency>
> <groupId>org.geotools</groupId>
> <artifactId>gt-opengis</artifactId>
> <version>${geotools.version}</version>
> </dependency>
> <dependency>
> <groupId>org.geotools</groupId>
> <artifactId>gt-referencing</artifactId>
> <version>${geotools.version}</version>
> </dependency>
> <dependency>
> <groupId>net.java.dev.jsr-275</groupId>
> <artifactId>jsr-275</artifactId>
> <version>1.0-beta-2</version>
> </dependency>
> <dependency>
> <groupId>java3d</groupId>
> <artifactId>vecmath</artifactId>
> <version>1.3.2</version>
> </dependency>
> <dependency>
> <groupId>org.geotools</groupId>
> <artifactId>gt-epsg-hsql</artifactId>
> <version>${geotools.version}</version>
> </dependency>
> <dependency>
> <groupId>org.hsqldb</groupId>
> <artifactId>hsqldb</artifactId>
> <version>2.3.2</version>
> </dependency>
>
> I have also removed the reflective code.
>
> Sincerely
> Stuart Adam
> ------------------------------------------------------------------------
> Date: Mon, 24 Nov 2014 15:49:27 +0100
> From: graphhopper at gmx.de
> To: graphhopper at openstreetmap.org
> Subject: Re: [GraphHopper] Ordnance Survey Loader
>
> Hi Stuart, hi Philip,
>
> thanks for the insights! Also thanks Philip for the suggestion!
>
> > Since once the import is done the graph is the same as one imported
> from osm the query time code is unchanged nd so should still be Java 6
> compatible.
>
> I see. Then we should probably just make the module separation and put
> all import stuff into another module or tools. E.g. the web module is
> also java 7
> Also I'm not 100% sure if java 7 is that problematic as e.g. Android
> development should be possible with java7 too I think.
>
>
> > As commented by Philip Welch on here the simplest one's to provide
> at runtime is either the hsqldb (remember to add the hsql-jdbc driver
> lib as well)
> > or the wkt based one.
>
> Do you have a pom snippet at hand :) ?
>
>
> > (OSITN = too much block caps for my liking)
>
> me too but no problem - just wanted to mention it
>
>
> > The sample data actually requires a license agreement so people
> should be directed to the website rather than trying to automate via
> wget.
> > If it were not for that stage I would have placed the sample data in
> git repo. I will try and negotiate for permission to include it which
> would simplify that stage.
>
> We can also do this license agreement on the (bash) shell.
>
>
> > At the moment execution time without contraction hierarchies (in
> order to support turn restrictions)
> > is very slow and so we would be interested in what work is going on
> to combine the two and if there is any way we can help
>
> What do you mean with 'very slow'? With turn restrictions it should be
> like 2 times slower than without.
>
> Combining the both is no magic but requires some time to work out the
> bits (although all the necessary steps are already defined -> #270).
> And of course this is important for us too but no starting date
> defined yet.
>
> Regards,
> Peter
>
>
> On 24.11.2014 14:52, Stuart Adam wrote:
>
> HI Peter
>
> The logging is so heavy as we are still learning our way around
> the api. Absolutely this should be reduced as time goes on and
> there is already a number of test cases specifically to do with os
> data. Unfortunately due to a late change in coordinate convertor
> (ITN uses British National Grid) some of these are currently
> broken. The tests are almost certainly not as clean as they could
> be. Only recently found the GraphInfo class that might help some
> with that :-)
>
> I used reflection as that would enable us to more dynamically
> extend the available readers but if it causes problems with the
> iOS port then absolutely happy to change that. iOS and android
> are important for us. The import code at the moment takes
> advantage of String based switches so is Java 7 specific. We can
> work on removing that but for now have been relying on using
> desktop class machines to pre import and then making the graph
> folder available to client machines. Since once the import is
> done the graph is the same as one imported from osm the query time
> code is unchanged nd so should still be Java 6 compatible.
>
> Yes we will standardise the naming scheme. Relatively new to git
> so was leaving such refactorings for now but was already unhappy
> with my initial naming (OSITN = too much block caps for my liking)
>
> The geotools supplies multiple different implementations of the
> CRS lookup. As commented by Philip Welch on here the simplest
> one's to provide at runtime is either the hsqldb (remember to add
> the hsql-jdbc driver lib as well) or the wkt based one. Only
> provide a single implementation at runtime. Given the preferences
> of deployment teams the actual implementation used wasn't
> hardwired in to the build.
>
> The sample data actually requires a license agreement so people
> should be directed to the website rather than trying to automate
> via wget. If it were not for that stage I would have placed the
> sample data in git repo. I will try and negotiate for permission
> to include it which would simplify that stage.
>
> At the moment execution time without contraction hierarchies (in
> order to support turn restrictions) is very slow and so we would
> be interested in what work is going on to combine the two and if
> there is any way we can help.
>
> Sincerely
> Stuart Adam
> ------------------------------------------------------------------------
> Date: Mon, 24 Nov 2014 12:26:31 +0100
> From: graphhopper at gmx.de <mailto:graphhopper at gmx.de>
> To: graphhopper at openstreetmap.org
> <mailto:graphhopper at openstreetmap.org>
> Subject: Re: [GraphHopper] Ordnance Survey Loader
>
> Hi Stuart,
>
> this is nice - thanks!
>
> I took a closer look at it. There seems to be an old
> graphhopper.sh script? Also I had a problem with the import when
> using the sample data you mentioned**
>
> What I did I described in the 'current instructions' of the issue
> #227 and let us move the discussion to it:
> https://github.com/graphhopper/graphhopper/issues/277
>
> Let me know when you think that this is stable enough, then we can
> work together on improving the abstraction and refactoring
> mentioned in the notes, if you like.
>
> Kind Regards,
> Peter
>
> **
> org.opengis.referencing.NoSuchAuthorityCodeException: No code
> "EPSG:27700" from authority "EPSG" found for object of type
> "EngineeringCRS".
> at
> org.geotools.referencing.factory.epsg.CartesianAuthorityFactory.noSuchAuthorityException(CartesianAuthorityFactory.java:136)
> at
> org.geotools.referencing.factory.epsg.CartesianAuthorityFactory.createEngineeringCRS(CartesianAuthorityFactory.java:130)
> at
> org.geotools.referencing.factory.epsg.CartesianAuthorityFactory.createCoordinateReferenceSystem(CartesianAuthorityFactory.java:121)
> at
> org.geotools.referencing.factory.AuthorityFactoryAdapter.createCoordinateReferenceSystem(AuthorityFactoryAdapter.java:801)
> at
> org.geotools.referencing.factory.ThreadedAuthorityFactory.createCoordinateReferenceSystem(ThreadedAuthorityFactory.java:731)
> at
> org.geotools.referencing.DefaultAuthorityFactory.createCoordinateReferenceSystem(DefaultAuthorityFactory.java:179)
> at org.geotools.referencing.CRS.decode(CRS.java:519)
> at org.geotools.referencing.CRS.decode(CRS.java:447)
> at
> uk.co.ordnancesurvey.api.srs.OpenCoordConverter.<clinit>(OpenCoordConverter.java:18)
> at
> com.graphhopper.reader.osgb.OSITNNode.parseCoordinateString(OSITNNode.java:143)
> at
> com.graphhopper.reader.osgb.OSITNNode.parseCoords(OSITNNode.java:131)
> at
> com.graphhopper.reader.osgb.OSITNElement.handleCoordinates(OSITNElement.java:289)
> at
> com.graphhopper.reader.osgb.OSITNElement.readTags(OSITNElement.java:89)
> at com.graphhopper.reader.osgb.OSITNNode.create(OSITNNode.java:55)
> at
> com.graphhopper.reader.osgb.OsItnInputFile.getNextXML(OsItnInputFile.java:209)
> at
> com.graphhopper.reader.osgb.OsItnInputFile.getNext(OsItnInputFile.java:174)
> at
> com.graphhopper.reader.osgb.OsItnReader.preProcessSingleFile(OsItnReader.java:294)
> at
> com.graphhopper.reader.osgb.OsItnReader.preProcessSingleFile(OsItnReader.java:284)
> at
> com.graphhopper.reader.osgb.OsItnReader.preProcessDirOrFile(OsItnReader.java:275)
> at
> com.graphhopper.reader.osgb.OsItnReader.preProcess(OsItnReader.java:260)
> at
> com.graphhopper.reader.osgb.OsItnReader.readGraph(OsItnReader.java:244)
> at com.graphhopper.GraphHopper.importData(GraphHopper.java:634)
> at com.graphhopper.GraphHopper.process(GraphHopper.java:603)
> at com.graphhopper.GraphHopper.importOrLoad(GraphHopper.java:576)
> at com.graphhopper.tools.Import.main(Import.java:16)
> Exception in thread "main" java.lang.RuntimeException: Problem
> while parsing file
> at
> com.graphhopper.reader.osgb.OsItnReader.preProcess(OsItnReader.java:262)
> at
> com.graphhopper.reader.osgb.OsItnReader.readGraph(OsItnReader.java:244)
> at com.graphhopper.GraphHopper.importData(GraphHopper.java:634)
> at com.graphhopper.GraphHopper.process(GraphHopper.java:603)
> at com.graphhopper.GraphHopper.importOrLoad(GraphHopper.java:576)
> at com.graphhopper.tools.Import.main(Import.java:16)
> Caused by: java.lang.NullPointerException
> at org.geotools.referencing.CRS.findMathTransform(CRS.java:1201)
> at
> uk.co.ordnancesurvey.api.srs.OpenCoordConverter.toWGS84(OpenCoordConverter.java:30)
> at
> com.graphhopper.reader.osgb.OSITNNode.parseCoordinateString(OSITNNode.java:143)
> at
> com.graphhopper.reader.osgb.OSITNNode.parseCoords(OSITNNode.java:131)
> at
> com.graphhopper.reader.osgb.OSITNElement.handleCoordinates(OSITNElement.java:289)
> at
> com.graphhopper.reader.osgb.OSITNElement.readTags(OSITNElement.java:89)
> at com.graphhopper.reader.osgb.OSITNNode.create(OSITNNode.java:55)
> at
> com.graphhopper.reader.osgb.OsItnInputFile.getNextXML(OsItnInputFile.java:209)
> at
> com.graphhopper.reader.osgb.OsItnInputFile.getNext(OsItnInputFile.java:174)
> at
> com.graphhopper.reader.osgb.OsItnReader.preProcessSingleFile(OsItnReader.java:294)
> at
> com.graphhopper.reader.osgb.OsItnReader.preProcessSingleFile(OsItnReader.java:284)
> at
> com.graphhopper.reader.osgb.OsItnReader.preProcessDirOrFile(OsItnReader.java:275)
> at
> com.graphhopper.reader.osgb.OsItnReader.preProcess(OsItnReader.java:260)
> ... 5 more
>
>
> On 19.11.2014 00:08, Stuart Adam wrote:
>
> Hi All
>
> The dependency on internal code has been removed from the OS
> ITN data loader by using geotools.
> This means you should be able to try it out now. Some of the
> tests currently fail but the basics work. To build you will
> want to skip the acceptancetesting module, since that is an
> internal test which will probably be moved out of this
> repository as it is data dependant. For now skip tests whilst
> I figure out what has broken the unit tests, hopefully tomorrow.
>
> To run graphhopper loading itn files supply the following
> configuration options
>
>
> osmreader.osm=<path to data file as normal but this should be
> either a directory or itn xml file rather than osm>
>
> reader.implementation=com.graphhopper.reader.osgb.OsItnReader
>
>
> the reader.implementation setting changes which DataReader
> implementation GraphHopper uses with
> com.graphhopper.reader.osgb.OsItnReader and
> com.graphhopper.reader.OSMReader now implementing an interface
> DataReader. if reader.implementation is not set
> com.graphhopper.reader.OSMReader is used by default.
>
> osmreader.osm setting should probably change to datareader.src
> with this change but for now I have left that alone for
> compatibility reasons.
>
> Once the data is imported it can be used in a standard
> GraphHopper server without having to switch reader
> implementation which means ingested data should work in a
> GraphHopper instance from the main repo without consuming the
> engaric/graphhopper branch though I believe at the moment I
> need to resync with graphhopper/graphhopper repo as I suspect
> the graph version has been changing rapidly recently.
>
> Sincerely
> Stuart Adam
>
>
>
> _______________________________________________ GraphHopper mailing
> list GraphHopper at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/graphhopper
>
>
> _______________________________________________
> GraphHopper mailing list
> GraphHopper at openstreetmap.org
> https://lists.openstreetmap.org/listinfo/graphhopper
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/graphhopper/attachments/20141125/49f1c058/attachment.html>
More information about the GraphHopper
mailing list