[GraphHopper] Ordnance Survey Loader

Mon Nov 24 13:52:21 UTC 2014

HI Peter

The logging is so heavy as we are still learning our way around the api.  Absolutely this should be reduced as time goes on and there is already a number of test cases specifically to do with os data. Unfortunately due to a late change in coordinate convertor (ITN uses British National Grid) some of these are currently broken.  The tests are almost certainly not as clean as they could be.  Only recently found the GraphInfo class that might help some with that :-)

I used reflection as that would enable us to more dynamically extend the available readers but if it causes problems with the iOS port then absolutely happy to change that.  iOS and android are important for us.  The import code at the moment takes advantage of String based switches so is Java 7 specific.  We can work on removing that but for now have been relying on using desktop class machines to pre import and then making the graph folder available to client machines.  Since once the import is done the graph is the same as one imported from osm the query time code is unchanged nd so should still be Java 6 compatible.

Yes we will standardise the naming scheme.  Relatively new to git so was leaving such refactorings for now but was already unhappy with my initial naming (OSITN = too much block caps for my liking) 

The geotools supplies multiple different implementations of the CRS lookup.  As commented by Philip Welch on here the simplest one's to provide at runtime is either the hsqldb (remember to add the hsql-jdbc driver lib as well) or the wkt based one.  Only provide a single implementation at runtime.  Given the preferences of deployment teams the actual implementation used wasn't hardwired in to the build.

The sample data actually requires a license agreement so people should be directed to the website rather than trying to automate via wget.  If it were not for that stage I would have placed the sample data in git repo.  I will try and negotiate for permission to include it which would simplify that stage.

At the moment execution time without contraction hierarchies (in order to support turn restrictions) is very slow and so we would be interested in what work is going on to combine the two and if there is any way we can help.  

Sincerely
Stuart Adam
Date: Mon, 24 Nov 2014 12:26:31 +0100
From: graphhopper at gmx.de
To: graphhopper at openstreetmap.org
Subject: Re: [GraphHopper] Ordnance Survey Loader

    Hi Stuart,

      this is nice - thanks!

      I took a closer look at it. There seems to be an old
      graphhopper.sh script? Also I had a problem with the import when
      using the sample data you mentioned**

      What I did I described in the 'current instructions' of the issue
      #227 and let us move the discussion to it:
      https://github.com/graphhopper/graphhopper/issues/277

      Let me know when you think that this is stable enough, then we can
      work together on improving the abstraction and refactoring
      mentioned in the notes, if you like.

      Kind Regards,

      Peter

      **

      org.opengis.referencing.NoSuchAuthorityCodeException: No code
      "EPSG:27700" from authority "EPSG" found for object of type
      "EngineeringCRS".

          at
org.geotools.referencing.factory.epsg.CartesianAuthorityFactory.noSuchAuthorityException(CartesianAuthorityFactory.java:136)

          at
org.geotools.referencing.factory.epsg.CartesianAuthorityFactory.createEngineeringCRS(CartesianAuthorityFactory.java:130)

          at
org.geotools.referencing.factory.epsg.CartesianAuthorityFactory.createCoordinateReferenceSystem(CartesianAuthorityFactory.java:121)

          at
org.geotools.referencing.factory.AuthorityFactoryAdapter.createCoordinateReferenceSystem(AuthorityFactoryAdapter.java:801)

          at
org.geotools.referencing.factory.ThreadedAuthorityFactory.createCoordinateReferenceSystem(ThreadedAuthorityFactory.java:731)

          at
org.geotools.referencing.DefaultAuthorityFactory.createCoordinateReferenceSystem(DefaultAuthorityFactory.java:179)

          at org.geotools.referencing.CRS.decode(CRS.java:519)

          at org.geotools.referencing.CRS.decode(CRS.java:447)

          at
uk.co.ordnancesurvey.api.srs.OpenCoordConverter.<clinit>(OpenCoordConverter.java:18)

          at
com.graphhopper.reader.osgb.OSITNNode.parseCoordinateString(OSITNNode.java:143)

          at
      com.graphhopper.reader.osgb.OSITNNode.parseCoords(OSITNNode.java:131)

          at
com.graphhopper.reader.osgb.OSITNElement.handleCoordinates(OSITNElement.java:289)

          at
      com.graphhopper.reader.osgb.OSITNElement.readTags(OSITNElement.java:89)

          at
      com.graphhopper.reader.osgb.OSITNNode.create(OSITNNode.java:55)

          at
com.graphhopper.reader.osgb.OsItnInputFile.getNextXML(OsItnInputFile.java:209)

          at
com.graphhopper.reader.osgb.OsItnInputFile.getNext(OsItnInputFile.java:174)

          at
com.graphhopper.reader.osgb.OsItnReader.preProcessSingleFile(OsItnReader.java:294)

          at
com.graphhopper.reader.osgb.OsItnReader.preProcessSingleFile(OsItnReader.java:284)

          at
com.graphhopper.reader.osgb.OsItnReader.preProcessDirOrFile(OsItnReader.java:275)

          at
      com.graphhopper.reader.osgb.OsItnReader.preProcess(OsItnReader.java:260)

          at
      com.graphhopper.reader.osgb.OsItnReader.readGraph(OsItnReader.java:244)

          at
      com.graphhopper.GraphHopper.importData(GraphHopper.java:634)

          at com.graphhopper.GraphHopper.process(GraphHopper.java:603)

          at
      com.graphhopper.GraphHopper.importOrLoad(GraphHopper.java:576)

          at com.graphhopper.tools.Import.main(Import.java:16)

      Exception in thread "main" java.lang.RuntimeException: Problem
      while parsing file

          at
      com.graphhopper.reader.osgb.OsItnReader.preProcess(OsItnReader.java:262)

          at
      com.graphhopper.reader.osgb.OsItnReader.readGraph(OsItnReader.java:244)

          at
      com.graphhopper.GraphHopper.importData(GraphHopper.java:634)

          at com.graphhopper.GraphHopper.process(GraphHopper.java:603)

          at
      com.graphhopper.GraphHopper.importOrLoad(GraphHopper.java:576)

          at com.graphhopper.tools.Import.main(Import.java:16)

      Caused by: java.lang.NullPointerException

          at
      org.geotools.referencing.CRS.findMathTransform(CRS.java:1201)

          at
uk.co.ordnancesurvey.api.srs.OpenCoordConverter.toWGS84(OpenCoordConverter.java:30)

          at
com.graphhopper.reader.osgb.OSITNNode.parseCoordinateString(OSITNNode.java:143)

          at
      com.graphhopper.reader.osgb.OSITNNode.parseCoords(OSITNNode.java:131)

          at
com.graphhopper.reader.osgb.OSITNElement.handleCoordinates(OSITNElement.java:289)

          at
      com.graphhopper.reader.osgb.OSITNElement.readTags(OSITNElement.java:89)

          at
      com.graphhopper.reader.osgb.OSITNNode.create(OSITNNode.java:55)

          at
com.graphhopper.reader.osgb.OsItnInputFile.getNextXML(OsItnInputFile.java:209)

          at
com.graphhopper.reader.osgb.OsItnInputFile.getNext(OsItnInputFile.java:174)

          at
com.graphhopper.reader.osgb.OsItnReader.preProcessSingleFile(OsItnReader.java:294)

          at
com.graphhopper.reader.osgb.OsItnReader.preProcessSingleFile(OsItnReader.java:284)

          at
com.graphhopper.reader.osgb.OsItnReader.preProcessDirOrFile(OsItnReader.java:275)

          at
      com.graphhopper.reader.osgb.OsItnReader.preProcess(OsItnReader.java:260)

          ... 5 more

      On 19.11.2014 00:08, Stuart Adam wrote:

      Hi All

        The dependency on internal code has been removed from the OS ITN
        data loader by using geotools.

        This means you should be able to try it out now.  Some of the
        tests currently fail but the basics work.  To build you will
        want to skip the acceptancetesting module, since that is an
        internal test which will probably be moved out of this
        repository as it is data dependant. For now skip tests whilst I
        figure out what has broken the unit tests, hopefully tomorrow.

        To run graphhopper loading itn files supply the following
        configuration options

        osmreader.osm=<path to data file as normal but this should be
        either a directory or itn xml file rather than osm>

        reader.implementation=com.graphhopper.reader.osgb.OsItnReader

        the reader.implementation setting changes which DataReader
        implementation GraphHopper uses with
        com.graphhopper.reader.osgb.OsItnReader and
        com.graphhopper.reader.OSMReader now implementing an interface
        DataReader.  if reader.implementation is not set
        com.graphhopper.reader.OSMReader is used by default.

        osmreader.osm setting should probably change to datareader.src
        with this change but for now I have left that alone for
        compatibility reasons.

        Once the data is imported it can be used in a standard
        GraphHopper server without having to switch reader
        implementation which means ingested data should work in a
        GraphHopper instance from the main repo without consuming the
        engaric/graphhopper branch though I believe at the moment I need
        to resync with graphhopper/graphhopper repo as I suspect the
        graph version has been changing rapidly recently.

        Sincerely

        Stuart Adam

      _______________________________________________
GraphHopper mailing list
GraphHopper at openstreetmap.org
https://lists.openstreetmap.org/listinfo/graphhopper

_______________________________________________
GraphHopper mailing list
GraphHopper at openstreetmap.org
https://lists.openstreetmap.org/listinfo/graphhopper 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/graphhopper/attachments/20141124/36a64ffe/attachment.html>