<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">Thanks for the hint!<br>
<br>
Is there a magic-free possibility :) ?<br>
<br>
Peter<br>
<br>
On 26.11.2014 01:01, me wrote:<br>
</div>
<blockquote
cite="mid:erjtturehki96jln6upuktks.1416960070746@email.android.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<div>
<div>By default geotools does not merge properly into an
uber-jar as some files override others in the meta-inf
directory. This is explained somewhere on the geotools site
and could be relevant to your problem? You can use the maven
shade plugin to do some automagic renaming to get round this.
I do it
here <a class="moz-txt-link-freetext" href="https://github.com/PGWelch/com.opendoorlogistics/blob/master/odl-geotools-fat-jar/pom.xml">https://github.com/PGWelch/com.opendoorlogistics/blob/master/odl-geotools-fat-jar/pom.xml</a>
based on one of their examples. </div>
<div><br>
</div>
<div>Phil Welch</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div style="font-size:75%;color:#575757">Sent from Samsung
Mobile</div>
</div>
</div>
<br>
<br>
<br>
-------- Original message --------<br>
From: Stuart Adam <a class="moz-txt-link-rfc2396E" href="mailto:engaric@hotmail.com"><engaric@hotmail.com></a> <br>
Date: <br>
To: <a class="moz-txt-link-abbreviated" href="mailto:graphhopper@openstreetmap.org">graphhopper@openstreetmap.org</a> <br>
Subject: Re: [GraphHopper] Ordnance Survey Loader <br>
<br>
<br>
<div dir="ltr">
<div dir="ltr">Hi Peter<br>
<br>
Yes as I suspected the uberjar causes problems. The
/META-INF/services/org.opengis.referencing.crs.CRSAuthorityFactory
, /META-INF/services/org.opengis.referencing.crs.CRSFactory,
/META-INF/services/org.opengis.referencing.datum.DatumAuthorityFactory
and<br>
/META-INF/services/org.opengis.referencing.datum.DatumFactory
files stored are only the versions from gt-referencing not the
ones from gt-epsg-hsql (or gt-epsg-wkt if you chose that
implementation)<br>
<br>
Not sure how we merge the content of these service registry
files as I believe we really want the content from all
supplied versions.<br>
<br>
Sincerely<br>
Stuart Adam<br>
<br>
<div>
<hr id="stopSpelling">Date: Tue, 25 Nov 2014 22:05:07 +0000<br>
From: <a class="moz-txt-link-abbreviated" href="mailto:engaric@hotmail.com">engaric@hotmail.com</a><br>
To: <a class="moz-txt-link-abbreviated" href="mailto:graphhopper@openstreetmap.org">graphhopper@openstreetmap.org</a><br>
Subject: Re: [GraphHopper] Ordnance Survey Loader<br>
<br>
<div
style="width:100%;font-size:initial;font-family:Calibri,
'Slate Pro', sans-serif;color:rgb(31, 73,
125);text-align:initial;background-color:rgb(255, 255,
255);">
Hi Peter </div>
<div
style="width:100%;font-size:initial;font-family:Calibri,
'Slate Pro', sans-serif;color:rgb(31, 73,
125);text-align:initial;background-color:rgb(255, 255,
255);">
<br>
</div>
<div
style="width:100%;font-size:initial;font-family:Calibri,
'Slate Pro', sans-serif;color:rgb(31, 73,
125);text-align:initial;background-color:rgb(255, 255,
255);">
I think I might have an idea what is happening when you
use the graphhopper.sh script (I have updated to something
closer to the version in upstream). The geotools libs use
a service provider pattern whereby properties files in the
META-INF/services folder of the implementing jar are read
for the name of the implementing class. It may be that the
repacked jar does not have the right layout for that to
work. I will try to investigate that a bit further.</div>
<div
style="width:100%;font-size:initial;font-family:Calibri,
'Slate Pro', sans-serif;color:rgb(31, 73,
125);text-align:initial;background-color:rgb(255, 255,
255);">
<span
style="font-size:initial;text-align:initial;line-height:initial;"><br
class="ecxmarkedForCaretMarkerRemoval">
</span></div>
<div
style="width:100%;font-size:initial;font-family:Calibri,
'Slate Pro', sans-serif;color:rgb(31, 73,
125);text-align:initial;background-color:rgb(255, 255,
255);">
<span
style="font-size:initial;text-align:initial;line-height:initial;">Sincerely </span></div>
<div
style="width:100%;font-size:initial;font-family:Calibri,
'Slate Pro', sans-serif;color:rgb(31, 73,
125);text-align:initial;background-color:rgb(255, 255,
255);">
<span
style="font-size:initial;text-align:initial;line-height:initial;">Stuart
Adam
</span></div>
<div
style="width:100%;font-size:initial;font-family:Calibri,
'Slate Pro', sans-serif;color:rgb(31, 73,
125);text-align:initial;background-color:rgb(255, 255,
255);">
<br style="display:initial;">
</div>
<div style="font-size:initial;font-family:Calibri, 'Slate
Pro', sans-serif;color:rgb(31, 73,
125);text-align:initial;background-color:rgb(255, 255,
255);">
Sent from my BlackBerry 10 smartphone on the EE network.</div>
<table style="background-color:white;border-spacing:0px;"
width="100%">
<tbody>
<tr>
<td colspan="2"
style="font-size:initial;text-align:initial;background-color:rgb(255,
255, 255);">
<div id="ecx_persistentHeader"
style="border-style:solid none
none;border-top-color:rgb(181, 196,
223);border-top-width:1pt;padding:3pt 0in
0in;font-family:Tahoma, 'BB Alpha Sans', 'Slate
Pro';font-size:10pt;">
<div><b>From: </b>Peter</div>
<div><b>Sent: </b>Tuesday, 25 November 2014 13:27</div>
<div><b>To: </b>GraphHopper Java routing engine</div>
<div><b>Reply To: </b>GraphHopper Java routing
engine</div>
<div><b>Subject: </b>Re: [GraphHopper] Ordnance
Survey Loader</div>
</div>
</td>
</tr>
</tbody>
</table>
<div style="border-style:solid none
none;border-top-color:rgb(186, 188,
209);border-top-width:1pt;font-size:initial;text-align:initial;background-color:rgb(255,
255, 255);">
</div>
<br>
<div style="background-color:#FFFFFF;">
<div class="ecxx_moz-cite-prefix">Hi Stuart,<br>
<br>
with your latest changes, which already include all
dependencies as you wrote I still get the exception.
Using a clean repo and following my instructions in the
issue.<br>
<br>
Do you have an idea how to fix this? I looked into the
created jar which the graphhopper.sh script is using
(tools/target/"gh-with-deps") and the content looks
fine: with hsqldb etc<br>
<br>
Regards,<br>
Peter<br>
<br>
On 24.11.2014 16:04, Stuart Adam wrote:<br>
</div>
<blockquote>
<style><!--
.ExternalClass .ecxx_hmmessage {
padding:0px;
}
.ExternalClass body.ecxx_hmmessage {
font-size:12pt;
font-family:Calibri;
}
--></style>
<div dir="ltr">Hi Guys<br>
<br>
I was using the Import class to pre prep the graph.
See itngen.sh for a command line. The pom in my fork
of core should already contain the following
dependency info which would ensure the correct jars
are downloaded to your local mvn repo.
gotools.version is 12.1 and you may need to add the
repo<br>
<br>
<repository><br>
<id>osgeo</id><br>
<name>Open Source Geospatial
Foundation Repository</name><br>
<url><a moz-do-not-send="true"
class="ecxx_moz-txt-link-freetext"
href="http://download.osgeo.org/webdav/geotools/"
target="_blank">http://download.osgeo.org/webdav/geotools/</a></url><br>
</repository><br>
<br>
<br>
Dependencies as follows (you might be able to use
gt-epsg-wkt in place of gt-epsg-hsql and thereby
remove the jdbc dependency as well, this worked for me
before but wasn't working when I was readying to
commit hence the more heavyweight dependency)<br>
<br>
<dependency><br>
<groupId>org.geotools</groupId><br>
<artifactId>gt-metadata</artifactId><br>
<version>${geotools.version}</version><br>
</dependency><br>
<dependency><br>
<groupId>org.geotools</groupId><br>
<artifactId>gt-opengis</artifactId><br>
<version>${geotools.version}</version><br>
</dependency><br>
<dependency><br>
<groupId>org.geotools</groupId><br>
<artifactId>gt-referencing</artifactId><br>
<version>${geotools.version}</version><br>
</dependency><br>
<dependency><br>
<groupId>net.java.dev.jsr-275</groupId><br>
<artifactId>jsr-275</artifactId><br>
<version>1.0-beta-2</version><br>
</dependency><br>
<dependency><br>
<groupId>java3d</groupId><br>
<artifactId>vecmath</artifactId><br>
<version>1.3.2</version><br>
</dependency><br>
<dependency><br>
<groupId>org.geotools</groupId><br>
<artifactId>gt-epsg-hsql</artifactId><br>
<version>${geotools.version}</version><br>
</dependency><br>
<dependency><br>
<groupId>org.hsqldb</groupId><br>
<artifactId>hsqldb</artifactId><br>
<version>2.3.2</version><br>
</dependency><br>
<br>
I have also removed the reflective code. <br>
<br>
Sincerely<br>
Stuart Adam<br>
<div>
<hr id="ecxx_stopSpelling">
Date: Mon, 24 Nov 2014 15:49:27 +0100<br>
From: <a moz-do-not-send="true"
class="ecxx_moz-txt-link-abbreviated"
href="mailto:graphhopper@gmx.de">graphhopper@gmx.de</a><br>
To: <a moz-do-not-send="true"
class="ecxx_moz-txt-link-abbreviated"
href="mailto:graphhopper@openstreetmap.org">
graphhopper@openstreetmap.org</a><br>
Subject: Re: [GraphHopper] Ordnance Survey Loader<br>
<br>
<div class="ecxx_ecxmoz-cite-prefix">Hi Stuart, hi
Philip,<br>
<br>
thanks for the insights! Also thanks Philip for
the suggestion!<br>
<br>
> Since once the import is done the graph is
the same as one imported from osm the query time
code is unchanged nd so should still be Java 6
compatible.<br>
<br>
I see. Then we should probably just make the
module separation and put all import stuff into
another module or tools. E.g. the web module is
also java 7<br>
Also I'm not 100% sure if java 7 is that
problematic as e.g. Android development should be
possible with java7 too I think.<br>
<br>
<br>
> As commented by Philip Welch on here the
simplest one's to provide at runtime is either the
hsqldb (remember to add the hsql-jdbc driver lib
as well)<br>
> or the wkt based one.<br>
<br>
Do you have a pom snippet at hand :) ?<br>
<br>
<br>
> (OSITN = too much block caps for my liking)<br>
<br>
me too but no problem - just wanted to mention it<br>
<br>
<br>
> The sample data actually requires a license
agreement so people should be directed to the
website rather than trying to automate via wget.
<br>
> If it were not for that stage I would have
placed the sample data in git repo. I will try
and negotiate for permission to include it which
would simplify that stage.<br>
<br>
We can also do this license agreement on the
(bash) shell.<br>
<br>
<br>
> At the moment execution time without
contraction hierarchies (in order to support turn
restrictions)
<br>
> is very slow and so we would be interested in
what work is going on to combine the two and if
there is any way we can help<br>
<br>
What do you mean with 'very slow'? With turn
restrictions it should be like 2 times slower than
without.<br>
<br>
Combining the both is no magic but requires some
time to work out the bits (although all the
necessary steps are already defined -> #270).
And of course this is important for us too but no
starting date defined yet.<br>
<br>
Regards,<br>
Peter<br>
<br>
<br>
On 24.11.2014 14:52, Stuart Adam wrote:<br>
</div>
<blockquote>
<style><!--
.ExternalClass .ecxx_ExternalClass .ecxx_ecxhmmessage {
padding:0px;
}
.ExternalClass .ecxx_ExternalClass body.ecxx_ecxhmmessage {
font-size:12pt;
font-family:Calibri;
}
--></style>
<div dir="ltr">HI Peter<br>
<br>
The logging is so heavy as we are still learning
our way around the api. Absolutely this should
be reduced as time goes on and there is already
a number of test cases specifically to do with
os data. Unfortunately due to a late change in
coordinate convertor (ITN uses British National
Grid) some of these are currently broken. The
tests are almost certainly not as clean as they
could be. Only recently found the GraphInfo
class that might help some with that :-)<br>
<br>
I used reflection as that would enable us to
more dynamically extend the available readers
but if it causes problems with the iOS port then
absolutely happy to change that. iOS and
android are important for us. The import code at
the moment takes advantage of String based
switches so is Java 7 specific. We can work on
removing that but for now have been relying on
using desktop class machines to pre import and
then making the graph folder available to client
machines. Since once the import is done the
graph is the same as one imported from osm the
query time code is unchanged nd so should still
be Java 6 compatible.<br>
<br>
Yes we will standardise the naming scheme.
Relatively new to git so was leaving such
refactorings for now but was already unhappy
with my initial naming (OSITN = too much block
caps for my liking)
<br>
<br>
The geotools supplies multiple different
implementations of the CRS lookup. As commented
by Philip Welch on here the simplest one's to
provide at runtime is either the hsqldb
(remember to add the hsql-jdbc driver lib as
well) or the wkt based one. Only provide a
single implementation at runtime. Given the
preferences of deployment teams the actual
implementation used wasn't hardwired in to the
build.<br>
<br>
The sample data actually requires a license
agreement so people should be directed to the
website rather than trying to automate via
wget. If it were not for that stage I would
have placed the sample data in git repo. I will
try and negotiate for permission to include it
which would simplify that stage.<br>
<br>
At the moment execution time without contraction
hierarchies (in order to support turn
restrictions) is very slow and so we would be
interested in what work is going on to combine
the two and if there is any way we can help.
<br>
<br>
Sincerely<br>
Stuart Adam<br>
<div>
<hr id="ecxx_ecxstopSpelling">
Date: Mon, 24 Nov 2014 12:26:31 +0100<br>
From: <a moz-do-not-send="true"
class="ecxx_ecxmoz-txt-link-abbreviated"
href="mailto:graphhopper@gmx.de">graphhopper@gmx.de</a><br>
To: <a moz-do-not-send="true"
class="ecxx_ecxmoz-txt-link-abbreviated"
href="mailto:graphhopper@openstreetmap.org">
graphhopper@openstreetmap.org</a><br>
Subject: Re: [GraphHopper] Ordnance Survey
Loader<br>
<br>
<div class="ecxx_ecxmoz-cite-prefix">Hi
Stuart,<br>
<br>
this is nice - thanks!<br>
<br>
I took a closer look at it. There seems to
be an old graphhopper.sh script? Also I had
a problem with the import when using the
sample data you mentioned**<br>
<br>
What I did I described in the 'current
instructions' of the issue #227 and let us
move the discussion to it:
<a moz-do-not-send="true"
class="ecxx_ecxmoz-txt-link-freetext"
href="https://github.com/graphhopper/graphhopper/issues/277"
target="_blank">
https://github.com/graphhopper/graphhopper/issues/277</a><br>
<br>
Let me know when you think that this is
stable enough, then we can work together on
improving the abstraction and refactoring
mentioned in the notes, if you like.<br>
<br>
Kind Regards,<br>
Peter<br>
<br>
**<br>
org.opengis.referencing.NoSuchAuthorityCodeException:
No code "EPSG:27700" from authority "EPSG"
found for object of type "EngineeringCRS".<br>
at
org.geotools.referencing.factory.epsg.CartesianAuthorityFactory.noSuchAuthorityException(CartesianAuthorityFactory.java:136)<br>
at
org.geotools.referencing.factory.epsg.CartesianAuthorityFactory.createEngineeringCRS(CartesianAuthorityFactory.java:130)<br>
at
org.geotools.referencing.factory.epsg.CartesianAuthorityFactory.createCoordinateReferenceSystem(CartesianAuthorityFactory.java:121)<br>
at
org.geotools.referencing.factory.AuthorityFactoryAdapter.createCoordinateReferenceSystem(AuthorityFactoryAdapter.java:801)<br>
at
org.geotools.referencing.factory.ThreadedAuthorityFactory.createCoordinateReferenceSystem(ThreadedAuthorityFactory.java:731)<br>
at
org.geotools.referencing.DefaultAuthorityFactory.createCoordinateReferenceSystem(DefaultAuthorityFactory.java:179)<br>
at
org.geotools.referencing.CRS.decode(CRS.java:519)<br>
at
org.geotools.referencing.CRS.decode(CRS.java:447)<br>
at
uk.co.ordnancesurvey.api.srs.OpenCoordConverter.<clinit>(OpenCoordConverter.java:18)<br>
at
com.graphhopper.reader.osgb.OSITNNode.parseCoordinateString(OSITNNode.java:143)<br>
at
com.graphhopper.reader.osgb.OSITNNode.parseCoords(OSITNNode.java:131)<br>
at
com.graphhopper.reader.osgb.OSITNElement.handleCoordinates(OSITNElement.java:289)<br>
at
com.graphhopper.reader.osgb.OSITNElement.readTags(OSITNElement.java:89)<br>
at
com.graphhopper.reader.osgb.OSITNNode.create(OSITNNode.java:55)<br>
at
com.graphhopper.reader.osgb.OsItnInputFile.getNextXML(OsItnInputFile.java:209)<br>
at
com.graphhopper.reader.osgb.OsItnInputFile.getNext(OsItnInputFile.java:174)<br>
at
com.graphhopper.reader.osgb.OsItnReader.preProcessSingleFile(OsItnReader.java:294)<br>
at
com.graphhopper.reader.osgb.OsItnReader.preProcessSingleFile(OsItnReader.java:284)<br>
at
com.graphhopper.reader.osgb.OsItnReader.preProcessDirOrFile(OsItnReader.java:275)<br>
at
com.graphhopper.reader.osgb.OsItnReader.preProcess(OsItnReader.java:260)<br>
at
com.graphhopper.reader.osgb.OsItnReader.readGraph(OsItnReader.java:244)<br>
at
com.graphhopper.GraphHopper.importData(GraphHopper.java:634)<br>
at
com.graphhopper.GraphHopper.process(GraphHopper.java:603)<br>
at
com.graphhopper.GraphHopper.importOrLoad(GraphHopper.java:576)<br>
at
com.graphhopper.tools.Import.main(Import.java:16)<br>
Exception in thread "main"
java.lang.RuntimeException: Problem while
parsing file<br>
at
com.graphhopper.reader.osgb.OsItnReader.preProcess(OsItnReader.java:262)<br>
at
com.graphhopper.reader.osgb.OsItnReader.readGraph(OsItnReader.java:244)<br>
at
com.graphhopper.GraphHopper.importData(GraphHopper.java:634)<br>
at
com.graphhopper.GraphHopper.process(GraphHopper.java:603)<br>
at
com.graphhopper.GraphHopper.importOrLoad(GraphHopper.java:576)<br>
at
com.graphhopper.tools.Import.main(Import.java:16)<br>
Caused by: java.lang.NullPointerException<br>
at
org.geotools.referencing.CRS.findMathTransform(CRS.java:1201)<br>
at
uk.co.ordnancesurvey.api.srs.OpenCoordConverter.toWGS84(OpenCoordConverter.java:30)<br>
at
com.graphhopper.reader.osgb.OSITNNode.parseCoordinateString(OSITNNode.java:143)<br>
at
com.graphhopper.reader.osgb.OSITNNode.parseCoords(OSITNNode.java:131)<br>
at
com.graphhopper.reader.osgb.OSITNElement.handleCoordinates(OSITNElement.java:289)<br>
at
com.graphhopper.reader.osgb.OSITNElement.readTags(OSITNElement.java:89)<br>
at
com.graphhopper.reader.osgb.OSITNNode.create(OSITNNode.java:55)<br>
at
com.graphhopper.reader.osgb.OsItnInputFile.getNextXML(OsItnInputFile.java:209)<br>
at
com.graphhopper.reader.osgb.OsItnInputFile.getNext(OsItnInputFile.java:174)<br>
at
com.graphhopper.reader.osgb.OsItnReader.preProcessSingleFile(OsItnReader.java:294)<br>
at
com.graphhopper.reader.osgb.OsItnReader.preProcessSingleFile(OsItnReader.java:284)<br>
at
com.graphhopper.reader.osgb.OsItnReader.preProcessDirOrFile(OsItnReader.java:275)<br>
at
com.graphhopper.reader.osgb.OsItnReader.preProcess(OsItnReader.java:260)<br>
... 5 more<br>
<br>
<br>
On 19.11.2014 00:08, Stuart Adam wrote:<br>
</div>
<blockquote>
<style><!--
.ExternalClass .ecxx_ExternalClass .ecxx_ecxhmmessage {
padding:0px;
}
.ExternalClass .ecxx_ExternalClass body.ecxx_ecxhmmessage {
font-size:12pt;
font-family:Calibri;
}
--></style>
<div dir="ltr">Hi All<br>
<br>
The dependency on internal code has been
removed from the OS ITN data loader by
using geotools.<br>
This means you should be able to try it
out now. Some of the tests currently fail
but the basics work. To build you will
want to skip the acceptancetesting module,
since that is an internal test which will
probably be moved out of this repository
as it is data dependant. For now skip
tests whilst I figure out what has broken
the unit tests, hopefully tomorrow.<br>
<br>
To run graphhopper loading itn files
supply the following configuration options<br>
<br>
<br>
osmreader.osm=<path to data file as
normal but this should be either a
directory or itn xml file rather than
osm><br>
<br>
reader.implementation=com.graphhopper.reader.osgb.OsItnReader<br>
<br>
<br>
the reader.implementation setting changes
which DataReader implementation
GraphHopper uses with
com.graphhopper.reader.osgb.OsItnReader
and com.graphhopper.reader.OSMReader now
implementing an interface DataReader. if
reader.implementation is not set
com.graphhopper.reader.OSMReader is used
by default.<br>
<br>
osmreader.osm setting should probably
change to datareader.src with this change
but for now I have left that alone for
compatibility reasons.<br>
<br>
Once the data is imported it can be used
in a standard GraphHopper server without
having to switch reader implementation
which means ingested data should work in a
GraphHopper instance from the main repo
without consuming the engaric/graphhopper
branch though I believe at the moment I
need to resync with
graphhopper/graphhopper repo as I suspect
the graph version has been changing
rapidly recently.<br>
<br>
Sincerely<br>
Stuart Adam<br>
</div>
</blockquote>
</div>
</div>
</blockquote>
<br>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</body>
</html>