[osmosis-dev] call osmosis from groovy

[Gubler, Ruediger]

Hi,

i have now an other problem.
I must call osmosis in the mid of my process chain and it will do a System.exit(0)

I found there is a run method in "org.openstreetmap.osmosis.core.Osmosis"  but it doesn't work with the following code:

  localDirectory = new File(getClass().protectionDomain.codeSource.location.path).parent
  File libFolder = new File(localDirectory, "/osmosis-0.39/lib/default/")
  File[] libs = libFolder.listFiles()
  List<URL> urlList = new ArrayList<URL>()
  for(File currLib : libs)
  {
    URL currUrl = new URL("file:/" + currLib.canonicalPath)
    urlList.add(currUrl)

    println(currUrl)
  }
  URL[] urls = new URL[urlList.size()]
  urls = urlList.toArray(urls)
  ClassLoader classLoader = new URLClassLoader(urls, getClass().getClassLoader());
  GroovyShell downloadShell = new GroovyShell(classLoader, binding);
  processErrorStatus = downloadShell.evaluate(new File(localDirectory, "/innerDownloadCall.groovy"))


19.09.2011 13:48:05 org.openstreetmap.osmosis.core.Osmosis run
INFO: Osmosis Version 0.39
19.09.2011 13:48:05 org.openstreetmap.osmosis.core.Osmosis run
INFO: Preparing pipeline.
org.openstreetmap.osmosis.core.OsmosisRuntimeException: Task type read-api doesn't exist.
        at org.openstreetmap.osmosis.core.pipeline.common.TaskManagerFactoryRegister.getInstance(TaskManagerFactoryRegister.java:60)
        at org.openstreetmap.osmosis.core.pipeline.common.Pipeline.buildTasks(Pipeline.java:50)
        at org.openstreetmap.osmosis.core.pipeline.common.Pipeline.prepare(Pipeline.java:112)
        at org.openstreetmap.osmosis.core.Osmosis.run(Osmosis.java:86)

The class path of the downloadShell contains
aopalliance-1.0.jar
commons-codec-1.4.jar
commons-compress-1.0.jar
commons-dbcp-1.4.jar
commons-logging-1.1.1.jar
commons-pool-1.5.4.jar
jpf-1.5.jar
mysql-connector-java-5.1.10.jar
osmosis-apidb-0.39.jar
osmosis-areafilter-0.39.jar
osmosis-core-0.39.jar
osmosis-dataset-0.39.jar
osmosis-extract-0.39.jar
osmosis-hstore-0.39.jar
osmosis-pbf-0.39.jar
osmosis-pgsimple-0.39.jar
osmosis-pgsnapshot-0.39.jar
osmosis-replication-0.39.jar
osmosis-set-0.39.jar
osmosis-tagfilter-0.39.jar
osmosis-xml-0.39.jar
osmpbf-1.1.1-754a33af.jar
plexus-classworlds-2.2.2.jar
postgis-jdbc-1.3.3.jar
postgresql-8.4-701.jdbc4.jar
protobuf-java-2.3.0.jar
spring-aop-3.0.3.RELEASE.jar
spring-asm-3.0.3.RELEASE.jar
spring-beans-3.0.3.RELEASE.jar
spring-context-3.0.3.RELEASE.jar
spring-core-3.0.3.RELEASE.jar
spring-expression-3.0.3.RELEASE.jar
spring-jdbc-3.0.3.RELEASE.jar
spring-tx-3.0.3.RELEASE.jar
stax2-api-3.0.2.jar
woodstox-core-lgpl-4.0.3.jar
xercesImpl-2.9.1.jar

What is going wrong or how can I setup plexus-classworlds to call the run not main method?



Yours Rüdiger


Von: Brett Henderson [mailto:brett at bretth.com]
Gesendet: Samstag, 17. September 2011 04:40
An: Gubler, Ruediger
Cc: osmosis-dev at openstreetmap.org
Betreff: Re: [osmosis-dev] call osmosis from groovy

Hi Ruediger,

8000 threads!  That's impressive :-)

Unfortunately there's not much you can do about it.  Osmosis will use one thread per xml reader, and one thread per merge task.  It's probably possible to develop a more efficient implementation but your use case sounds fairly unusual.  If it won't run properly with 8000 threads you could try running Osmosis in several passes by combining smaller sets together then combining merged files into larger merged files.

Cheers,
Brett
On Fri, Sep 16, 2011 at 12:17 AM, Gubler, Ruediger <rgubler at init-ka.de<mailto:rgubler at init-ka.de>> wrote:
Oh no,

I found the error:

 for(String fileStr : subFiles)
 {
   argsList.add("--fast-read-xml " + new File(dir, fileStr).canonicalPath)
   count++
 }
 for(int ii = 1; ii < count; ii++)
 {
   argsList.add("--merge")
 }
 argsList.add("--wx " + new File(localDirectory, "tmp/osm/merged.osm"))


must be


 for (String fileStr: subFiles)
 {
   argsList.add("--fast-read-xml")
   argsList.add(new File(dir, fileStr).canonicalPath)
   count++
 }
 for (int ii = 1; ii < count; ii++)
 {
   argsList.add("--merge")
 }
 argsList.add("--wx")
 argsList.add(new File(localDirectory, "tmp/osm/merged.osm").canonicalPath)


The arguments must different entries not "--fast-read-xml c:\...." but "--fast-read-xml" "c:\...."


Now it works but osmosis is using over 8000 threads for 1904 *.osm files to merge ?!



Yours Rüdiger


_______________________________________________
osmosis-dev mailing list
osmosis-dev at openstreetmap.org<mailto:osmosis-dev at openstreetmap.org>
http://lists.openstreetmap.org/listinfo/osmosis-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/osmosis-dev/attachments/20110919/07245818/attachment-0001.html>