[OSM-dev] Is there a way to use simple schema without hstore

Andreas Kalsch andreaskalsch at gmx.de
Sat Nov 20 13:54:35 GMT 2010


Hi,

thank you, Brett, for me this is the perfect setup. I hope that others will find it useful as well. 
The names are OK. Two problems:

1) I read XML or PBF, dump it to CSV and then read the dump, but now my feature tables are blank 
because of these errors:

ERROR:  extra data after last expected column
CONTEXT:  COPY nodes, line 1: "125799    5    45445    2010-09-30 21:23:30+0200    5922698        
0101000020E610000034034B64D592214088BE164F98894A40"
ERROR:  extra data after last expected column
CONTEXT:  COPY ways, line 1: "3999478    17    4473    1970-01-07 21:53:09+0100    4957195    
"mapping_status"=>"incomplete","highway"=>"secondar..."
ERROR:  extra data after last expected column
CONTEXT:  COPY relations, line 1: "2952    23    29975    1970-01-15 10:56:44+0100    5014762    
"type"=>"multipolygon""

I use COPY instead of \copy like this:

COPY nodes FROM 'path/to/nodes.txt';
...

I don't know why this happens. The number and type of columns of the simple schema table and the CSV 
do fit.

Reading straight into database works (but is much slower).

2) --write-pgsimp-changeresults in this error log:

Nov 20, 2010 2:49:20 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Osmosis Version SNAPSHOT-r24310
Nov 20, 2010 2:49:20 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Preparing pipeline.
Nov 20, 2010 2:49:20 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Launching pipeline execution.
Nov 20, 2010 2:49:20 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Pipeline executing, waiting for completion.
Nov 20, 2010 2:49:23 PM org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager 
waitForCompletion
SEVERE: Thread for task 1-read-replication-interval failed
org.openstreetmap.osmosis.core.OsmosisRuntimeException: Unable to insert action with type=NODE, 
action=MODIFY and id=996097244.
     at org.openstreetmap.osmosis.pgsimple.v0_6.impl.ActionDao.addAction(ActionDao.java:80)
     at org.openstreetmap.osmosis.pgsimple.v0_6.impl.EntityDao.modifyEntity(EntityDao.java:263)
     at org.openstreetmap.osmosis.pgsimple.v0_6.impl.NodeDao.modifyEntity(NodeDao.java:74)
     at org.openstreetmap.osmosis.pgsimple.v0_6.impl.ChangeWriter.write(ChangeWriter.java:123)
     at 
org.openstreetmap.osmosis.pgsimple.v0_6.impl.ActionChangeWriter.process(ActionChangeWriter.java:48)
     at org.openstreetmap.osmosis.core.container.v0_6.NodeContainer.process(NodeContainer.java:58)
     at 
org.openstreetmap.osmosis.pgsimple.v0_6.PostgreSqlChangeWriter.process(PostgreSqlChangeWriter.java:71)
     at org.openstreetmap.osmosis.core.sort.v0_6.ChangeSorter.complete(ChangeSorter.java:64)
     at 
org.openstreetmap.osmosis.replication.v0_6.ReplicationDownloader.processComplete(ReplicationDownloader.java:93)
     at 
org.openstreetmap.osmosis.replication.v0_6.BaseReplicationDownloader.runImpl(BaseReplicationDownloader.java:284)
     at 
org.openstreetmap.osmosis.replication.v0_6.BaseReplicationDownloader.run(BaseReplicationDownloader.java:345)
     at java.lang.Thread.run(Thread.java:619)
Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint 
"pk_actions"
     at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2062)
     at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1795)
     at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
     at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:479)
     at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:367)
     at org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:321)
     at org.openstreetmap.osmosis.pgsimple.v0_6.impl.ActionDao.addAction(ActionDao.java:77)
     ... 11 more
Nov 20, 2010 2:49:23 PM org.openstreetmap.osmosis.core.Osmosis main
SEVERE: Execution aborted.
org.openstreetmap.osmosis.core.OsmosisRuntimeException: One or more tasks failed.
     at org.openstreetmap.osmosis.core.pipeline.common.Pipeline.waitForCompletion(Pipeline.java:146)
     at org.openstreetmap.osmosis.core.Osmosis.run(Osmosis.java:92)
     at org.openstreetmap.osmosis.core.Osmosis.main(Osmosis.java:37)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
     at java.lang.reflect.Method.invoke(Method.java:597)
     at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:329)
     at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239)
     at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
     at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
     at org.codehaus.classworlds.Launcher.main(Launcher.java:31)

Is it always guaranteed that a change for datatype/id is unique?

Am 20.11.10 07:27, schrieb Brett Henderson:
> On Sat, Nov 20, 2010 at 12:09 PM, Brett Henderson <brett at bretth.com <mailto:brett at bretth.com>> wrote:
>
>     Hmm, I've given all of this a bit more thought.  Perhaps there is a need for a "simple" schema
>     that is easy for people to populate and utilise.  I'm quite happy with hstore, but it's not as
>     simple for those familiar with basic SQL.
>
>     The original reason I created the so-called simple schema was to support improved bounding box
>     functionality because I couldn't do it via flat files efficiently.  It was called "simple"
>     because I was also working on a full history schema that I never found time to complete.  My
>     intent has always been to optimise for accurate bounding box query performance and not
>     simplicity so the name is something of a misnomer.
>
>     Anyway, perhaps I should re-instate the old tasks and run them alongside the new ones.  I'll
>     have to re-think the naming of these tasks and schemas.  Perhaps "simple" and "snapshot" or
>     something ...
>
>     But I don't know when I'll get to do this.  I'm very time poor at the moment.
>
>
> Okay, that wasn't too painful.  I found an hour or two to today so I've created new (actually old) 
> tasks and documented them here:
> http://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage_0.39#PostGIS_Tasks_.28Simple_Schema.29
>
> It's available in SVN for now.  These "new" tasks are just the old tasks retrieved out of SVN 
> history and renamed.  Note that these tasks are effectively at a dead end because any new 
> development is likely to proceed on the new hstore schema.  They should continue to work for the 
> foreseeable future.
>
> I'll probably cause further confusion with schema names.  But I'm now calling the new style hstore 
> schema the "pgsnapshot" schema, and the older tag table style schema the "pgsimple" schema.  I'm 
> open to suggestions, but any change will need to be reflected on the wiki and in the codebase.
>
> Brett
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20101120/f8f5441d/attachment-0001.html>


More information about the dev mailing list