[OSM-dev] Again: Importing planet file with osmosis into PostgreSQL

[Brett Henderson]

On Wed, Jan 6, 2010 at 9:48 PM, Peter Körner <osm-lists at mazdermind.de>wrote:

> Frank Bielig schrieb:
> > Thank you for reply. But your import script uses osm2pgsql. This script
> > expects the "simple" database schema for storing data. I really want to
> > have an 1:1 mirror using the "official" schema. The only tool I found
> > for that was osmosis.
> Oh, I'm sorry, I missed this.
>
>
> Does it work when you skip the merge of the replicate changesets (ie
> with a plain planet dump). If this works, wouldn't it be a solution to
> apply the replicate changesets afterwards onto the database?
>

That will probably work.

I believe that the reason for the failure is that replicate diffs can
contain multiple changes per entity per change file.  In other words, node
1234 may have a version 4 and 5 in the same change file.  If you merge these
change files into a planet file you could end up with duplicate entries for
some entities.

Many Osmosis tasks were never designed for full history change files.  I
*think* the --write-apidb-change task will work, but I haven't tried it.

There is another task that is useful here, the --simplify-change task.  It
takes as input a full history change stream, and converts it into a delta
style change stream with a maximum of one change per entity.  If you are
using the --apply-change task to patch a planet file you will need to feed
the change stream through --simplify-change first.

Again, first choice should be to import a planet and then apply replication
files directly to the database, and only use --simplify-change if the
--write-apidb-change task fails.  Second choice is to fully patch a planet
file first and use the --simply-change task in front of the --apply-change
task.

I haven't tested any of this out on full planet files though, only on small
test files.

Hope that helps,
Brett

PS.  Frank, I may have misunderstood but you mentioned something about the
"simple" schema.  The simple schema is something else entirely.  The apidb
tasks support the rails port schema which is not the simple schema.  The
"simple" schema is an Osmosis specific PostGIS schema supported by the pgsql
tasks.  Simple refers to the fact that it has no history tables.  To
complete the picture, the osm2pgsql tool has its own separate PostGIS schema
which has no Osmosis support.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/dev/attachments/20100107/153ab69f/attachment.html>