[osmosis-dev] Updating database with partial data

[Brett Henderson]

Hi Nuno,

On 10 October 2013 22:18, Nuno Miguel Lourenço <
nuno-miguel-lourenco at telecom.pt> wrote:

> Hi,****
>
> ** **
>
> I’m updating a OpenStreetMaps based DB, but we are doing incremental steps
> to the DB data.****
>
> We are importing just a few countries and then proceed to continents and
> at the end the planet file.****
>
> ** **
>
> We are already using the country data! NOTE: The countries are not
> neighbors!****
>
> We are importing some Europe countries from
> http://download.geofabrik.de/europe.html getting the several *.osm.pbf *files
> for the ones we want!****
>
> ** **
>
> We are coming across an issue…****
>
> We are getting a lot of ****
>
> *Detail: Key (id)=(xxxxx) is duplicated*.; nested exception is
> org.postgresql.util.PSQLException: ERROR: could not create unique index "*
> pk_relations*"****
>
> We are getting this for the relations table, for the nodes, ways tables
> also!!!
>

It sounds like your .osm.pbf files contain duplicate data.  The countries
may not be neighbours, but there may still be data relationships that
connect them.

I assume (providing your exact command line and exception messages would be
very helpful) you're trying to use a task like --write-pgsql which assumes
that the database is empty.  In your case you're trying to run it multiple
times against a non-empty database which will fail if the input files
contain duplicate data.


> ****
>
> ** **
>
> Is there any way we can avoid this?
>

Perhaps.  Combining all .osm.pdf files together into a single file and
remove duplicate data before importing should fix it.


> ****
>
> We tried to remove the duplicates on the *osmosisUpdate* function but it
> did not work due to this being called after all the pushes of data to the
> DB.
>

The osmosisUpdate function is only called by the --write-pgsql-change task,
but I don't believe you're using that task ...


> ****
>
> ** **
>
> Is there any other way to do this with osmosis?
>

Merge all the .osm.pbf files together before importing.


> ****
>
> Merging the several .*osm.pbf* files to generate a unique one is not a
> solution.
>

Okay.  Can you please explain why this is not an option?

Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/osmosis-dev/attachments/20131013/851e7829/attachment.html>