[openstreetmap/openstreetmap-website] osmosis import : duplicate key value violates unique constraint (#1071)

Matt Amos notifications at github.com
Fri Dec 15 10:52:10 UTC 2017


Here are some tips for trying to fix the duplicate key issue. There might be more answers relevant to this on [help.openstreetmap.org](https://help.openstreetmap.org), and it's worth asking follow-up questions there in case someone in the community has already figured it out.

* The first thing to check is whether the input file has duplicate nodes. If it is a ["history"](https://wiki.openstreetmap.org/wiki/Planet.osm/full) file, then it is *not* suitable for loading with Osmosis' `--write-apidb` command. If it is a "normal" non-history file, it is possible that the file itself is broken. If you can get the same extract from somewhere else, then it might be worth trying that.
* It has happened to me that I've tried to import into a database which I thought was empty, but actually contained data (in my case, from a previous failed run). It's worth checking that the database is truly empty of all fixtures, test data, previous runs, and so forth, before starting the import. If in doubt, drop and re-create the database. I don't use Rails itself to do this, as I don't trust it not to put in data that I don't know about; I just load the `db/structure.sql` file directly. This might be me cargo-culting, I'm not sure :wink:
* Adding data to an existing database [is not supported](https://help.openstreetmap.org/questions/51995/duplicate-key-value-violates-unique-constraint-relations_pkey) by `--write-apidb`. If you do want to add data on top of existing data then you'll need to [derive a change](https://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage_0.46#--derive-change_.28--dc.29) between your two inputs and [add only the change](https://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage_0.46#--write-apidb-change_.28--wdc.29) to the database.
*  Finally, there is a complex method which can be used to clean the data, but I do not recommend it! It might cause more problems than it solves, but if you feel confident then give it a go. It's possible to remove duplicates by [deriving the change](https://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage_0.46#--derive-change_.28--dc.29) between [an empty input](https://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage_0.46#--read-empty_.28--rem.29) and your file, then [simplifying the change](https://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage_0.46#--simplify-change_.28--simc.29) and [applying the change](https://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage_0.46#--apply-change_.28--ac.29) to [an empty input](https://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage_0.46#--read-empty_.28--rem.29).


-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/openstreetmap-website/issues/1071#issuecomment-351975991
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/rails-dev/attachments/20171215/ab06d267/attachment-0001.html>


More information about the rails-dev mailing list