[openstreetmap/openstreetmap-website] Idempotency for API 0.6 (#2201)

Andy Allan notifications at github.com
Wed Apr 10 13:54:43 UTC 2019


I'm reasonably keen to see the API become fully idempotent. Network requests / responses / connections are lost for all kinds of reasons, and are more likely to happen the worse your internet connection (mobile, rural, far from OSMF's datacentres etc).

I know we've been planning on making a changeset upload queue, in order to avoid having to keep the connection open for significant amounts of time until the diff is finished being processed. It's still worth doing. But that doesn't fully solve the problem, since the response to the initial upload might not be received, and the client won't know whether the diff was put in to the queue or not.

We will, eventually, need to go through the API to make every request idempotent i.e. can be repeated as many times as you like with no ill effects. Setting aside changesets for the moment (since they are complex and arouse passions over multiple diffs in a changeset and whatnot), even simple things like creating a note or creating a trace can be changed to become fully idempotent. If we take the view that we shouldn't mess around with non-idempotent HTTP data upload methods like POST and PATCH, that leaves PUT.

But it's not easy to make a PUT request for a new note (or trace) idempotent. One way would be to have a specific url for creating a given note, like PUT /api/x/notes/uuid-goes-here. We can make it so that the client can PUT to that url twice or more, if required, with no ill effects. Of course this still has complexity - the first request received should succeed, the second should also succeed, but PUTing something different should get a 400, and if the trace is now different (e.g. the description was changed in another request or via the web) then PUTing the original data to the original URL should still succeed but without stomping on changes that were made subsequently.

This uuid could be generated by the server (and obtained via a different call) but that's not really necessary - the client could just make one up. And at that point this is just the same as having an idempotency key in a header anyway, which I think is a slightly more standard way to do it.

So in general I support this overall topic, and I don't think it really matters whether we use POST or PUT since, as far as I can tell, creating new resources needs an idempotency key regardless of whether the method is idempotent itself.

(P.S. There's a small opportunity to say that we can make life easier for ourselves for creation by saying that everything needs to be unique, i.e. you can't have two identical notes (or traces), and so you can just PUT without an idempotency key and the server can avoid creating duplicates by assuming duplicate requests are just resent requests. But that still doesn't work, since if the server receives a new trace, and then the owner changes the description 2 seconds later on the website, and the server receives the same creation request 2 seconds after that, it doesn't have enough information to know if it's an intentional recreation or just a resent request.)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/openstreetmap-website/issues/2201#issuecomment-481700850
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/rails-dev/attachments/20190410/565e7c12/attachment.html>


More information about the rails-dev mailing list