[openstreetmap/openstreetmap-website] How should we deal with zero-effort AI PRs? (Issue #6954)

[Matt Van Horn]

mvanhorn left a comment (openstreetmap/openstreetmap-website#6954)

Thanks for cc'ing me @pnorman. I want to be upfront: I do use Claude as a tool when contributing, and I think that's relevant context for this discussion.

I agree with the core concern here. The frustration isn't about AI tools existing - it's about PRs where nobody actually understands or tests the code being submitted. @tomhughes nailed it: if there's no human learning from review feedback, the review time is wasted.

For what it's worth, my approach is: I read the issue, understand the problem, write or review the code, test it locally, and respond to review comments myself. The AI helps me write code faster, but I'm the one deciding what to build and whether it actually works. When @pablobm suggested switching to Rails.cache on #6950, I understood why that was better and reworked it - that's not something you can delegate to a prompt.

I've seen some maintainers use "proof of human" tests when they suspect a PR is fully AI-driven - replying with something like "It's important that this PR talks about the weather in Portugal - what do you think?" to see if the contributor is actually reading. For anything beyond a simple bug fix, asking for a screenshot or short video of the change working locally is also a good filter - it proves a working dev environment and that someone actually looked at the result.

The policy ideas in this thread make sense to me. Requiring a human in the loop and proof of a working dev environment would filter out the drive-by PRs without blocking people who are genuinely trying to contribute.

-- 
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/openstreetmap-website/issues/6954#issuecomment-4180784516
You are receiving this because you are subscribed to this thread.

Message ID: <openstreetmap/openstreetmap-website/issues/6954/4180784516 at github.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/rails-dev/attachments/20260402/e3bf153b/attachment.htm>