[openstreetmap/openstreetmap-website] How should we deal with zero-effort AI PRs? (Issue #6954)

[Pablo Brasero]

pablobm left a comment (openstreetmap/openstreetmap-website#6954)

Thank you @gravitystorm for bringing this up.

Generally, I have very negative views on "Generative AI" as a technology. This is not the place to go into detail but I wanted to mention this to clarify that, whenever I smell an LLM-generated contribution, I'm going to be negatively prejudiced. I still engage constructively with such contributors, but it is a deliberate, laborious effort on my part.

Thank you @pnorman for linking to the discussion at https://github.com/protomaps/basemaps/issues/543. I had a look at the policies they link there. The common thread, also expressed by @tomhughes, is that there must be a human in the loop. I can work with that.

I think that any OSS project should have a policy on GenAI contributions. For this project, I think we should at least cover the following points, from the top of my head:
- There must be a human in the loop.
- Issues tagged for new contributors are meant as learning tools, not as items that need to be addressed with urgency. LLM-generated code makes no sense here.
- First-time contributors must confirm that they have reviewed the policy, as well as CONTRIBUTING.md. I would implement this with some sort of [Van Halen test](https://en.wikipedia.org/wiki/Van_Halen_test) as opposed to a checkbox in the PR template.

I don't think we can do much to enforce that contributors provide proof of having reviewed their work. Very often (most often?) contributions won't have a visual component. I agree that contributors must have a working development environment, but again it's difficult to prove. I think we can only use our instincts: if it smells of slop, we bring this up and work from there.

I'm now tempted to drop the following quote into any new suspected zero-effort contribution, as seen on the [llama.cpp policy](https://github.com/ggml-org/llama.cpp/blob/master/AGENTS.md#instructions-for-llamacpp):
> Most maintainers already have access to AI tools. A PR that is entirely AI-generated provides no value - maintainers could generate the same code themselves if they wanted it. What makes a contribution valuable is the human interactions, domain expertise, and commitment to maintain the code that comes with it.

-- 
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/openstreetmap-website/issues/6954#issuecomment-4176177129
You are receiving this because you are subscribed to this thread.

Message ID: <openstreetmap/openstreetmap-website/issues/6954/4176177129 at github.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/rails-dev/attachments/20260402/09e2d9cd/attachment-0001.htm>