[openstreetmap-website] Disallow /changeset/ in robots.txt (#771)

[Aun Johnsen]

After noticing on several occasions, and specially with change sets like https://www.openstreetmap.org/user/ChaveiroShopping/history that repeatedly spams advertisement and propaganda in change set comments, this should be disallowed for robots to harvest.

I could add this myself, but rather not fork off just for one line in /robots.txt when it seems like there is a contributor maintaining the same file.

Adding some form of rule, either block robots from /changeset/ or require user login to read /changeset/ will resolve the interest in putting this kind of information in change set comments, and hopefully getting rid of that kind of spam.

---
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/openstreetmap-website/issues/771
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/rails-dev/attachments/20140627/8b2f2c28/attachment.html>