[openstreetmap/openstreetmap-website] WIP: Reduce use of Selenium in favour of rack-test (PR #6497)

[Pablo Brasero]

pablobm left a comment (openstreetmap/openstreetmap-website#6497)

100% agree with leaving controller/integration tests for low-level testing. My experience is that these are typically used for testing specific technical details, variants of "happy paths" reflected in system tests, and malicious usage of APIs.

My main experience is also with RSpec with the syntax shared by @gravitystorm. I want to think that with minitest it can be done with something like `driven_by :rack_test` similar to how there are `driven_by` declarations on https://github.com/openstreetmap/openstreetmap-website/blob/master/test/system/embed_test.rb, but I haven't gone that far into testing this idea yet.

Re: Turbo, yes, that's a tradeoff. With rack-test, links and form submissions will always trigger full-page loads. This creates a difference between what the tests do and real users do. At this point the question is: how much do we trust Rails to do what it promises to do?

When using an external tool, we want to think that we don't need to test the tool itself, but just the things we do with the tool, if that makes sense. Having said that, things can go wrong for example if something is misconfigured, etc. Eg: if we don't label a turbo frame correctly by mistake. Still we can go on a case-by-case basis. For example using Selenium only for the happy paths, or for specific situations where there's a known risk.

In exchange, we gain a faster execution of the test suite, a reduction of flaky tests, and more easily debuggable system tests.

-- 
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/openstreetmap-website/pull/6497#issuecomment-3485319404
You are receiving this because you are subscribed to this thread.

Message ID: <openstreetmap/openstreetmap-website/pull/6497/c3485319404 at github.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/rails-dev/attachments/20251104/54c5028a/attachment-0001.htm>