[openstreetmap/openstreetmap-website] Cucumber tests (#2545)

Andy Allan notifications at github.com
Wed Feb 19 13:45:31 UTC 2020

I've been aware of cucumber for years, and I deliberately avoid using it for any project that I'm involved in. I don't think we should use it here either.

I think a lot of my feelings are also described in these two posts:

* https://www.jimmycuadra.com/posts/please-don-t-use-cucumber/
* https://www.codewithjason.com/recommend-against-cucumber/

Basically, I feel like cucumber is a solution looking for a problem.

First, the natural language syntax isn't actually natural language in any way. It's just a thin textual layer above a bunch of hand-written regular expressions to match the sentences and run some actual test code. So if you make a small change to the sentence, you need to make the corresponding change to the regexp. If you try to make the regexps flexible (e.g. handling both "I am on the page" and "I am looking at the page"), they quickly get overcomplicated. Every new sentence you make needs a corresponding regular expression and implementation. So you end up writing just the same capybara-based tests (like `visit users_path`), but then you also have to wrap them in a regexp-based abstraction layer:

Given("I am on the microcosms page") do
  visit microcosms_path

Given("I am on the microcosm {string} page") do |name|
  visit microcosm_path(Microcosm.find_by(:name => name))

Secondly, we don't have any "Product Owners" or "Business Analysts" or anyone in a similar position to that. Even if we did, I doubt that they would want to read or contribute to the tests (something that cucumber claims to enable, but in the real world I think rarely actually happens). They would be writing text into github issue comments and developers would then be writing the relevant tests. So it feels like the layer of abstraction - which I still believe isn't genuinely useful anyway - isn't even necessary in our situation. 

Third, it's just a whole bunch of extra work when creating or reviewing our tests. Any time I see something like `Given there is a microcosm "MappingDC", "Washington, DC, USA", "38.9", "-77.03", "38.516", "39.472", "-77.671", "-76.349"` then I need to find the definition of that sentence, figure out what bits get parsed out into which variables, read the underlying definition to find out what it's really doing (and hope there isn't another definition with a similar regular expression that I should be reading instead), and so on. So I believe it increases the work needed to write tests and brings no benefits, and we'd be better just continuing to write code-based tests.

So I think we should continue to avoid introducing cucumber tests. I'm interested to hear what other people think.

You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstreetmap.org/pipermail/rails-dev/attachments/20200219/021ee63a/attachment-0001.htm>

More information about the rails-dev mailing list