Cucumbers Have Layers: A Love Story (AATC 2017)

1. @geeksam CUCUMBERS HAVE LAYERS Sam Livingston-Gray 💚 1 A Love Story With that out of the way, I'd like to get a quick [ADVANCE] show of hands.

2. @geeksam AUDIENCE POLL How many people in this audience... Have used Cucumber (on any project)? Have used Cucumber more than once? Would use Cucumber again? Any ﬁrst-timers? ❓ 2 [REVEAL EACH] - Who here has used Cucumber on any project, big or small? - Who's used Cucumber on more than one project? - And regardless of how many times you've used it before, how many would use it again? - Finally, do we have anybody here who's new to Cucumber? [IF SO:] Welcome! I'll try not to leave you behind, but if you need me to clear anything up, please don't hesitate to ask me after the talk.

3. @geeksam3 For those of you who haven't used Cucumber, you should go get The Cucumber Book before you start. It'll give you a much better introduction than I possibly could, even if this were a full 45 minutes of "Cucumber 101." But, just so you're not completely lost...

4. @geeksam CUCUMBER 101 ❓ 4 Cucumber is often referred to as a tool for writing automated acceptance tests. What I prefer to say is that Cucumber lets you describe the behavior of your software in a domain-speciﬁc language called [ADVANCE] Gherkin.

5. @geeksam CUCUMBER 101 GHERKIN ❓ 5 Each separate Gherkin ﬁle is called a feature, and here's a feature that I pulled straight from the Cucumber website.

6. @geeksam Feature: Addition In order to avoid silly mistakes As a math idiot I want to be told the sum of two numbers Scenario: Add two numbers Given I have entered 50 into the calculator And I have entered 70 into the calculator When I press add Then the result should be 120 on the screen 6 A feature has one or more scenarios, and a scenario has one or more steps: those are the given/when/then that you see toward the bottom left of the screen. Aside from a few keywords, which I've highlighted here in green, everything else is written in whatever natural language works for you. Gherkin's grammar is extremely simple: everything from a keyword to the end of the line is basically treated as a single token by the Gherkin parser. This is quite useful just as documentation, but Cucumber also lets you use these feature files to automate tests, which is why Cucumber folks tend to talk about "executable specifications". To go from human-readable documents to running automated tests, you have to write a bunch of [ADVANCE] step definitions.

7. @geeksam CUCUMBER 101 STEP DEFINITIONS ❓ 7 Given /I have entered (.*) into the calculator/ do |n| @calculator = Calculator.new calculator.push(n) end This is what a step deﬁnition looks like in Ruby; your local implementation may vary. This boils down to a [REVEAL] regular expression that gets associated with [REVEAL] a chunk of code. This is how you translate from human-friendly blobs of text to something that Ruby can execute. When Cucumber wants to run a step, it tests it against every one of the regular expressions you've given it. When it ﬁnds an expression that matches, it executes the corresponding code. There's more, but that's all you need to know to follow along with this talk.

8. @geeksam Gherkin Features Scenarios Steps  Cucumber Parses Gherkin Steps → Step deﬁnitions Runs each Scenario  step by step CUCUMBER 101 ❓ 8 In my mind, Gherkin and Cucumber are almost, but not quite, two separate things. [REVEAL EACH] - Gherkin gives you a human-friendly way to describe software, - and Cucumber interprets your Gherkin ﬁles and uses them as a script for automating tests. So that's the Cucumber crash course. Let's dig into Gherkin a little bit more.

9. @geeksam GHERKIN IS:  AWESOME 😍 9 [PAUSE BRIEFLY] Gherkin is a domain-speciﬁc language where the domain is "talking to other humans about software." It's very free-form, so it lets you talk about your application using whatever natural language makes sense to you and your team. Gherkin has just enough structure that Cucumber can use it to drive a lot of machinery for automating tests. But...

10. @geeksam GHERKIN IS NOT:  CODE 💚 10 Gherkin is NOT a programming language! This is a critically important point that can be easy to overlook when you're starting out with Cucumber. Programming languages are great, but they require us to get a bunch of details right all at once, and getting into that headspace tends to shift our focus onto how to do a thing. Gherkin, on the other hand, exists to help us think about what thing to do, why we're doing it, and who we're doing it for.

11. @geeksam CUCUMBER IS NOT: TDD 💚 11 I think it's also important to realize that Cucumber is not a tool for doing test-driven development. Cucumber and TDD complement each other nicely, but it's been my experience that Cucumber works on a very diﬀerent rhythm and timescale than TDD.

12. @geeksam Cucumber Scenario Unit TestFail Pass Refactor Fail Pass Refactor 12 I think of Cucumber as a set of guide rails for TDD, and my workflow for using it goes something like this... [REVEAL EACH] - I start with a Cucumber scenario. I run the scenario... - and watch it fail. I look at the error message to find out why it failed, and use that information to go... - write a unit test. - I watch the unit test fail... - make it pass... - and refactor. At this point, I have a choice. If I know what the next unit test I need to write is, - I do that, and go back around the red/green/refactor cycle again, usually several times. But if I'm stuck... - I go back and run the Cucumber test again. It's probably still failing, but it's failing for a different reason. So I go back into that fast TDD cycle again. At some point, I run the Cucumber test and... - it passes. I do a little dance, - refactor, - and move to the next scenario.

13. @geeksam Cucumber Scenario Unit TestFail Pass Refactor Fail Pass Refactor 13 When I'm working this way, I spend most of my time in that tight inner loop doing TDD: red/green/refactor. The fact that the tests and the code under test are both written in the same language makes it easy to go around the inner TDD loop very quickly—sometimes writing a new test every minute or so. This is where I'm focusing on how the thing works, and it's good, satisfying, detail-oriented work.

14. @geeksam Cucumber Scenario Unit TestFail Pass Refactor Fail Pass Refactor 14 But when I start to lose sight of the forest for the trees, I jump back up to Cucumber. The shift from a programming language back to Gherkin helps remind me to get out of that hyperfocused how mode, and come back to thinking about what, why, and who... and that helps me ﬁgure out the next thing to do.

15. @geeksam "For me, Cucumber works  more like a mind hack than a testing tool.  It reminds and invites me to think about the experience of using software separately from the details of its implementation." -Tom Stuart http://codon.com/notes-on-counting-tree-nodes MIND HACK 15 Tom Stuart wrote something about Cucumber that really resonated with me. He described it as [REVEAL] "more like a mind hack than a testing tool", because it helps him think about the big picture rather than the details.

16. @geeksam "I wish more people knew it works best  as a thinking and collaboration tool,  not just an automated checking tool" -@mattwynne A THINKING TOOL 🎓 16 When I first put this talk together, I asked Matt Wynne, co-author of The Cucumber Book, if there was anything he wanted people to know about Cucumber. He tweeted back that he wished more people viewed Cucumber as [REVEAL] a thinking and collaboration tool, not just something for test automation. Both of these quotes lead back to something I said a few slides ago. I asserted that Gherkin is not a programming language, and that Cucumber is not a TDD tool. But negative definitions aren't very useful. Or, to reframe that, a positive definition is much more useful than a negative one. What are these things for? Well, I already talked about how I use Cucumber as guide rails around an inner TDD loop, but let's talk about Gherkin some more.

17. @geeksam Describing software  At the level of  USER INTENT  (and maybe automating  tests with Cucumber) GHERKIN IS FOR: 💚 17 In contrast to the quotes I just showed you, this part is purely my own opinion, based on my own experience. I want to make it clear that I do not speak for the Cucumber team here. I think that Gherkin is for: [REVEAL EACH] - describing software  - at the level of user intent. And, at some point, you might choose to use Cucumber to turn your Gherkin artifacts [REVEAL] into automated tests. Let's unpack that one piece at a time.

18. @geeksam Acceptance Criteria DESCRIBING SOFTWARE If this doesn't work,  our users will abandon us. 💚 18 By "describing software", I mean that Gherkin lets you capture [REVEAL] "acceptance criteria". And by "acceptance criteria", I mean [REVEAL] "the system has to do these things or you don't get paid."

19. @geeksam When I add a new widget USER INTENT NOT: When I visit "/widgets"  And I click the "New Widget" link  And I ﬁll in "Widget Name" with "My Widget"  And I click the "Create Widget" button 💚 19 By "user intent", I mean that Gherkin lets you paint your picture [REVEAL] in broad strokes, without getting bogged down in a lot of details. Details are what TDD is all about. [REVEAL, PAUSE] I've worked with scenarios that look like this, and what I found was that every time I tweaked my user interface, twenty of my Cucumber scenarios would explode, and I'd have to spend an hour or three adjusting everything, which... is not the best use of my time.

20. @geeksam (maybe) AUTOMATING TESTS 💚 20 Finally, just because Cucumber is often pitched as a tool for writing automated tests, you are under no obligation whatsoever to use it to automate your tests. Personally, I think Cucumber's greatest value comes from using Gherkin to facilitate conversations between developers and the people who pay us. I've written Gherkin ﬁles, thrown them away, and felt like my time was very well spent.

21. @geeksam 💚 21 Describing software  At the level of  USER INTENT  (and maybe automating  tests with Cucumber) So this is what I now think Cucumber should be used for. But it took me a long time to ﬁgure this out, and I made a lot of mistakes along the way. Some of those mistakes were rather painful. In the hope that you can learn from them, I'm going to share my mistakes with you.

22. @geeksam Inconceivable! CLASSIC BLUNDERS 💔 22 Yes, this is the part of the talk where you all get to laugh at me. [REVEAL] This is by no means an exhaustive list of my Cucumber fuckups; these are just some of the more interesting, entertaining, or educational ones.

23. @geeksam When I visit "/test/js-tests" Then I see 44 passing QUnit tests WTF 💔 23 I'm going to show you a Cucumber scenario that I helped write in a real live codebase. Before I show it to you, let me reiterate: it is okay to laugh at me. Everybody ready? [REVEAL AND... WAIT FOR IT] Pro tip: just because you've already got Cucumber set up to drive a browser doesn't mean you should use it for something like this! Going back to what I said Gherkin was for...this deﬁnitely describes software, and oh boy is it automated, but it completely fails the "user intent" test. Here's a quick question that should have clued me in that this did not belong in Cucumber: who cares about this? Is this important to someone using my software?

24. @geeksam Given the Rails application Then CRUD works for the Widgets controller ACCEPTANCE ASTRONAUT 💔 24 Here's something else I did in an actual project. [REVEAL] What this did was - visit the new page for the widgets controller, - ﬁll out the form with randomly generated data, - submit the form, - check that the random data it submitted was displayed on the show page, - click the edit link, - change each value on the edit form, click "Save", make sure that the changes were visible... and so on. Seriously, this was a lot of fun, but while we were gold-plating our Cucumber suite, we were avoiding writing actual features that our actual customer actually cared about, and pretty soon they actually ﬁred us.

25. @geeksam CUKING REGRESSION TESTS 💔 25 When you get a bug report, it's a good idea to write an automated test to reproduce the bug. You might even want to commit that test to your repository so you can detect any future regressions. However, it is my considered opinion that you should not put these tests in Cucumber. As with so many things in software, it comes down to communication. Gherkin is a great way for you to tell the story of your application. Ideally, when you bring on a new developer or manager, you can print out all of your feature ﬁles and hand them to your new hire. They should be able to read through those in an hour or two and come away with a pretty good high-level idea of what your software does. Regression tests interrupt that narrative with a bunch of digressions into "remember the bug when?" Doing this is also a good way to commit the next blunder, which is just plain [ADVANCE] having too many scenarios.

26. @geeksam *(to run on every commit) TOO MANY SCENARIOS* 💔 26 Opinions vary on how long is a reasonable time to wait for a test suite. Personally, I'm willing to wait about ﬁve minutes, up to three or four times a day. Any more than that and my tests might as well not be running. If you do ﬁnd yourself in this situation, you might consider tagging a critical subset of your tests to run before every commit, then let your CI server run everything else after you push. Another way to commit the "too many scenarios" blunder is [ADVANCE] to automate every feature you write.

27. @geeksam @FYI / @TBD AUTOMATING EVERY FEATURE 💔 27 It's perfectly okay to use Gherkin to facilitate a conversation with someone—possibly even yourself, if you don't happen to have a rubber duck handy. Once you've learned what you needed to learn from the feature ﬁle, you have my permission to just delete it. It's okay! Now, if do you feel the need to hang on to it for posterity, you still don't have to automate it. Before you commit your changes, tag your features as [REVEAL] "@FYI" or "@TBD", and change your conﬁguration so that scenarios with that tag never run. So far, all of these blunders can be boiled down to a loss of focus on the user intent. But Cucumber also gives you plenty of room for implementation blunders, so I want to talk about a those before we move on.

28. @geeksam ABUSING STEP DEFINITIONS Too many step definitions Huge step definitions Too many huge step definitions Step definitions that call other s͘te͡p d e҉ fin̕it̕i͏o̕n̨ s Just about any l ͔̭͎̘͓̣ o̥̯̯ ̭g̝i ̘̪̯̱̙̟ c̗͓̟͕̭̬ wh̛ a̢ ts ̀oe̢ v͡er͡ in a̙̖̙̖͔̤ ͓͍s̗͎̩ ̩t͓̥͖ e̗̹p͝ ̗̮̩̰̜͍ ͉͕͉̲͍͚̞͟ d̀͒ ̳̩ͅ ę̭̹̥͎͈̲ ͋́ͯ̾ f ̋̚ ͒ͥ͑͛ ̷ ̖̼͎̞͓͕ i ̨͔̈́̈́̌̇͗͐ņ̴̖̯̦̞ ͗̾͆̆ ͛͊ i͐ͮ̔̄̿ͮͩ͊ ̬͉̖ t͌̆̑̄ ҉ ͈͙̻̥͚͉̩ i͐́̓́ ̦̗̳ ô͆̿̉ͭ͂͊ ̶ ́ ͈͓n̎̆ͥ͐ ͏̵̖̗ 😡 Given /some rubbish/ do # rubbish code... end 28 Basically, step definitions are terrible. They're a necessary implementation detail, but they will get you in SO MUCH TROUBLE.   If you get the Cucumber book—and you should get the Cucumber book—it will tell you to define a bunch of helper functions and invoke them from your step definitions, and that does help. But... I started using Cucumber years before the Cucumber book was published. Which means I've made mistakes like: [REVEAL EACH. DO NOT AD-LIB NUMBERS; THEY'RE COMING SOON] Doing either of these last two things turns out to be an excellent way to summon the Elder Gods. Ask me how I know this.

29. @geeksam </RANT> 💔 29 After making all of those mistakes and more, I found myself feeling very conﬂicted about Cucumber. I really loved the expressiveness of Gherkin, and I wanted to believe in this idea that programmers and managers could sit down in a room and write acceptance tests together in universal harmony. But I struggled to reconcile that with the project I'd been working on, which had hundreds of scenarios backed by 750 step deﬁnitions that contained almost 5,000 lines of Ruby code, and the whole test suite took about 90 minutes to run. It sucked. Eventually, I found myself asking an interesting question...

30. @geeksam ❓ THE QUESTION How would you write scenarios if you didn't know what the UI was going to be? 30 [REVEAL AND READ] [PAUSE] I assert that, if you can tell from reading your Cucumber features whether it refers to a web app or a desktop app or a CLI... you're probably letting too much detail leak into your features. And even though this question doesn't directly say anything about user intent, it turns out to be a great way to drag your attention back to what really matters about your software. Here's an example.

31. @geeksam ❓ When I go to the home screen And I click on the "Widgets" button vs. When I view the list of available widgets INTERFACE VS. INTENT 31 [PAUSE 5 SECONDS. THE SILENCE WILL NOT KILL YOU.] Here's another.

32. @geeksam ❓ When I POST <some JSON> to "/widgets/42" vs. When I save my changes to the current widget INTERFACE VS. INTENT 32 [PAUSE 5 SECONDS. BREATHE.]

33. @geeksam ❓ INTERFACE VS. INTENT How would you write scenarios if you didn't know what the UI was going to be? 33 This question ﬂoated around in my head for a while as I worked on other things, until...

34. @geeksam THE PROJECT Salesperson Commissions Multiple compensation schemes Schemes change every quarter or two Described in dense, imprecise quasi-legalese 💚 34 Once upon a time, I was brought in to work on one part of a rather large monolithic Rails app that calculated [REVEAL] salesperson commissions. Now, you might hear "salesperson commissions" and think "okay, so you add up how much each person sold, multiply the total by some percentage, and cut a check, right?" But, no. That would be far too simple. There were usually [REVEAL] half a dozen compensation schemes in effect at any one time, with very different incentives for different kinds of sales staff. These schemes changed [REVEAL] a couple of times a year, sometimes quite dramatically. And they were worked out by the sales department, who would put together something like fifteen pages of [REVEAL] dense, confusing quasi-legalese to describe each plan, which we then had to read through and somehow translate into working code. So, one of my goals as I worked on this project was to be able to describe every aspect of these schemes using Gherkin. I figured that if I could show them how I translated their documents into clear, precise examples, maybe in a year or two they'd work out how to do it themselves.

35. @geeksam This is a work of fiction. DISCLAIMER Any resemblance to actual  sales compensation schemes is...  unfortunate. 💚 35 I'm going to talk about a simplified, fictionalized version of how just one of those compensation schemes worked. It's not super important for you to catch all of the details here. I'm just trying to give you a quick taste of how I wrote and organized my features for this project.

36. @geeksam TARGETS Sales Target: $100,000 / month Target Bonus: $100 💚 36 We'll start with the concept of a "sales target" and a "target bonus". This is, basically, the company saying "if you sell $100k of widgets in a month [REVEAL] (that's the sales target), we'll pay you $100 over your base salary [REVEAL] (that's the target bonus)." By the way, these numbers aren't realistic; I just picked them to make it easy to convert between percentages and dollars in my head. Anyway, there's a scaling factor here: if you miss your target, you get paid less. If you exceed your sales target, you get paid more. So far, so good—but there's a catch.

37. @geeksam PAY CURVE 💚 % to Sales Target % of Target Bonus paid 0% 0% 50% 25% 100% 100% 150% 175% 37 The catch is this little thing called a "pay curve". The pay curve is a simple function: you put in what percentage of the sales target you hit, and you get back the percentage of your target bonus that you'll get paid. To describe the pay curve, the sales department actually gave us a spreadsheet with example rows for every possible input value from zero to 250. Fortunately, since it was already in a spreadsheet, it was easy to build a chart so we could see the curve.

38. @geeksam PAY CURVE 💚 PercentofBonus 0% 75% 150% 225% 300% Percent to Target 0% 25% 50% 75% 100% 125% 150% 175% 200% 225% 250% 275% 38 Looking at the chart shows that this is a "piecewise linear function", and that helped me wrap my head around what was happening. So [ADVANCE] I went back to the spreadsheet...

39. @geeksam PAY CURVE 💚 % to Sales Target % of Target Bonus paid 0% 0% 50% 25% 100% 100% 150% 175% 39 and from there, it was quite straightforward to convert that spreadsheet into a Cucumber table, and use it to drive a scenario outline.

40. @geeksam Feature: Pay Curve for Sales Associates  Scenario Outline: "Sales Associate" Pay Curve Given the "Sales Associate" pay curve When the rep hits <Percent to Target> of their sales targets Then their monthly payout is <Percent of Bonus> of target monthly bonus  Examples: | Percent to Target | Percent of Bonus | | 0 | 0.0 | | 1 | 0.5 | | 50 | 25.0 | | 51 | 26.5 | 40 In case you're not familiar with it, a scenario outline is basically a [REVEAL] template for a scenario, followed by [REVEAL] a table. The template gets executed once for each row in the table, with [REVEAL] values from the appropriate column ﬁlled in wherever you put a placeholder value. Now, because I knew that this was a piecewise linear function, I was able to get away with not turning all 251 rows of the spreadsheet into a giant Cucumber table. I put in a few examples around each of the inﬂection points just to make sure I got the boundaries right, and then moved on to the compensation scheme...

41. @geeksam Feature: Compensation Scheme for Sales Associates    Background: * I am a sales rep using the "Sales Associate" scheme * I have a monthly sales target of $100,000 * I have a monthly target bonus of $100    Scenario Outline: Using the right pay curve When I make <Sales> in sales Then my monthly sales bonus is <Bonus> Examples: | Sales | Bonus | | $ 0 | $ 0 | | $ 50,000 | $ 25 | | $ 100,000 | $ 100 | 41 At ﬁrst glance, this looks remarkably similar to the scenarios for the pay curves. When I ﬁrst wrote this, it felt like I was repeating myself, but this actually introduces quite a few new concepts: [REVEAL EACH] - compensation schemes, - sales target and target bonus in dollars instead of percentages, - actual sales in dollars, and - bonus amount in dollars. With those concepts in place, I could then introduce the next feature of this scheme.

42. @geeksam Feature: Commissions Plan for Sales Associates    Scenario Outline: Safety Net Given my safety net IS deployed When I make <Sales> in sales Then my monthly sales bonus is <Bonus> And my safety net bonus is <Safety Net>  Examples: | Sales | Bonus | Safety Net | | $ 0 | $ 0 | $ 100 | | $ 50,000 | $ 25 | $ 75 | | $ 100,000 | $ 100 | $ 0 | | $ 175,000 | $ 225 | $ 0 | 42 This is the last one, I promise! The safety net is a feature to help out new hires as they're getting up to speed for the ﬁrst few months. This is basically a guarantee that you'll always get paid at least the amount of your target bonus. If you don't hit your sales target, we'll kick in the diﬀerence. If you do better than your sales target, we'll pay you more than your target bonus -- but you'll never make less, at least until we take your safety net away. Anyway, I think that's enough to give you a sense for the project. Before I get to the fun twist, though, I want to talk about an underutilized element of Gherkin's grammar.

43. @geeksam Feature: Compensation Scheme for Sales Associates   FREE-FORM TEXT GOES HERE!  Scenario: Dolor sit amet 43 I omitted this on earlier slides so I could make the text bigger, but Gherkin gives you some space at the top of the ﬁle where you can write whatever you want. A lot of Cucumber examples show that space being used for [ADVANCE] "As a / I want / so that"...

44. @geeksam Feature: Lorem Ipsum   As a ______ I want ______ So that ______  Scenario: Dolor sit amet 44 ...but in practice, I ﬁnd that people tend to [ADVANCE] ﬁll in that template without really thinking about it...

45. @geeksam Feature: Lorem Ipsum   As a manager ﬁlling in this form I want to make up some plausible BS So that the developers do what I tell them to, damn it  Scenario: Dolor sit amet 45 [WAIT FOR THE CHUCKLE] So sometimes I just skip this part. For this project, though...

46. @geeksam Feature: Compensation Scheme for Sales Associates   This scheme is based on gross sales, and uses the "Sales Associate" pay curve.  This scheme features a "safety net". If the appropriate ﬂag is set, sales reps on this scheme are guaranteed at least 100% of their target bonus. This tends to be used in a new hire's ﬁrst few months so they can earn a reasonable living while they get up to speed. Background: * I am a sales rep using the "Sales Associate" scheme *[...] 46 ...I used that space to provide some context about why this feature exists, or what makes this feature interesting in comparison to other features that may be similar. [KEEP GOING]

47. @geeksam Feature: Commissions Plan for Sales Associates   This plan is based on gross sales, and uses the "Sales Associate" pay curve.  This plan features a "safety net". If the appropriate flag is set, sales reps on this plan are guaranteed at least 100% of their target bonus. This tends to be used in a new hire's first few months so they can earn a reasonable living while they get up to speed. Background: * I am a sales rep using the "Sales Associate" plan *[...] 47 For this scheme, I was worried that the examples of how the safety net worked in specific cases might not fully explain what that aspect of the scheme was for. So I took a few lines to explain it, using the simplest language I could. After I handed this project off, one of the bits of feedback I got was that this documentation, in particular, was extremely helpful in making sense of these frankly ridiculous compensation schemes. Again, this is the sort of thing you may overlook if you're trying to treat Gherkin as a programming language. If you're thinking of Gherkin as code, then this section of the file just feels like a big block comment. But if you're thinking about Gherkin as a medium for communicating with other people about your project, this freeform text area can be really useful, because it gives you a place to talk about things without having to fit it into a step-by-step recipe.

48. @geeksam Feature: Compensation Scheme for Sales Associates    Scenario Outline: Safety Net Given my safety net IS deployed When I make <Sales> in sales Then my monthly sales bonus is <Bonus> And my safety net bonus is <Safety Net>  Examples: | Sales | Bonus | Safety Net | | $ 0 | $ 0 | $ 100 | | $ 50,000 | $ 25 | $ 75 | | $ 100,000 | $ 100 | $ 0 | | $ 175,000 | $ 225 | $ 0 | 48 The last thing I want you to notice about these features is that absolutely every word of this is expressed in terms of the domain, not the interface. If you sat down and read through all of these features, you'd learn a lot about how this organization thinks it can motivate its salespeople, but you won't have any idea what kind of app they're using to do it. Now, let's talk very brieﬂy about [ADVANCE] architecture.

49. @geeksam ARCHITECTURE 📐 Controllers &Views ActiveRecord & Services Core logic in POROs 49 I normally present at Ruby conferences, which means I mostly talk with Rails developers, and while I love Ruby, the fact that Rails dominates the Ruby market means that the level of discourse around software architecture is... not always the most sophisticated. The short version is that I departed from Rails orthodoxy by introducing service objects and a layer of plain old Ruby objects. I organized these into three logical layers as follows: [REVEAL EACH] - The UI is standard Rails controllers and views. - The UI layer talks to a mix of ActiveRecord objects and some service objects. - And that layer, in turn, talks to a set of plain old Ruby objects, or POROs, that model the rules for the compensation schemes themselves. Anyway, this is a little bit formal by Rails standards, but it's nothing earth-shattering.

50. @geeksam TESTING LAYERS 💚 ActiveRecord & Services Core logic in POROs 50 Controllers &Views @model @ui @core The interesting thing I did for this project was to reuse my Cucumber scenarios, running them at both the UI layer and the ActiveRecord layer. Using the tagging feature of Cucumber, I wanted to be able to mark a scenario as being [REVEAL] "@ui", [REVEAL] "@model", or both. The scenarios tagged with "@ui" would drive a web browser, which is the way most people in the Rails world think about using Cucumber. But the scenarios tagged with "@model" would run directly against the model layer, so they could be faster. The ActiveRecord layer, in turn, would exercise the POROs indirectly, and I thought that would be good enough. However, I discovered very quickly that the PORO layer was complicated enough on its own that I didn't want to have to think about the relational data model at the same time, so almost immediately, I added [REVEAL] a "@core" tag as well.

51. @geeksam Feature: Pay Curve for Sales Associates  Scenario Outline: "Sales Associate" Pay Curve Given the "Sales Associate" pay curve When the rep hits <Percent to Target> of their sales targets Then their monthly payout is <Percent of Bonus> of their target monthly bonus  Examples: | Percent to Target | Percent of Bonus | | 0 | 0.0 | | 1 | 0.5 | | 2 | 1.0 | 51 Here's how that played out in one of the feature ﬁles I showed earlier. Once I had written a scenario, I would start by tagging it with the name of the layer I wanted to run it at, plus a [ADVANCE] "WIP" suﬃx to indicate that it was work in progress.

52. @geeksam Feature: Pay Curve for Sales Associates  @core_WIP Scenario Outline: "Sales Associate" Pay Curve Given the "Sales Associate" pay curve When the rep hits <Percent to Target> of their sales targets Then their monthly payout is <Percent of Bonus> of their target monthly bonus  Examples: | Percent to Target | Percent of Bonus | | 0 | 0.0 | | 1 | 0.5 | | 2 | 1.0 | 52 I'd run the scenario and watch it fail, and from there, I'd drop down to RSpec and do the usual small, fast TDD cycles until the scenario passed. Once the scenario was passing, I'd [ADVANCE] remove the "WIP" suﬃx.

53. @geeksam Feature: Pay Curve for Sales Associates  @core Scenario Outline: "Sales Associate" Pay Curve Given the "Sales Associate" pay curve When the rep hits <Percent to Target> of their sales targets Then their monthly payout is <Percent of Bonus> of their target monthly bonus  Examples: | Percent to Target | Percent of Bonus | | 0 | 0.0 | | 1 | 0.5 | | 2 | 1.0 | 53 If I wanted to reuse the scenario at the next layer up, I would then add another tag for that layer, again with the [ADVANCE] "WIP" suﬃx.

54. @geeksam Feature: Pay Curve for Sales Associates  @core @model_WIP Scenario Outline: "Sales Associate" Pay Curve Given the "Sales Associate" pay curve When the rep hits <Percent to Target> of their sales targets Then their monthly payout is <Percent of Bonus> of their target monthly bonus  Examples: | Percent to Target | Percent of Bonus | | 0 | 0.0 | | 1 | 0.5 | | 2 | 1.0 | 54 And again, I'd write a bunch of RSpec tests until I got the same scenario passing at the new higher layer, at which point I would [ADVANCE] remove the "WIP" suﬃx from that layer.

55. @geeksam Feature: Pay Curve for Sales Associates  @core @model Scenario Outline: "Sales Associate" Pay Curve Given the "Sales Associate" pay curve When the rep hits <Percent to Target> of their sales targets Then their monthly payout is <Percent of Bonus> of their target monthly bonus  Examples: | Percent to Target | Percent of Bonus | | 0 | 0.0 | | 1 | 0.5 | | 2 | 1.0 | 55 So that's how I approached this from the Gherkin side of things. But it took me a while to ﬁgure out a good way to implement this...

56. @geeksam Messing with Cucumber load path THROWING A SWITCH 💚 56 I wanted Cucumber to run all of the @core scenarios with the load path set to "features/core" to load the core set of step definitions, then run all of the @model scenarios with the load path set to "features/model", which had its own set of step definitions, and then run all of the @ui scenarios with a *third* load path pointing to a third independent set of step definitions. Manipulating the load path was painful, but it basically worked. The problem was that, any time I changed a step, I had to edit three different regular expressions in three different files and make sure they all matched. Which...was not very much fun.

57. @geeksam Swapping in different step drivers THROWING A SWITCH 💚 Given /(.*?) in sales/ do |amount| step_driver.given.sales_total(amount) end 57 So instead, I wound up consolidating down to one set of step definitions that invoked a step driver. [REVEAL, PAUSE] Again, this is nothing earth shattering. A basic principle of object-oriented programming is that if you send messages to an object that plays a role, you can swap in a different object without modifying the calling code. It's called "polymorphism." As an experienced Cucumber user, though, I did find it nicely refreshing that being forced to do this made my step definitions very small and simple. With this change, I was then able to define three different step drivers to interface with each layer of my application. I put all three of them on the load path, and used an environment variable to decide which step driver to instantiate. I had a lot of fun doing this, and used literally every single Ruby trick I had learned in almost a decade to do it... but further details about that are beyond the scope of this talk. Feel free to ask me about them later if you're curious.

58. @geeksam OBSERVATIONS 🔬 58 I just have a few observations to make before I wrap up. First, a piece of advice from my inner ﬁve-year-old:

59. @geeksam THE STEP DEFINITIONS ARE LAVA 59 🔬 Given /(.*?) in sales/ do |amount| step_driver.given.sales_total(amount) end The step definitions are lava. Because of the way Cucumber works, the step definitions have to be there, but they should be a very thin adapter between Gherkin steps and a custom driver for automating your application. [REVEAL] Ideally, a step definition should be one annoyingly obvious line of code. Step definitions exist in a flat namespace with one global scope for sharing variables between them. Moving all of the interesting logic out into a step driver lets me use the full set of tools my programming language gives me to organize and refactor code, and lets me keep the step definitions so simple that I never have to think about them. Using a step driver worked so well for me that I'd do it again on a new project even if I wasn't using a multi-layered approach.

60. @geeksam BREAK IT DOWN 🔬 60 These commissions plans involved something like fifteen pages of legalese, which I then had to translate into code. It was just too complicated for me to hold all of it in my head at once. Breaking the problem down across architectural layers allowed me to focus on just the core logic, then figure out how to adapt it to my persistence layer, and then fill in the UI once everything else was already working. And honestly, looking back at it, this was probably the only way I could have completed this app in any reasonable amount of time.

61. @geeksam TAG DISTRIBUTION 🔬 Feature @core @model @ui Pay Curve ✓ Associate scheme uses right pay curve ✓ ✓ Safety net ✓ ✓ "Big Deal" bonus ✓ ✓ Total monthly bonus ✓ ✓ 61 Some scenarios I only tagged at one level, and some of them I tagged at two. In practice, it turned out that the ones that ran at multiple layers always ran at two adjacent layers. So: @core and @model, or @model and @ui, but I never had a single scenario that ran at both @core and @ui. I think this was because I had some features that described some lower-level concepts, like the pay curve, that never showed up directly in the user interface. So it would make sense to describe those concepts at the core and model layers—and I still got a lot of value from describing them in Gherkin—but by the time I made it up to the UI, they just weren't worth mentioning.

62. @geeksam SUSTAINABILITY "I can see why you did it that way" "It was really nice to have such clear documentation" 🔬 62 The developers who took over the project from me weren't as enthusiastic about this Cucumber setup as I was... but they did say things like [REVEAL EACH] - "Yeah, I can see why you did it that way" and - "it was really nice to have such clear documentation about what these business terms meant." So I'll call that a win.

63. @geeksam 🔬 PERFORMANCE 63 I do have to talk about performance. At Cascadia RubyConf in 2011, Ryan Davis gave a talk called "Size Doesn't Matter," in which he talked about the speed and relative size of various testing frameworks in Ruby. Cucumber shows up in nine diﬀerent slides from Ryan's slide deck, and they basically all look like this:

64. @geeksam64Cascadia Ruby Conf 2011 @ Seattle, Cascadia Ryan Davis, Seattle.rbSize Doesn’t Matter Lines of Code (including dependencies) bacon minitest shoulda test-unit2 rspec2 cucumber 17430 7484 6381 4236 1166 380 + 62,985 lines of C! Monday, August 1, 11 In every single metric Ryan chose to present, Cucumber came in dead last. When I saw his talk, I laughed and winced, because at the time, I was dealing with that 90- minute test suite. So when I started this project, I was fully expecting to pay a huge performance penalty.

65. @geeksam PERFORMANCE Layer # Scenarios Time Core 64 0.7s Model 118 7.2s UI 11 3.0s Total wall clock time (WIP + done) ~40s 🔬 65 With that in mind, here are the numbers from this project. There were 64 scenarios tagged with "@core", and they ran in [REVEAL] under a second. At the model layer, 118 scenarios ran in [REVEAL] under eight seconds. At the UI layer, I used Capybara to drive the application. I didn't care about Javascript, so I was able to just look at the HTML, but even with that advantage, these were by far the slowest: [REVEAL] 11 scenarios in three seconds. Of course, these are the times reported by Cucumber, so they don't include the time it takes to load all of Rails. I had a build task set up to run the WIP versions of all three layers, then run the non-WIP versions, for a total of six separate test runs, so I loaded Rails six times. The total wall clock time for that build task came in at [REVEAL] about 40 seconds. So, yeah. Cucumber isn't the fastest thing out there. It's never gonna run thousands of tests per second. But I was... pretty happy with these numbers.

66. @geeksam 🎓 IN SUMMATION... 66 I know that was a lot of information to take in. Just as a reminder, my slides are available on the conference website, so feel free to supplement your notes with those. Before I let you go, I just want to reiterate what I think are the two most important ideas from this talk...

67. @geeksam67 Describing software  At the level of  USER INTENT  (and maybe automating  tests with Cucumber) 🎓 The ﬁrst of those is, again, what I think is really the sweet spot of Cucumber usage: that focus on communicating with other people about the actual value that your application is supposed to bring to your users.

68. @geeksam THE QUESTION How would you write scenarios if you didn't know what the UI was going to be? 68 🎓 And the second thing is this question, which you can use to evaluate your Cucumber scenarios and make sure they're focused at that right level of abstraction.

69. @geeksam README 💚 69 @geeksam Podcast: >Code GreaterThanCode.com Finally, If you enjoyed this talk and want to hear more from me, you can ﬁnd me on Twitter at @geeksam. I'm also on a podcast called Greater Than Code, which is largely about people in tech and how we can get better at the "people" part of that.

70. @geeksam FIN. 💚 70 @geeksam Podcast: >Code GreaterThanCode.com Thank you!

71. @geeksam Q&A 💚 71 @geeksam Podcast: >Code GreaterThanCode.com We've got a few minutes for Q&A. Before we launch into that, I know some of you are ready to get up and move around a bit, so if you need to leave, please do. For those of you who are sticking around, I ask that you please be respectful of everyone's time, and remember to phrase your question in the form of a question. If you have a longer topic you'd like to chat about, feel free to come talk to me after the session. Let's have some questions!

Cucumbers Have Layers: A Love Story (AATC 2017)

Cucumbers Have Layers: A Love Story (AATC 2017)

Recommended

More Related Content

Similar to Cucumbers Have Layers: A Love Story (AATC 2017)

Similar to Cucumbers Have Layers: A Love Story (AATC 2017)(20)

Recently uploaded

Recently uploaded(20)

Cucumbers Have Layers: A Love Story (AATC 2017)