BDD & Beyond: The Past, Present, & Future of Test Automation
•Download as PPTX, PDF•
2 likes•107 views
A tour of the history of agile, test automation, and BDD frameworks, plus a prediction for the future.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to BDD & Beyond: The Past, Present, & Future of Test Automation
Similar to BDD & Beyond: The Past, Present, & Future of Test Automation (20)
More from Ford Prior
More from Ford Prior (12)
Recently uploaded
Recently uploaded (20)
BDD & Beyond: The Past, Present, & Future of Test Automation
- 1. BDD & Beyond Past, Present Future
- 2. ABOUT ME CONTACT • Multidisciplinary background • QA practitioner for 5 years • Corporate, startup, mid-size • Agile & waterfall • Stoked about: • Agile • CI/CD • AWS • Open-source • Wife/kid/jiu-jitsu/eating Email (fordprior@gmail.com) GitHub (fordprior) LinkedIn (Ford Prior)
- 3. TOC Agenda. Agile. Unit testing. 3 Selenium. WebDriver. BDD & friends. Frameworks. Getting started. The Future.
- 4. NASA’s Mercury Project, 1961
- 6. Monolithic architecture = monologue One guy, totally opaque, barely making sense
- 7. Client-server architecture = two people talking Two people with obvious intentions, personalities, concerns
- 8. procedural 8 linear • functions • imperative
- 10. AND THAT’S WHEN THE AMERICANS HEARD ABOUT IT, CALLED IT “SCRUM”, AND STARTED SELLING IT.
- 11. Rugby analogy: Small teams + self-organization + objectives (instead of tasks) = Success!
- 12. ✓ Prof. Ogunnaike Tunde ✓ 1995 ✓ “Agile is statistically better than waterfall for making software.”
- 13. Extreme! ✓ 1997 ✓ Kent Beck ✓ Chrysler ✓ Small teams, changing requirements ✓ Code reviews & testing?!?!
- 15. xUnit, et. al. ✓ Smalltalk ✓ Self-contained ✓ SUnit, JUnit, etc...
- 16. So 2004… ✓ iMac UI ✓ MySpace ✓ Ask Jeeves ✓ Trashy interstitials ✓ 1010XOXO_v26?
- 17. Hugs! ✓ Jason Huggins ✓ ThoughtWorks ✓ CI/CD testing ✓ Boo Mercury! ✓ Yay Selenium!
- 18. Yay!: ✓ Supports most browsers ✓ Supports most languages ✓ Popular w/ big community ✓ IDE allows bootstrapping test creation ✓ Selenium Grid = massive parallelization
- 19. Not so good: ○ SOP compliance ○ Limitations of JavaScript ○ Infrequent releases ○ Doesn’t like Safari
- 20. So 2008… ✓ Internet presidents ✓ Netbooks ✓ Twitter is cool
- 21. Plucky! ✓ 2008 ✓ ThoughtWorks ✓ WebDriver Project ✓ Simon Stewart ✓ Fan of Selenium
- 22. Yay!: ✓ Small, comprehensible API ✓ Durable to change b/c uses native OS ✓ Speed & performance (w/ trade-offs) ✓ Tiny (no server process) ✓ Accommodates Selenium API
- 23. Not so good: ○ Writing native browser code ○ Doesn’t like IE ○ Only supports Java?! ○ <?>
- 25. 25 They join forces: ✓ 2010 ✓ Google ✓ JSON
- 26. Why JSON?: ✓ W3C standard ✓ Every lingo has JSON parsing lib ✓ Every lingo has HTTP library
- 28. Ugh! Lots of code, until... ✓ 2011 ✓ SeleniumConf ✓ OperaDriver ✓ ChromeDriver
- 29. Selenium 3.x ✓ 2016 ✓ Default = browser’s maintained driver ✓ More API-based ✓ W3C spec (almost)
- 30. Today! ✓ Still open-source ✓ RIP Selenium IDE ✓ W3C spec (still almost)
- 31. Still bad ● Documentation ● Specific language support ● IDE Driver (EdgeDriver is focus of Microsoft)
- 32. Selenium 4.x & Beyond ■ ALL browser drivers spec compliant ■ Vendor adoption ■ Only bindings & Js Common Code
- 33. PhantomJs! ✓ 2011 ✓ Ariya Hidayat ✓ Browser w/out UI ✓ Webscraping
- 34. Chromium! ✓ WebKit fork (Blink & v8) ✓ Google, but open-source ✓ 2017 (v59)
- 36. Tech is neat-o, but what about a framework?
- 37. TDD, BDD, ATDD HUGE POTENTIAL, JUST TREMENDOUS.
- 38. TDD, BDD, ATDD New software development methodologies They do just 2 things: Problem Solution How Changing, complex requirements 1. Makes it easy to talk about specs & stories universal language and format High velocity 2. Helps developer “flow” w/ a fast, elegant, automated feedback loop
- 39. RED Outline an aspect of the product that’s failing the customer in some way. GREEN Come up with a solution so that the customer wins instead of loses. REFACTOR Streamline and slim down experience while making sure customer is still winning. BDD (red, green, refactor) For business people:
- 40. RED Write some unit test code that fails because the feature it’s testing is non- existent. GREEN Write some real code until that test code is passing instead of failing. REFACTOR Refactor the real code as much as possible while unit tests still pass. TDD (red, green, refactor) For programmers:
- 41. RED Write some acceptance test code that fails because the feature it’s testing is non- existent. GREEN Give it to the programmer and let him write real code until your tests pass. REFACTOR Add test to “regression” list. Programmer refactors/adds new code without regressing. ATDD (red, green, refactor) For testers:
- 42. RED Outline an aspect of the product that’s failing the customer in some way. GREEN Come up with a solution so that the customer wins instead of loses. REFACTOR Streamline and slim down experience while making sure customer is still winning. BDD (red, green, refactor) Put ‘em together: RED Write some test code that fails because the feature it’s testing is non-existent. GREEN Write some real code until that test code is passing instead of failing. REFACTOR Refactor the real code as much as possible while still passing tests. TDD (red, green, refactor) REFACTOR Add test to “regression” list. Programmer refactors/adds new code without regressing. RED Write some acceptance test code that fails because the feature it’s testing is non-existent. GREEN Give it to the programmer and let him write real code until your tests pass. ATDD (red, green, refactor)
- 43. Gherkin ✓ 2005 ✓ Chris Matts ✓ ThoughtWorks “Let’s start using a common way of talking about requirements between Devs, PO’s, and QA’s.”
- 44. JBehave ✓ 2006 ✓ Dan North ✓ ThoughtWorks ✓ Agiledox (JUnit utility) ✓ TDD “...And this format is specific enough for us to write into test code.”
- 45. RSpec ✓ 2006 ✓ Aslak Hellesøy ✓ ThoughtWorks ✓ BDD “...And it can also be a way to communicate specifications before they’re even coded.”
- 47. RSpec Story Runner ✓ 2007 ✓ David Chelimsky ✓ ThoughtWorks ✓ BDD “In fact, let’s abstract it all so that it runs off of plain text in a Given/When/Then format!” StoryRunner
- 48. Cucumber ✓ 2008 ✓ Aslak Hellesøy ✓ ThoughtWorks ✓ Forked from RSpec ✓ ATDD “...and now it’s even better, plus in a bunch of different languages!”
- 55. What about the future?
- 56. More commoditization. ✓ More SaaS tooling ✓ Data-driven testing (canaries) ✓ Less of a “discipline”
- 57. AI & ML are here. ✓ Trained on 1000’s of apps ✓ Less CSS targeting ✓ Based on natural language
- 58. Regression Classification Anomaly detection Pattern recognition The high tomorrow will probably be 84.6 F. Tomorrow will be muggy. Tomorrow will be abnormally hot. Wear a raincoat tomorrow, no need for rain boots, and maybe an umbrella. This type of bug will probably take 3 weeks to fix and impact 1,000 users per day. This page is a form, and these are the relevant fields and that’s the Submit button. This shouldn’t exist. That shouldn’t have happened. That is a user profile, and here’s how you update it.
- 59. Here to stay (for now): ✓ “QA brain” ✓ Domain knowledge ✓ HTTP/Web (Selenium WebDriver)
Editor's Notes
- API testing 101 About Me About Snagajob HTTP Is how the web communicates. Super quick over the wire, 234 ms. API's An API isn’t the same as the remote server — rather it is the part of the server that receives requests and sends responses. The code that takes the HTTP requests and turns it into business vaue. Stores in database so another app can access it, Applies some logic to transform it into a number someone really cares about Feeds it into some proprietary algo to drive insight All of the above Public & Private (amazon button story) Microservices are new hottness HTTP basics this is a protocol, just like a language. 1) headers (includes auth, tells service what type of content it's delivering, and what to accept in terms of a response) 2) querystrings (optional) which are on the actual route and specify what to ask the application for (e.g. `http://gateway.api.snagQA.corp/customers?isActive=true` is asking for customers where isActive=true) 3) body (also optional) which is giving some detailed data to the service so, those 3 things the BODY is usually part of anything related to creation, but it can also be passed to an existing entity to overwrite some existing data. so new customers will have large bodies
- Project Mercury (NASA) software development, ran with half-day iterations. “All of us, as far as I can remember, thought waterfalling of a huge project was rather stupid, or at least ignorant of the realities.” — Weinberg G. M. (Project Mercury)
- Monolithic architecture = monologue One guy, totally opaque, barely making sense
- Client-server architecture = two people talking Two people with obvious intentions, personalities, concerns
- Software development in the 1990s was shaped by two major influences: internally, object-oriented programming replaced procedural programming as the programming paradigm favored by some in the industry; externally, the rise of the Internet and the dot-com boom emphasized speed-to-market and company growth as competitive business factors. Rapidly changing requirements demanded shorter product life-cycles, and were often incompatible with traditional methods of software development.
- 1986
- While in the process of developing and using early versions of Scrum, Ken asked Professor Babatunde A. Ogunnaike Tunde, a famous process control research engineer, to look at software development processes. Tunde investigated several commercial software-development methodologies to conclude that the waterfall and predictive process is not a good fit for the work of software development. He confirmed the empirical approach of Scrum to be the preferred process.
- “The first time I was asked to lead a team, I asked them to do a little bit of the things I thought were sensible, like testing and reviews. The second time there was a lot more on the line. I thought, "Damn the torpedoes, at least this will make a good article," [and] asked the team to crank up all the knobs to 10 on the things I thought were essential and leave out everything else.”
- http://slideplayer.com/slide/6356916/ https://en.wikipedia.org/wiki/Unit_testing https://stackoverflow.com/questions/652292/what-is-unit-testing-and-how-do-you-do-it
- Why Cambrian? ~Oxygenation ~Two continents colliding ~Large extinction allowed open field ~All three? Despite their proliferation, however, many Cambrian creatures eventually went extinct, including the exotic Opabinia, which had five eyes and a nose like a fire hose, and Wiwaxia, an armored slug with two rows of upright scales.
- (Selenium Core replaced by WebDriver API’s in RC) Wc3 stuff: -2 independent implementations (Gecko + IE) -Test Suite
- Get Appium up to speed (it’s lingering behind) Session Protocol Payload (currently, you can send pretty much anything into Desired_Capabilities, but with W3C it’s regimented Commercial tool vendors adopting open-source WebDriver EdgeDriver is focus of Microsoft; but IE Driver will stay open-source
- Only spec-dialect version of protocol (no open-source drivers, browsers must comply) Tighter Desired_Capabilities
- https://intoli.com/blog/running-selenium-with-headless-chrome/ ...So Google goes its own way with HEADLESS BROWSERS, too. 2017 With versions 59 and onwards, it will now be possible to harness the power of V8, Blink, and the rest of Chrome in a non-graphical server environment. It has long been rumored that Google uses a headless variant of Chrome for their web crawls. Over the last two years or so it had started looking more and more like this functionality would eventually make it into the public releases and, as of this week, that has finally happened. With versions 59 and onwards, it will now be possible to harness the power of V8, Blink, and the rest of Chrome in a non-graphical server environment. This may not sound earth-shattering if you don’t deal with headless browsers very often but it’s actually a pretty big deal. To put this into context: PhantomJS, one of the current leaders in the space, has over 21k stars on GitHub and is used by companies such as Netflix and Twitter for both unit and performance testing. Vitaly Slobodin, the former maintainer of PhantomJS, had this to say after hearing the news. I think people will switch to it, eventually. Chrome is faster and more stable than PhantomJS. And it doesn’t eat memory like crazy. I don’t see any future in developing PhantomJS. Developing PhantomJS 2 and 2.5 as a single developer is a bloody hell. Even with recently released 2.5 Beta version with new and shiny QtWebKit, I can’t physically support all 3 platforms at once (I even bought the Mac for that!). We have no support. From now, I am stepping down as maintainer. If someone wants to continue - feel free to reach me. – Vitaly Slobodin
- We just do “D”, sometimes it’s “DD”.
- https://www.quora.com/Why-is-the-Cucumber-tool-for-BDD-named-as-such
- Given When Then
- IIRC Cucumber was a re-write of the RSpec Story Runner, which was a re-write of JBehave. AFAIK GWT came from Chris Matts, who was a business analyst working at Throughtworks with Dan North and @lunivore
- IIRC Cucumber was a re-write of the RSpec Story Runner, which was a re-write of JBehave. AFAIK GWT came from Chris Matts, who was a business analyst working at Throughtworks with Dan North and @lunivore