Agile testing for mere mortalsPresentation Transcript
ALN DC Chapter Meeting March 2012Agile Testing for Mere Mortals Presented by Dave Haeffner
ALL BOOKS OR SERVICES MENTIONED ORRECOMMENDED ARE COOL AND WORTH SHARING. DAVE IS NOT AFFILIATED WITH THEM IN ANY WAY SHAPE OR FORM. IF ANYONE HAS A MEDICAL CONDITION, OR ISALLERGIC TO FIRE, DONT SIT IN THE FRONT ROW. AND PLEASE, FEEL FREE TO ASK QUESTIONS THROUGHOUT.
Show of hands -- How many of you are humanbeings?Keep them up. Ah, good we have our control.Keep your hand up if youve had experience withAgile Testing, QA… or Automated Web Testing.OK. Keep your hand up if the thought of it leaves abad taste in your mouth.Look around. Youre not alone.Thanks for coming out.
A little bit about me…While working at The Motley Fool, I accidentally ended up inthis profession.I used to be in IT Operations -- a Systems Administrator.Through a twist of fate, I ended up trying out QA for a changeof pace. It was supposed to be temporary…That was over 3 years ago…When I started out, I had a lot of questions. A big one was --"Ummmm, what is Agile Testing?”Id like to share with you some of what Ive learned sincethen. It should only take about 5 minutes.And Ill deliver it in 3 parts so its more digestable -- So Ill giveyou the text book version, the "street" version, andAutomation.
I tend to leave Automation last because its a good excuse to read you a quote. Thisones from Randy Pauschs "The Last Lecture":"No, I did not make it to the National Football League, but I probably got more fromthat dream and not accomplishing it than I got from any of the ones that I didaccomplish. I had a coach, I signed up when I was nine years old. I was the smallestkid in the league, by far. And I had a coach, Jim Graham, who was six-foot-four, hehad played linebacker at Penn State. He was just this hulk of a guy and he was oldschool. And I mean really old school. Like he thought the forward pass was a trickplay. So, we showed up for practice the first day, and you know, there’s big hulkingguy, and we were all scared to death of him. And he hadn’t brought any footballs.One of the other kids said, excuse me coach, but there’s no football. And CoachGraham said, right, how many men are on a football field at a time? Eleven on ateam, twenty-two total. And Coach Graham said, all right, and how many peopleare touching the football at any given time? One of them. And he said, right, sowe’re going to work on what those other twenty-one guys are doing. And that’s areally good story because it’s all about fundamentals. Fundamentals, fundamentals,fundamentals. You’ve got to get the fundamentals down because otherwise thefancy stuff isn’t going to work.”Lets dig in.
Ah, Agile Testing -- The White Whale of the AgileWorld. Everyone knows it exists, but it can be hellon earth to catch it. If youre successful, thebenefits are *massive*. If you fail, then yourabilities as a team can be severely limited, even ifyou dont know it.So lets unpack the idea of "Agile Testing" and focuson what it is, its key elements, and the key playersinvolved in its delivery. For this, Im going to consult"The Book"
Agile Testing -- what is it?Heres a good definition -- "doing the simplest testspossible to verify that a piece of functionality existsor that the customers quality standard (e.g.performance) has been met”. Sounds simpleenough, but theres obviously more to it, or elseyou wouldnt all be here.I look at that definition as the outcome you want.But you cant get there by dictating it from the top-down. It requires more... finesse.
Crispin & Gregory advocate that you should put it tothe team, letting them work together to come upwith a "software quality strategy". They go on toabstract this concept further, saying you need tokeep stoking that initial fire, and evolve it into a"Quality Philosophy" -- making software quality apart of the teams DNA -- or else your efforts willnot be as effective.
While a main tenant of Agile Testing is a whole teamfocus, Crispin & Gregory really emphasize theimportance of the "Tester" or "QA” role (they use bothnames interchangeably). Their primary responsibilitiesare varied, and include:* Seeing the big picture, looking at the applicationmore from a user or customer point of view* Being an integral member of the customer team --helping elicit requirements and examples, and helpingthe customers express their requirements as tests* Being embedded on the developer team* Should look for unique ways to facilitatecommunication
One tool for facilitating communication is ThePower of Three, or (since I was just in Costa Rica)The Three Amigos. NEXT SLIDE
It means that all discussions about a featureneed to have a Developer, a QA, and the ProductOwner present. And it is the responsibility ofeach person to make sure that each group isproperly represented.
Value & RiskTeam Bridge
So if you follow their model and have successfully ignited apassion for quality -- then you should have an autonomouswhole team approach to quality and someone on the teamworking to help bridge the gap between tech & biz. If youtake this and keep it simple, focusing on value for thebusiness, then things start to get interesting.(Heres a quote -- "while we have skills to identify test casesbeyond the happy path we still need to start by making surethe happy path works.")Seems simple enough. Lets all do that.While it sounds simple, its not easy. Because lets be honest,Agile Testing is a lot like the classic 1981 video game "TheOregon Trail”
Think about it…Its a long journey (takes time)You think you know where youre heading -- greener pastures andso on (everyone talks about it – a UTOPIA)You try to make it while keeping your family together (wholeteam)Only to be attacked by natives along the way (funding, politics,other priorities, technology woes)And youll likely end up dying of dissentary (give up and stick withthe status quo)Let’s draw some parallels (above in parens)But all joking aside, there are additional challenges that preventAgile Testing from working. Here are some more subtle ones
When dealing with building out QA, you’ll find youneed both a technical and analytical set ofresources. Such skills exist in a single person, butit’s hard to find. And odds are, they won’t stay inthe QA role for long. Unless they’re 6’2” withbrown hair and has a coffee obsessionsAn approach to work around this is, pick one, andleverage other resources to fill in the gaps. (e.g.what The Fool did) – makes staffing *a lot* easier
Need more QA’s and having trouble sourcingexternal candidates?Higher from w/in – Customer Service is anexcellent placeAnd why not? They know your product insideand out and interact with your customers on aregular basis
Throwing it over the wall *shudder*The problem is that often times the feedback thatQA’s give to developers is out of band with theirwork flowAnd there’s often not enough QA’s to go around,they are spread too thin, and not truly embeddedon the teamThe short game - use a CI serverThe long game – use a CI server and get the teamusing ATDD and well written step definitions (moreon this later)
“The one thing worse than screwing up, isscrewing up and thinking you’re doing well.” – J.B. Rainsberger
In antithesis of this, Devs treat QA Regression asa safety net w/o knowing what is actually covered,QAs do the same for unit testsI say we get rid of the words "regression" and"smoke". Ive found that they tend to meandifferent things to different people
Effectiveness = Quality * Acceptance
But also, you need to tie behavioral and processchange back to valueThat way you can motivate people in the rightdirection w/o forcing it – autonomy will reignsupremeStart with why, then sort out the how and whatThis will help your effectiveness formula – haveyou heard of this formula?(Effectiveness = Quality * Acceptance)
QA as a gatekeeper & defects still getting out?Try a Proper process
Pre Often times, things get released Stories reviewed , if issues found, that bypass the previous steps kicked back to Devs Story Discussion, Sprint Goals, This reinforces a passive acceptance Happens more often that you think of the inefficiencies of our process Sizing, Tasking, and a dash ofDefinition of Done (depending on Automated tests written here It’s time for a change the team) IDEATION Planning WIP QA Review Done Released EXISTENCE Work’s done! Time to throw it over Business Owner review, if they find the wall to be “QA’d” an issue – it gets kicked back to Devs Sometimes happens without QA knowing – causing duplicative effort and additional context switching for Devs
Post Create Acceptance Tests Can be done at the same time Becomes the Specification Batches feedback for DevMagically becomes an automated web test Expl. Testing done by QA (or someone that didn’t work on the story)Require rep’s. from each group (Biz Owner, Review done by Stakeholder Dev, QA, UXD) Shovel Exploratory IDEATION Planning WIP Review Done Released EXISTENCE Ready Testing This is *the* most important PART Automated Acceptance Tests need to pass Everyone can rest easy, because this in order to proceed software is high quality baby!
My train of thought spawned from atremendously good read by Gojko Adzic wherehe introduced me to the concept of ATDDIt solves so many problems and just makes somuch sense.Different dialects from team to team & btwn biz& tech? create a ubiquitous languageIf you get this right, then you win the game, youhave completed the oregon trail and did not dieof dysentary
REDTest Code GREEN REFACTOR (repeat) Commit Given <precondition(s)> When <action> Then <result>
Automation frees you to do more exploratory testing! Acceptance Test Driven Development (ATDD) or BDD Test Driven Development (TDD)http://watirmelon.com/2012/01/31/introducing-the-software-testing-ice-cream-cone/
Not ready to automate? That’s fine.If you start to capture new features in the gherkinsyntax, you will be heading in the right direction.It will act as a communication and collaborationtool between your team and the business.And it will enable you to have a shared, ubiquitouslanguage that business and tech can both agree onand understandIf you add automation later, then you get theretroactive benefit of already captured features.
Traditionally w/ automated web testing using afree tool like Selenium there were two pathsIDE -> big sweet -> slow -> brittle -> falsepositivesIDE -> RC -> abstraction, logic, smarter ->more technical (read “custom testing harness”)And the big complaint among Devs is that it’stoo slow – they call it full stack testing since itloads the browser to execute
The funny thing about SeleniumJason Huggins built Se to solve a problemmuch like Nike, then founded a company tosolve the problem he created.
By Aslak Hellesøy
Right around the same time, Cucumber cameabout. It’s a ruby implementation of BDD. It enablesyou to automate gherkin feature setsWhere Gojko evolved the approach of Crispin &GregoryAslak provided automation for Gojko’s approach tocapturing featuresAnd you can drive Selenium through CucumberAnd push it to the cloud, or run things parallellocally
By taking the ATDD & Cucumber (or other BDDimplement) route, you short-circuit the entireheadache chain that has plagued Agile Testers thathave gone before youGranted, there are new challenges, given that thestep definitions need to be crafted to fit the contextof your environment, and that is code. But thebenefits, IMO, greatly outweigh the cost
I often get the question – but what if we need tomove fast? Or have a legacy code base? Wedon’t have time for this, what do yourecommend?
Focus on the 3 buckets1) SETI Are the foundational pillars of your system alive and responding?2) Money Makers What in your app is responsible for directlygenerating value to the business? Focus on that.3) Back breakers Watch out for that hole in the Death Star
Keep it simpleThis advice is more for people who aregoing to be writing the glue codeFocus on IDs -- XPath is a last resortHard to test apps are a symptom of poordesign
Take steps towards speed (and long termsanity)Make each test scenario fast – 1-2 minKeep them autonomous (e.g. unique accounts,set up built in, no cross dependencies, etc.)Single assertions per scenariosOne interpretation of test results
The prior will make parallelization possibleOptions abound for both local and cloudofferings
But all that feedback is pretty useless unlessyou do something with itBuild a feedback loopMake sure that developers are gettingfeedback in line with their work flow, elsehear the dreaded words "Works on mymachine"
A checklist for youMake Quality The Whole Team’s responsibilityTie things back to WhyBuild a ubiquitous languagePromote from withinKeep it simpleFocus on the 3 buckets (SETI, Money Makers, Back Breakers)Parallelize for speed