St Louis Day of .NET

Angela Dugan
ALM Practice Manager
Polaris Solutions
Of course this has NEVER happened to you... Right?
It is plan-driven, and plans are good right?
Pert charts, Gaant charts, Critical paths, OH MY!

Rules with an Iron Fist (A.K.A Microsoft Project)
Pre-defined Start Dates & End Dates
Teams operate in silos (Centers of Excellence)
It is not the devil, but it CAN be evil if its
prescribed techniques are abused
Individuals and interactions over processes and tools

Working software over comprehensive documentation
Customer collaboration over contract negotiation

Responding to change over following a plan
Embraces uncertainty, software IS
uncertain
Empirical (based on experience and
observation)
Continuous improvement
“Forecast” rather than “commitment”
Self-organization and estimation by
the “do-ers”
It is not the devil, but it CAN be evil
if its prescribed techniques are
abused
Daily standup INCLUDES people from multiple disciplines
Agile estimation leverages INSTINCT and EXPERIENCE to provide realistic
expectations and more confident forecasts

Backlog grooming focuses team’s efforts on customer’s current PRIORITIES
An iterative process fueled by customer FEEDBACK ensures the team delivers
the right functionality

A constant FOCUS ON QUALITY ensures that quality is built-in, not tested in
Retrospectives foster CONTINUOUS IMPROVEMENT by inspecting outcomes,
sharing of best practices and honing the process
Waterfall

Agile

Requirements documents

Just-in-time, informal requirements

Occasional “customer” involvement

Frequent “customer” involvement

Start-to-finish Project Plan

Plan for Sprint.
Details are sketchy beyond that.
Priorities shift based on new data.

Tasks are assigned

Assigned tasks are a bottleneck

Potentially large team size

Teams of 3 – 9 people

Multiple phases, eventual delivery

Working software each Sprint / Iteration

Resistant to change

Change is expected

Contract says what we build, deliver

Contract is a lot closer to T&E
Waterfall

Agile

Test cases created from

Specifications

Acceptance criteria

Test cases are created

Manually

Manual
Automate stubs from acceptance criteria

Test cases are created

Up front

Started up front, continually refined

Time commitment

Large

Still a lot, but a huge improvement

Text execution is

Well defined steps
Some automation
Near end of project

Some defined steps
Scenario-based/Exploratory
Automation
Executed early, often, continuously

Tests executed by

QA Team

Everyone

Weaknesses

Documentation overhead
Regression often squeezed
Sensitive to change

Coordination can be challenging
Requires skilled automation resources
More collaboration
Better overall visibility of status, progress, quality
Less bureaucracy to get in your way
Less impact from requirement churn
Testing is EVERYBODY’S concern, ALL the time!
Reduces resource bottlenecks
Less focus on output, more focus on quality
Everyone feels IS invested in the deliverable
More meetings (kind of)
Less (perceived) accountability
Less (unnecessary) documentation

More requirement churn
Shorter runway for writing tests
May require a new “toolbox”
Change is hard, and this could be a BIG one
FAR greater levels of discipline required by
EVERYONE on an agile team (yes, really)

Far more responsibility on Stakeholders and endusers
Management support can be difficult to achieve &
maintain, and it is CRITICAL for success!
Agile shines a light on existing dysfunction
Agile for day-to-day dev/test activities
Detect problems and continuously improve with Sprints
Focus on Definition Of Done & delivering working
software (a.k.a. value to customers)
Waterfall-ish style for multi-team coordination
Waterfall for environment release planning/management
Agile testing starts with requirements (ATDD)
Continues throughout development (TDD, Unit Test Automation, CI)
QA validates delivered functionality every day (Scenario/Exploratory Testing)

Culminated with customers (UAT)

Testing is not a practice, it is an attitude!
Teams may struggle to adapt to changes because significant effort has already
gone into the initial requirements development and testing a.k.a “Throwing good
money after bad.”
Engineers may not be used to being “responsible for quality”. QA should never
be testing code that has not already passed unit testing in the development
phase.
QA is still logically the last task in marking a user story done. Delays in
development tasks still impact QA timelines.
QA may not be used to inspecting requirements and asking questions up front.
Addition of new user stories into the current iteration impacts EVERYONE.
Include QA to ensure appropriate commitments and estimations are built in
Collaborate: daily stand-ups should include
testers

Adopt a process (if it’s all ad-hoc today)
Adopt an integrated ALM tool (if you don’t have
one)

Shorten delivery cycles
Question anything that “smells”
Continuously improve, even if it is just the little
things
Get your developers involved (TDD, unit testing)
Automate regression tests
Scenario based testing when appropriate
Generate test case documentation whenever possible (from exploratory
tests or acceptance criteria)
Involve stakeholders in testing (UAT)

Adopt a good toolset to assist with collaboration and automation
Gartner’s “Magic Quadrant” 2012

Ovum Decision Matrix for ALM 2013
Read what Forrester and Gartner have to say, then sh*t-can the reports and make your
own decision
Focus on tools that foster collaboration
Many tools can fit the bill, use what feels good
Best fit is not always “Best of Breed”
Tools can foster efficiency and collaboration
Tools cannot fix your people or process issues, they just automate them :-

Expensive tools and fancy practices are useless if they aren't supportive of the
approach you are willing to adopt.
No more Magna Carta Requirements documents
Tests automated in VS and run via CI builds in TFS
Tests created and managed in MTM
Lab Management – ‘nuf said!
Rich bugs, OMG
Web tools
Drive: The Surprising Truth About
What Motivates Us
Daniel Pink
Under $10 on Amazon
http://www.amazon.com/Drive-Surprising-Truth-AboutMotivates/dp/1594484805/
Succeeding with Agile
Mike Cohn

$35 on Amazon
http://www.amazon.com/Succeeding-Agile-SoftwareDevelopment-Using/dp/0321579364
Agile Testing
Lisa Crispin
Janet Gregory

$40 on Amazon
http://www.amazon.com/Agile-Testing-PracticalGuide-Testers/dp/0321534468
Visual Studio Team Foundation
Server 2012: Adopting Agile
Software Practices: From Backlog
to Continuous Feedback
Sam Guckenheimer
Neno Loje
$30 on Amazon
http://www.amazon.com/Visual-Studio-Team-FoundationServer/dp/0321864875
Agile Software Testing in a Large Scale Project:
http://www.slideshare.net/Softwarecentral/agile-software-testing-in-a-largescaleproject
Great Testing Blog: http://blogs.msdn.com/b/anutthara/
Another Great Testing Blog: http://www.clemensreijnen.nl/search.aspx?q=testing

Forrester ALM Blogs: http://blogs.forrester.com/category/alm
Free ALM Images with HOL: http://blogs.msdn.com/b/briankel/archive/2013/04/17/listof-all-visual-studio-alm-virtual-machines.aspx
ALM Summit Video: Testing and Agile: The Team Approach http://channel9.msdn.com/Events/ALM-Summit/ALM-Summit-3/Testing-and-AgileThe-Team-Approach
ALM Summit Video: Agile Testing: http://channel9.msdn.com/Events/ALMSummit/ALM-Summit-3/Agile-Testing
ALM Summit Video: Exploratory Testing: http://channel9.msdn.com/Events/ALMSummit/2011/Exploratory-Testing
http://www.eventboardmobile.com/download
Platinum Sponsors

Gold Sponsors

Silver Sponsors
STLDODN - Agile Testing in a Waterfall World

STLDODN - Agile Testing in a Waterfall World

  • 1.
    St Louis Dayof .NET Angela Dugan ALM Practice Manager Polaris Solutions
  • 2.
    Of course thishas NEVER happened to you... Right?
  • 3.
    It is plan-driven,and plans are good right? Pert charts, Gaant charts, Critical paths, OH MY! Rules with an Iron Fist (A.K.A Microsoft Project) Pre-defined Start Dates & End Dates Teams operate in silos (Centers of Excellence) It is not the devil, but it CAN be evil if its prescribed techniques are abused
  • 4.
    Individuals and interactionsover processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan
  • 5.
    Embraces uncertainty, softwareIS uncertain Empirical (based on experience and observation) Continuous improvement “Forecast” rather than “commitment” Self-organization and estimation by the “do-ers” It is not the devil, but it CAN be evil if its prescribed techniques are abused
  • 6.
    Daily standup INCLUDESpeople from multiple disciplines Agile estimation leverages INSTINCT and EXPERIENCE to provide realistic expectations and more confident forecasts Backlog grooming focuses team’s efforts on customer’s current PRIORITIES An iterative process fueled by customer FEEDBACK ensures the team delivers the right functionality A constant FOCUS ON QUALITY ensures that quality is built-in, not tested in Retrospectives foster CONTINUOUS IMPROVEMENT by inspecting outcomes, sharing of best practices and honing the process
  • 7.
    Waterfall Agile Requirements documents Just-in-time, informalrequirements Occasional “customer” involvement Frequent “customer” involvement Start-to-finish Project Plan Plan for Sprint. Details are sketchy beyond that. Priorities shift based on new data. Tasks are assigned Assigned tasks are a bottleneck Potentially large team size Teams of 3 – 9 people Multiple phases, eventual delivery Working software each Sprint / Iteration Resistant to change Change is expected Contract says what we build, deliver Contract is a lot closer to T&E
  • 8.
    Waterfall Agile Test cases createdfrom Specifications Acceptance criteria Test cases are created Manually Manual Automate stubs from acceptance criteria Test cases are created Up front Started up front, continually refined Time commitment Large Still a lot, but a huge improvement Text execution is Well defined steps Some automation Near end of project Some defined steps Scenario-based/Exploratory Automation Executed early, often, continuously Tests executed by QA Team Everyone Weaknesses Documentation overhead Regression often squeezed Sensitive to change Coordination can be challenging Requires skilled automation resources
  • 9.
    More collaboration Better overallvisibility of status, progress, quality Less bureaucracy to get in your way Less impact from requirement churn Testing is EVERYBODY’S concern, ALL the time! Reduces resource bottlenecks Less focus on output, more focus on quality Everyone feels IS invested in the deliverable
  • 10.
    More meetings (kindof) Less (perceived) accountability Less (unnecessary) documentation More requirement churn Shorter runway for writing tests May require a new “toolbox”
  • 11.
    Change is hard,and this could be a BIG one FAR greater levels of discipline required by EVERYONE on an agile team (yes, really) Far more responsibility on Stakeholders and endusers Management support can be difficult to achieve & maintain, and it is CRITICAL for success! Agile shines a light on existing dysfunction
  • 12.
    Agile for day-to-daydev/test activities Detect problems and continuously improve with Sprints Focus on Definition Of Done & delivering working software (a.k.a. value to customers) Waterfall-ish style for multi-team coordination Waterfall for environment release planning/management
  • 13.
    Agile testing startswith requirements (ATDD) Continues throughout development (TDD, Unit Test Automation, CI) QA validates delivered functionality every day (Scenario/Exploratory Testing) Culminated with customers (UAT) Testing is not a practice, it is an attitude!
  • 14.
    Teams may struggleto adapt to changes because significant effort has already gone into the initial requirements development and testing a.k.a “Throwing good money after bad.” Engineers may not be used to being “responsible for quality”. QA should never be testing code that has not already passed unit testing in the development phase. QA is still logically the last task in marking a user story done. Delays in development tasks still impact QA timelines. QA may not be used to inspecting requirements and asking questions up front. Addition of new user stories into the current iteration impacts EVERYONE. Include QA to ensure appropriate commitments and estimations are built in
  • 15.
    Collaborate: daily stand-upsshould include testers Adopt a process (if it’s all ad-hoc today) Adopt an integrated ALM tool (if you don’t have one) Shorten delivery cycles Question anything that “smells” Continuously improve, even if it is just the little things
  • 16.
    Get your developersinvolved (TDD, unit testing) Automate regression tests Scenario based testing when appropriate Generate test case documentation whenever possible (from exploratory tests or acceptance criteria) Involve stakeholders in testing (UAT) Adopt a good toolset to assist with collaboration and automation
  • 17.
    Gartner’s “Magic Quadrant”2012 Ovum Decision Matrix for ALM 2013
  • 18.
    Read what Forresterand Gartner have to say, then sh*t-can the reports and make your own decision Focus on tools that foster collaboration Many tools can fit the bill, use what feels good Best fit is not always “Best of Breed” Tools can foster efficiency and collaboration Tools cannot fix your people or process issues, they just automate them :- Expensive tools and fancy practices are useless if they aren't supportive of the approach you are willing to adopt.
  • 19.
    No more MagnaCarta Requirements documents Tests automated in VS and run via CI builds in TFS Tests created and managed in MTM Lab Management – ‘nuf said! Rich bugs, OMG Web tools
  • 20.
    Drive: The SurprisingTruth About What Motivates Us Daniel Pink Under $10 on Amazon http://www.amazon.com/Drive-Surprising-Truth-AboutMotivates/dp/1594484805/
  • 21.
    Succeeding with Agile MikeCohn $35 on Amazon http://www.amazon.com/Succeeding-Agile-SoftwareDevelopment-Using/dp/0321579364
  • 22.
    Agile Testing Lisa Crispin JanetGregory $40 on Amazon http://www.amazon.com/Agile-Testing-PracticalGuide-Testers/dp/0321534468
  • 23.
    Visual Studio TeamFoundation Server 2012: Adopting Agile Software Practices: From Backlog to Continuous Feedback Sam Guckenheimer Neno Loje $30 on Amazon http://www.amazon.com/Visual-Studio-Team-FoundationServer/dp/0321864875
  • 24.
    Agile Software Testingin a Large Scale Project: http://www.slideshare.net/Softwarecentral/agile-software-testing-in-a-largescaleproject Great Testing Blog: http://blogs.msdn.com/b/anutthara/ Another Great Testing Blog: http://www.clemensreijnen.nl/search.aspx?q=testing Forrester ALM Blogs: http://blogs.forrester.com/category/alm
  • 25.
    Free ALM Imageswith HOL: http://blogs.msdn.com/b/briankel/archive/2013/04/17/listof-all-visual-studio-alm-virtual-machines.aspx ALM Summit Video: Testing and Agile: The Team Approach http://channel9.msdn.com/Events/ALM-Summit/ALM-Summit-3/Testing-and-AgileThe-Team-Approach ALM Summit Video: Agile Testing: http://channel9.msdn.com/Events/ALMSummit/ALM-Summit-3/Agile-Testing ALM Summit Video: Exploratory Testing: http://channel9.msdn.com/Events/ALMSummit/2011/Exploratory-Testing
  • 26.
  • 27.