Successfully reported this slideshow.

BTDConf: Breaking Illusions: Testing is Your Most Valuable Asset

0

Share

Loading in …3
×
1 of 11
1 of 11

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

Related Audiobooks

Free with a 14 day trial from Scribd

See all

BTDConf: Breaking Illusions: Testing is Your Most Valuable Asset

  1. 1. Breaking Illusions: Testing is your most valuable asset! Maaret Pyhäjärvi Email: <maaret@iki.fi> | Twitter: maaretp Maaret Pyhäjärvi Nimeä | Attribution (Finland) http://creativecommons.org/licenses/by/1.0/fi/ http://creativecommons.org/licenses/by/1.0/fi/deed.en
  2. 2. Testing is about Feedback Checking • Machines can do with algorithmic decision rules • Focuses on specific facts • Cmp. “compiling”, “generating” Exploring • Skilled humans can do • Open-ended search for information • Cmp. “programming” Testing not testers – but becoming a skilled tester requires time and focus
  3. 3. Context Matters – Expectations Differ • Testing in Production? – Rolling releases: limit the extent of impact – Product lock-in: limit the options • Testing in the delivery chain? – Development teams skilled in exploring (as well as checking) – Internal releases – Beta releases – Releases • Available skills? – Intentional vs. accidental production feedback
  4. 4. Testers don’t break your code, they break your illusions about the code. -- adapted from James Bach
  5. 5. Dispelling Illusions: Empirical Evidence over Speculation VALU E ILLUSIONS
  6. 6. The World Has Already Changed COMMODITY TESTERS • Manual checkers • Tests are an artifact SKILLED TESTERS • Explorers of products and businesses • Testing is a performance • DEVELOPERS IN TEST • All development is automating, testing is just a domain of development
  7. 7. Exploration Skills Source: Adapted from James Bach, Jon Bach, Michael Bolton. Exploratory Testing Dynamics. v.2.2. 2009 Self- managemen t Developing ideas Examining product Done To Do Issues Coverage All sources available Best use of time – effective and efficient work Making models Tool support – creative solutions Risk-based testing – scientific approach Keeping one’s eyes open
  8. 8. Things Can Look Different from Different Perspectives
  9. 9. Culture of Feedback and Experiments • Illusions are meant to be broken • Teams over individuals – Diverse teams (personalities, skills) deliver better results – Social dissonance gets the team talking and voices out uncertainties to address on time • Experiment: try something different
  10. 10. Summary 1. TESTING is FEEDBACK and LEARNING. EMPIRICAL supports learning over theoretical. FAST is better than slower. Not outsourcing the cost of feedback could be better. 2. CONTEXT matters: sometimes the “best testing” happens live – as long as you’re covering the checking part; testing != testers 3. BUGS not defects: anything this might bug a user, there’s many types of illusions we have 4. COMMODITY testers and SKILLED testers are two completely different breeds 5. SKILL to break DIVERSE illusions – skill in team, not in individual 6. DIVERSITY brings good results: people who choose to love testing tend to be different from those who choose to love checking and development

Editor's Notes

  • This presentation is about FEEDBACK. I don’t care if you have specialized testers to help you work out the multidimensional feedback or not. Feedback is still valuable, and for all contexts, asking for all the feedback from the end users / product management just isn’t the thing to do…

    I come from an organization where (some) developers strongly still believe that it would be better to hear about bugs from the end users so that we would only hear about “real bugs” and that regardless of me showing I do understand what they complain on, it would still be different (there would be less of it…)
  • Point of this slide: TESTING is FEEDBACK and LEARNING. EMPIRICAL supports learning over theoretical. FAST is better than slower. Not outsourcing the cost of feedback could be better.


    MENTION: Testing looks into more than functionality. Other quality criteria and stakeholder views are particular relevance.

    Emergent information, you do not know what you are looking for



    For more info, see http://www.satisfice.com/blog/archives/856

    Checking
    the process of making evaluations by applying algorithmic decision rules to specific observations of a product.
    Testing
    the process of evaluating a product by learning about it through exploration and experimentation, which includes to some degree: questioning, study, modeling, observation, inference, etc.
  • Point of this slide: CONTEXT matters: sometimes the “best testing” happens live – as long as you’re covering the checking part; testing != testers

    Example: Security vulnerabilities
    Vulnerabilities are considered real problems
    Still only the black market pays real money for reporting them
    Functional bugs / missing value reporting is actual effort too, and wastes time for people
    Something we as end users can do: reclamation

    I tend to work with PRODUCT BUSINESS and CUSTOMER ORGANIZATIONS, not delivering bespoke software.

    Inability to produce quality vs. productivity service – different stakeholder considerations, incl. “design with a change in mind’

    I belong to context-driven school of testing – I know things can be different!
    History of having things tested in production – wasting a lot of time.

    Not every team has the most skilled developers but they can still have diverse teams delivering excellent results
  • Point of this slide: Testing helps with EFFECTIVENESS, as a productivity service. Your users might not really like reporting your bugs for you, as it wastes their time.
    It’s co-creation. The cake example. You can’t separate the ideas, ideas feeding the ideas and implementation.
  • Illusion stories:
    “I tested it, it works”
    “reporting engine”
    “no time for refactoring” / “releasing as scheduled over done”
    “Need document grouping feature over others”
    ---

    Ray Arell (Intel) giving apple example: showing you’re living with an illlusion. Tester mentality!!
    ----

    Point of this slide: BUGS not defects: anything this might bug a user, there’s many types of illusions we have

    Finding really expensive bugs: missing value, wrong features

    Illusion: releasing as scheduled (sprint) vs. releasing when done (kanban)

    EMPIRICAL evidence trumps speculation. Every Single Time. – testobsessed
  • Point of this slide: COMMODITY testers and SKILLED testers are two completely different breeds
    The spider web idea.


    Your ideas of testers may be outdated. You get what you ask for, and if you ask for test cases, you get commodity approaches.
  • Point of this slide: SKILL to break DIVERSE illusions – skill in team, not in individual
  • Exploratory is a MINDSET / APPROACH, not an activity.
    Testing is PERFORMANCE, not ARTIFACT.
    Testing is testing even if context is agile.
  • Point of this slide: DIVERSITY brings good results: people who choose to love testing tend to be different from those who choose to love checking and development


    Not every team has the most skilled developers but they can still have diverse teams delivering excellent results

  • Creating culture of feedback and experiments with diversity of personalities and skills in the teams
  • ×