SlideShare a Scribd company logo
small 
controlled 
experiments 
@mathiasverraes
small 
uncontrolled 
experiments 
@mathiasverraes
Mathias Verraes 
Independent Consultant 
Value Object Comm.V 
Student of Systems 
Meddler of Models 
Labourer of Legacy 
verraes.net
Fitness landscape
Continuous 
Improvement
"When changing teams or organizations, the 
trick is not to try and push them out of their 
current behavior. (...) A better idea is to 
change parameters in the environment so 
that their current situation becomes 
unstable and disappears all by itself." 
4 Jurgen Appelo6 
6 Management 3.0: Leading Agile Developers, Developing Agile Leaders
Heavyweight
Retrospectives are too slow
Unproductive pressure to 
improve
Daily 
Two minutes, after standup
Brainstorm rules 
"Yes, and... " 
Divergence 
Convergence7 
7 "Thinking in New Boxes", Alan Iny & Luc de Brabandere
Avoid upfront consensus 
"A meeting is where ideas go to die" 
Experiments over opinions
"If we have data, let's look at 
data. If all we have are 
opinions, let's go with mine." 
4 Jim Barksdale
Avoid upfront expectations 
Expectations determine outcomes
Low impact 
Small, cheap, reversible, low-risk
No backlog 
Backlogs kill motivation
Timeline 
Stickies
Guarantee veto 
Everybody must be heard
Measure 
selectively & intentionally 
Avoid optimising for the metrics
Accept uncertainty 
Non-scientific 
Exposes invisible problems
Accept gut feeling 
Emotional response is fine
Accept failed experiments 
Welcome failures as new data points
Kaizen Mind 
The urgency to improve
Climate of Doubt 
Assume everything is broken and fixable
"If an idea is obviously bad, 
find a quick way to test it, 
because if it's not bad, 
then it's really interesting." 
4 Kent Beck
Experiment 
Deliver 
one story a day
Experiment 
Atomically scoped 
stories
Experiment 
Start every story 
in pair
Experiment 
Testers deploy 
independently
Experiment 
Core Protocols
Experiment 
Syncing physical 
boards
If it's not 
on a wall or a board, 
it's not visual.
Experiment 
Measure by hand
Experiment 
Hide the estimate 
from the board
Experiment 
No more 
sprint deadlines
Experiment 
No interrupts after 
lunch
Experiment 
Vizualize cost of 
interrupts
Experiment 
Wall of 
Technical Debt8 
8 http://verraes.net/2013/07/managed-technical-debt/
Use experiments to 
detect problems
@mathiasverraes 
http://verraes.net/2014/03/small-controlled-experiments/ 
http://verraes.net/workshops

More Related Content

What's hot

Design of experiments
Design of experimentsDesign of experiments
Design of experiments
design501
 
Design of experiments
Design of experimentsDesign of experiments
Design of experiments
design501
 
Bryan.oconnor
Bryan.oconnorBryan.oconnor
Bryan.oconnor
NASAPMC
 

What's hot (19)

#ATAGTR2018 Presentation "The Subtle Influence of Cognitive Biases on Testing...
#ATAGTR2018 Presentation "The Subtle Influence of Cognitive Biases on Testing...#ATAGTR2018 Presentation "The Subtle Influence of Cognitive Biases on Testing...
#ATAGTR2018 Presentation "The Subtle Influence of Cognitive Biases on Testing...
 
Vibrant Performance Today-Travel on a Trip to Self-Discovery!
Vibrant Performance Today-Travel on a Trip to Self-Discovery!Vibrant Performance Today-Travel on a Trip to Self-Discovery!
Vibrant Performance Today-Travel on a Trip to Self-Discovery!
 
Information radiators
Information radiatorsInformation radiators
Information radiators
 
Learning From Failure & How You Can Too
Learning From Failure & How You Can TooLearning From Failure & How You Can Too
Learning From Failure & How You Can Too
 
Framework for Decision Making-NeVC2010
Framework for Decision Making-NeVC2010Framework for Decision Making-NeVC2010
Framework for Decision Making-NeVC2010
 
Integrating Design Using the Native Language of Healthcare
Integrating Design Using the Native Language of HealthcareIntegrating Design Using the Native Language of Healthcare
Integrating Design Using the Native Language of Healthcare
 
Myths of Cognitive Science and Psychology
Myths of Cognitive Science and PsychologyMyths of Cognitive Science and Psychology
Myths of Cognitive Science and Psychology
 
Chantelle Morrissette - Scanning the Future to Influence it
Chantelle Morrissette - Scanning the Future to Influence itChantelle Morrissette - Scanning the Future to Influence it
Chantelle Morrissette - Scanning the Future to Influence it
 
Problem solving decisions
Problem solving decisionsProblem solving decisions
Problem solving decisions
 
How to Run Structured Growth Tests
How to Run Structured Growth TestsHow to Run Structured Growth Tests
How to Run Structured Growth Tests
 
Design of experiments
Design of experimentsDesign of experiments
Design of experiments
 
Design of experiments
Design of experimentsDesign of experiments
Design of experiments
 
Orientation and alignment in a vuca world
Orientation and alignment in a vuca worldOrientation and alignment in a vuca world
Orientation and alignment in a vuca world
 
On Norman doors, sign design, and why you need to think like a designer!
On Norman doors, sign design, and why you need to think like a designer!On Norman doors, sign design, and why you need to think like a designer!
On Norman doors, sign design, and why you need to think like a designer!
 
What is the similarity between a hopsital patient and an oil rig
What is the similarity between a hopsital patient and an oil rigWhat is the similarity between a hopsital patient and an oil rig
What is the similarity between a hopsital patient and an oil rig
 
Action Plan: Turning Your Ideas into Action
Action Plan: Turning Your Ideas into ActionAction Plan: Turning Your Ideas into Action
Action Plan: Turning Your Ideas into Action
 
Causal Tree Analysis (Root Cause)
Causal Tree Analysis (Root Cause)Causal Tree Analysis (Root Cause)
Causal Tree Analysis (Root Cause)
 
Graham Thomas - The Testers Toolbox - EuroSTAR 2010
Graham Thomas - The Testers Toolbox - EuroSTAR 2010Graham Thomas - The Testers Toolbox - EuroSTAR 2010
Graham Thomas - The Testers Toolbox - EuroSTAR 2010
 
Bryan.oconnor
Bryan.oconnorBryan.oconnor
Bryan.oconnor
 

Similar to Small Controlled Experiments

A Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software TestingA Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software Testing
TechWell
 
Research Challenges – Am I Doing “Real” Research?
Research Challenges – Am I Doing “Real” Research?Research Challenges – Am I Doing “Real” Research?
Research Challenges – Am I Doing “Real” Research?
Dr. Mazlan Abbas
 

Similar to Small Controlled Experiments (20)

Reducing Test Case Bloat2 1
Reducing Test Case Bloat2 1Reducing Test Case Bloat2 1
Reducing Test Case Bloat2 1
 
Using Problem Solving Skills To Get A Job
Using Problem Solving Skills To Get A JobUsing Problem Solving Skills To Get A Job
Using Problem Solving Skills To Get A Job
 
Testing Business Models
Testing Business ModelsTesting Business Models
Testing Business Models
 
Michael Bolton - Heuristics: Solving Problems Rapidly
Michael Bolton - Heuristics: Solving Problems RapidlyMichael Bolton - Heuristics: Solving Problems Rapidly
Michael Bolton - Heuristics: Solving Problems Rapidly
 
Conduct a psychology experiment
Conduct a psychology experimentConduct a psychology experiment
Conduct a psychology experiment
 
Experiment Driven Product Innovation
Experiment Driven Product InnovationExperiment Driven Product Innovation
Experiment Driven Product Innovation
 
Productive collaboration
Productive collaborationProductive collaboration
Productive collaboration
 
How should we build that? Evolving a development environment that's suitable ...
How should we build that? Evolving a development environment that's suitable ...How should we build that? Evolving a development environment that's suitable ...
How should we build that? Evolving a development environment that's suitable ...
 
Research and Commercialisation Challenges
Research and Commercialisation ChallengesResearch and Commercialisation Challenges
Research and Commercialisation Challenges
 
A Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software TestingA Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software Testing
 
A Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software TestingA Rapid Introduction to Rapid Software Testing
A Rapid Introduction to Rapid Software Testing
 
SINY Leanstartup Introduction | Reduce waste, run experiments!
SINY Leanstartup Introduction | Reduce waste, run experiments!SINY Leanstartup Introduction | Reduce waste, run experiments!
SINY Leanstartup Introduction | Reduce waste, run experiments!
 
Research Challenges – Am I Doing “Real” Research?
Research Challenges – Am I Doing “Real” Research?Research Challenges – Am I Doing “Real” Research?
Research Challenges – Am I Doing “Real” Research?
 
Agile Values
Agile ValuesAgile Values
Agile Values
 
Analytical Thinking And Innovation
Analytical Thinking And InnovationAnalytical Thinking And Innovation
Analytical Thinking And Innovation
 
Johan Jonasson - Test Strategy, Why Should You Care - EuroSTAR 2013
Johan Jonasson - Test Strategy, Why Should You Care - EuroSTAR 2013Johan Jonasson - Test Strategy, Why Should You Care - EuroSTAR 2013
Johan Jonasson - Test Strategy, Why Should You Care - EuroSTAR 2013
 
Evolve or Die: A3 Thinking and Popcorn Flow in Action (#LKCE14)
Evolve or Die: A3 Thinking and Popcorn Flow in Action (#LKCE14)Evolve or Die: A3 Thinking and Popcorn Flow in Action (#LKCE14)
Evolve or Die: A3 Thinking and Popcorn Flow in Action (#LKCE14)
 
Exploratory Testing Explained and Experienced
Exploratory Testing Explained and ExperiencedExploratory Testing Explained and Experienced
Exploratory Testing Explained and Experienced
 
Heuristics, bias and critical thinking in testing distribution
Heuristics, bias and critical thinking in testing   distributionHeuristics, bias and critical thinking in testing   distribution
Heuristics, bias and critical thinking in testing distribution
 
Ppt methods of acquiring knowledge
Ppt methods of acquiring knowledgePpt methods of acquiring knowledge
Ppt methods of acquiring knowledge
 

More from Mathias Verraes

Why Domain-Driven Design Matters
Why Domain-Driven Design MattersWhy Domain-Driven Design Matters
Why Domain-Driven Design Matters
Mathias Verraes
 

More from Mathias Verraes (12)

Towards Modelling Processes
Towards Modelling ProcessesTowards Modelling Processes
Towards Modelling Processes
 
Modelling Heuristics
Modelling HeuristicsModelling Heuristics
Modelling Heuristics
 
Managed Technical Debt
Managed Technical DebtManaged Technical Debt
Managed Technical Debt
 
DDD Basics: Bounded Contexts, Modelling - Kortrijk Edition
DDD Basics: Bounded Contexts, Modelling - Kortrijk EditionDDD Basics: Bounded Contexts, Modelling - Kortrijk Edition
DDD Basics: Bounded Contexts, Modelling - Kortrijk Edition
 
Why Domain-Driven Design Matters
Why Domain-Driven Design MattersWhy Domain-Driven Design Matters
Why Domain-Driven Design Matters
 
Practical Event Sourcing
Practical Event SourcingPractical Event Sourcing
Practical Event Sourcing
 
Unbreakable Domain Models PHPUK 2014 London
Unbreakable Domain Models PHPUK 2014 LondonUnbreakable Domain Models PHPUK 2014 London
Unbreakable Domain Models PHPUK 2014 London
 
Domain-Driven Design Basics
Domain-Driven Design BasicsDomain-Driven Design Basics
Domain-Driven Design Basics
 
Model Storming Workshop PHP Benelux 2014
Model Storming Workshop PHP Benelux 2014Model Storming Workshop PHP Benelux 2014
Model Storming Workshop PHP Benelux 2014
 
Fighting Bottlencks with CQRS - ResearchGate
Fighting Bottlencks with CQRS - ResearchGateFighting Bottlencks with CQRS - ResearchGate
Fighting Bottlencks with CQRS - ResearchGate
 
DDDBE Modellathon 2013
DDDBE Modellathon 2013DDDBE Modellathon 2013
DDDBE Modellathon 2013
 
Unbreakable Domain Models - DPC13
Unbreakable Domain Models - DPC13Unbreakable Domain Models - DPC13
Unbreakable Domain Models - DPC13
 

Recently uploaded

AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...
AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...
AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...
Alluxio, Inc.
 

Recently uploaded (20)

How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?
 
Accelerate Enterprise Software Engineering with Platformless
Accelerate Enterprise Software Engineering with PlatformlessAccelerate Enterprise Software Engineering with Platformless
Accelerate Enterprise Software Engineering with Platformless
 
AI/ML Infra Meetup | ML explainability in Michelangelo
AI/ML Infra Meetup | ML explainability in MichelangeloAI/ML Infra Meetup | ML explainability in Michelangelo
AI/ML Infra Meetup | ML explainability in Michelangelo
 
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
 
Large Language Models and the End of Programming
Large Language Models and the End of ProgrammingLarge Language Models and the End of Programming
Large Language Models and the End of Programming
 
A Comprehensive Appium Guide for Hybrid App Automation Testing.pdf
A Comprehensive Appium Guide for Hybrid App Automation Testing.pdfA Comprehensive Appium Guide for Hybrid App Automation Testing.pdf
A Comprehensive Appium Guide for Hybrid App Automation Testing.pdf
 
First Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User EndpointsFirst Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User Endpoints
 
How Recreation Management Software Can Streamline Your Operations.pptx
How Recreation Management Software Can Streamline Your Operations.pptxHow Recreation Management Software Can Streamline Your Operations.pptx
How Recreation Management Software Can Streamline Your Operations.pptx
 
SOCRadar Research Team: Latest Activities of IntelBroker
SOCRadar Research Team: Latest Activities of IntelBrokerSOCRadar Research Team: Latest Activities of IntelBroker
SOCRadar Research Team: Latest Activities of IntelBroker
 
De mooiste recreatieve routes ontdekken met RouteYou en FME
De mooiste recreatieve routes ontdekken met RouteYou en FMEDe mooiste recreatieve routes ontdekken met RouteYou en FME
De mooiste recreatieve routes ontdekken met RouteYou en FME
 
AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...
AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...
AI/ML Infra Meetup | Improve Speed and GPU Utilization for Model Training & S...
 
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdfDominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
 
Breaking the Code : A Guide to WhatsApp Business API.pdf
Breaking the Code : A Guide to WhatsApp Business API.pdfBreaking the Code : A Guide to WhatsApp Business API.pdf
Breaking the Code : A Guide to WhatsApp Business API.pdf
 
Facemoji Keyboard released its 2023 State of Emoji report, outlining the most...
Facemoji Keyboard released its 2023 State of Emoji report, outlining the most...Facemoji Keyboard released its 2023 State of Emoji report, outlining the most...
Facemoji Keyboard released its 2023 State of Emoji report, outlining the most...
 
BoxLang: Review our Visionary Licenses of 2024
BoxLang: Review our Visionary Licenses of 2024BoxLang: Review our Visionary Licenses of 2024
BoxLang: Review our Visionary Licenses of 2024
 
Using IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New ZealandUsing IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New Zealand
 
Developing Distributed High-performance Computing Capabilities of an Open Sci...
Developing Distributed High-performance Computing Capabilities of an Open Sci...Developing Distributed High-performance Computing Capabilities of an Open Sci...
Developing Distributed High-performance Computing Capabilities of an Open Sci...
 
Cyaniclab : Software Development Agency Portfolio.pdf
Cyaniclab : Software Development Agency Portfolio.pdfCyaniclab : Software Development Agency Portfolio.pdf
Cyaniclab : Software Development Agency Portfolio.pdf
 
Globus Connect Server Deep Dive - GlobusWorld 2024
Globus Connect Server Deep Dive - GlobusWorld 2024Globus Connect Server Deep Dive - GlobusWorld 2024
Globus Connect Server Deep Dive - GlobusWorld 2024
 
Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus Compute wth IRI Workflows - GlobusWorld 2024
 

Small Controlled Experiments