e 
   
 
Presented by: 
Ray  rell 
Brought to you by: 
 
 
340 Corporate Way, Suite   Orange Park, FL 32073 
888‐2
K...
Ray Arell
Intel
 
As director of Intel Emergent Systems and Coaching, Ray Arell is a transformative
leader in the adoption...
2/26/2014
1
THE ART OF COMPLEX SYSTEM THE ART OF COMPLEX SYSTEM TESTINGTESTING
Ray Arell, Director 
Intel Emergent Systems...
2/26/2014
2
Source: http://en.wikipedia.o
2/26/2014
3
2/26/2014
4
Source: Flickr.com 
2/26/2014
5
nts
Many
Chaotic
Requireme
Fe
Complex
Simple
Complicated
Technology Far from
certainty
Close to
certainty
Few
...
2/26/2014
6
2/26/2014
7
Source: http://www.calresco.org/lucas/cas.htm
You can't connect the dots looking 
forward; you can only connec...
2/26/2014
8
We manage the emergence of beneficial coherence within 
attractors, within boundaries—Dave Snowden
Voltage
Fre...
2/26/2014
9
Source: http://www.mdofpc.com
2/26/2014
10
CYNEFIN FRAMEWORK
Complicated
C E
Complex
C E Dave Snowden
Simple
C E
good practiceemergent practice
Chaotic
...
2/26/2014
11
Source: www.thephonetown.com; with a fish m
Complex Complicated
c
a
Disorder
b
c
SimpleChaos
Source: Cynefin ...
2/26/2014
12
D i d St t
dge 
Desired State
Tyranny of the
expert
Knowle
PROBEPROBE--SENSESENSE--RESPONDRESPOND
emergent pr...
2/26/2014
13
2/26/2014
14
Source: Ylvis; The Fox (What Does The Fox Say?)
“It takes a thousand voices to tell a single story”
- Native ...
2/26/2014
15
Prompting Question
Story Capture 
Act on Signals
Stimulate/Dampen 
Patterns
Self‐Signification
Discussion Vis...
2/26/2014
16
Poor Performance
N/A
Poor Reliability Poor Usability
Quality
More info: cognitive‐edge.com 
2/26/2014
17
EXPANDING OUR TEST METHODS TO ADDRESS 
THE COMPLEX SPACE 
Sources: right http://www.ece.lsu.edu; left Paul Ma...
2/26/2014
18
ABSTRACT
It’s one week after your product’s launch, and everyone is happy. After all, for the 
first time in ...
2/26/2014
19
As director of Intel Emergent Systems and Coaching, Ray Arell is a g y g, y
transformative leader in the adop...
Upcoming SlideShare
Loading in …5
×

The Art of Complex System Testing

251 views

Published on

It’s one week after your product’s launch, and everyone is happy. After all, for the first time in years, your product development exceeded expectations. Coding was completed on time with very few defects. Suddenly, the report of a major usability and security flaw destroys the euphoria and sends everything into chaos. Unfortunately, this is not uncommon in our industry. So, how can we mitigate such things from happening? As he shares stories about the complex domain of product delivery, Ray Arell introduces a framework with associated emergent practices that enable you to better guide your product to success. He presents an overview of the Cynefin model, a description of complicated and complex systems, and discusses how to use it to establish an effective testing strategy. Ray describes how to identify key patterns of product usage to establish a robust defect-prevention system that reduces product development costs. Lastly, Ray describes how to interview customers to identify key quality expectations, ensuring that your testing focuses on producing the highest value for your customers.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
251
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

The Art of Complex System Testing

  1. 1.       e        Presented by:  Ray  rell  Brought to you by:      340 Corporate Way, Suite   Orange Park, FL 32073  888‐2 K3  Keynot 4/9/2014 8:30 AM          “The Art of Complex System Testing”      A Intel                    300, 68‐8770 ∙ 904‐278‐0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com 
  2. 2. Ray Arell Intel   As director of Intel Emergent Systems and Coaching, Ray Arell is a transformative leader in the adoption of agile, lean, and complex system methods inside Intel. Ray’s group is currently coaching a community of practice of more than ten-thousand people who are moving to a continuous value delivery culture. Prior to this role, he spent several decades as both an engineer and engineering manager of teams focused on CPU, chipset, graphics, wireless, and software development. Ray is a popular speaker at events worldwide and coauthor of Change-Based Test Management: Improving the Software Validation Process.  
  3. 3. 2/26/2014 1 THE ART OF COMPLEX SYSTEM THE ART OF COMPLEX SYSTEM TESTINGTESTING Ray Arell, Director  Intel Emergent Systems and Coaching  Copyright © 2014 Intel Corporation. All rights reserved. *Other names and brands may be claimed as the property of others.  
  4. 4. 2/26/2014 2 Source: http://en.wikipedia.o
  5. 5. 2/26/2014 3
  6. 6. 2/26/2014 4 Source: Flickr.com 
  7. 7. 2/26/2014 5 nts Many Chaotic Requireme Fe Complex Simple Complicated Technology Far from certainty Close to certainty Few KnowledgeHigh Low Simple Source: Flickr.com 
  8. 8. 2/26/2014 6
  9. 9. 2/26/2014 7 Source: http://www.calresco.org/lucas/cas.htm You can't connect the dots looking  forward; you can only connect them  looking backwards—Steve Jobs
  10. 10. 2/26/2014 8 We manage the emergence of beneficial coherence within  attractors, within boundaries—Dave Snowden Voltage Frequency 
  11. 11. 2/26/2014 9 Source: http://www.mdofpc.com
  12. 12. 2/26/2014 10 CYNEFIN FRAMEWORK Complicated C E Complex C E Dave Snowden Simple C E good practiceemergent practice Chaotic probe-sense-respond sense-analyze-respond C E Disorder Dave Snowden Disorder best practice Cause Effect novel practice act-sense-respond sense-categorize-respond C E Source: cognitive‐edge.com 
  13. 13. 2/26/2014 11 Source: www.thephonetown.com; with a fish m Complex Complicated c a Disorder b c SimpleChaos Source: Cynefin by David Snowden http://cognitive‐edge.com/
  14. 14. 2/26/2014 12 D i d St t dge  Desired State Tyranny of the expert Knowle PROBEPROBE--SENSESENSE--RESPONDRESPOND emergent practiceemergent practice
  15. 15. 2/26/2014 13
  16. 16. 2/26/2014 14 Source: Ylvis; The Fox (What Does The Fox Say?) “It takes a thousand voices to tell a single story” - Native American proverb Source: Competition among memes in a world with limited attention; L. Weng, A. Flammini, A. Vespignani & F. Menczer
  17. 17. 2/26/2014 15 Prompting Question Story Capture  Act on Signals Stimulate/Dampen  Patterns Self‐Signification Discussion Visualization Deepen Insight Source: cognitive‐edge.com  SELF‐SIGNIFICATION IS THE KEY Poor Performance Exciting  This story is about:  The product talked about is:  l b l b l g d bl N/AN/A Poor Reliability Poor Usability Quality Expected Desirable Qualities  More info: cognitive‐edge.com 
  18. 18. 2/26/2014 16 Poor Performance N/A Poor Reliability Poor Usability Quality More info: cognitive‐edge.com 
  19. 19. 2/26/2014 17 EXPANDING OUR TEST METHODS TO ADDRESS  THE COMPLEX SPACE  Sources: right http://www.ece.lsu.edu; left Paul Mathews  Complicated  Complexto C E C E FINAL THOUGHTS • We need to start to develop new test methods  to address complex products • Testing the rules is fine but it needs to be with  real world complex stimulus  • High satisfaction can only be understood if we  listen closer to the stories our customers are  telling
  20. 20. 2/26/2014 18 ABSTRACT It’s one week after your product’s launch, and everyone is happy. After all, for the  first time in years your product development exceeded expectations Coding wasfirst time in years, your product development exceeded expectations. Coding was  completed on time with very few defects. Suddenly, the report of a major usability  and security flaw destroys the euphoria and sends everything into chaos.  Unfortunately, this is not uncommon in our industry. So, how can we mitigate such  things from happening? As he shares stories about the complex domain of product  delivery, Ray Arell introduces a framework with associated emergent practices that  enable you to better guide your product to success. He presents an overview of  the Cynefin model, a description of complicated and complex systems, and  discusses how to use it to establish an effective testing strategy. Ray describes how  to identify key patterns of product usage to establish a robust defect‐prevention  system that reduces product development costs. Lastly, Ray describes how to  interview customers to identify key quality expectations, ensuring that your testing  focuses on producing the highest value for your customers.
  21. 21. 2/26/2014 19 As director of Intel Emergent Systems and Coaching, Ray Arell is a g y g, y transformative leader in the adoption of agile, lean, and complex  system methods inside Intel. Ray’s group is currently coaching a  community of practice of more than ten‐thousand people who are  moving to a continuous value delivery culture. Prior to this role, he  spent several decades as both an engineer and engineering  manager of teams focused on CPU, chipset, graphics, wireless, and  software development. Ray is a popular speaker at events  worldwide and coauthor of Change‐Based Test Management:  Improving the Software Validation Process.

×