This presentation takes the seminal work structured analytic techniques work pioneered within US intelligence, and proposes adaptions and simplifications for use within commercial enterprises

1. Giving Organisations
new Capabilities
to ask the Right
Business Questions
Stephen Simpson
stephen@ssimpson.net
@sharplyunclear

2. Making Data Work is Hard
Value Captured
Outputs
Outcomes
Sales Growth
Profit Growth
Sales Growth to Existing
Customers
Product Performance
Technology
Leadership
New Customers
Process
Improvement
Effective Project Execution
Processes
Inputs
Balanced Innovation
Portfolio
Supportive Strategy,
Structure, & Systems
Partners’
Value-add
Employee
Commitment to
Innovation
Quality of Innovation
Pipeline
Access to Talent

3. The “All In” Approach
Ron Johnson, CEO
Myron Ullman, CEO

4. You start out thinking you have a sales problem but might find
it is not really sales but marketing or customer retention...
…you could spent a lot of time on analysis that doesn’t lead to
solving the right problem.”

5. The Experimental Approach
"We did a Hadoop trial last year, it didn't go very far
because we weren't getting the intelligence out of it
that we thought we would. So we are looking at some
other initiatives with different vendors this year.
"We tried to put three different data sets together, and
then tried to see if we could find some causality
between the data sets that would gives us intelligence
that would allow us to manage our operations better…
"Whether that was how we set the trial up or the
software I don't know, so we are going to try
some different things.”

6. The “Wait and See” Approach
Incumbents are rarely disrupted by new technologies they can't
catch up to, but instead by new business models they can't match.
Institutions will try to preserve the problem to which they are
the solution.

7. Satisficing
 Can rarely evaluate all outcomes with sufficient precision
 Usually don’t know relevant probabilities of outcomes
 Possess limited memory

8. Results are often Modest

9. “All other things being equal”
When we sacrifice dealing with detail complexity to focus on dynamic
complexity, the solutions don’t produce the outcomes that we really want.
http://blogs.hbr.org/2013/09/our-self-inflicted-complexity/

10. Obtaining new insights
Business
Strategy
We need to make
sure that we’re
asking people to
research the right
questions
Domain
Expertise
Company Systems
& Data
And then we iterate to improve the
insight gained, or address the next
business question…
Data Mining
Agile
Experimentation
We need to choose the
right storage technologies,
integration services &
architecture
Sourcing
We need to look in
many more places to
find data…
Extraction
…and it will take a lot of
different skills and
approaches to bring it
together
We need to perform analysis
quickly inside small projects,
with a specific business goal.
Some of these will fail.
We need to be careful to
curb our enthusiasm and
separate out the signal
from the noise
Interpretation
Implementation
We need simple, easy to
use production tools to act
upon the new insights.
Authority needs to be
delegated to where the
information is captured
Visualisation
We need new techniques to
interpret and manipulate vast
numbers of data points on a
single surface

12. Candidate Sources
CRISP-DM
Richards J. Heuer &
Randolph H. Pherson

13. Analytic Methods
Decomposition &
Visualisation
Idea Generation
Expert Judgment
Scenarios & Indicators
Quantitative
Methods using
Expert-Generated
Data
Quantitative
Methods using
Empirical Data
Hypothesis Generation
& Testing
Assessment of
Cause & Effect
Challenge Analysis
Conflict Management
Structured Analysis
Decision Support

14. 14

15. Structured Analysis
a step by step process for analyzing the kind of
incomplete, ambiguous and sometimes deceptive
information that analysts must deal with.

16. Structured Analytic Techniques contain
Diagnostic + Contrarian + Imagination
elements
16

17. The Techniques

18. Choosing what you want to do
1. Define the
project?
2. Get started?
Decomposition & Visualisation
Decomposition & Visualisation
3. Examine & make
sense of the data?
Figure out what is
going on?
Idea Generation
Scenarios & Indicators
4. Assess the most
likely outcome of an
evolving situation?
5. Monitor a
situation to avoid
surprise?
6. Generate and test
hypotheses?
Hypothesis Generation
& Testing
Assessment of
Cause & Effect
7. Assess the
possibility of
deception?
8. Foresee the
future?
9. Challenge your
own mental model?
Challenge Analysis
Conflict Management
10. See events from
the perspective of
other players?
11. Managing
conflicting mental
models or
opinions?
12. Support a
manager in
deciding course of
action?
Decision Support

20. Template Structure
Overview
When to Use It
Value Added
The Method
Relationship to other Techniques
Origins of this Technique
20

21. Long term
personal healthcare
Branded Currency
Personalised Interactions
21

22. 1. Decomposition & Visualisation
When forced to work within a strict framework the imagination
is taxed to its utmost – and will produce its richest ideas.
Given freedom the work is likely to sprawl.

24. Value Proposition
Understand your clients’ needs at the finest level of detail
Client micro-segmentation using multiple sources of data
Description
FOR marketing operations
WHO want to understand the growth potential for each identified customer subdivision
THE understand your clients’ needs at the finest level of detail solution
PROVIDES understanding of the root causes for your current share of each identified slice
THAT lets you act on the information quickly with targeted retail product placement & location selling
UNLIKE your existing solution
WHICH is coarse-grained and retrospective
Scenarios
•
•
•
•
•
Retail product placement & location selling
Counteracting effectiveness of competitors
Understanding local reputation via ”voice of the customer”
Real-time decision making such as mobile-based coupon positioning to particular segments
Partner organisations’ service effectiveness

25. 2. Idea Generation
The best way to have a good idea is to have a lot of ideas

27. Creativity
Value Creation
 Out-of-box thinking
 In-the-box thinking
 Raw & refined ideas
 Experimentation
 Engineering/process
improvement
 Ambiguity/uncertainty
 Precision
 Research
 Well-calculated trade-offs
 Intuition
 Buying/selling of ideas
 Surprise
 Do things right
 Courage
 Answer questions & verify
solutions
 Find the right things
 Ask questions & explore
unknown innovation
 Seize opportunities
 Visualize future & consider
all options
 Include incremental &
radical ideas
 Avoid major risks
 Get product into the
marketplace
 Bias for incremental

28. Cross Impact Matrix
For when “Everything is connected to everything else”
 Business is in flux
 Context for discussion of interactions
 System is stable
 Discover variables once thought to be simple
- Need to identify and monitor all
factors that might upset this
 A significant event has occurred
- Need to understand implications
& independent are actually interrelated
 Focus on
- Interactions that may have been overlooked
- Variables that might reinforce each other

29. Cross Impact Matrix
A
B
C
C. Existing core banking solutions
D. Apps & Cloud Service interaction
E
F
++ ++
A. Personalised Interactions
B. Existing mobile solutions
D
--- ++
+ -
E. Offers
++
F. Analytics
++
-
+
+
++
++
++

30. 3. Scenarios & Indicators
Scenarios are plausible &
provocative stories about how
the future might unfold

32. Indicators
Observable Phenomena that can periodically be reviewed to help track events
 Make humans recognize early signs
significant change
 Spot emerging trends
 Quality indicators are critical
- If narrowly defined or out of date
- Reinforce bias
- Warn unanticipated changes
- Discard new evidence
- Avoid surprise
- Lull people inappropriately
 Forward looking, predictive
 Objective baseline for tracking
- Dashboards…
 Indicators Validator
 Instil rigour into analytic process
- Quality and strength of indicator
 Enhance credibility of what delivered
- Whether appears in all scenarios
 Exchange knowledge between experts
from different domains

33. Indicators
2013
Q4
Q3
Mobile Offers
Reaching right segment
People engaged
Volunteering information
Infrastructure
Holding initiative back
Cloud
Security, regulatory, compliance
Service
Take-up standard services
3rd party composing new apps
Industry Trends
Personalised CRM
Branded Currency
Device as Bank
Ecosystem
Retailers using your backbone
Competitive launches
Q1
□
●
▫
□
▫
▫
□
○
○
○
Q2
●
▪
□
▫
▫
Q1
●
▪
□
●
▫
Q4
2015
Q3
□
●
□
●
□
□
Q2
2014
Q3
Neglible concern
Low concern
Moderate
Substantial
Strong
▫
▪
□
○
●
Q4

34. 4. Hypothesis Generation & Testing
A possible explanation of the past or a judgment about
the future is a hypothesis that needs to be tested by
collecting and presenting evidence

36. 5. Assessment of Cause & Effect
We are slow to accept the reality
of simple mistakes, accidents,
unintended consequences,
coincidences, or small causes
leading to large effects

38. Personalised Interactions will increase: Key Assumptions check
 Legal and privacy – Caveated.
 Components available across entire chain – Caveated.
 Customers want seamless, personally relevant services – Solid
 Devices will progress sufficiently – Solid
 Analytics techniques are sufficiently refined, accurate and timely – Caveated
 Back-end systems will support workload – Solid
 Systems will be cost effective – Caveated. What’s the ROI of something you don’t know?
 Employees trained and authority delegated to act – Unsupported

39. 6. Challenge Analysis
It is the mark of an educated mind to be able to entertain a
thought without accepting it.

41. Pre-mortem analysis
Imagine the future where your plan has been implemented, but has failed
Advantages:
 Take people out of perspective of
defending their plan & shielding
themselves from its flaws
 Increase level of candour
 Can be used to show decision makers
that are typically over-confident that
their decisions and plans will work
 Questions re-framed, to elicit different
responses to original ones
 Legitimises dissent – asked to make a
positive contribution by identifying
weaknesses in previous analysis
 Examples
- Internal inertia or uneven execution
- Competitors’ actions
- Law of unintended consequences
- Economic changes

43. 7. Conflict Management
Disagreements sparked by differences in perspective, competencies,
& access to information… actually generate much of the value that can
come from collaboration across organisational boundaries.
http://hbr.org/2005/03/want-collaboration-accept-and-actively-manage-conflict

45. 8. Decision Support
…without overstepping the limits of their role…; just structures all the
relevant information in a format that makes it easier for the decision
maker to make a choice.

47. A word on Dashboards
It is also unfortunate to see how many business intelligence
and enterprise data warehousing projects get waylaid by the
singular pursuit of pretty dashboards…

48. Iterating Quickly

49. Time is Key

50. In Summary
 Does provide new capabilities to ask right questions
- Offers path to clearer business goals
- Discourages “wait and see” approaches
 Encourages cross-organisational linkages
 Validates or challenges experts’ “hunches”
 More limited use in monitoring subsequent change
12