Frame, Build, Measure, Learn:

Responding to Change"
Zach Nies – Chief Technologist!

www.rallydev.com

@zachnies!

©2014!
Zach Nies
Chief Technologist
Rally Software

Faculty
University of Colorado at Boulder

@zachnies
www.rallydev.com

@zachnies!

©2014!
The biggest risk
you face

is changing"
www.rallydev.com

@zachnies!

©2014!
Risk:

failing to respond
to market shifts"
www.rallydev.com

@zachnies!

©2014!
ANOTHER RISK…?

www.rallydev.com

@zachnies!

©2014!
An example…"

www.rallydev.com

@zachnies!

©2014!
Local consumer
data company"

www.rallydev.com

@zachnies!

©2014!
90%

We have 80%
market share"

80%

70%

Market Share

60%

50%

40%

30%

20%

10%

0%
Q3 2011

Q4 2011

Q1 2012

Q2 2012

Q3 2012

Q4 2012

Desktop
www.rallydev.com

@zachnies!

Q1 2013

Q2 2013

Q3 2013

Q4 2013

Q1 2014

Mobile
©2014!
90%

We need

a mobile app"

80%

70%

Market Share

60%

50%

40%

30%

20%

10%

0%
Q3 2011

Q4 2011

Q1 2012

Q2 2012

Q3 2012

Q4 2012

Desktop
www.rallydev.com

@zachnies!

Q1 2013

Q2 2013

Q3 2013

Q4 2013

Q1 2014

Mobile
©2014!
90%

Mobile app
specification
begins"

80%

70%

Market Share

60%

50%

40%

30%

20%

10%

0%
Q3 2011

Q4 2011

Q1 2012

Q2 2012

Q3 2012

Q4 2012

Desktop
www.rallydev.com

@zachnies!

Q1 2013

Q2 2013

Q3 2013

Q4 2013

Q1 2014

Mobile
©2014!
90%

80%

70%

Market Share

60%

50%

40%

30%

20%

10%

Startup
competitor
ships 1.0"

0%
Q3 2011

Q4 2011

Q1 2012

Q2 2012

Q3 2012

Q4 2012

Desktop
www.rallydev.com

@zachnies!

Q1 2013

Q2 2013

Q3 2013

Q4 2013

Q1 2014

Mobile
©2014!
90%

80%

70%

Market Share

60%

50%

40%

30%

20%

10%

Startup
competitor
ships 1.1"

0%
Q3 2011

Q4 2011

Q1 2012

Q2 2012

Q3 2012

Q4 2012

Desktop
www.rallydev.com

@zachnies!

Q1 2013

Q2 2013

Q3 2013

Q4 2013

Q1 2014

Mobile
©2014!
90%

App

defined



Dev

begins"

80%

70%

Market Share

60%

50%

40%

30%

20%

10%

0%
Q3 2011

Q4 2011

Q1 2012

Q2 2012

Q3 2012

Q4 2012

Desktop
www.rallydev.com

@zachnies!

Q1 2013

Q2 2013

Q3 2013

Q4 2013

Q1 2014

Mobile
©2014!
90%

80%

70%

Market Share

60%

50%

40%

30%

20%

10%

Startup
competitor
ships 1.2"

0%
Q3 2011

Q4 2011

Q1 2012

Q2 2012

Q3 2012

Q4 2012

Desktop
www.rallydev.com

@zachnies!

Q1 2013

Q2 2013

Q3 2013

Q4 2013

Q1 2014

Mobile
©2014!
90%

Mobile app ships"

80%

70%

Market Share

60%

50%

40%

30%

20%

10%

0%
Q3 2011

Q4 2011

Q1 2012

Q2 2012

Q3 2012

Q4 2012

Desktop
www.rallydev.com

@zachnies!

Q1 2013

Q2 2013

Q3 2013

Q4 2013

Q1 2014

Mobile
©2014!
90%

80%

70%

Market Share

60%

50%

40%

30%

20%

10%

We forever
lost half our
market share
in one year"

0%
Q3 2011

Q4 2011

Q1 2012

Q2 2012

Q3 2012

Q4 2012

Desktop
www.rallydev.com

@zachnies!

Q1 2013

Q2 2013

Q3 2013

Q4 2013

Q1 2014

Mobile
©2014!
Gut wrenching"

www.rallydev.com

@zachnies!

©2014!
What’s

going on here?"

www.rallydev.com

@zachnies!

©2014!
Time needed

to change

market expectations"

www.rallydev.com

@zachnies!

©2014!
Some examples…"

www.rallydev.com

@zachnies!

©2014!
1984!
www.rallydev.com

@zachnies!

©2014!
2006!
www.rallydev.com

@zachnies!

©2014!
About 20 years!

www.rallydev.com

@zachnies!

©2014!
2007!
www.rallydev.com

@zachnies!

©2014!
2010!
www.rallydev.com

@zachnies!

©2014!
3 years!

www.rallydev.com

@zachnies!

©2014!
2010!
www.rallydev.com

@zachnies!

©2014!
2012!
www.rallydev.com

@zachnies!

©2014!
2 years!

www.rallydev.com

@zachnies!

©2014!
1979!
www.rallydev.com

@zachnies!

©2014!
1989!
www.rallydev.com

@zachnies!

©2014!
10 years!

www.rallydev.com

@zachnies!

©2014!
5 years!
www.rallydev.com

@zachnies!

©2014!
2.5 years!
www.rallydev.com

@zachnies!

©2014!
1.5 years!
www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!

http://hbr.org/2013/03/big-bang-disruption/
Huge market shifts
can happen

in a year"
www.rallydev.com

@zachnies!

©2014!
Are you

Agile enough

to balance"
www.rallydev.com

@zachnies!

©2014!
operating the business

and

responding to change?"

www.rallydev.com

@zachnies!

©2014!
Typical approach"

www.rallydev.com

@zachnies!

©2014!
Create a plan to
predict the future"
www.rallydev.com

@zachnies!

©2014!

http://www.flickr.com/photos/isaacmao/380257543
Market research"
Product definition"
Product requirements"
Detailed plan"
Alpha release"
Beta release"
Final release"

FIRST CONTACT WITH

CUSTOMERS"

www.rallydev.com

@zachnies!

©2014!
Market research"
Project definition"
Product requirements"
Detailed plan"
Alpha release"
Beta release"
Final release"
FIRST CONTACT WITH CUSTOMERS"
www.rallydev.com

@zachnies!

©2014!
Market research"
Project definition"
Project requirements"
Detailed plan"
Alpha release"
Beta release"
Final release"

FIRST CONTACT WITH

CUSTOMERS"

www.rallydev.com

@zachnies!

©2014!
Market research"
Project definition"
Project requirements"
Detailed plan"
Alpha release"
Beta release"
Final release"

FIRST CONTACT WITH

CUSTOMERS"

www.rallydev.com

@zachnies!

©2014!
Market research"
Project definition"
Project requirements"
Detailed plan"
Development"
"
Final release"

FIRST CONTACT WITH

CUSTOMERS"

www.rallydev.com

@zachnies!

©2014!
Market research"
Project definition"
Project requirements"
Detailed plan"
Development"
User acceptance testing"
"

FIRST CONTACT WITH

CUSTOMERS"

www.rallydev.com

@zachnies!

©2014!
Market research"
Project definition"
Project requirements"
Detailed plan"
Development"
User acceptance testing"
Final release"

FIRST CONTACT WITH

CUSTOMERS"

www.rallydev.com

@zachnies!

©2014!
HOW MANY
GUESSES?"
www.rallydev.com

@zachnies!

©2014!
FIRST CONTACT
WITH USERS"
www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!

http://www.flickr.com/photos/oregondot/4132135156
Obscure

the uncertainty

with details of a
false precision"
www.rallydev.com

@zachnies!

©2014!
Embrace, reveal
and mitigate

the uncertainty"
www.rallydev.com

@zachnies!

©2014!
Why does this

feel unnatural?"
www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!
“the greatest prosperity
can exist only when that
individual has reached
his highest state of
efficiency; that is, when
he is turning out his
largest daily output.”"
www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!
Assumes:

Certainty is high"



Goal: 

Maximize individual
efficiency"
www.rallydev.com

@zachnies!

©2014!
100 years later"

www.rallydev.com

@zachnies!

©2014!
Status Quo

Budgets"
"
Comprehensive

Analysis"
"
Detailed

Plans"
"
Waterfall

Execution"
www.rallydev.com

@zachnies!

©2014!
Works when:"

"
Certainty is high"
www.rallydev.com

@zachnies!

©2014!
Is your business
getting

more certain?"
www.rallydev.com

@zachnies!

©2014!
Agile Execution



Lean Thinking"

www.rallydev.com

@zachnies!

©2014!
Balance

execution and
exploration"

www.rallydev.com

@zachnies!

©2014!
Assumes:

Uncertainty exists"
"
Goal:

Maximize value
delivery"
www.rallydev.com

@zachnies!

©2014!
Adaptive

Steering"
"
Disciplined

Exploration"
"
Empirical

Learning"
"
Agile

Execution"
www.rallydev.com

@zachnies!

©2014!
Adaptive

Steering"
"
Disciplined

Exploration"
"
Empirical

Learning"
"
Agile

Execution"
www.rallydev.com

@zachnies!

©2014!
Turn Uncertainty"

Into Knowledge"
www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!

http://www.flickr.com/photos/memekiller/4564410767
www.rallydev.com

@zachnies!

©2014!
Business"
"
"
"
Initiative"
www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!
Assumptions

Canvas Name:
Key Assumption

Reminders will
help gift givers

Dependent Assumptions

Key Assumption

Suggestions
will motivate
givers

Dependent Assumptions

We know when

We can give

giving events

reasonable

will happen

suggestions

Text messages
are acceptable
reminders

We can send
timely text
messages

We can get
receiver
preference
data

Date:

Key Assumption

Falsified Assumptions

We can make
money through
partner
referrals
Dependent Assumptions

Partners want
referral leads

We can get

Other Assumptions

Other Assumptions

Facebook adds

Recipients

will drive

want gifts on

traffic

time

A mobile first

enough parter

product

inventory

strategy

Our leads will
buy gifts

I-1

www.rallydev.com

@zachnies!

©2014!
Key assumptions"

Knowledge"
www.rallydev.com

@zachnies!

©2014!
Run experiments"

www.rallydev.com

@zachnies!

©2014!
Frame"

www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!
Build"

www.rallydev.com

@zachnies!

©2014!
Scrum Machine
Daily
Scrum
Product Backlog
Refinement
Sprint
Planning

Sprint
1-2 Weeks
Burndown
Chart

Product
Backlog
www.rallydev.com

Sprint
Backlog
@zachnies!

Sprint Demo & Review
Sprint Retrospective

Product
Increment
©2014!
Measure"

www.rallydev.com

@zachnies!

©2014!
Collect"

www.rallydev.com

@zachnies!

©2014!

http://www.flickr.com/photos/mrpetersononline/4977025601
Analyze"

www.rallydev.com

@zachnies!

©2014!

http://www.flickr.com/photos/mrpetersononline/4992505907
Organize"

www.rallydev.com

@zachnies!

©2014!

http://www.flickr.com/photos/mrpetersononline/4992505641
Learn"

www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!
25%
www.rallydev.com

@zachnies!

©2014!
25%
www.rallydev.com

@zachnies!

55%
©2014!
Empirical Learning"

www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!
Advice

from a master…"

www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

http://www.richard-feynman.net/gallery.htm

©2014!
www.rallydev.com

@zachnies!

©2014!
One last thing to
keep in mind"

www.rallydev.com

@zachnies!

©2014!
Technology adoption
is a human
phenomenon "

www.rallydev.com

@zachnies!

©2014!
An example…"

www.rallydev.com

@zachnies!

©2014!
www.rallydev.com

@zachnies!

©2014!
Need an Agile and
human centered
approach!
www.rallydev.com

@zachnies!

©2014!
Balance execution
and exploration"

www.rallydev.com

@zachnies!

©2014!
LEARN MORE…
rallydev.com/els
rallydev.com/frame

www.rallydev.com

@zachnies!

©2014!

Frame, Build, Measure, Learn: Responding to Change