Creating Persuasive Technologies An Eight-Step Design PrCruzIbarra161
Creating Persuasive Technologies:
An Eight-Step Design Process
BJ Fogg
Persuasive Technology Lab
Stanford University
captology.stanford.edu
www.bjfogg.com
[email protected]
ABSTRACT
This paper outlines eight steps to follow as best practices in the
early stages of persuasive technology design. The eight-step
process, drawn from demonstrated successes in industry practice,
begins with defining the persuasion goal to match a target
audience with an appropriate technology channel. Subsequent
steps include imitating successful examples of persuasive design,
performing rapid trials, measuring behavioral outcomes, and
building on small successes.
General Terms
Design, Measurement, Experimentation, Human Factors.
Keywords
Persuasion, design, persuasive technology, prototyping, iteration,
behavior change, captology, behavior model.
INTRODUCTION
Fifteen years ago, there were relatively few examples of
persuasive technologies in our lives. The web wasn’t ubiquitous,
and software wasn’t designed to change behaviors; it was focused
more on crunching data and boosting productivity. But today
persuasive technologies are ubiquitous; we are surrounded by
digital products designed to change what we think and do.
Persuasive technology experiences come to us through the web
(from commerce sites to social networking), video games (e.g.,
Wii Fit and Dance Dance Revolution), mobile phones (e.g., health
applications for iPhone and commercial texting services), and
specialized consumer electronic device, from “talking”
pedometers to bathroom scales that track body mass.
Increasingly, the living room TV and even automobiles are
channels for persuasive experiences. For instance, TiVo not only
suggests programs to watch but integrates Netflix and encourages
customers to make purchases on Amazon. As for automobiles, one
feature of the Toyota Prius is a miles-per-gallon meter that
motivates owners to adopt more eco-friendly driving habits.
Today those of us who are interested in the design and study of
persuasive technologies have a wealth of examples from which to
choose. The existence of so many successful examples changes
the study of persuasive technology in significant ways. We no
longer have to invent new persuasive solutions out of whole cloth.
Instead, we can focus on existing persuasive technology products
and techniques, varying those systems to understand the dynamics
and principles of persuasive design. In this way, we can learn most
rapidly about the psychology of persuasion and persuasive
technology by working with existing solutions.
That said, there still will be times, either for commercial purposes
or for our own academic research, when we want to create an
entirely new persuasive technology for which there is no good
prototype. This can be a challenge, given that that many people
have little or no experience in creating products with a persuasive
goal, and our emerging field doe ...
An Experimentation Framework: How to Position for Triple Digit GrowthOptimizely
You’ve done the button color A/B test, you’ve optimized your landing pages for better conversion. What next? At B2B organizations large and small, there is still tremendous potential for experimentation to drive innovation and growth. Learn how Brion’s growth team enables rapid iteration across a variety of different domains, teams, and organizations within Cisco. With an organization of 70,000 employees and many distributed divisions, enabling experimentation can be a complex initiative. Learn the framework for upleveling from random testing to
explicit strategy to position your org for triple digit growth.
Creating Persuasive Technologies An Eight-Step Design PrCruzIbarra161
Creating Persuasive Technologies:
An Eight-Step Design Process
BJ Fogg
Persuasive Technology Lab
Stanford University
captology.stanford.edu
www.bjfogg.com
[email protected]
ABSTRACT
This paper outlines eight steps to follow as best practices in the
early stages of persuasive technology design. The eight-step
process, drawn from demonstrated successes in industry practice,
begins with defining the persuasion goal to match a target
audience with an appropriate technology channel. Subsequent
steps include imitating successful examples of persuasive design,
performing rapid trials, measuring behavioral outcomes, and
building on small successes.
General Terms
Design, Measurement, Experimentation, Human Factors.
Keywords
Persuasion, design, persuasive technology, prototyping, iteration,
behavior change, captology, behavior model.
INTRODUCTION
Fifteen years ago, there were relatively few examples of
persuasive technologies in our lives. The web wasn’t ubiquitous,
and software wasn’t designed to change behaviors; it was focused
more on crunching data and boosting productivity. But today
persuasive technologies are ubiquitous; we are surrounded by
digital products designed to change what we think and do.
Persuasive technology experiences come to us through the web
(from commerce sites to social networking), video games (e.g.,
Wii Fit and Dance Dance Revolution), mobile phones (e.g., health
applications for iPhone and commercial texting services), and
specialized consumer electronic device, from “talking”
pedometers to bathroom scales that track body mass.
Increasingly, the living room TV and even automobiles are
channels for persuasive experiences. For instance, TiVo not only
suggests programs to watch but integrates Netflix and encourages
customers to make purchases on Amazon. As for automobiles, one
feature of the Toyota Prius is a miles-per-gallon meter that
motivates owners to adopt more eco-friendly driving habits.
Today those of us who are interested in the design and study of
persuasive technologies have a wealth of examples from which to
choose. The existence of so many successful examples changes
the study of persuasive technology in significant ways. We no
longer have to invent new persuasive solutions out of whole cloth.
Instead, we can focus on existing persuasive technology products
and techniques, varying those systems to understand the dynamics
and principles of persuasive design. In this way, we can learn most
rapidly about the psychology of persuasion and persuasive
technology by working with existing solutions.
That said, there still will be times, either for commercial purposes
or for our own academic research, when we want to create an
entirely new persuasive technology for which there is no good
prototype. This can be a challenge, given that that many people
have little or no experience in creating products with a persuasive
goal, and our emerging field doe ...
An Experimentation Framework: How to Position for Triple Digit GrowthOptimizely
You’ve done the button color A/B test, you’ve optimized your landing pages for better conversion. What next? At B2B organizations large and small, there is still tremendous potential for experimentation to drive innovation and growth. Learn how Brion’s growth team enables rapid iteration across a variety of different domains, teams, and organizations within Cisco. With an organization of 70,000 employees and many distributed divisions, enabling experimentation can be a complex initiative. Learn the framework for upleveling from random testing to
explicit strategy to position your org for triple digit growth.
Creating Actionable Product Strategy by Turo Director of ProductProduct School
Main Takeaways:
- Measure what matters – Establishing the right metrics and KPIs early on can provide tremendous clarity. Driving towards the wrong goals can result in team misalignment, at best, and a failed product strategy, at worst.
- Distinguish the highest impact ideas from the good ideas
- Most companies have lots of good ideas. PMs must separate the great from the good, and craft product strategies that yield the highest impact outcomes for their customers and business.
Iterate, based on customer feedback & data – Great product strategies should evolve over time, with the ongoing incorporation of customer feedback, data, and stakeholder input. Strategies developed in a vacuum are unlikely to succeed, as are strategies that fail to evolve with the changing needs of customers.
Brent Summers, Director of Marketing at Digital Telepathy Using Data and Design toDrive Your Business June 25, 2015
Data is All Around You 1
Quantitative Data Sales Reports Data is All Around
Quantitative Data Application Performance Data Data is All Around You Quantitative Data Search Engine Optimization Data is All Around
Quantitative Web Analytics Data is All Around You
Qualitative Data Customer Surveys Data is All Around You Qualitative Data Customer Interviews Data is All Around You Get more info at: goo.gl/Jeol7v
Qualitative Data Personas Data is All Around You Get more info at: goo.gl/UW8mgQ
Observation Heat Mapping & Scroll Mapping Data is All Around You Observation User Behavior Data is All Around You
Data Already 
 Informs Design 2
A/B Testing Optimize for conversions. Data Already Informs Design
Eye Tracking People read in F-Shaped Pa erns Data Already Informs Design
Eye Tracking People look where people look. Data Already Informs Design h
Vertical Rhythm There’s a reason paper is ruled. Data Already Informs Design
Color Psychology What does your brand color say about your business?
The Golden Ratio 1.618 —
Consider the Entire 
 User Journey 3
Identify the Friction Evaluate sentiment/friction at each stage of the user journey. Consider the Entire User Journey
Designing for
 Business Objectives 4
Identify the Friction Where can you make the biggest impact? Designing for Business Objectives
User Journey Consideration
Landing Pages Incremental improvements can drive exponential results.
Be er Social Sharing Social sharing + content performance insights.
Animations Scroll is the new click.
Change Language Try different value proposition, calls to action, etc.
Change Layout Use behavior patterns to drive decisions.
User Journey Conversion: The act of purchasing a product or service through self service or a sales process.
Content Marketing Share knowledge to establish trust. Onboarding Step-by-step walkthroughs for new users.
Get the First Click Break through psychological barriers. User Journey Retention: Post-purchase. Activities that drive further product engagement, adoption and upgrades. Designing for Business Objectives
Reduce cognitive load: hide data until a user requests it.
Simplify your user interface for experienced users
Testimonials “Who doesn’t love social proof?” - Brent Summers
Prioritizing Your Backlog
Keep Track of Experiments Practical Advice Use a formula to assess which experiments to do first.
Sample Experiments Which of these experiments should be implemented Paid conversions
What does the data tell you? Identify where can design make the biggest impact.
Rounding Out the Process Your implementation method is unique. Measure the results. Repeat.
Measuring Success 6
Good Design is Great for Business Design lead firms out-perform the S&P 500 by 228%. Measuring Success
The talk has three parts : the first part gives an overview of data science work, including roadmap of data science team, responsibility and value of data scientists; the second part talks about pitfalls in analysis and teaches some common analysis methods; the third part takes decision support, metrics and AB testing as examples to explain the data science work and how they are translated to business value.
On March 30, the Corporate Learning Network held its long awaited Drucker Master Class Day – led by celebrated Drucker management guru, Dr. Bernard Jaworski, Professor at the Peter F. Drucker and professor at the Peter F. Drucker and Masatoshi Ito Graduate School of management.
skip to main contentmenuMy HomeTIM-7001 V3 Changing Times Ma.docxjennifer822
skip to main content
menu
My HomeTIM-7001 V3: Changing Times: Managing Technology & Innovation in the 21st Century (9045874881)
Select a course...
Message alertsMessage alerts - You have new alerts
Subscription alertsSubscription alerts - You have new alerts
Update alertsUpdate alerts - You have new alerts
Mala Samaroo
ProfileNotificationsAccount SettingsProgressLog OutCourse HomeContentDropboxGradesBookshelfLibraryThe CommonsCalendar
More
Course HomeContentDropboxGradesBookshelfLibraryThe CommonsCalendar
Menu Start
Menu Start
close
My HomeTIM-7001 V3: Changing Times: Managing Technology & Innovation in the 21st Century (9045874881)
Select a course...
TIM-7001 V3: Changing Times: Managing Technology & Innovation in the 21st Century (9045874881)TIM-7001 V3: Changing Times: Managing Technology & Innovation in the 21st Century (9045874881)
TIM-7001 V3: Changing Times: Managing Technology & Innovation in the 21st Century (9045874881)Course HomeContentDropboxGradesBookshelfLibraryThe CommonsCalendar
Menu End
Are You Still There?
Your session expires after 180 minutes of inactivity, which protects your information in case you've left your device without logging out.
Hit a key or click anywhere to stay logged in.
Oh, There You Are!
Side PanelExpand side panelCollapse side panel
Loading...
Breadcrumb:Table of ContentsSection 4: Technology and Innovation: ApplicationWeek 7Books and Resources for this WeekWeek 7 Assignment Signature Assignment RubricWeek 7 Assignment Signature Assignment RubricSend to BinderSubmit FeedbackDownload
PDF documentPrevious Next
YOU
NEED
AN
INNOVATION
STRATEGY
It’s the only way to make sound trade-off
decisions and choose the right practices.
BY GARY P. PISANO
THE BIG IDEA
44 Harvard Business Review June 2015
Gary P. Pisano is the
Harry E. Figgie Professor
of Business Administration
and a member of the U.S.
Competitiveness Project at
Harvard Business School.
G
U
ST
AV
O
B
R
IG
A
N
TE
HBR.ORG
June 2015 Harvard Business Review 45
DESPITE MASSIVE
INVESTMENTS OF
MANAGEMENT
TIME AND MONEY,
INNOVATION
REMAINS A
FRUSTRATING
PURSUIT
IN MANY
COMPANIES.
my more than two decades studying and consulting
for companies in a broad range of industries, I have
found that firms rarely articulate strategies to align
their innovation efforts with their business strategies.
Without an innovation strategy, innovation
improvement efforts can easily become a grab bag
of much-touted best practices: dividing R&D into
decentralized autonomous teams, spawning inter-
nal entrepreneurial ventures, setting up corporate
venture-capital arms, pursuing external alliances,
embracing open innovation and crowdsourcing,
collaborating with customers, and implementing
rapid prototyping, to name just a few. There is noth.
How To Build a Winning Experimentation Program & Team | Optimizely ANZ Webinar 8Optimizely
Watch Dan Ross, Managing Director for Optimizely ANZ in our latest webinar from the Experimentation Insights Tour -- "How To Build a Winning Experimentation Program & Team."
View the presentation here: https://optimizely.wistia.com/medias/1o6xy4j0xm
Take Optimizely's Maturity Assessment here: https://www.optimizely.com/maturity-model/
DESCRIPTION: The world’s leading companies utilise experimentation to build a culture that fosters innovation and agility. The key to experimentation is to have both the right tools (software) in combination with the right people and processes
In this webinar, you will learn:
* Why experimentation is central to competing and innovating
* Areas to assess when building your experimentation capability
* How organisational culture helps scale an experimentation program
About Optimizely:
Optimizely is the world's leading experimentation platform, enabling businesses to deliver continuous experimentation and personalisation across websites, mobile apps and connected devices. Optimizely enables businesses to experiment deeply into their technology stack and broadly across the entire customer experience.
The platform’s ease of use and speed of deployment empower organisations to create and run bold experiments that help them make data-driven decisions and grow faster.
To date, marketers, developers and product managers have delivered over 700 billion experiences tailored to the needs of their customers. Optimizely’s global client base includes Atlassian, eBay, Fox, IBM, The New York Times, LendingClub, Hotwire, Microsoft and many more leading businesses.
To learn more about customer experience optimisation, visit optimizely.com
Turning Failed Tests into Big Wins at BoxOptimizely
Last year the growth team at Box revisited a design layout from their Plans page. A page that they had been excited about redesigning in the past, but had failed to show the measurable improvements needed to key business metrics.
With a new hypothesis and experiment implementation — the team found that this redesign not only delivered more sign ups, it also made a significant improvement to their annual recurring revenue (ARR). In this webinar, Box's Renny Chan will share their philosophy of revisiting tests that have failed with a fresh perspective.
In this webinar we share:
- A new process for evaluating failed tests in your program
- How to think about the win/loss rate of your program
- New ideation tactics that will inform the way you create new experiments
Your company's identity (what you do) and implementation (how you do it) should be closely linked. Here are the precepts to keep in mind as you bring them together: Aim high. Build on your strengths. Be ambidextrous (sophisticated at both strategy and execution). Clarify everyone's strategic role. Align structures to strategy. Transcend functional barriers. Become a fully digital enterprise. Keep it simple, sometimes. Shape your value chain. And cultivate collective mastery. Do all those things, and your company will be on its way to effectively executing its strategy.
Creating Actionable Product Strategy by Turo Director of ProductProduct School
Main Takeaways:
- Measure what matters – Establishing the right metrics and KPIs early on can provide tremendous clarity. Driving towards the wrong goals can result in team misalignment, at best, and a failed product strategy, at worst.
- Distinguish the highest impact ideas from the good ideas
- Most companies have lots of good ideas. PMs must separate the great from the good, and craft product strategies that yield the highest impact outcomes for their customers and business.
Iterate, based on customer feedback & data – Great product strategies should evolve over time, with the ongoing incorporation of customer feedback, data, and stakeholder input. Strategies developed in a vacuum are unlikely to succeed, as are strategies that fail to evolve with the changing needs of customers.
Brent Summers, Director of Marketing at Digital Telepathy Using Data and Design toDrive Your Business June 25, 2015
Data is All Around You 1
Quantitative Data Sales Reports Data is All Around
Quantitative Data Application Performance Data Data is All Around You Quantitative Data Search Engine Optimization Data is All Around
Quantitative Web Analytics Data is All Around You
Qualitative Data Customer Surveys Data is All Around You Qualitative Data Customer Interviews Data is All Around You Get more info at: goo.gl/Jeol7v
Qualitative Data Personas Data is All Around You Get more info at: goo.gl/UW8mgQ
Observation Heat Mapping & Scroll Mapping Data is All Around You Observation User Behavior Data is All Around You
Data Already 
 Informs Design 2
A/B Testing Optimize for conversions. Data Already Informs Design
Eye Tracking People read in F-Shaped Pa erns Data Already Informs Design
Eye Tracking People look where people look. Data Already Informs Design h
Vertical Rhythm There’s a reason paper is ruled. Data Already Informs Design
Color Psychology What does your brand color say about your business?
The Golden Ratio 1.618 —
Consider the Entire 
 User Journey 3
Identify the Friction Evaluate sentiment/friction at each stage of the user journey. Consider the Entire User Journey
Designing for
 Business Objectives 4
Identify the Friction Where can you make the biggest impact? Designing for Business Objectives
User Journey Consideration
Landing Pages Incremental improvements can drive exponential results.
Be er Social Sharing Social sharing + content performance insights.
Animations Scroll is the new click.
Change Language Try different value proposition, calls to action, etc.
Change Layout Use behavior patterns to drive decisions.
User Journey Conversion: The act of purchasing a product or service through self service or a sales process.
Content Marketing Share knowledge to establish trust. Onboarding Step-by-step walkthroughs for new users.
Get the First Click Break through psychological barriers. User Journey Retention: Post-purchase. Activities that drive further product engagement, adoption and upgrades. Designing for Business Objectives
Reduce cognitive load: hide data until a user requests it.
Simplify your user interface for experienced users
Testimonials “Who doesn’t love social proof?” - Brent Summers
Prioritizing Your Backlog
Keep Track of Experiments Practical Advice Use a formula to assess which experiments to do first.
Sample Experiments Which of these experiments should be implemented Paid conversions
What does the data tell you? Identify where can design make the biggest impact.
Rounding Out the Process Your implementation method is unique. Measure the results. Repeat.
Measuring Success 6
Good Design is Great for Business Design lead firms out-perform the S&P 500 by 228%. Measuring Success
The talk has three parts : the first part gives an overview of data science work, including roadmap of data science team, responsibility and value of data scientists; the second part talks about pitfalls in analysis and teaches some common analysis methods; the third part takes decision support, metrics and AB testing as examples to explain the data science work and how they are translated to business value.
On March 30, the Corporate Learning Network held its long awaited Drucker Master Class Day – led by celebrated Drucker management guru, Dr. Bernard Jaworski, Professor at the Peter F. Drucker and professor at the Peter F. Drucker and Masatoshi Ito Graduate School of management.
skip to main contentmenuMy HomeTIM-7001 V3 Changing Times Ma.docxjennifer822
skip to main content
menu
My HomeTIM-7001 V3: Changing Times: Managing Technology & Innovation in the 21st Century (9045874881)
Select a course...
Message alertsMessage alerts - You have new alerts
Subscription alertsSubscription alerts - You have new alerts
Update alertsUpdate alerts - You have new alerts
Mala Samaroo
ProfileNotificationsAccount SettingsProgressLog OutCourse HomeContentDropboxGradesBookshelfLibraryThe CommonsCalendar
More
Course HomeContentDropboxGradesBookshelfLibraryThe CommonsCalendar
Menu Start
Menu Start
close
My HomeTIM-7001 V3: Changing Times: Managing Technology & Innovation in the 21st Century (9045874881)
Select a course...
TIM-7001 V3: Changing Times: Managing Technology & Innovation in the 21st Century (9045874881)TIM-7001 V3: Changing Times: Managing Technology & Innovation in the 21st Century (9045874881)
TIM-7001 V3: Changing Times: Managing Technology & Innovation in the 21st Century (9045874881)Course HomeContentDropboxGradesBookshelfLibraryThe CommonsCalendar
Menu End
Are You Still There?
Your session expires after 180 minutes of inactivity, which protects your information in case you've left your device without logging out.
Hit a key or click anywhere to stay logged in.
Oh, There You Are!
Side PanelExpand side panelCollapse side panel
Loading...
Breadcrumb:Table of ContentsSection 4: Technology and Innovation: ApplicationWeek 7Books and Resources for this WeekWeek 7 Assignment Signature Assignment RubricWeek 7 Assignment Signature Assignment RubricSend to BinderSubmit FeedbackDownload
PDF documentPrevious Next
YOU
NEED
AN
INNOVATION
STRATEGY
It’s the only way to make sound trade-off
decisions and choose the right practices.
BY GARY P. PISANO
THE BIG IDEA
44 Harvard Business Review June 2015
Gary P. Pisano is the
Harry E. Figgie Professor
of Business Administration
and a member of the U.S.
Competitiveness Project at
Harvard Business School.
G
U
ST
AV
O
B
R
IG
A
N
TE
HBR.ORG
June 2015 Harvard Business Review 45
DESPITE MASSIVE
INVESTMENTS OF
MANAGEMENT
TIME AND MONEY,
INNOVATION
REMAINS A
FRUSTRATING
PURSUIT
IN MANY
COMPANIES.
my more than two decades studying and consulting
for companies in a broad range of industries, I have
found that firms rarely articulate strategies to align
their innovation efforts with their business strategies.
Without an innovation strategy, innovation
improvement efforts can easily become a grab bag
of much-touted best practices: dividing R&D into
decentralized autonomous teams, spawning inter-
nal entrepreneurial ventures, setting up corporate
venture-capital arms, pursuing external alliances,
embracing open innovation and crowdsourcing,
collaborating with customers, and implementing
rapid prototyping, to name just a few. There is noth.
How To Build a Winning Experimentation Program & Team | Optimizely ANZ Webinar 8Optimizely
Watch Dan Ross, Managing Director for Optimizely ANZ in our latest webinar from the Experimentation Insights Tour -- "How To Build a Winning Experimentation Program & Team."
View the presentation here: https://optimizely.wistia.com/medias/1o6xy4j0xm
Take Optimizely's Maturity Assessment here: https://www.optimizely.com/maturity-model/
DESCRIPTION: The world’s leading companies utilise experimentation to build a culture that fosters innovation and agility. The key to experimentation is to have both the right tools (software) in combination with the right people and processes
In this webinar, you will learn:
* Why experimentation is central to competing and innovating
* Areas to assess when building your experimentation capability
* How organisational culture helps scale an experimentation program
About Optimizely:
Optimizely is the world's leading experimentation platform, enabling businesses to deliver continuous experimentation and personalisation across websites, mobile apps and connected devices. Optimizely enables businesses to experiment deeply into their technology stack and broadly across the entire customer experience.
The platform’s ease of use and speed of deployment empower organisations to create and run bold experiments that help them make data-driven decisions and grow faster.
To date, marketers, developers and product managers have delivered over 700 billion experiences tailored to the needs of their customers. Optimizely’s global client base includes Atlassian, eBay, Fox, IBM, The New York Times, LendingClub, Hotwire, Microsoft and many more leading businesses.
To learn more about customer experience optimisation, visit optimizely.com
Turning Failed Tests into Big Wins at BoxOptimizely
Last year the growth team at Box revisited a design layout from their Plans page. A page that they had been excited about redesigning in the past, but had failed to show the measurable improvements needed to key business metrics.
With a new hypothesis and experiment implementation — the team found that this redesign not only delivered more sign ups, it also made a significant improvement to their annual recurring revenue (ARR). In this webinar, Box's Renny Chan will share their philosophy of revisiting tests that have failed with a fresh perspective.
In this webinar we share:
- A new process for evaluating failed tests in your program
- How to think about the win/loss rate of your program
- New ideation tactics that will inform the way you create new experiments
Your company's identity (what you do) and implementation (how you do it) should be closely linked. Here are the precepts to keep in mind as you bring them together: Aim high. Build on your strengths. Be ambidextrous (sophisticated at both strategy and execution). Clarify everyone's strategic role. Align structures to strategy. Transcend functional barriers. Become a fully digital enterprise. Keep it simple, sometimes. Shape your value chain. And cultivate collective mastery. Do all those things, and your company will be on its way to effectively executing its strategy.
www.elsevier.comlocatecompstrucComputers and Structures .docxjeffevans62972
www.elsevier.com/locate/compstruc
Computers and Structures 85 (2007) 235–243
On the treatment of uncertainties in structural mechanics and analysis q
G.I. Schuëller *
Institute of Engineering Mechanics, Leopold-Franzens University Innsbruck, Technikerstr. 13, 6020 Innsbruck, Austria
Received 9 August 2006; accepted 31 October 2006
Available online 22 December 2006
Abstract
In this paper the need for a rational treatment of uncertainties in structural mechanics and analysis is reasoned. It is shown that the
traditional deterministic conception can be easily extended by applying statistical and probabilistic concepts. The so-called Monte Carlo
simulation procedure is the key for those developments, as it allows the straightforward use of the currently used deterministic analysis
procedures.
A numerical example exemplifies the methodology. It is concluded that uncertainty analysis may ensure robust predictions of vari-
ability, model verification, safety assessment, etc.
� 2006 Elsevier Ltd. All rights reserved.
Keywords: Uncertainty; Monte Carlo simulaton; Finite elements; Response variability; Model verification; Robustness
1. Introduction
Structural mechanics analysis up to this date, generally is
still based on a deterministic conception. Observed varia-
tions in loading conditions, material properties, geometry,
etc. are taken into account by either selecting extremely
high, low or average values, respectively, for representing
the parameters. Hence, this way, uncertainties inherent in
almost every analysis process are considered just intuitively.
Observations and measurements of physical processes,
however, show not only variability, but also random char-
acteristics. Statistical and probabilistic procedures provide
a sound frame work for a rational treatment of analysis
of these uncertainties. Moreover there are various types of
uncertainties to be dealt with. While the uncertainties in
mechanical modeling can be reduced as additional knowl-
edge becomes available, the physical or intrinsic uncertain-
ties, e.g. of environmental loading, can not. Furthermore,
0045-7949/$ - see front matter � 2006 Elsevier Ltd. All rights reserved.
doi:10.1016/j.compstruc.2006.10.009
q Plenary Keynote Lecture presented at the 3rd MIT Conference on
Computational Fluid and Solid Mechanics, Boston, MA, USA, June 14–
17, 2005.
* Tel.: +43 512 507 6841; fax: +43 512 507 2905.
E-mail address: [email protected]
the entire spectrum of uncertainties is also not known. In
reality, neither the true model nor the model parameters
are deterministically known. Assuming that by finite ele-
ment (FE) procedures structures and continua can be repre-
sented reasonably well the question of the effect of the
discretization still remains. It is generally expected, that
an increase in the size of the structural models, in terms of
degrees of freedom, will increase the level of realism of the
model. Comparisons with measurements, however, clearly
show that this expect.
www.ebook3000.comList of Cases by ChapterChapter 1.docxjeffevans62972
www.ebook3000.com
List of Cases by Chapter
Chapter 1
Development Projects in Lagos, Nigeria 2
“Throwing Good Money after Bad”: the BBC’s
Digital Media Initiative 10
MegaTech, Inc. 29
The IT Department at Hamelin Hospital 30
Disney’s Expedition Everest 31
Rescue of Chilean Miners 32
Chapter 2
Tesla’s $5 Billion Gamble 37
Electronic Arts and the Power of Strong Culture
in Design Teams 64
Rolls-Royce Corporation 67
Classic Case: Paradise Lost—The Xerox Alto 68
Project Task Estimation and the Culture of “Gotcha!” 69
Widgets ’R Us 70
Chapter 3
Project Selection Procedures: A Cross-Industry
Sampler 77
Project Selection and Screening at GE: The Tollgate
Process 97
Keflavik Paper Company 111
Project Selection at Nova Western, Inc. 112
Chapter 4
Leading by Example for the London Olympics—
Sir John Armitt 116
Dr. Elattuvalapil Sreedharan, India’s Project
Management Guru 126
The Challenge of Managing Internationally 133
In Search of Effective Project Managers 137
Finding the Emotional Intelligence to Be a Real Leader 137
Problems with John 138
Chapter 5
“We look like fools.”—Oregon’s Failed Rollout
of Its ObamacareWeb Site 145
Statements of Work: Then and Now 151
Defining a Project Work Package 163
Boeing’s Virtual Fence 172
California’s High-Speed Rail Project 173
Project Management at Dotcom.com 175
The Expeditionary Fighting Vehicle 176
Chapter 6
Engineers Without Borders: Project Teams Impacting
Lives 187
Tele-Immersion Technology Eases the Use of Virtual
Teams 203
Columbus Instruments 215
The Bean Counter and the Cowboy 216
Johnson & Rogers Software Engineering, Inc. 217
Chapter 7
The Building that Melted Cars 224
Bank of America Completely Misjudges Its Customers 230
Collapse of Shanghai Apartment Building 239
Classic Case: de Havilland’s Falling Comet 245
The Spanish Navy Pays Nearly $3 Billion for a Submarine
That Will Sink Like a Stone 248
Classic Case: Tacoma Narrows Suspension Bridge 249
Chapter 8
Sochi Olympics—What’s the Cost of National
Prestige? 257
The Hidden Costs of Infrastructure Projects—The Case
of Building Dams 286
Boston’s Central Artery/Tunnel Project 288
Chapter 9
After 20 Years and More Than $50 Billion, Oil is No Closer
to the Surface: The Caspian Kashagan Project 297
Chapter 10
Enlarging the Panama Canal 331
Project Scheduling at Blanque Cheque Construction (A) 360
Project Scheduling at Blanque Cheque Construction (B) 360
Chapter 11
Developing Projects Through Kickstarter—Do Delivery
Dates Mean Anything? 367
Eli Lilly Pharmaceuticals and Its Commitment to Critical
Chain Project Management 385
It’s an Agile World 396
Ramstein Products, Inc. 397
Chapter 12
Hong Kong Connects to the World’s Longest Natural
Gas Pipeline 401
The Problems of Multitasking 427
Chapter 13
New York City’s CityTime Project 432
Earned Value at Northrop Grumman 451
The IT Department at Kimble College 463
The Superconducting Supercollider 464
Boeing’s 787 Dreamliner: Failure to Launch 465
Chapter 14.
www.AEP-Arts.org | @AEP_Arts
EDUCATION TRENDS www.ecs.org | @EdCommission
TUNE IN.
Explore emerging
education developments.
SEPT 2017
ESSA creates
flexibility allowing
states and
schools to more
fully explore and
leverage the arts in
K-12 teaching and
learning.
Research
indicates that
deeper learning
skills contribute
significantly
to a student’s
college, career
and citizenship
readiness.
Thirty years ago, in response to a K-12
public education system defined by
mediocrity1, with low student test scores
and widening gaps in achievement, the
accountability movement was born.
Federal and state education policies
focused on raising standards and
regularly assessing students. However,
over the years, many policymakers
and the public observed a connection
between the accountability movement
and an overemphasis on testing in
core subjects, such as English and
math, a narrowing of curricula and the
elimination of many important subjects,
including the arts.
Arts education
fosters critical deeper
learning skills, such
as collaboration and
perseverance, in
students.
Yet, research consistently shows that
arts education and the integration of
the arts into core subjects can have
dramatic effects on student success
— defined not just by student test
scores, but also critical skills, such as
creativity, teamwork and perseverance.
Research indicates that these skills
can be as effective predictors of long-
term success in college, careers and
citizenship as test scores.2,3
The Every Student Succeeds Act
(ESSA), which passed in late 2015, is
the first major federal law in more than
30 years offering states a significant
degree of flexibility to broaden —
rather than narrow — curricula, and
strongly encourages states to ensure all
students have access to a well-rounded
education, which includes the arts
and music.4 Armed with the evidence
presented in this report highlighting
the impressive effects education in and
through the arts can have on student
Beyond the Core: Advancing
student success through the arts
EMILY WORKMAN
EDUCATION
TRENDS
www.AEP-Arts.org | @AEP_Arts
2
EDUCATION TRENDS www.ecs.org | @EdCommission
success, state policymakers have an opportunity and
incentive to take advantage of the flexibility awarded
under ESSA related to the arts.
“Despite [deeper learning] skills’
central roles in our education and,
more broadly, our lives, education
policy has tended to overlook their
importance.”5
Bolstering Deeper
Learning Through Arts in
Education
Deeper Learning
The arts — including dance, music, theatre, media arts
and visual arts — bolster the development of what are
commonly referred to as deeper learning skills. Deeper
learning is an umbrella term defining the skills and
knowledge students need to attain success in college,
career and citizenship. Students that possess deeper
learning skills6:
1. Master core academic content.
2. Think criti.
wsb.to&NxQXpTHEME Leading with LoveAndreas J. Kӧste.docxjeffevans62972
wsb.to/&NxQXp
THEME: Leading with Love
Andreas J. Kӧstenberger & David Crowther
Introduction
At the outset of this chapter, it should be frankly acknowledged that the Johannine Letters were not originally intended primarily to provide a theology of leadership. Nevertheless, a closer examination of these three letters reveals the way in which the author relates to and provides leadership for the people in the congregations to which the letters are written. The author’s relationship with his recipients in these three letters does not directly correspond to a modern model of leadership because of his unique role in the churches to which he is writing. Yet his faithful and caring relationship can provide an example to Christian leaders in every age. In order to grasp the lessons on leadership in the Johannine Epistles, one must consider the identity of the author of these letters, the source of his authority, his relationship with his audience, and the nature of the conflict addressed in his third letter.
Original Setting
The Authorship of the Letters
The author of 1, 2 and 3 John is never named except for the title “elder” in 2 and 3 John. The early church accepted all three letters into the canon in the belief that John the apostle, the son of Zebedee, was the author.[1] While the author of these letters was doubtless known to his initial readers, the modern reader is indebted to the early church for preserving the tradition of authorship. Sources from the late second and early third centuries, such as the Muratorian Fragment (c. ad 180) and church fathers Tertullian (c. ad 160–215) and Clement of Alexandria (c. ad 155–220), ascribe authorship to John the son of Zebedee.
However, not only the external but also the internal evidence points to Johannine authorship. First, in 1 John 1:1–4 the author claims to be an eyewitness of Jesus. Although the first-person plural reference (“we”) in the author’s description of what he has heard, seen, and touched may include his audience because they share in the tradition that was handed down (alternatively, the reference is to the apostles; cf. John 1:14; 2:11), there is a clear distinction between the author and his recipients with regard to their firsthand knowledge of Jesus (cf.1 John 1:2–3). While the author may use the first-person plural reference to identify with his audience, 1 John 1:1–2 indicates that the author is a personal eyewitness of the incarnate Christ.[2]
Second, all three of the Johannine letters contain similar vocabulary, style, and theology. In fact, the relationship between the letters is so strong that the majority of modern scholars view them as coming from one author—albeit not all agree that their author is the same as the author of the Fourth Gospel.[3] For instance, among the Johannine letters one can identify a common background in which itinerant teachers with competing theological agendas threatened the confession of the Johannine churches.[4] In response to such threa.
WSJ Executive Adviser (A Special Report) TheCase Against .docxjeffevans62972
WSJ Executive Adviser (A Special Report): The
Case Against Corporate Social Responsibility:
The idea that companies have a duty to address
social ills is not just flawed, argues Aneel
Karnani; It also makes it more likely that we'll
ignore the real solutions to these problems
Karnani, Aneel . Wall Street Journal , Eastern edition; New York, N.Y. [New York, N.Y]23 Aug 2010: R.1.
ProQuest document link
ABSTRACT
[...] the fact is that while companies sometimes can do well by doing good, more often they can't. Because in most
cases, doing what's best for society means sacrificing profits.
FULL TEXT
Can companies do well by doing good? Yes -- sometimes.
But the idea that companies have a responsibility to act in the public interest and will profit from doing so is
fundamentally flawed.
Large companies now routinely claim that they aren't in business just for the profits, that they're also intent on
serving some larger social purpose. They trumpet their efforts to produce healthier foods or more fuel-efficient
vehicles, conserve energy and other resources in their operations, or otherwise make the world a better place.
Influential institutions like the Academy of Management and the United Nations, among many others, encourage
companies to pursue such strategies.
It's not surprising that this idea has won over so many people -- it's a very appealing proposition. You can have
your cake and eat it too!
But it's an illusion, and a potentially dangerous one.
Very simply, in cases where private profits and public interests are aligned, the idea of corporate social
responsibility is irrelevant: Companies that simply do everything they can to boost profits will end up increasing
social welfare. In circumstances in which profits and social welfare are in direct opposition, an appeal to corporate
social responsibility will almost always be ineffective, because executives are unlikely to act voluntarily in the
public interest and against shareholder interests.
Irrelevant or ineffective, take your pick. But it's worse than that. The danger is that a focus on social responsibility
will delay or discourage more-effective measures to enhance social welfare in those cases where profits and the
public good are at odds. As society looks to companies to address these problems, the real solutions may be
ignored.
http://ezproxy.library.berkeley.org/login?qurl=https%3A%2F%2Fsearch.proquest.com%2Fdocview%2F746396923%3Faccountid%3D38129
http://ezproxy.library.berkeley.org/login?qurl=https%3A%2F%2Fsearch.proquest.com%2Fdocview%2F746396923%3Faccountid%3D38129
To get a better fix on the irrelevance or ineffectiveness of corporate social responsibility efforts, let's first look at
situations where profits and social welfare are in synch.
Consider the market for healthier food. Fast-food outlets have profited by expanding their offerings to include
salads and other options designed to appeal to health-conscious consu.
WRTG 293 students, Your first writing assignment will be .docxjeffevans62972
WRTG 293 students,
Your first writing assignment will be to rewrite a set of instructions. The scenario for this
assignment is described below.
________________________
You have just taken a position as a student worker for the Communications Arts Department at
Anderson College. You began your job last week.
Anderson College has an enrollment of 10,000 students. Among this student population, 20% of
the students are international students for whom English is not a native language, 10% of the
students are dual-enrollment high school students, 20% of the students are graduate students, and
the remaining 50% of the student population consists of a mixture of adult learners and
traditional students.
Anderson adopted LEO as its learning management system two years ago. Anderson uses LEO
for both its online classes and its hybrid classes.
Since moving to LEO, Dr. Richard Johnson, Dean of the Undergraduate School at Anderson, and
Dr. Lynn Peterson, Dean of the Graduate School at Anderson, have noticed that both students
taking classes at Anderson and instructors teaching at Anderson are often not aware of the
different settings one can choose to view discussions in LEO. This lack of awareness has caused
confusion and frustration as students and faculty members have attempted to navigate through
the discussions in their classes.
Dr. Johnson and Dr. Peterson tried to address this problem two months ago. At that time, they
asked the previous student worker to write instructions on how to change the settings for
discussions in LEO for the optimal viewing arrangement.
The previous student worker wrote some instructions. However, the worker wrote them very
unprofessionally and poorly. They cannot be distributed to students in their current form.
Moreover, shortly after the student worker finished the instructions, he left his position for
another job.
As a result, Anderson College now has a set of poorly designed instructions that it cannot send
out to students and faculty members. Meanwhile, students and faculty members are still
experiencing frustration with the system, and they need a document that guides them through
how to adjust their settings in LEO for viewing discussions.
Dr. Johnson, who is your immediate supervisor, has now asked you, the new student worker, to
rewrite the instructions that the previous student worker wrote. He has asked you to use the
same graphics the previous student worker used. He has also suggested that you use arrows to
point to sections of the graphics if such arrows can help in understanding specific steps in the
instructions.
Keep in mind that potentially 10,000 students will be using the instructions, in addition to
various faculty members. The instructions should be clear, professional, and well designed.
Moreover, you will want to consider the different types of students at Anderson College,
including their backgrounds and their var.
Writtenn papers include the following minimum elementsCompany.docxjeffevans62972
Writtenn papers include the following minimum elements:
Company Background
Evaluation of the Supply Chain Processes
Drivers of Supply Chain Performance
Network Design
Risk Mitigation within the Supply Chain
Forecasting Practices
Sales & Operations Planning
Inventory Management Practices
Use of Transportation
Decisions in Sourcing
Use of Information Technology for Supply Chain Optimization
Supply Chain Sustainability with Learning Outcomes & Recommendations
.
The French Revolution, which began in 1789, was a period of radical social and political upheaval in France. It marked the decline of absolute monarchies, the rise of secular and democratic republics, and the eventual rise of Napoleon Bonaparte. This revolutionary period is crucial in understanding the transition from feudalism to modernity in Europe.
For more information, visit-www.vavaclasses.com
How to Make a Field invisible in Odoo 17Celine George
It is possible to hide or invisible some fields in odoo. Commonly using “invisible” attribute in the field definition to invisible the fields. This slide will show how to make a field invisible in odoo 17.
Unit 8 - Information and Communication Technology (Paper I).pdfThiyagu K
This slides describes the basic concepts of ICT, basics of Email, Emerging Technology and Digital Initiatives in Education. This presentations aligns with the UGC Paper I syllabus.
How to Create Map Views in the Odoo 17 ERPCeline George
The map views are useful for providing a geographical representation of data. They allow users to visualize and analyze the data in a more intuitive manner.
We all have good and bad thoughts from time to time and situation to situation. We are bombarded daily with spiraling thoughts(both negative and positive) creating all-consuming feel , making us difficult to manage with associated suffering. Good thoughts are like our Mob Signal (Positive thought) amidst noise(negative thought) in the atmosphere. Negative thoughts like noise outweigh positive thoughts. These thoughts often create unwanted confusion, trouble, stress and frustration in our mind as well as chaos in our physical world. Negative thoughts are also known as “distorted thinking”.
Students, digital devices and success - Andreas Schleicher - 27 May 2024..pptxEduSkills OECD
Andreas Schleicher presents at the OECD webinar ‘Digital devices in schools: detrimental distraction or secret to success?’ on 27 May 2024. The presentation was based on findings from PISA 2022 results and the webinar helped launch the PISA in Focus ‘Managing screen time: How to protect and equip students against distraction’ https://www.oecd-ilibrary.org/education/managing-screen-time_7c225af4-en and the OECD Education Policy Perspective ‘Students, digital devices and success’ can be found here - https://oe.cd/il/5yV
Synthetic Fiber Construction in lab .pptxPavel ( NSTU)
Synthetic fiber production is a fascinating and complex field that blends chemistry, engineering, and environmental science. By understanding these aspects, students can gain a comprehensive view of synthetic fiber production, its impact on society and the environment, and the potential for future innovations. Synthetic fibers play a crucial role in modern society, impacting various aspects of daily life, industry, and the environment. ynthetic fibers are integral to modern life, offering a range of benefits from cost-effectiveness and versatility to innovative applications and performance characteristics. While they pose environmental challenges, ongoing research and development aim to create more sustainable and eco-friendly alternatives. Understanding the importance of synthetic fibers helps in appreciating their role in the economy, industry, and daily life, while also emphasizing the need for sustainable practices and innovation.
2024.06.01 Introducing a competency framework for languag learning materials ...Sandy Millin
http://sandymillin.wordpress.com/iateflwebinar2024
Published classroom materials form the basis of syllabuses, drive teacher professional development, and have a potentially huge influence on learners, teachers and education systems. All teachers also create their own materials, whether a few sentences on a blackboard, a highly-structured fully-realised online course, or anything in between. Despite this, the knowledge and skills needed to create effective language learning materials are rarely part of teacher training, and are mostly learnt by trial and error.
Knowledge and skills frameworks, generally called competency frameworks, for ELT teachers, trainers and managers have existed for a few years now. However, until I created one for my MA dissertation, there wasn’t one drawing together what we need to know and do to be able to effectively produce language learning materials.
This webinar will introduce you to my framework, highlighting the key competencies I identified from my research. It will also show how anybody involved in language teaching (any language, not just English!), teacher training, managing schools or developing language learning materials can benefit from using the framework.
Instructions for Submissions thorugh G- Classroom.pptxJheel Barad
This presentation provides a briefing on how to upload submissions and documents in Google Classroom. It was prepared as part of an orientation for new Sainik School in-service teacher trainees. As a training officer, my goal is to ensure that you are comfortable and proficient with this essential tool for managing assignments and fostering student engagement.
Model Attribute Check Company Auto PropertyCeline George
In Odoo, the multi-company feature allows you to manage multiple companies within a single Odoo database instance. Each company can have its own configurations while still sharing common resources such as products, customers, and suppliers.
www.hbr.orgHow to Design Smart Business Experiments.docx
1. www.hbr.org
How to Design Smart
Business Experiments
by Thomas H. Davenport
Included with this full-text
Harvard Business Review
article:
Idea in Brief—the core idea
Idea in Practice—putting the idea to work
1
Article Summary
2
2. How to Design Smart Business Experiments
Managers now have the tools
to conduct small-scale tests
and gain real insight. But too
many “experiments” don’t
prove much of anything.
Reprint R0902E
For the exclusive use of L. WATSON, 2019.
This document is authorized for use only by LANA WATSON in
2019.
http://hbr.org/search/R0902E
http://www.hbr.org
How to Design Smart Business Experiments
page 1
Idea in Brief Idea in Practice
6. expect truly valid tests.
•
With a small investment in training,
readily available software, and the right
encouragement, an organization can
build a “test and learn” capability.
•
Companies that equip managers to per-
form small-scale yet rigorous experi-
ments don’t only save themselves from
expensive mistakes—they also make it
more likely that great ideas will see the
light of day.
You or someone on your team is suggesting a change that just
might work. But why act on a
hunch when you can hold out for evidence? According to the
author, the best way to support
decision making on potential innovations is to...
•
Design an experiment.
Start with a hypothesis about how the change
7. will help the business. If it’s a good one, you’ll
learn as much by disproving it as you would
by proving it. Put it to the test by measuring
what happens in a test group versus a control
group. From the outset, be clear on what you
need to measure to produce a decisive re-
sult—and whether that’s a metric you even
have the capability to track.
•
Act on the facts.
Nothing but a success in a testing environ-
ment should be rolled out more broadly. But
neither should failures simply be scrapped.
Refine the hypothesis on the basis of the re-
sults, and consider testing a variation. Most
important, capture what’s been learned, and
make it available to others in the organization
through a “learning library,” so resources aren’t
wasted proving the same thing again.
Example:
Marketers at the Subway restaurant chain
wanted to drum up business by putting
foot-long subs on sale for only $5, but fran-
chise owners worried that the promotion
would lure existing customers away from
higher-priced menu items. An experiment
pitting test sites against control sites
8. proved that the promotion would pay off—
which it subsequently did.
•
Make testing the norm.
Create the training and infrastructure that will
enable nonexperts in statistics to oversee rig-
orous experiments. Off-the-shelf software can
walk them through the steps and help them
analyze results. A core group of experts can
lend resources and expertise and maintain the
learning library. Leadership must cultivate a
test-and-learn culture, in part by penalizing
those who act without sufficient evidence.
As your managers become more comfortable
with testing, they’ll discover that it paves the
way for, rather than throwing up barriers to,
promising new ideas.
For the exclusive use of L. WATSON, 2019.
This document is authorized for use only by LANA WATSON in
2019.
How to Design Smart
Business Experiments
12. Managers now have the tools to conduct small-scale tests and
gain real
insight. But too many “experiments” don’t prove much of
anything.
Every day, managers in your organization take
steps to implement new ideas without having
any real evidence to back them up. They fiddle
with offerings, try out distribution ap-
proaches, and alter how work gets done, usu-
ally acting on little more than gut feel or seem-
ing common sense—“I’ll bet this” or “I think
that.” Even more disturbing, some wrap their
decisions in the language of science, creating
an illusion of evidence. Their so-called experi-
ments aren’t worthy of the name, because
they lack investigative rigor. It’s likely that the
resulting guesses will be wrong and, worst of
all, that very little will have been learned in
the process.
Take the example of a major retail bank that
set the goal of improving customer service. It
embarked on a program hailed as scientific:
Some branches were labeled “laboratories”;
the new approaches being tried were known as
“experiments.” Unfortunately, however, the
methodology wasn’t as rigorous as the rhetoric
implied. Eager to try out a variety of ideas, the
bank changed many things at once in its “labs,”
making it difficult if not impossible to deter-
mine what was really driving any improved re-
13. sults. Branches undergoing interventions
weren’t matched to control sites for the most
part, so no one could say for sure that the out-
comes noted wouldn’t have happened anyway.
Anxious to head off criticism, managers did
provide a control in one test, which was de-
signed to see if placing video screens showing
television news over waiting lines would
shorten customers’ perceived waiting time.
But rather than looking at control and test
groups, they compared just one control site
with one test site. That wasn’t enough to en-
sure statistically valid results. Perceived wait-
ing time did drop in the test branch, but it
went up substantially in the control branch,
despite no changes there. Those confounding
data kept the test from being at all conclu-
sive—but that’s not how the findings were pre-
sented to top management.
It doesn’t have to be this way. Thanks to
new, broadly available software and given
some straightforward investments to build ca-
For the exclusive use of L. WATSON, 2019.
This document is authorized for use only by LANA WATSON in
2019.
How to Design Smart Business Experiments
harvard business review • february 2009 page 3
14. pabilities, managers can now base consequen-
tial decisions on scientifically valid experi-
ments. Of course, the scientific method is not
new, nor is its application in business. The
R&D centers of firms ranging from biscuit bak-
ers to drug makers have always relied on it, as
have direct-mail marketers tracking response
rates to different permutations of their pitches.
To apply it outside such settings, however, has
until recently been a major undertaking. Any
foray into the randomized testing of manage-
ment ideas—that is, the random assignment of
subjects to test and control groups—meant
employing or engaging a PhD in statistics or
perhaps a “design of experiments” expert
(sometimes seen in advanced TQM programs).
Now, a quantitatively trained MBA can over-
see the process, assisted by software that will
help determine what kind of samples are nec-
essary, which sites to use for testing and con-
trols, and whether any changes resulting from
experiments are statistically significant.
Consumer-facing companies rich in transac-
tion data are already routinely testing innova-
tions well outside the realm of product R&D.
They include banks such as PNC, Toronto-
Dominion, and Wells Fargo; retailers such as
CKE Restaurants, Famous Footwear, Food
Lion, Sears, and Subway; and online firms such
as Amazon, eBay, and Google. As randomized
testing becomes standard procedure in certain
settings—website analysis, for instance—firms
build the capabilities to apply it in other cir-
15. cumstances as well. (See the sidebar “Stop
Wondering” for a sampling of tests conducted
recently.) To be sure, there remain many busi-
ness situations where it is not easy or practical
to structure a scientifically valid experiment.
But while the “test and learn” approach might
not always be appropriate (no management
method is), it will doubtless gain ground over
time. Will it do so in your organization? If it’s
like many companies I have studied, an invest-
ment in software and training will yield quick
returns of the low-hanging-fruit variety. The
real payoff, however, will happen when the or-
ganization as a whole shifts to a test-and-learn
mind-set.
When Testing Makes Sense
Formalized testing can provide a level of un-
derstanding about what really works that puts
more intuitive approaches to shame. In the-
ory, it makes sense for any part of the business
in which variation can lead to differential re-
sults. In practice, however, there are times
when a test is impossible or unnecessary.
Some new offerings simply can’t be tested on a
small scale. When Best Buy, for example, ex-
plored partnering with Paul McCartney on an
exclusively marketed CD and a sponsored con-
cert tour, neither component of the promo-
tion could be tested on a small scale, so the
company’s managers went with their intu-
ition. At Toronto-Dominion, one of the largest
16. and most profitable banks in Canada, testing is
so well established that occasionally managers
are reminded that, in the interests of speed,
they can make the call without a test when
they have a great deal of experience in the rel-
evant business domain.
Generally speaking, the triumphs of testing
occur in strategy execution, not strategy for-
mulation. Whether in marketing, store or
branch location analysis, or website design, the
most reliable insights relate to the potential
impact and value of tactical changes: a new
store format, for example, or marketing pro-
motion or service process. Scientific method is
not well suited to assessing a major change in
business models, a large merger or acquisition,
or some other game-changing decision.
Capital One’s experience hints at the natural
limits of experimental testing in a business. The
company has been one of the world’s most ag-
gressive testers since 1988, when its CEO and
cofounder, Rich Fairbank, joined its predeces-
sor firm, Signet Bank. You could even say the
firm was founded on the concept. One thing
that appealed to Fairbank about the credit card
industry was its “ability to turn a business into a
scientific laboratory where every decision
about product design, marketing, channels of
communication, credit lines, customer selec-
tion, collection policies and cross-selling deci-
sions could be subjected to systematic testing
using thousands of experiments.”
17. 1
Capital One
adopted what Fairbank calls an information-
based strategy, and it paid off: The company be-
came the fifth-largest provider of credit cards in
the United States.
Yet when it came time to make the largest
decision the company had faced in recent
years, Capital One’s management concluded
that testing would not be useful. Realizing that
the business would need other sources of capi-
tal to remain independent, the team consid-
ered acquiring some regional banks in order to
Thomas H. Davenport
([email protected]
babson.edu) is the President’s Distin-
guished Professor of Information Tech-
nology and Management at Babson
College in Babson Park, Massachusetts.
His newest book is
Competing on Ana-
lytics: The New Science of Winning
, with
Jeanne G. Harris (Harvard Business
Press, 2007).
18. For the exclusive use of L. WATSON, 2019.
This document is authorized for use only by LANA WATSON in
2019.
mailto:[email protected]
mailto:[email protected]
How to Design Smart Business Experiments
harvard business review • february 2009 page 4
The real payoff will
happen when the
organization as a whole
shifts to a test-and-learn
mind-set.
transform itself from a monoline credit pro-
vider into a full-service bank. The decision was
not tested for a couple of important reasons.
First, the nature of the opportunity made it im-
perative to move quickly; no time was avail-
able for even a small-scale test. Second, and
more critical, it was impossible to design an ex-
periment that could reliably predict the out-
comes of such a major change in business di-
19. rection. Still, after making the acquisitions,
Capital One reaffirmed its commitment to
information-based strategy. Its managers im-
mediately set about translating that ethos into
the full-service banking context, which re-
quired pushing the method further, into tests
involving customer service and employee be-
havior. As one employee told me, “It’s much
easier to do randomized testing with direct-
mail envelopes than with branch bankers.”
Sears Holdings provides another example of
what can reasonably be tested and what can’t.
Interestingly, this is another business with a
heritage of testing. Robert E. Wood, who origi-
nally moved Sears out of the catalog business
and into retail stores, said his favorite book was
the
Statistical Abstract of the United States
.
When he opened Sears’s first free-standing re-
tail stores, in 1928, he placed two in Chicago.
Asked why he needed two in one city, Wood
said it was to reduce the risk of choosing a
wrong location or store manager.
Today Sears Holdings has embarked upon a
new era: Its primary owner, financier Edward
Lampert, who has been its chairman since
Kmart acquired Sears, is exploring alternative
ways to combine the two troubled chains. To
my knowledge, Lampert didn’t test the idea of
20. combining the retailers. That would have been
difficult if not impossible to do (and the jury is
still out on whether the acquisition was a good
decision). However, he’s a strong advocate of
testing at the tactical level. He wrote in a 2006
letter to shareholders, “One of the great advan-
tages of having approximately 2,300 large-
format stores at Sears Holdings is that we can
test concepts in a few stores before undertak-
ing the risk and capital associated with rolling
out the concept to a larger number of stores or
to the entire chain.” The retailer has tested, for
example, various formats for including Sears
merchandise in Kmart stores, and vice versa, as
well as other formats, such as the arrangement
of merchandise in Sears stores by rooms in a
consumer’s home (kitchen, laundry room, bed-
room, and so on).
Beyond using the tactical-versus-strategic cri-
terion, there are other ways to decide whether
formal testing makes sense. For instance, it is
useful only in situations where desired out-
comes are defined and measurable. A new
sales training program might be proposed, but
before you can test its efficacy, you’ll need to
identify a goal (such as “We want to increase
cross-selling”), and you must be able to mea-
sure that change (do you even track cross-
selling?). Sales and conversion-rate changes are
frequently used as dependent variables in tests
and are reliably measured for separate pur-
poses. Other outcomes, such as customer satis-
faction and employee engagement, may re-
quire more effort and invasiveness to measure.
21. Tests are most reliable where many roughly
equivalent settings can be observed. This might
mean physical sites, as with Sears’s stores, or it
might mean more ephemeral settings, such as
alternative website versions. Among the earli-
est and most extensive users of testing are retail
and restaurant chains. Because so much is held
constant among their multitudinous sites, it is
easy to designate which ones will serve as ex-
periments and which will serve as controls and
to attribute cause to effect. By the same token,
workplace design changes are most readily
tested in companies that have offices in many
cities. Drawing statistical inferences from small
numbers of test sites is much more difficult and
represents the leading edge of the test-and-
learn approach.
Finally, formal testing makes sense only if a
logical hypothesis has been formulated about
how a proposed intervention will affect a busi-
ness. Although it’s possible to just make a
change and then sit back and observe what
happens, that process will inevitably lead to a
hypothesis—and often the realization that it
could have been formulated in advance and
tested more precisely.
The Process of Testing
To begin incorporating more scientific man-
agement into your business, you’ll need to ac-
quaint managers at all levels with your organi-
22. zation’s process of testing. It is probably
simple to grasp (a typical depiction is shown in
the exhibit “Put Your Ideas to the Test”), but it
must be communicated in the same terms to
people across the organization. Having a
shared understanding of what constitutes a
For the exclusive use of L. WATSON, 2019.
This document is authorized for use only by LANA WATSON in
2019.
How to Design Smart Business Experiments
harvard business review • february 2009 page 5
valid test enables the innovators to deliver on
it and the senior executives to demand it.
The process always begins with the creation
of a testable hypothesis. (It should be possible
to pass or fail the test based on the measured
goals of the hypothesis.) Then the details of
the test are designed, which means identifying
sites or units to be tested, selecting the control
groups, and defining the test and control situa-
tions. After the test is carried out for the speci-
fied period—which sometimes can take several
months but is usually done in less time—the
data are analyzed to determine the results and
appropriate actions. The results are ideally put
23. into some sort of “learning library” (although,
unfortunately, many organizations skip this
step). They might lead to a wider rollout of the
experiment or further testing of a revised
hypothesis.
More broadly, managers must understand
how the testing process fits in with other busi-
ness processes. They conduct tests in the context
of, for example, order management, or site se-
lection, or website development, and the testing
feeds into various subprocesses. At CKE Restau-
rants, which includes the Hardee’s and Carl’s Jr.
quick-service restaurant chains, the process for
new product introduction calls for rigorous test-
ing at a certain stage. It starts with brainstorm-
ing, in which several cross-functional groups de-
velop a variety of new product ideas. Only some
of them make it past the next phase, judgmen-
tal screening, during which a group of market-
ing, product development, and operations peo-
ple will evaluate ideas based on experience and
intuition. Those that make the cut are actually
developed and then tested in stores, with well-
defined measures and control groups. At that
point, executives decide whether to roll out a
product systemwide, modify it for retesting, or
kill the whole idea.
CKE has attained an enviable hit rate in new
product introductions—about one in four new
products is successful, versus one in 50 or 60
for consumer products—and executives say
that their rigorous testing process is part of the
reason why. If you have had occasion to enjoy
24. a Monster Thickburger at Hardee’s, or a Philly
Cheesesteak Burger or a Pastrami Burger at
Carl’s Jr., you’ve been the beneficiary of CKE’s
efforts. These are just three of the successful
new products that were rolled out after testing
proved they would sell well.
At eBay, there is an overarching process for
making website changes, and randomized test-
ing is a key component. Like other online busi-
nesses, eBay benefits greatly from the fact that
it is relatively easy to perform randomized
tests of website variations. Its managers have
conducted thousands of experiments with dif-
ferent aspects of its website, and because the
site garners over a billion page views per day,
they are able to conduct multiple experiments
concurrently and not run out of treatment and
control groups. Simple A/B experiments (com-
paring two versions of a website) can be struc-
tured within a few days, and they typically last
at least a week so that they cover full auction
periods for selected items. Larger, multivariate
experiments may run for more than a month.
Online testing at eBay follows a well-defined
process that consists of the following steps:
• Hypothesis development
• Design of the experiment: determining
test samples, experimental treatments, and
other factors
• Setup of the experiment: assessing costs,
determining how to prototype, ensuring fit
25. with the site’s performance (for example, mak-
ing sure the testing doesn’t slow down user re-
sponse time)
• Launch of the experiment: figuring out
how long to run it, serving the treatment to
users
• Tracking and monitoring
Stop Wondering
Testing is used to make tactical decisions in a range of business
settings, from banks
to retailers to dot-coms. Here are some questions various
companies are examining:
•
Do lobster tanks increase lobster
sales at Food Lion supermarkets?
•
Does a Kmart with a Sears store
inside sell more than an all-Kmart
format?
•
26. Do eBay users bid higher in auc-
tions when they can pay by credit
card?
•
What’s the optimum number of
loose checks for a Wells Fargo ATM
to accept?
•
Do Subway promotions on low-fat
sandwiches increase sandwich
sales?
•
Does a Famous Footwear store sell
fewer shoes when there is a compet-
itor in the same mall?
•
Does a Toronto-Dominion branch
27. get significantly more deposits
when open 60 hours a week com-
pared with 40?
•
Which promotional offers will most
efficiently drive checking account
acquisition at PNC Bank?
As a result of their testing, these organizations are finding out
whether supposedly
better ways of doing business are actually better. Once they
learn from their tests, they
can spread confirmed better practices throughout their business.
For the exclusive use of L. WATSON, 2019.
This document is authorized for use only by LANA WATSON in
2019.
How to Design Smart Business Experiments
harvard business review • february 2009 page 6
• Analysis and results
The company has also built its own applica-
tion, called the eBay Experimentation Plat-
28. form, to lead testers through the process and
keep track of what’s being tested at what times
on what pages.
As with CKE’s new product introductions,
however, this online testing is only part of the
overall change process for eBay’s website. Ex-
tensive offline testing also takes place, includ-
ing lab studies, home visits, participatory de-
sign sessions, focus groups, and trade-off
analysis of website features—all with custom-
ers. The company also conducts quantitative
visual-design research and eye-tracking studies
as well as diary studies to see how users feel
about potential changes. No significant change
to the website is made without extensive study
and testing. This meticulous process is clearly
one reason why eBay is able to introduce most
Put Your Ideas to the Test
1: Create or Refine Hypothesis
Ascertain
that the hypothesized relationships
haven’t already been tested and measured—
and that they can be.
Make
29. sure the hypothesis could generate
substantial economic value.
Determine
whether it suggests an actual de-
cision or action. (If not, go no further.)
2: Design Test
Ensure
that the number of test and control
sites is sufficient for statistical significance.
Use
simulation to explore multiple strategies
for creating control groups (for instance, they
may be nearly identical but different on one
key variable).
Assess
whether control group strategies pre-
30. viously used for similar tests will suffice; they
usually do.
Conduct
statistical analysis to minimize the
number of test cells needed.
Extend
testing period if key metrics are
highly variable.
3: Execute Test
Meet
with test and control site managers
and analytical experts to discuss what might
go wrong and what would constitute test-
confounding events.
Instruct
field personnel to report abnormal
events.
31. Remove
sites from test if test-confounding
events occur.
Adjust
evaluation and compensation plans
for managers so that they are not negatively
affected by tests.
4: Analyze Test
Ensure
that “lift” from interventions is statis-
tically significant.
Use
software to analyze results and manage
complex data from multiple test and control
sites.
Determine
32. need for further testing.
Examine
as many site attributes as possible
to see how key variables interact.
5: Plan Rollout
Study
attributes of test sites to determine
whether rollout should be universal or differ-
entiated.
Balance
complexity of rollout with ease of
implementation and management.
6: Rollout
Stagger
33. the rollout and view it as a test in it-
self. (Are early-adopting sites yielding the de-
sired result? If not, modify the approach in
later-adopting sites.)
Encourage
site managers to share rollout
strategies and tactics.
Learning Library
Develop
a summary of each test: hypotheses,
test dimensions, key results, interactions, and
rollout strategies and results.
Employ
standard business taxonomy to allow
easy searching of library.
Make
library widely accessible to employees;
34. publicize tests and results of important stud-
ies to encourage a test-and-learn culture.
ANALYZE
TEST
CREATE
OR REFINE
HYPOTHESIS
DESIGN
TEST
ROLLOUT
PLAN
ROLLOUT
EXECUTE
TEST
Adapted from Applied Predictive Technologies’ “Test and
Learn” Wheel
LEARNING
LIBRARY
2
1
456
3
For the exclusive use of L. WATSON, 2019.
35. This document is authorized for use only by LANA WATSON in
2019.
How to Design Smart Business Experiments
harvard business review • february 2009 page 7
changes with no backlash from its potentially
fractious seller community. The online retailer
now averages more than 113 million items for
sale in more than 50,000 categories at any
given time.
EBay performed extensive online and offline
testing, for example, in 2007 and 2008, when it
changed its page for viewing items on sale. The
page had not been redesigned since 2003, and
both customers and eBay designers felt it
lacked organization, had inadequate photo-
graphs of items, and suffered from haphazard
item placement and redundant functionality.
After going through all the testing steps, eBay
adopted a new site design. It posted photos
200% larger than those in the previous design,
added a countdown timer for auctions with 24
hours or less to go, made more prominent the
item condition and return policy, and included
tabs to make shipping and payment fields eas-
ier to navigate. It also included new security
features to prevent unauthorized changes in
36. site content. Each new feature and function
was tested independently with control pages.
Measures of page views and bid counts suggest
that the redesign was very successful.
Building a Testing Capability
Establishing a standard process is the first step
toward building an organizational test-and-
learn capability, but it isn’t sufficient unto itself.
Companies that want testing to be a reliable, ef-
fective element of their decision making need
to create an infrastructure to make that hap-
pen. They need training programs to hone com-
petencies, software to structure and analyze
the tests, a means of capturing learning, a pro-
cess for deciding when to repeat tests, and a
central organization to provide expert support
for all the above.
Managerial training.
At the very least,
managers should learn what constitutes a ran-
domized test and when to employ it. Capital
One, for example, offers a professional educa-
tion program on testing and experiment de-
sign through its internal training function
known as Capital One University. One benefit
of hosting a program like this, rather than
sending managers outside for training, is the
greater emphasis on how the testing connects
37. to upstream and downstream activities in the
business.
Test-and-learn software.
Some firms, such
as Capital One and eBay, have built their own
software for managing experiments, but sev-
eral off-the-shelf options exist—the most com-
mon ones being broad statistical packages and
analytical tools like SAS. With every passing
year, these tools make it more possible for nu-
merate—but not statistically expert—users to
conduct truly defensible experiments. Ease of
design and analysis has been a particular focus
at Applied Predictive Technologies, whose
product leads users through the test-and-learn
process, keeps track of test and control groups,
and provides a repository for findings to be
usefully accessed in the future.
Some software tools are tailored to particu-
lar problems or industries. Several packaged
tools, for example, are available for the analy-
sis of manufacturing-quality experiments.
Likewise, highly specialized tools exist for on-
line-usage testing, such as the web analytics
software sold by Omniture and WebTrends
and the free tools provided by Google Analyt-
ics. As of yet, unfortunately, no single soft-
ware tool can help organizations with all test-
ing types and contexts.
38. Learning capture.
If a firm does a substan-
tial amount of testing, it will generate a sub-
stantial amount of learning about what works
and what doesn’t. Ideally employees through-
out the company would share that knowledge
and use it to guide future initiatives. But that
happens at few organizations. The head of
testing at one online firm admitted, “All of that
knowledge is in my head, and we’d be in tough
shape if I were hit by a bus.” One bank execu-
tive justified a lack of shared learning, com-
menting, “We should probably do more, but
we’ve found that people need to learn from
doing the test themselves, even if we’ve done
it before many times.” People do learn
through personal experience, but one would
hope that it’s not the only possible way.
Some organizations, however, have begun to
address the issue. Capital One captures the
learning from its thousands of tests in an on-
line knowledge management system and has
experimented with an even more ambitious
system that would use such learning to guide
product managers as they develop new offer-
ings. Famous Footwear takes a “billboard” ap-
proach; for each test, it captures the results in a
one-page document, circulates that through-
out the organization, and posts it on the wall
outside the testing office.
39. Regular revisiting.
One tricky aspect of es-
For the exclusive use of L. WATSON, 2019.
This document is authorized for use only by LANA WATSON in
2019.
How to Design Smart Business Experiments
harvard business review • february 2009 page 8
tablishing a long-term testing approach is de-
termining when to retest. There is no way to
know for sure when a test has become obso-
lete; an experienced analyst needs to assess
whether enough factors have changed in the
environment to make previous results suspect.
Famous Footwear executives feel that the re-
tail store location context—their primary ap-
plication area for testing—changes enough to
merit retesting after about a year. Netflix con-
cluded in 2006 that its five-year-old customer
tests needed to be redone; the user base had
evolved in that time from internet pioneers to
mainstream society members. CKE Restau-
rants has difficulty deciding whether to retest
pricing, particularly in times when commod-
ity prices are increasing fast. Ironically, it is
40. human intuition, not testing or analytics, that
must be applied to determine the need for re-
testing.
Core resource group.
Most of the firms that
do extensive testing have established a small,
somewhat centralized organization to super-
vise it. The group either actually does the test-
ing, as at PNC Bank, Subway, and Famous
Footwear, or—if testing is employed through-
out the organization—serves as a resource for
methodological and statistical questions, as at
Capital One. At PNC Bank, the test-and-learn
group (part of the bank’s knowledge manage-
ment function, which reports to Marketing)
views the promotion of its own services
around the bank as a priority. It tries to build
relationships and trust with key executives so
that no major initiatives are undertaken with-
out testing. Without a central coordination
point, testing methods may not be sufficiently
rigorous, and test and control groups across
multiple experiments may confound one an-
other. That said, it’s not always easy to influ-
ence or coordinate testing even when a central
group exists.
Creating a Testing Mind-Set
In addition to making the requisite changes in
41. process, technology, and infrastructure, organi-
zations also need to establish a testing culture.
Testing costs money (though not as much as
widespread rollouts of new tactics that don’t
work), and it takes time. Senior managers have
to become accustomed to, and even passionate
about, the idea that no major change in tactics
should be adopted without being tested by peo-
ple who understand testing.
Ask for evidence.
CEOs who firmly believe
in testing can change their entire organiza-
tion’s perspective on the issue. When people
claim that testing has confirmed the wisdom
of their idea, have them walk you through the
process they used, and demand at least the
level of rigor outlined in the exhibit “Put Your
Ideas to the Test.”
Give it teeth.
Gary Loveman at Harrah’s En-
tertainment has said that “not using a control
group” is sufficient rationale for termination at
the company. Jeff Bezos of Amazon reportedly
fired a group of web designers for changing the
website without testing. Toronto-Dominion
has a culture in which managers insist on tests
for every major initiative involving customers
or branches. The CEO, Ed Clark, is a PhD econ-
42. omist who once noted that although the bank
might not be perfect, “nobody ever criticizes us
for not running the numbers.”
Sponsor tests yourself.
The best manage-
ment teams in this regard have institutional-
ized the process of doing and reviewing tests.
At Famous Footwear, Joe Wood and his senior
management team meet with the testing head
every two weeks to discuss past tests, upcom-
ing tests, and preliminary and final results.
Wood says that the company has made testing
a part of management’s dialogue and the orga-
nization’s culture.
• • •
Testing may not be appropriate for every busi-
ness initiative, but it works for most tactical
endeavors. And it just isn’t that difficult any-
more. It needs to come out of the laboratory
and into the boardroom. The key challenges
are no longer technological or analytical; they
have more to do with simply making manag-
ers familiar with the concepts and the process.
Testing, and learning from testing, should be-
come central to any organization’s decision
making. The principles of the scientific
method work as well in business as in any
other sector of life. It’s time to replace “I’ll bet”
43. with “I know.”
1. “Capital One Financial Corporation,” HBS case no. 9-700-
124.
Reprint R0902E
To order, call 800-988-0886 or 617-783-7500 or
go to www.hbr.org
For the exclusive use of L. WATSON, 2019.
This document is authorized for use only by LANA WATSON in
2019.
http://www.hbr.org
http://hbr.org/search/R0902E