The document outlines a hypothesis-driven development process involving six steps to validate new strategies through small experiments and gathering feedback. It describes various mechanisms for feedback, including feature flags and JavaScript experimentation platforms, emphasizing the need for a suitable codebase and staging environment. The document also highlights the importance of real user testing to ensure engagement, flexibility, and cost-effectiveness in product development.

1. Hypothesis Driven
Development
Small Experiments to Validate the New Strategy, Demonstrate Concrete Action
Items, Forecast ROI

3. Define the
Hypothesis
Step 1 of 6
(for each experiment)
We believe <this capability>
Will result in <this outcome>
We will have confidence to proceed when
<we see this measurable signal>
Seamless Teamwork
Customer Centered
Experiment

4. Design the
Solution
Step 2 of 6
Lightweight Docs
Lightweight Design

5. Build
Step 3 of 6
Together...

6. Test
Step 4 of 6
Test with Real Users

7. Controlled Release
Step 5 of 6
Gather Facts

8. Analytics &
Feedback
Step 6 of 6
Fact Based Decision

9. What do we need for testing?

10. Feature Flags
Option 1: Mechanism for gathering
feedback
What it is
A technique to turn some functionality of your
application off, via configuration, without deploying
new code
Why use it
Efficiently and seamlessly manage the
[Dev]-[Test]-[ControlledRelease] lifecycle
We are engaging in actual development so we can
address all three outcomes at once (engagement,
flexibility, cost)
Real Users
Requirements
● Access to Codebase
● Codebase must be capable of supporting it
(needs analysis)
● Able to create a staging environment

11. JS Experimentation
Platform
Option 2: Mechanism for gathering
feedback
What it is
A tool which adjusts design via JavaScript, post-render
of the page
Why use it
● Lightweight
● Real Users
Limitations
● Does not modify actual site code
● Primarily for content and visual changes
(functionality changes are limited)
Requirements
● 1 Line of Code in website
● Optimizely Account (Paid or Trial)

12. Internal Guided &
Unguided User
Testing
Option 3: Mechanism for gathering
Feedback
What it is
A tool which adjusts design via JavaScript, post-render
of the page
Why use it
● Able to gather anecdotal, qualitative feedback
● Able to gather emotional feedback
Limitations
● Feedback can be biased
● Can be time consuming
Requirements
● Access to Codebase
● Able to create a staging environment

13. External Unguided
User Testing
Option 4: Mechanism for gathering
Feedback
What it is
A tool which adjusts design via JavaScript, post-render
of the page
Why use it
● Able to gather anecdotal, qualitative feedback
● Able to gather emotional feedback
Limitations
● Feedback can be biased
● Can be time consuming
Requirements
● Access to Codebase
● Able to create a staging environment
● Starting at $99 / Test