business model, customer development, e245, engr245, lean launchpad, lean startup, stanford, startup, steve blank

1. Video

2. Self-driving Ads
A browser plug-in that
observes knowledge
workers’ workflow, and
automates repetitive
processes
End-to-end advertisement
generation and experimentation
platform that automates
content iteration
Start of the Class
Self-driving Browser 126 interviews
Today
Behzad Haghgoo Chenyu Ren Ying Hang SeahTim SpencerGirish Kumar

3. Our 10-week Rollercoaster Ride
Contextual
Search
Pilot!
Week 2
“Words don’t matter”
- Some marketers
Words DO
Matter!!
Complete
Reboot!
Unified Search
Ad Copy
Generation
Week
4
Week
0
Week 10
Week
6
Week
8
“We need case studies”
- CMO

4. Part I: Why Contextual
Search?
Week 2
Week
4
Week 10
Week
0
Week
0

5. Week 2
Week
4
Week 10
Week
0
Knowledge work is dominated by repetitive tasks
- mainly consisting of pulling information from
several sources
Our Hypothesis: A context-based information
retrieval system can dramatically speed up this
process.
Part I: Why Contextual
Search?
Week
0

6. - BPOs: Channel partnership
- Enterprise: CIO
relationship; higher/lower-
touch depending on
account size (best practice
SaaS account mgmt.)
Key Activities
Key Resources
Cost Structure Revenue Streams
Key Partners
Channels
Customer Relationships
- Business process outsourcing
firms (BPOs)
- Consulting firms
- Cloud Computing Services:
AWS/Azure
- Open-source ML frameworks
(Keras, Tensorflow)
- Marketing agencies
- Cloud computing resources
- Strong engineering team
- Aggregation of user logs for
better algorithms
- Server and infrastructure cost
- Sales commision
- Marketing campaigns
- Engineering for product development
- Direct Sales
- Third-party Companies
(e.g. consulting firms)
- Website
- Build a low-touch user
interface
- Demonstrate a significant
speed-up in menial
workflows
- ML research to parse noisy
user logs
- Recurring annual/monthly subscription (SaaS model)
Private Equity
Analysts
Value Proposition Customer Segments
Less time
wasted on
finding
information
trapped in tons
of documents
Part I: Why Contextual
Search?

7. Week 2
Week
4
Week 10
Week
0
We built a functional MVP to see if it’s useful
Week 1

8. Week
0
Week 2
Week
4
Week 10
But after 29 Interviews our hypothesis was
rejected
“I sit down with the Office Manager every quarter to
review what enterprise tools junior employees need,
and make the purchase decision. It’s a long process.”
-Investment Banking Managing
Director
“Usually I know where to look for
the files. The hard part is using the
data to build the models.”
-PE Analyst
Week 1

9. Part II: Pivot to Unified Search
Week 2
Week
4
Week 10
Week
0
Week
2

10. Part II: Pivot to Unified Search
Week 2
Week
4
Week 10
Week
0
Knowledge workers have information stored
in multiple siloed tools - pulling information
from the right sources is hard and inefficient
Our Hypothesis: Unified search will greatly
boost productivity by making search easier
Week
2

11. - BPOs: Channel partnership
- Enterprise: CIO
relationship; higher/lower-
touch depending on
account size (best practice
SaaS account mgmt.)
Key Activities
Key Resources
Cost Structure Revenue Streams
Key Partners
Channels
Customer Relationships
- Business process outsourcing
firms (BPOs)
- Consulting firms
- Cloud Computing Services:
AWS/Azure
- Open-source ML frameworks
(Keras, Tensorflow)
- Marketing agencies
- Cloud computing resources
- Strong engineering team
- Aggregation of user logs for
better algorithms
- Server and infrastructure cost
- Sales commision
- Marketing campaigns
- Engineering for product development
- Direct Sales
- Third-party Companies
(e.g. consulting firms)
- Website
- Build a low-touch user
interface
- Demonstrate a significant
speed-up in menial
workflows
- ML research to parse noisy
user logs
- Recurring annual/monthly subscription (SaaS model)
Cross-functional
roles:
Sales
Professionals
Product
Marketing
Value Proposition Customer Segments
“Home-screen”
to search and
access
information
stuck in different
enterprise silos
(e.g. Slack, Jira,
Salesforce)
Part II: Unified Search for Cross-Functional Role

12. Week 2
Week
4
Week 10
Week
0
After 8 more interviews, our unified search
idea was rejected :(
“I have a good sense of where the
information is stored - unified search
might be more useful for
newcomers”
- Sales
Engineer
“I care much more about lead
generation than productivity.”
- Sales Manager
Week 3

13. Week 2
Week
4
Week 10
Week
0
Part III: Reboot TIME!!!!!
Week 4
Complete
Reboot!

14. Week 2
Week
4
Week 10
Week
0
Writing
Ad Copy!
If you had a magic
wand, what would
you automate?
Product Marketing Manager
Week 4

15. Week 2
Week
4
Week 10
Week
0
New Value Proposition
For B2C growth marketing teams,
who have to optimize thousands of text ads,
Manna generates lift through an end-to-end ad copy
optimization platform.
Unlike our competitors, our system will close the A/B testing
loop by automatically generating thousands of ad copy
alternatives.
Week 4

16. Week 2
Week
4
Week 10
Week
0
We emailed every Stanford Alum Marketer
there ever was!
Week 4

17. And got out of the building to understand
the search marketing ecosystem
Week 4-8

18. We spoke to marketers from 65 companies!!!
B2C
B2B
Agency
or
Martech
CEO / CMO Digital Marketing Director Marketer
Week 4-
8

19. B2C
B2B
Agency
CEO / CMO Digital Marketing Director Marketer
Performance Marketers are
metrics driven
⟹ our target customers!
Our key learnings
Compliance can be a
bottleneck in A/B testing
Marketers want to know why
certain ads are performing
better for whom.
B2C marketers with
huge paid search channel
are the most excited.
Week 4-
8

20. B2C
B2B
Agency
CEO / CMO Digital Marketing Director Marketer
We narrowed down to customers with the
biggest pain point
Large amount of ad groups
⇒ insufficient bandwidth to
optimize every ad group
Companies with:
Week 4-
8
Less brand-driven strategy
⇒ can rapidly experiment
with different ad copies

21. Week 2
Week
4
Week 10
Week
0
However, most marketers believed that
words would NOT make a difference...
Week
6
Week 7-8

22. Week 2
Week
4
Week 10
Week
0
Why not test it ourselves?
Week
6
Week 7-8
Henry Ford

23. Proof of lift generation: Coffee Pod demo
Week 7-8

24. Copikup - World-class Coffee. Enjoy gourmet
capsules, sent right to your home.
Copikup - Artful Exquisite Coffee. Enjoy gourmet
capsules, sent right to your home.
Copikup - Easy Subscriptions. Enjoy gourmet
capsules, sent right to your home.
Copikup - Free Two-Day Shipping. Enjoy
gourmet capsules, sent right to your home.
Copikup - Limited Edition Coffee. Enjoy gourmet
capsules, sent right to your home.
83% Lift! Words DO Matter!
11% lift with 61%
significance
29% lift with 74%
significance
13% lift with 65%
significance
83% lift with 98%
significance
Baseline
Week 7-8

25. - BPOs: Channel partnership
- Enterprise: CIO
relationship; higher/lower-
touch depending on
account size (best practice
SaaS account mgmt.)
Key Activities
Key Resources
Cost Structure Revenue Streams
Key Partners
Channels
Customer Relationships
- Business process outsourcing
firms (BPOs)
- Consulting firms
- Cloud Computing Services:
AWS/Azure
- Open-source ML frameworks
(Keras, Tensorflow)
- Marketing agencies
- Cloud computing resources
- Strong engineering team
- Aggregation of user logs for
better algorithms
- Server and infrastructure cost
- Sales commision
- Marketing campaigns
- Engineering for product development
- Direct Sales
- Third-party Companies
(e.g. consulting firms)
- Website
- Build a low-touch user
interface
- Demonstrate a significant
speed-up in menial
workflows
- ML research to parse noisy
user logs
- Recurring annual/monthly subscription (SaaS model)
Decision Makers:
CMOs at Medium
and Large B2C
Enterprises
Users: Growth Marketers at
B2C companies
Partners: Facebook
Marketing Partners:
Campaign Managers
Value Proposition Customer Segments
- Automatically generates
lift through an end-to-end
ad copy optimization
platform
- Causal analytics to
identify the types of
messaging that work better
for cross-channel usage
Part III: Sales Mode to CMOs

26. CMOs want it but not badly enough to pilot...
Week
8
“I would rather test this on a high
performing ad group but need to know
this won’t have a negative effect.”
- CMO Hatch Baby / Yelp
“I would love to have a tool like that, but I
will need case studies before I can
recommend it to my clients.”
- Director of Search Marketing
AKQA
Week 8

27. Week 8
Start with
someone
Junior

28. My first sale at
Rocketfuel was to
someone very
junior
Week 8

29. We went after junior folks and got pilots!!!
Week
9
WINGMAN
Week 9
179% Lift

30. What’s next for us?
Following through with
pilots in pipeline
We are excited about
the progress we’ve
made and hope to
decide on a plan
after the pilots!
WINGMAN
Week 10

31. We want to thank our Mentor

32. And our teaching team for their guidance