Main takeaways: -How to use random processes to break out of biases you don't know you have -How to use noise to understand the world and interpret the ambiguity you see -How to use chaos to build more resilient teams

1. www.productschool.com
Breaking Down Biases w/ Adversarial
PM Techniques by Philosophie Dir of
AI

2. FREE INVITE
Join 30,000+ Product Managers on

3. COURSES
Product Management
Learn the skills you need to land a product manager job

4. COURSES
Coding for Managers
Build a website and gain the technical knowledge to lead software engineers

5. COURSES
Data Analytics for Managers
Learn the skills to understand web analytics, SQL and machine learning concepts

6. COURSES
Digital Marketing for Managers
Learn how to acquire more users and convert them into clients

7. COURSES
Blockchain for Managers
Learn how to trade cryptocurrencies and build products using the blockchain

8. Chris Butler
TONIGHT’S SPEAKER

9. Breaking Down Biases with
Adversarial Product Management
Chris Butler
chrisbutler@philosophie.is
@chrizbot

10. Chris Butler
Director of AI @ Philosophie NYC
TBPP 2016, TPM 4/6/7/8/9
18 years of product and BD
Microsoft, Waze, Horizon Ventures,
KAYAK, and started my own company
(failed)
chrisbutler@philosophie.is
@chrizbot

11. Some things product teams struggle with:
● Writing good stories
● Creating a real roadmap
● Aligning the team
● Working through conflict
● Balancing trade offs with engineering
● Being data focused
● ...

12. What product teams really struggle with:
● Innate or unknown biases
● Bad assumptions
● Uncomfortable ambiguity
● Unjustified uncertainty
● Overly or incorrectly constrained
● Maladapted mindsets

13. Let’s start with biases...

14. Biases help us...
● Reduce the amount of information to consider
● Make more sense of an incredibly complex world
● Ability to act fast in the face of uncertainty
● Remember things that we can find use for later

15. ...but biases keep us constrained
● Naïve realism
● Attentional bias
● Confirmation bias
● Availability heuristic
● Status quo bias
● Loss aversion
● Bias blind spot (aka bias blindness bias)

17. So how do you ‘debias’ yourself?

18. ??? ???

19. Adversarial Thinking

20. Red teaming

23. The practice of viewing a
problem from an
adversary or competitor’s
perspective.
- Red Team Journal

24. Who is a product manager’s adversary?

25. Competitors?

27. Time?

29. Customer complacency or disinterest?

31. You?

33. Your past mindsets

34. Adversarial Product Manager Toolbox

35. Adversarial Product Manager Toolbox
● Tight feedback loops
● Contrarian thinking
● Randomness

36. Feedback loops

40. What makes it so good?

41. Feedback loops

42. What makes good feedback?

43. Good feedback
● Personalized
● Contextual and situational
● Timely
● Actionable

44. How do we accept feedback?

45. Chris Butler’s writing...
“Could do with more
imagination/inspiration…”
“At parts feel a bit pedantic, there's some
profanity we'd need to remove...”
“Is all substance, no style.”
“Please stop using hyped topics (like AI)
just as advertisements!”

46. If it doesn’t “hurt” [it] is
not feedback.
- Gustavo Razzetti

48. Feedback triggers

49. Feedback triggers
● Truth triggers
● Relationship triggers
● Identity triggers

50. How do you get feedback?

52. Product critiques
● Engineers and designers critique their work—why don't we?
● Our artifacts are presentations, documents, etc.
● Each round
○ 5 min - help you need, audience, and the work
○ 5 min - critique from your peers and leaders
○ 5 min more, as needed

53. Rules
Presenters
● No sales pitches
● Talk less, not more
● Leave with a list — you take your
own notes
● It is up to you to decide to take
your feedback (or ignore it)
Critiquers
● Critique, not criticism
● Talk about the work, not the person
● Ask more questions, make less
assertions
● It doesn’t matter what you like or
dislike
● Don’t ‘product’ in the meeting

54. Feedback loops

55. Contrarian thinking

56. Also known as being a “devil’s advocate”

58. How do you do it?
Introduction
Private ideation of assumptions for the industry/customer/problem
Review for clarification
Dot vote issues that you ‘believe to be most true’
Private ideation of what it means to do the ‘opposite’ for top item(s)
Discuss
Repeat as necessary
5 min
5 min
5 min
2 min
5 min
5 min

60. How do you do it?
Introduction
Private ideation of problems
Review for clarification
Dot vote issues that are most likely to cause problems
Discuss and assign top items
5 min
5 min
8 min
2 min
10 min

61. “What is a ‘bad’ idea?”

62. Contrarian thinking

63. Randomness

66. How do we not miss something?

68. We don’t have time for all of the items
we actually want to check!

69. Product randomization

70. For ideation...

78. For interpreting...

80. “[I Ching] prompts me to go
beyond false certainties
and to create new and
unexpected possibilities.”
- Will Buckingham

83. Potential
Present FuturePast
Question

84. How do you do it?
Ask a product question
Draw cards
Interpret the cards based on your context
1 min
1 min
3 min

85. For making choices...

89. Randomness

90. Closing

91. Adversarial Product Manager Toolbox
● Tight feedback loops
● Contrarian thinking
● Randomness

92. It is sufficient for the ‘devil’s
advocate’ to be right in just
one instance in order to
justify its existence.
- “Shmuel,” IDI, Revision Dept.

93. Homework
Next time you have to make a decision try an adversarial method:
● Make a decision how your competitor would make it.
● Make what you consider a “bad” decision.
● Make what you think your company would not think is “good” decision.
● Make what you consider a random decision.

94. Victorious warriors win first
and then go to war, while
defeated warriors go to war
first and then seek to win.
- Sun Tzu

95. Everybody has a plan until
they get punched in the
mouth.
- Mike Tyson

96. Thank you
chrisbutler@philosophie.is
@chrizbot

97. www.productschool.com
Part-time Product Management, Coding, Data, Digital
Marketing and Blockchain courses in San Francisco, Silicon
Valley, New York, Santa Monica, Los Angeles, Austin, Boston,
Boulder, Chicago, Denver, Orange County, Seattle, Bellevue,
Toronto, London and Online