#atlassian
Scaling Product Experimentation 
with JIRA and Confluence 
ZANE SALIM • SENIOR TECHNICAL PM • TWITTER • @ZANES
Technology is wiping out 
companies faster than ever
Companies that stop 
innovating die… fast
How are you 
innovating?
1. Our approach @twitter 
2. Challenges at scale 
3. How we scaled 
4. What got better 
5. Final takeaways
1. Our approach @Twitter
@Twitter we innovate 
through experimentation
Anyone can launch 
an experiment
Unleashes creativity
Drives user 
insights at scale
100s every quarter
Experimentation is the new black
… and the old black 
“Negative results are just what I want. They’re just as valuable to me as 
positive results. I can ne...
Both refinement and exploration
Example: which drives more clicks?
Example: which drives more retention?
2. Challenges at scale
Visibility
Learning
Collaboration
Rigor
3. How we scaled
COMMON JIRA 
WORKFLOW FOR 
COLLABORATION
Lifecycle of an Experiment
Lifecycle of an Experiment in JIRA
Lifecycle of an Experiment in JIRA
KNOWLEDGE 
MANAGEMENT IN 
CONFLUENCE
Confluence: Knowledge Management
Confluence: Experiment Brief Templates
Confluence: Experiment Brief Templates 
Experiment Brief 
Experiment Write-up 
Retrospective Analysis 
Describe the proble...
Confluence: Reviewed Experiments & Learning
HYPOTHESIS 
TREES
Visualizing Focus Areas & Insights
4. What got better
Doubled completed 
experiments
Reduced cycle 
time up to 50%
Learning org à 
smarter experiments
5. Final takeaways
Experimentation 
drives innovation
Collaboration & 
learning essential
Smart use of tools can 
make you 4x more effective
Thank you! 
ZANE SALIM • SENIOR TECHNICAL PM • TWITTER • @ZANES
Example: <made up example experiment>
Managing Product Growth Experiments With JIRA and Confluence - Zane Salim
Upcoming SlideShare
Loading in …5
×

Managing Product Growth Experiments With JIRA and Confluence - Zane Salim

2,144 views

Published on

Experiments and A/B tests are at the heart of innovation at Twitter. A culture of experimentation empowers the organization to try new things with minimal downside. Twitter uses Confluence and JIRA to manage hundreds of experiments at any given time. Learn Twitter's philosophy on experimentation, how we enhance organizational learning, and uncover deeper insights that help everyone become smarter with every new experiment.

Published in: Software
  • Be the first to comment

Managing Product Growth Experiments With JIRA and Confluence - Zane Salim

  1. 1. #atlassian
  2. 2. Scaling Product Experimentation with JIRA and Confluence ZANE SALIM • SENIOR TECHNICAL PM • TWITTER • @ZANES
  3. 3. Technology is wiping out companies faster than ever
  4. 4. Companies that stop innovating die… fast
  5. 5. How are you innovating?
  6. 6. 1. Our approach @twitter 2. Challenges at scale 3. How we scaled 4. What got better 5. Final takeaways
  7. 7. 1. Our approach @Twitter
  8. 8. @Twitter we innovate through experimentation
  9. 9. Anyone can launch an experiment
  10. 10. Unleashes creativity
  11. 11. Drives user insights at scale
  12. 12. 100s every quarter
  13. 13. Experimentation is the new black
  14. 14. … and the old black “Negative results are just what I want. They’re just as valuable to me as positive results. I can never find the thing that does the job best until I find the ones that don’t.” ― Thomas Edison, Harper’s Magazine (Vol. 165, p 405) “We must conduct research and then accept the results. If they don't stand up to experimentation, Buddha's own words must be rejected.” ― Dalai Lama XIV, A Policy of Kindness (p 31) “There must be some ability to experiment to advance the state of the art… In the early days of aviation there was a great deal of experimentation and a high death rate. We don't want that … but by the same token we can't have a situation where no deaths are ever allowed, because that would put innovation in a coffin too.” – Elon Musk, L.A. Times interview (August 1, 2012) “There’s a wonderful thing about learning as a group. A fundamental part of that is making mistakes together. There’s no learning without trying lots of ideas and failing lots of times.” ― Jony Ive, Telegraph interview (May 23, 2012)
  15. 15. Both refinement and exploration
  16. 16. Example: which drives more clicks?
  17. 17. Example: which drives more retention?
  18. 18. 2. Challenges at scale
  19. 19. Visibility
  20. 20. Learning
  21. 21. Collaboration
  22. 22. Rigor
  23. 23. 3. How we scaled
  24. 24. COMMON JIRA WORKFLOW FOR COLLABORATION
  25. 25. Lifecycle of an Experiment
  26. 26. Lifecycle of an Experiment in JIRA
  27. 27. Lifecycle of an Experiment in JIRA
  28. 28. KNOWLEDGE MANAGEMENT IN CONFLUENCE
  29. 29. Confluence: Knowledge Management
  30. 30. Confluence: Experiment Brief Templates
  31. 31. Confluence: Experiment Brief Templates Experiment Brief Experiment Write-up Retrospective Analysis Describe the problem Describe the experiment Present retrospective analysis Define a hypothesis Present the results Present the results Design the experiment Offer take-away’s Offer take-away’s
  32. 32. Confluence: Reviewed Experiments & Learning
  33. 33. HYPOTHESIS TREES
  34. 34. Visualizing Focus Areas & Insights
  35. 35. 4. What got better
  36. 36. Doubled completed experiments
  37. 37. Reduced cycle time up to 50%
  38. 38. Learning org à smarter experiments
  39. 39. 5. Final takeaways
  40. 40. Experimentation drives innovation
  41. 41. Collaboration & learning essential
  42. 42. Smart use of tools can make you 4x more effective
  43. 43. Thank you! ZANE SALIM • SENIOR TECHNICAL PM • TWITTER • @ZANES
  44. 44. Example: <made up example experiment>

×