Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Managing Product Growth Experiments With JIRA and Confluence - Zane Salim


Published on

Experiments and A/B tests are at the heart of innovation at Twitter. A culture of experimentation empowers the organization to try new things with minimal downside. Twitter uses Confluence and JIRA to manage hundreds of experiments at any given time. Learn Twitter's philosophy on experimentation, how we enhance organizational learning, and uncover deeper insights that help everyone become smarter with every new experiment.

Published in: Software
  • Be the first to comment

Managing Product Growth Experiments With JIRA and Confluence - Zane Salim

  1. 1. #atlassian
  2. 2. Scaling Product Experimentation with JIRA and Confluence ZANE SALIM • SENIOR TECHNICAL PM • TWITTER • @ZANES
  3. 3. Technology is wiping out companies faster than ever
  4. 4. Companies that stop innovating die… fast
  5. 5. How are you innovating?
  6. 6. 1. Our approach @twitter 2. Challenges at scale 3. How we scaled 4. What got better 5. Final takeaways
  7. 7. 1. Our approach @Twitter
  8. 8. @Twitter we innovate through experimentation
  9. 9. Anyone can launch an experiment
  10. 10. Unleashes creativity
  11. 11. Drives user insights at scale
  12. 12. 100s every quarter
  13. 13. Experimentation is the new black
  14. 14. … and the old black “Negative results are just what I want. They’re just as valuable to me as positive results. I can never find the thing that does the job best until I find the ones that don’t.” ― Thomas Edison, Harper’s Magazine (Vol. 165, p 405) “We must conduct research and then accept the results. If they don't stand up to experimentation, Buddha's own words must be rejected.” ― Dalai Lama XIV, A Policy of Kindness (p 31) “There must be some ability to experiment to advance the state of the art… In the early days of aviation there was a great deal of experimentation and a high death rate. We don't want that … but by the same token we can't have a situation where no deaths are ever allowed, because that would put innovation in a coffin too.” – Elon Musk, L.A. Times interview (August 1, 2012) “There’s a wonderful thing about learning as a group. A fundamental part of that is making mistakes together. There’s no learning without trying lots of ideas and failing lots of times.” ― Jony Ive, Telegraph interview (May 23, 2012)
  15. 15. Both refinement and exploration
  16. 16. Example: which drives more clicks?
  17. 17. Example: which drives more retention?
  18. 18. 2. Challenges at scale
  19. 19. Visibility
  20. 20. Learning
  21. 21. Collaboration
  22. 22. Rigor
  23. 23. 3. How we scaled
  25. 25. Lifecycle of an Experiment
  26. 26. Lifecycle of an Experiment in JIRA
  27. 27. Lifecycle of an Experiment in JIRA
  29. 29. Confluence: Knowledge Management
  30. 30. Confluence: Experiment Brief Templates
  31. 31. Confluence: Experiment Brief Templates Experiment Brief Experiment Write-up Retrospective Analysis Describe the problem Describe the experiment Present retrospective analysis Define a hypothesis Present the results Present the results Design the experiment Offer take-away’s Offer take-away’s
  32. 32. Confluence: Reviewed Experiments & Learning
  34. 34. Visualizing Focus Areas & Insights
  35. 35. 4. What got better
  36. 36. Doubled completed experiments
  37. 37. Reduced cycle time up to 50%
  38. 38. Learning org à smarter experiments
  39. 39. 5. Final takeaways
  40. 40. Experimentation drives innovation
  41. 41. Collaboration & learning essential
  42. 42. Smart use of tools can make you 4x more effective
  44. 44. Example: <made up example experiment>