Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Data Science at Atlassian 

The transition towards a data-driven
organisation
Dr Ilias Flaounas
Capital Markets CRC, Sydne...
Atlassian - Trivia
• Atlassian is an Australian enterprise software company
specialised in products that support project m...
• ~43,000 enterprise
customers
• Tens of thousands
more own $10
starter and free
community
licenses
JIRA
JIRA
Key Requirements:!
• Support from upper
management!
• Some minimum
critical mass of
customers!
• Serious investment

in in...
Product Growth Team
• Team was formed ~2 years ago
• Main mission: growth!
• Discover value in heaps of behavioural data!
...
Analytics Infrastructure (v.2)
~3-4TB overall (highly compressed)!
800GB per day produced (uncompressed)
We analyse events, i.e., user actions
Timestamp Instance User Event Attributes
1356958862 mySuperCompany John Login
135696...
The Challenges
• Build infrastructure!
- break silos!
- replace existing “solutions”!
- add instrumentation!
- improve dat...
The Challenges
• Educate!
- “analytics is counting”!
- help people ask the
correct questions
• Build infrastructure!
- bre...
JIRA Agile!
Which Agile report is used the most?
However, counting is hard…
Report 1
Report 2
Report 3
Report 4
Report 5
Report 6
Report 7
Report 8
Report 9
No usage
No usage Low usage
No usage Low usage Medium usage High usage
… and beyond counting…
Predicting conversions
How many variables can you measure?
Top customers
none, n/a, home, self, test,
personal, atlassian, “-”, na, private,
abc, personal use, freelance,
my company...
Predicting conversions
• Baseline set at 50%
• Using only the sign-up information we have an
accuracy of 65%
The Challenges
• Convince

- challenge gut feelings

- demonstrate value
• Educate!
- “analytics is counting”!
- help peop...
The argument of common sense
How Common Sense Fails
Duncan Watts, 2011
κοινὴ αἲσθησις
Aristotle, 384–322 BCE
Source: http://www.tylervigen.com/
Correlation and causation
• Source: P. Chopra, The ultimate guide to A/B testing, 

http://www.smashingmagazine.com, 2010
A/B Tests
• A/B tests allow a constant improvement of
products in a measurable and predictive way
• Last year we deployed ...
Subject: %%first_name%%,
assemble the
bitbucketeers!
Subject: Grab the
Bitbucketeers...4, 3, 2, 1
LAUNCH!
Do A/B tests slow down
development?
“Move fast with stable infra”
Mark Zuckerberg
Facebook F8, 2014
“Pauca sed matura”
i.e...
Instead of a Conclusion: We simply recycle the scientific method
from other fields, e.g., medicine…
Patient User
Medical doc...
Reed
Johnson
Ilias
Flaounas
Stephen
Lee
Shaun
Clowes
Graeme
Smith
Tim
Garnsey
Fardin
Sarker
Theo
Voronov
Herman
Chow
Leo
B...
Thank you!
Dr Ilias Flaounas
!
ilias [dot] flaounas [at] atlassian.com
Data Science at Atlassian: 
The transition towards a data-driven organisation
Data Science at Atlassian: 
The transition towards a data-driven organisation
Data Science at Atlassian: 
The transition towards a data-driven organisation
Data Science at Atlassian: 
The transition towards a data-driven organisation
Data Science at Atlassian: 
The transition towards a data-driven organisation
Data Science at Atlassian: 
The transition towards a data-driven organisation
Data Science at Atlassian: 
The transition towards a data-driven organisation
Data Science at Atlassian: 
The transition towards a data-driven organisation
Data Science at Atlassian: 
The transition towards a data-driven organisation
Data Science at Atlassian: 
The transition towards a data-driven organisation
Upcoming SlideShare
Loading in …5
×

Data Science at Atlassian: 
The transition towards a data-driven organisation

789 views

Published on

This was an invited talk at Capital Markets CRC, presented on May 2015.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Data Science at Atlassian: 
The transition towards a data-driven organisation

  1. 1. Data Science at Atlassian 
 The transition towards a data-driven organisation Dr Ilias Flaounas Capital Markets CRC, Sydney 13 May 2015
  2. 2. Atlassian - Trivia • Atlassian is an Australian enterprise software company specialised in products that support project management and collaboration. • ~12 years old • Latest valuation at $US3.3 billion in 2014, (x8 since 2010). • >1.5K employees • Headquarters in Sydney • Best place to work in Australia 
 (The Australian Financial Review, 2014)
  3. 3. • ~43,000 enterprise customers • Tens of thousands more own $10 starter and free community licenses
  4. 4. JIRA
  5. 5. JIRA
  6. 6. Key Requirements:! • Support from upper management! • Some minimum critical mass of customers! • Serious investment
 in infrastructure, people,
 time
  7. 7. Product Growth Team • Team was formed ~2 years ago • Main mission: growth! • Discover value in heaps of behavioural data! • Champion data-driven decision making
  8. 8. Analytics Infrastructure (v.2) ~3-4TB overall (highly compressed)! 800GB per day produced (uncompressed)
  9. 9. We analyse events, i.e., user actions Timestamp Instance User Event Attributes 1356958862 mySuperCompany John Login 1356968863 mySuperCompany Anne Login 1356978862 mySuperCompany John dashboard view 1356988862 mySuperCompany Anne dashboard view 1357884442 anotherCompany John view issue {issueID = 1234} 1357956862 mySuperCompany John create issue {issueID = 1000} 1357957862 mySuperCompany Anne view issue {issueID = 1000} 1357958862 mySuperCompany Anne comment issue {issueID = 1000} 1357959862 mySuperCompany John create issue {issueID = 1001} … … … … …
  10. 10. The Challenges • Build infrastructure! - break silos! - replace existing “solutions”! - add instrumentation! - improve data quality! - deal with catch-22…

  11. 11. The Challenges • Educate! - “analytics is counting”! - help people ask the correct questions • Build infrastructure! - break silos! - replace existing “solutions”! - add instrumentation! - improve data quality! - deal with catch-22…

  12. 12. JIRA Agile! Which Agile report is used the most?
  13. 13. However, counting is hard… Report 1 Report 2 Report 3 Report 4 Report 5 Report 6 Report 7 Report 8 Report 9
  14. 14. No usage
  15. 15. No usage Low usage
  16. 16. No usage Low usage Medium usage High usage
  17. 17. … and beyond counting…
  18. 18. Predicting conversions How many variables can you measure?
  19. 19. Top customers none, n/a, home, self, test, personal, atlassian, “-”, na, private, abc, personal use, freelance, my company,student, testing, company, no company, private, mycompany, a, no, test company, noname, testcompany, self employed, individual, tester, me, “.” , independent, trial, myself, nocompany, xyz
  20. 20. Predicting conversions • Baseline set at 50% • Using only the sign-up information we have an accuracy of 65%
  21. 21. The Challenges • Convince
 - challenge gut feelings
 - demonstrate value • Educate! - “analytics is counting”! - help people ask the correct questions • Build infrastructure - replace existing “solutions” - data quality - catch-22…

  22. 22. The argument of common sense How Common Sense Fails Duncan Watts, 2011 κοινὴ αἲσθησις Aristotle, 384–322 BCE
  23. 23. Source: http://www.tylervigen.com/ Correlation and causation
  24. 24. • Source: P. Chopra, The ultimate guide to A/B testing, 
 http://www.smashingmagazine.com, 2010
  25. 25. A/B Tests • A/B tests allow a constant improvement of products in a measurable and predictive way • Last year we deployed ~118 experiments. • The volume of customers, evaluators, visitors, has become a limited and valuable resource
  26. 26. Subject: %%first_name%%, assemble the bitbucketeers! Subject: Grab the Bitbucketeers...4, 3, 2, 1 LAUNCH!
  27. 27. Do A/B tests slow down development? “Move fast with stable infra” Mark Zuckerberg Facebook F8, 2014 “Pauca sed matura” i.e., “Few, but mature” Carl Friedrich Gauss 1777-1855 “Move Fast and Break Things”“Move Fast and Break Things”
  28. 28. Instead of a Conclusion: We simply recycle the scientific method from other fields, e.g., medicine… Patient User Medical doctor Product manager Illness Bad User Experience Diagnosis Interviews, surveys… Treatment UX modification Medical researchers Growth team Observations Data mining Medical trials A/B tests Ethics Privacy Policy Evidence-based
 medicine Data-driven
 development
  29. 29. Reed Johnson Ilias Flaounas Stephen Lee Shaun Clowes Graeme Smith Tim Garnsey Fardin Sarker Theo Voronov Herman Chow Leo Balan John Jiang Rob Sangster Andrew Wakeling Martin Smyrk Kostas Servis Houda Chehab Lakshan Perera We want YOU!
  30. 30. Thank you! Dr Ilias Flaounas ! ilias [dot] flaounas [at] atlassian.com

×