Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

SplunkLive! - Want to Turbocharge your Developer Pipeline?

540 views

Published on

Help your developers waste less time & optimize their build times using Splunk. A story of how we optimized Maven build times at Atlassian for developers working on Jira Cloud

Published in: Technology
  • Be the first to comment

  • Be the first to like this

SplunkLive! - Want to Turbocharge your Developer Pipeline?

  1. 1. Want to Turbocharge your Developer Pipeline? Help your developers waste less time & optimize their build times using Splunk VIKTOR ADAM | SOFTWARE ENGINEER @ ATLASSIAN | @RYCUS86
  2. 2. Jira Confluence Bitbucket Trello
  3. 3. OUR MISSION We believe behind every great human achievement, there is a team. Our mission is to unleash the potential in every team.
  4. 4. Developer Experience Advocates team
  5. 5. Scaling up and its challenges Splunk to the rescue Evolution of dashboards Achievements & failures Takeaways Agenda
  6. 6. Growing teams
  7. 7. 128%more lines of Java code
  8. 8. Collecting metrics A B C D E F G H I J Directed Acyclic Graph
  9. 9. 2.5xmore Maven modules
  10. 10. Increased complexity Inefficient builds Long build times Challenges with growth
  11. 11. Increased complexity Jmake, a wrapper tool around Maven invocations Inefficient builds Maven incremental build extension Long build times Challenges with growth
  12. 12. https://github.com/jcgay/maven-profiler
  13. 13. Collecting metrics
  14. 14. Maven profiler results in Datadog
  15. 15. Scaling up and its challenges Splunk to the rescue Evolution of dashboards Achievements & failures Takeaways Agenda
  16. 16. Metrics pipeline DevMetrics Publisher
  17. 17. Structured events Not aggregated Can aggregate however you want later Plus it supports ad-hoc queries High cardinality No need to worry about reporting on properties with a big range of values
  18. 18. Surfacing information Initial dashboard Lots of data, very busy Aggregated in different ways Central overview Single place for all events Easy to explore
  19. 19. Scaling up and its challenges Splunk to the rescue Evolution of dashboards Achievements & failures Takeaways Agenda
  20. 20. Session explorer
  21. 21. Session explorer (cont.)
  22. 22. Timeline (initially) Complete view Has all the information we need Start & finish times of each module Hard to understand The actual view is too long, needs scrolling Does not help to draw conclusions easily ... ...
  23. 23. Timeline (per thread) Complete view Same information as before But now it fits on a single screen Reveals connections Highlights inefficiencies Easy to spot them for the human eye
  24. 24. Internal dependencies
  25. 25. Critical path of internal dependencies
  26. 26. Critical path of internal dependencies (cont.)
  27. 27. Eliminate unnecessary dependencies Eliminate artificial dependencies Splitting code into multiple modules Splitting expensive tasks Changing test dependencies Our plan
  28. 28. Scaling up and its challenges Splunk to the rescue Evolution of dashboards Achievements & failures Takeaways Agenda
  29. 29. Starting point (last December)
  30. 30. Removing integration test dependencies And here And here Less modules here And here
  31. 31. Cleaning up unnecessary and artificial dependencies
  32. 32. Refactoring test-only dependencies
  33. 33. Making tasks more parallel
  34. 34. Splitting long tasks
  35. 35. Before After
  36. 36. Achievements Overall build time reduced from 10:40 to 6:30
  37. 37. Tracking progress
  38. 38. Tracking progress (cont.) time
  39. 39. Not everything worked well
  40. 40. Difficulties Complex problem space Noisy data Changing behaviour Key metric Different build environments Different build targets Geographical locations
  41. 41. Difficulties Complex problem space Noisy data Changing behaviour Key metric Not enough data points External factors Correlating changes
  42. 42. Difficulties Complex problem space Noisy data Changing behaviour Key metric N-th percentile Overall vs. Machine type Excluding our team
  43. 43. Difficulties Complex problem space Noisy data Changing behaviour Key metric Wanted more incremental builds Hoped for less full builds
  44. 44. Scaling up and its challenges Splunk to the rescue Evolution of dashboards Achievements & failures Takeaways Agenda
  45. 45. FIND A METRIC YOU CAN COMMIT TO AND WORRY LESS ABOUT WHETHER IT’S THE BEST ONE.
  46. 46. HAVE TRUST THAT YOU’RE MOVING IN THE RIGHT DIRECTION AND METRICS WILL FOLLOW.
  47. 47. COLLECT AS MANY STRUCTURED EVENTS AS YOU CAN. YOU CAN AGGREGATE THEM LATER IN SPLUNK.
  48. 48. COLLECT AS MANY PROPERTIES AS YOU CAN. YOU CAN FIGURE OUT LATER WHICH OF THEM ARE USEFUL.
  49. 49. BUT MAKE SURE THAT YOU DON’T DUMP ALL THE METRICS ARE ON A SINGLE DASHBOARD.
  50. 50. USE VISUALIZATIONS THAT ARE EASY TO UNDERSTAND WITH THE HUMAN EYES.
  51. 51. WE’RE HIRING
  52. 52. Thank you! VIKTOR ADAM | SOFTWARE ENGINEER @ ATLASSIAN | @RYCUS86

×