Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Best Practices for Measuring your Code Pipeline

150 views

Published on

Best Practices for Measuring your Code Pipeline - Eric Mittelhammer

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Best Practices for Measuring your Code Pipeline

  1. 1. ©2008–18 New Relic, Inc. All rights reserved Best practices for measuring your code pipeline Eric Mittelhammer Solutions Consultant, New Relic
  2. 2. ©2008–18 New Relic, Inc. All rights reserved 2 Safe Harbor This presentation and the information herein (including any information that may be incorporated by reference) is provided for informational purposes only and should not be construed as an offer, commitment, promise or obligation on behalf of New Relic, Inc. (“New Relic”) to sell securities or deliver any product, material, code, functionality, or other feature. Any information provided hereby is proprietary to New Relic and may not be replicated or disclosed without New Relic’s express written permission. Such information may contain forward-looking statements within the meaning of federal securities laws. Any statement that is not a historical fact or refers to expectations, projections, future plans, objectives, estimates, goals, or other characterizations of future events is a forward- looking statement. These forward-looking statements can often be identified as such because the context of the statement will include words such as “believes,” “anticipates,” “expects” or words of similar import. Actual results may differ materially from those expressed in these forward-looking statements, which speak only as of the date hereof, and are subject to change at any time without notice. Existing and prospective investors, customers and other third parties transacting business with New Relic are cautioned not to place undue reliance on this forward-looking information. The achievement or success of the matters covered by such forward-looking statements are based on New Relic’s current assumptions, expectations, and beliefs and are subject to substantial risks, uncertainties, assumptions, and changes in circumstances that may cause the actual results, performance, or achievements to differ materially from those expressed or implied in any forward-looking statement. Further information on factors that could affect such forward-looking statements is included in the filings New Relic makes with the SEC from time to time. Copies of these documents may be obtained by visiting New Relic’s Investor Relations website at ir.newrelic.com or the SEC’s website at www.sec.gov. New Relic assumes no obligation and does not intend to update these forward-looking statements, except as required by law. New Relic makes no warranties, expressed or implied, in this presentation or otherwise, with respect to the information provided.
  3. 3. ©2008–18 New Relic, Inc. All rights reserved Origin Story 3
  4. 4. ©2008–18 New Relic, Inc. All rights reserved 4 We Had to Change From To Ruby Monolith Siloed Teams Infrequent Releases Reactive Monitoring 200+ Microservices 50+ Engineering Teams with Embedded SREs 20-70 Deploys a Day Teams Proactively Watching What they deploy
  5. 5. ©2008–18 New Relic, Inc. All rights reserved What Is a Code Pipeline? 5
  6. 6. ©2008–18 New Relic, Inc. All rights reserved 6 What is a code pipeline? The set of tools used to automate your Continuous Integration/Continuous Delivery (CI/CD) process Manages the incremental workflow as a series of steps required to deliver features and fixes to end users Failures cascade (all or nothing).
  7. 7. ©2008–18 New Relic, Inc. All rights reserved 7 What is a code pipeline? Action: Source (GitHub) STAGE STAGE STAGE Action: Build (AWS CodeBuild) Action: Build (AWS CodeBuild) Action: Deploy (AWS ElasticBeanstalk)
  8. 8. ©2008–18 New Relic, Inc. All rights reserved 8 Why Measure It? Successful DevOps is based in automation and measurement Your automation tools are software too Correlate business and team KPIs to operational changes
  9. 9. ©2008–18 New Relic, Inc. All rights reserved 9 How Do We Measure It? AWS CodePipeline TagBuil d RunTests Test results Deploy Webhook Webhook Insights API Insights API State Change State Change State Change State Change Insights API Record Deployment
  10. 10. ©2008–18 New Relic, Inc. All rights reserved Send Events from SCM 10
  11. 11. ©2008–18 New Relic, Inc. All rights reserved 11 Send Events from SCM Use SCM repository hooks Ex: GitHub’s PUSH event Webhook At minimum, use attributes: commit id (hash) user timestamp Propagate wherever possible!
  12. 12. ©2008–18 New Relic, Inc. All rights reserved 12 GitHub Push Event One “changeset” made of several commits Additional attributes: Number of files changed Lines of code (LOC) changed (yes, really) push to master == deploy
  13. 13. ©2008–18 New Relic, Inc. All rights reserved 13 What Can We Measure? “Look for patterns, correlations, or outliers (anonymize if you have to). “Why are there so many pushes on weekends?” “Why did we ship so little on Monday?”
  14. 14. ©2008–18 New Relic, Inc. All rights reserved 14 What Can We Measure? More LOC = More productivity! Right??? (No.)
  15. 15. ©2008–18 New Relic, Inc. All rights reserved 15 What Can We Measure? “Are we really making small, incremental changes?”
  16. 16. ©2008–18 New Relic, Inc. All rights reserved 16 What Can We Measure? “How many pizzas do we need to order?”
  17. 17. ©2008–18 New Relic, Inc. All rights reserved 17 What Can We Measure? “Are our stories/epics/feature requests right-sized?”
  18. 18. ©2008–18 New Relic, Inc. All rights reserved Capture State Change Data from the Pipeline 18
  19. 19. ©2008–18 New Relic, Inc. All rights reserved 19 Capture State Change Data from the Pipeline Send the status of the pipeline at the beginning and end of each Stage and Action Attributes: Pipeline stage/action name Current state (“started”, “succeeded”, “failed”, etc) Timestamp Duration Source code revision and build id (you propagated it, remember? :) )
  20. 20. ©2008–18 New Relic, Inc. All rights reserved 20 What Can We Measure? “Move faster with confidence.”
  21. 21. ©2008–18 New Relic, Inc. All rights reserved 21 What Can We Measure?
  22. 22. ©2008–18 New Relic, Inc. All rights reserved 22 What Can We Measure? “Where are our pipelines failing?”
  23. 23. ©2008–18 New Relic, Inc. All rights reserved 23 What Can We Measure? “DevOps is too risky, we can’t do it.” (“Is it tho?”) (Deploys vs. any performance or quality metric: Apdex, Error Rates, etc)
  24. 24. ©2008–18 New Relic, Inc. All rights reserved Tag Your Build 24
  25. 25. ©2008–18 New Relic, Inc. All rights reserved 25 Tag Your Build Attach metadata about the build itself to your build artifact Make the runtime application aware of the build state Lets you relate metrics and performance data to code revisions Invaluable for retroactive investigation Attributes: Source code revision Build number
  26. 26. ©2008–18 New Relic, Inc. All rights reserved 26 Tag Your Build As a build step, pull metadata from the build environment and insert it into the artifact Place it somewhere that your app can pick it up at runtime Attach it as a custom attribute using the New Relic APM library
  27. 27. ©2008–18 New Relic, Inc. All rights reserved 27 Tag Your Build
  28. 28. ©2008–18 New Relic, Inc. All rights reserved 28 What Can We Measure? Every event and metric is versioned to your code
  29. 29. ©2008–18 New Relic, Inc. All rights reserved Record Test Results 29
  30. 30. ©2008–18 New Relic, Inc. All rights reserved 30 Record Unit Test Results Most test runners will provide status metrics via a “reporter” Attributes: Total test cases/suites # passed # failed # skipped (tech debt here) duration code coverage
  31. 31. ©2008–18 New Relic, Inc. All rights reserved 31 What Can We Measure?
  32. 32. ©2008–18 New Relic, Inc. All rights reserved 32 What Can We Measure?
  33. 33. ©2008–18 New Relic, Inc. All rights reserved 33 What Can We Measure? “How much of our code remains untested?”
  34. 34. ©2008–18 New Relic, Inc. All rights reserved Record Deployments 34
  35. 35. ©2008–18 New Relic, Inc. All rights reserved 35 Record Deployments Use the same state change event, but only when it’s a successful deploy Attributes: SCM revision change log user
  36. 36. ©2008–18 New Relic, Inc. All rights reserved 36 Record Deployments
  37. 37. Thank You ©2008–18 New Relic, Inc. All rights reserved Webinar: https://newrelic.com/webinar/best-practices-code-pipeline-180912 Source code: https://github.com/newrelic/webinar

×