AI-Powered DevOps: Injecting Speed & Quality Across Verizon’s Cloud Pipelines
•Download as PPTX, PDF•
2 likes•682 views
Customer experience is a top priority for Verizon, so they turned to DevOps best practices to address technical issues. The result was tremendous – 3x faster build times and a 50% drop in reported bugs – all within the first six months! This success evolved to an automated delivery pipeline approach that leveraged cloud and container technology, and teams were able to deploy new features faster, directly into production. With faster releases came vast technical complexity. Using Artificial Intelligence (AI)-powered monitoring, Verizon was able to transcend this problem, and today easily manages complex, web-scale DevOps in the cloud. • Learn the DevOps “shift-left” quality model Verizon embraced to find issues before they reached production. • Discover best practices that accelerated Verizon’s build and test cycle times by 3x in just six months. • Understand what AI-powered technology is, and how it can help you master the complexity of DevOps cloud environments. • Learn how Verizon uses DevOps and cloud monitoring as part of one integrated application delivery chain. You’ll gain best practices and insights you can use immediately!
Recommended
More Related Content
What's hot
What's hot (20)
Similar to AI-Powered DevOps: Injecting Speed & Quality Across Verizon’s Cloud Pipelines
Similar to AI-Powered DevOps: Injecting Speed & Quality Across Verizon’s Cloud Pipelines (20)
More from Dynatrace
More from Dynatrace (20)
Recently uploaded
Recently uploaded (20)
AI-Powered DevOps: Injecting Speed & Quality Across Verizon’s Cloud Pipelines
- 1. Andreas GrabnerAnil Chintalapudi Director, Verizon Enterprise Solutions grabnerandi DevOps Activist, Dynatrace AI-Powered DevOps Injecting Speed & Quality Across Verizon’s Cloud Pipelines @grabnerandianilchintalapudi
- 2. #2 Status Quo: Frictions#1 First DevOps Successes #4 AI-Powered DevOps Tool Chain #3 Next: Empowerment
- 3. #1 First DevOps Successes
- 4. What have we done and achieved so far? •3xfaster build and test cycles •50% faster deployment (every two weeks, rather than once a month) •33% faster revenue realization •50% reduction in issues #1 Deployment & Pipeline #2 Feedback Loops #3 Cultural Change AWS Webinar: https://pages.awscloud.com/devops_dynatrace_june_2017.html
- 5. #1: Pipeline & Deployments
- 7. 1 Monolithic App 6 Weeks to DevOps’ify App Goal: Cloud Continuous Delivery Dev take Ops Ownership #3: Culture
- 8. #2 Status Quo: Frictions
- 9. Status Quo: Challenges ALERT DROWNING
- 11. Empowering DevOps Operation Teams One Click/Auto Mitigating Action
- 12. #4 AI-Powered DevOps Tool Chain
- 13. Artificial Intelligence – Verizon's Own Approach
- 15. Dev Perf/Test Ops Biz Shift-Left: Break Pipeline Earlier Improve MTTR: Auto-Mitigation Shift-Right: Tags, Deploys, Events Actionable Feedback Loops
- 16. Shift-Right: Tags, Deployments & Events docker run –e DT_TAGS=BLUE dtcli tag srv CartServicev2 GREEN dtcli evt push host .*demo version=123 source={git_commit} dtcli evt push pg tomcat1 desc=JVMMemIncr hint=+100MB Dynatrace Smartscape Release Automation Dynatrace Automation API, CLI, Auto-Detection
- 17. Improve MTTR: Automate Mitigate with AI Data Auto Mitigate! 1 CPU Exhausted? Add a new service instance! 3 Issue with BLUE only? Switch back to GREEN! ?Escalate at 2AM? 2 High Garbage Collection? Adjust/Revert Memory Settings! 4 Hung threads? Restart Service! 5 Still ongoing? Initiate Rollback! Escalate ? Still ongoing?5 1 2 3 4 Mark Bad Commits Update Dev Tickets … … Impact Mitigated?? ?
- 18. Shift-Left: Break Pipeline Earlier c0123bd nov17 myservice:nov17 myservice:nov17 space:UAT space:PERF myservice:nov17 Selenium Perf Data space:PERF myservice:nov17myservice:nov16 space:PROD myservice:BLUE myservice:GREEN myservice:nov17 space:PROD myservice:BLUE myservice:GREEN space:PROD myservice:nov16
- 19. Shift-Left: Performance as Self-Service myservice:tmp57 myservice:tmp57 space:PERF c0123bd
- 20. Actionable Feedback Loops: Business Success Criteria Labels become Key User Action Live Data Queries New Requirement Definition
- 21. Dev Perf/Test Ops Biz Shift-Left: Break Pipeline Earlier Improve MTTR: Auto-Mitigation Shift-Right: Tags, Deploys, Events Actionable Feedback Loops
- 22. DevOps is about … … creating more value … … making money faster … … by also making your life easier!
- 23. Questions? Give this a try and see for yourself! Free Trial: http://bit.ly/dtsaastrial Andreas GrabnerAnil Chintalapudi Director, Verizon Enterprise Solutions grabnerandi DevOps Activist, Dynatrace @grabnerandianilchintalapudi
- 24. Connect with us! Participate in our Forum : community.dynatrace.com Like us on Facebook : facebook.com/dynatrace Follow us on LinkedIn : linkedin.com/company/dynatrace Follow us on Twitter : twitter.com/dynatrace Watch our Videos & Demos : youtube.com/dynatrace Read our Blog : https://www.dynatrace.com/blog/
Editor's Notes
- Lets start with what Verizon has already achieved. If you want to learn more – we talked about this during our AWS Webinar
- At PERFORM I remember you talking a lot about how much you invested in your Pipeline, Automated Deployments as well as in your whole DevOps Tool chain
- I also remember you using Hygieia and actually extending it to pull in Dynatrace UEM Data
- I was also fascinated about how you changed culturally by allowing development teams to learn what it means to do “Test Driven Development”, “Deploying into the Cloud” and “Operating an Application”
- We have seen many new technologies on the horizon. And with DevOps promoting “Choose the best technology for your problem” you quickly end up with a lot of moving pieces
- The problem is that most monitoring tools simply aren’t good enough anymore. That’s not to say they have zero value, they’re just not good enough. Why is that?
- Despite expensive investments in multiple monitoring tools, we still see companies doing this bad behavior… war rooms, reactive, finger pointing... You can’t afford to have people watching dashboards all the time. You can’t afford to have 50 people in a war room or on bridge call responding to a problem. If you’re doing this, you’re not innovating. And if you’re not innovating, you will become digital prey.
- DevOps Operations Teams Have a lot of “Feedback Loop data” but not the expertise to understand and act on it on their own Feedback Loop is not Actionable! Alert Drowning! Results in Large Meetings during “Crisis Management” Actual Numbers from Verizon on # of People & Hours involved in Root Cause Analysis and Remediating Actions Explain the typical “MANUAL ACTIONS” of a “Crisis Management Situation” Fault Domain Isolation: App, Infra or Configuration Problem Get evidence on potential root cause, e.g: look up log files, code or config changes … After consulting with “domain experts” decide on whether to A) FIX B) ROLLBACK
- Now lets learn from some of those companies that made a transformation and those where DevOps was always in their DNA
- Better Root Cause Information Linking all the evidence from Code Change to Impact to Problem Ticket “One Click” Remediation Action Automating manual Remediation Actions to be executed by DevOps Operations Team “One Click” follow up actions Open Tickets in JIRA, … Post Mortems in Confluence, …
- Quickly Explain the approach Verizon itself took
- And how that has inspired Dynatrace to build our own AI to support exactly these use cases for better production monitoring
- But there is more than production! There is more we can do throughout the whole DevOps Toolchain
- But there is more than production! There is more we can do throughout the whole DevOps Toolchain