This is the presentation given for the Docker Meetup in Cordoba, Argentina. Recording should soon be up on http://www.meetup.com/Docker-Cordoba-ARG/events/226995018/
Key Takeaways: Pick your Metrics! Automate It! Fail Bad Builds Faster! Deliver Faster with Better Quality!
To the Docker Audience my main point was that: Just adding Docker doesn't give you free performance and scalability of your app. I walk through many examples of failing apps. What are the metrics that highlight the problem and how to automatically detect bad builds by looking at these Metrics along your Pipeline.
BTD2015 - Your Place In DevTOps is Finding Solutions - Not Just Bugs!Andreas Grabner
This is about leveling-up and REVOLUTIONIZING Testing as part of your Agile/DevOps Transformation.
You can contribute more than testing functionality. You need to Level-Up your skill set by understanding the apps you are testing. # Images, # JS Files, # SQL Statements, Connection Pool Utilization and Garbage Collection Activity have to be added to your portfolio.
Check these metrics when you do your functional testing and report regressions to your engineers even though the functionality is still good. But you just uncovered an Architectural regression that will lead to a scalabilty and performance problem.
Finding these problems early will eliminate a lot of wasted and unplanned time later on in the lifecycle. that is your contribution to delivering software faster with better quality
Mobile User Experience:Auto Drive through Performance MetricsAndreas Grabner
Believe it or not - 85% of mobile apps are removed after first usage! In this presentation - given at the APM Meetup in Singapore in April 2015 - I talked about the challenges, best practices and especially metrics to avoid this situation.
Key Points of the Presentation
The two key trends "Internet of Things" and "DevOps" play a big role in our life when we talk about User Experience and especially mobile user experience. In this presentation I tell you what metrics to use to make sure you deliver your ideas faster to your mobile end users but also ensuring the right quality and user experience so that your users stay loyal and dont delete the mobile app after first usage.
Hugs instead of Bugs: Dreaming of Quality Tools for Devs and TestersAndreas Grabner
I have a Dream that Testers extend their horizon and toolsets and not only test for functional correctness but make a step towards what developers need in order to fix critical issues. I am talking about architectural, scalability and performance metrics such as # of JS Files on a page, Page Size, # of SQL Statements, # of Log Messages Written.
If Testers start to capture this information as well and share it with their bug description I am sure it will both increase the value of testers as well as reduce the total time it takes to fix problems.
Automate Application Quality Detection. Use Key Application Quality Metrics (# of SQL, Memory Allocated, CPU & GC Times, ...) captured during Automated Test Executions.
Let these Metrics act as Quality Gates. Leads to better quality software reaching the end of the Pipeline
Web and App Performance: Top Problems to avoid to keep you out of the NewsAndreas Grabner
As presented at Boston and NYC Web Perf Meetup.
Its time to level up Web Performance Optimization started by Steve Souders. We need to look beyond the rim of the browser as there are many problems happenig from browser to database.
In this presentation I showed how Browser Diagnostics needs to evolve into End-to-End Application Diagnostics and Monitoring. Showing 5 real life examples on why applications failed and the metrics to look at to identify these problems early on
Top Java Performance Problems and Metrics To Check in Your PipelineAndreas Grabner
Why is Performance Important? What are the most common reasons applications dont scale and perform well. Which technical metrics to look at. How to check it automated in the pipeline
Four Practices to Fix Your Top .NET Performance ProblemsAndreas Grabner
Inefficient Database Access, Inefficien Pool usage and Sizing, Bad Synchronization, Bad Web Page Design - these are the problems that crash .NET Apps. Learn how to analyze them and fix these problems
BTD2015 - Your Place In DevTOps is Finding Solutions - Not Just Bugs!Andreas Grabner
This is about leveling-up and REVOLUTIONIZING Testing as part of your Agile/DevOps Transformation.
You can contribute more than testing functionality. You need to Level-Up your skill set by understanding the apps you are testing. # Images, # JS Files, # SQL Statements, Connection Pool Utilization and Garbage Collection Activity have to be added to your portfolio.
Check these metrics when you do your functional testing and report regressions to your engineers even though the functionality is still good. But you just uncovered an Architectural regression that will lead to a scalabilty and performance problem.
Finding these problems early will eliminate a lot of wasted and unplanned time later on in the lifecycle. that is your contribution to delivering software faster with better quality
Mobile User Experience:Auto Drive through Performance MetricsAndreas Grabner
Believe it or not - 85% of mobile apps are removed after first usage! In this presentation - given at the APM Meetup in Singapore in April 2015 - I talked about the challenges, best practices and especially metrics to avoid this situation.
Key Points of the Presentation
The two key trends "Internet of Things" and "DevOps" play a big role in our life when we talk about User Experience and especially mobile user experience. In this presentation I tell you what metrics to use to make sure you deliver your ideas faster to your mobile end users but also ensuring the right quality and user experience so that your users stay loyal and dont delete the mobile app after first usage.
Hugs instead of Bugs: Dreaming of Quality Tools for Devs and TestersAndreas Grabner
I have a Dream that Testers extend their horizon and toolsets and not only test for functional correctness but make a step towards what developers need in order to fix critical issues. I am talking about architectural, scalability and performance metrics such as # of JS Files on a page, Page Size, # of SQL Statements, # of Log Messages Written.
If Testers start to capture this information as well and share it with their bug description I am sure it will both increase the value of testers as well as reduce the total time it takes to fix problems.
Automate Application Quality Detection. Use Key Application Quality Metrics (# of SQL, Memory Allocated, CPU & GC Times, ...) captured during Automated Test Executions.
Let these Metrics act as Quality Gates. Leads to better quality software reaching the end of the Pipeline
Web and App Performance: Top Problems to avoid to keep you out of the NewsAndreas Grabner
As presented at Boston and NYC Web Perf Meetup.
Its time to level up Web Performance Optimization started by Steve Souders. We need to look beyond the rim of the browser as there are many problems happenig from browser to database.
In this presentation I showed how Browser Diagnostics needs to evolve into End-to-End Application Diagnostics and Monitoring. Showing 5 real life examples on why applications failed and the metrics to look at to identify these problems early on
Top Java Performance Problems and Metrics To Check in Your PipelineAndreas Grabner
Why is Performance Important? What are the most common reasons applications dont scale and perform well. Which technical metrics to look at. How to check it automated in the pipeline
Four Practices to Fix Your Top .NET Performance ProblemsAndreas Grabner
Inefficient Database Access, Inefficien Pool usage and Sizing, Bad Synchronization, Bad Web Page Design - these are the problems that crash .NET Apps. Learn how to analyze them and fix these problems
OOP 2016 - Building Software That Eats The WorldAndreas Grabner
According to VC and web pioneer Marc Andreessen software is eating the world. Evidence proves he is right. Uber, the biggest taxi company, has no cars, AirBnB, the biggest hotel service, has no rooms and there are many more examples. Looking at these success stories there is a clear blueprint how to build software that eats the world. Just a quick heads up: It is not about building your typical web application any more.
From Zero to Performance Hero in Minutes - Agile Testing Days 2014 PotsdamAndreas Grabner
As a Tester you need to level up. You can do more than functional verification or reporting Response Time
In my Performance Clinic Workshops I show you real life exampls on why Applications fail and what you can do to find these problems when you are testing these applications.
I am using Free Tools for all of these excercises - especially Dynatrace which gives full End-to-End Visibility (Browser to Database). You can test and download Dynatrace for Free @ http://bit.ly/atd2014challenge
Too many database queries, too much data loaded into memory, overloaded html pages, bad architectural decisions, ...
These are all reasons why Java Applications are slow. In this presentation - first given at Boston Java Meetup - shows 6 real life examples on why Java-based Applications failed - and you may even heard about this in the news.
All examples and the technical details were captured using Dynatrace which is available as a 30 Day Free Trial - http://bit.ly/dttrial - with an option to extend it for another 180 Days in case you share some of your results with us
I gave this presentation at the Sydney Continuous Delivery Meetup Group. The main goal was to talk about Performance Metrics that you should monitor along the pipeline. I examples in 4 different areas where deployments failed and how metrics would have helped preventing these problems
Top .NET, Java & Web Performance Mistakes - Meetup Jan 2015Andreas Grabner
Top .NET, Java & Web Performance Problems. Why these apps failed, how to avoid it and which metrics to look at, e.g: # of Busy vs. Idle Worker Threads, Connection Pool Acquisition Time, # Exceptions Thrown, ...
This presentation was given as part of a Dynatrace Lunch & Learn event. APM (=Application Performance Management) allows us to transform the way we develop, deploy and run software.
Here are some ideas how APM can be (r)evolutionized
Deploy Faster Without Failing Faster - Metrics-Driven - Dynatrace User Groups...Andreas Grabner
Do it like the "DevOps Unicorns" Etsy, Facebook and Co: Deploy more frequently. But how and why? Challenges?
Deploying Software Faster without Failing Faster is possible through Metrics driven Engineering. Identify problems early on using a "Shift-Left in Quality". This requires a Level-Up of Dev, Test, Ops, Biz
See some of the metrics that I think you need to look at and how to upgrade your engineering team to produce better quality right from the start
How to keep you out of the News: Web and End-to-End Performance TipsAndreas Grabner
Too many websites make it too the news when they fail to deliver, e.g: eCommerce when they go down on Cyber Monday, Tax Software on Tax Day or Online Banking when people want to check on their latest pay check.
In this presentation - presented at several Web Performance, Java, .NET, ... Meetups I walk through the most common performance mistakes people made in recent history. I explain in technical detail what the problem was and how to find these problems earlier as you dont want to wait until your site crashes and you end up in the news.
These are the slides used in my #devone (www.devone.at) keynote presentation:
DevOps is one of the most abused and overrated marketing terms in the last years! That’s not an alternative fact! It’s just Andi’s opinion! Yet - it is a very real thing that allowed many software companies to transform the way they think about software engineering. DevOps can mean something totally different thought depending on who you are and what type of business your company is doing. To clarify things, Andi gives us insights on how he explains the benefits to “DevOps Newbies” and how software companies around the world implement it in their own ways. Andi will answer: What does it really mean for developers, testers and operators? What will change? How does Facebook deploy twice a day without big issues? How does DevOps work in financial, government or healthcare where you have tight regulations? Does it mean Devs are responsible for Ops? Does it only work in the cloud? Or can we apply it to “old fashioned” on premise software as well? Learn for yourself and make up your own mind on whether DevOps is just a marketing term or something that can benefit you!
Boston DevOps Days 2016: Implementing Metrics Driven DevOps - Why and HowAndreas Grabner
How can we detect a bad deployment before it hits production? By automatically looking at the right architectural metrics in your CI/CD and stop a build before its too late. Lets hook up your test automation with app metrics and use them as quality gates to stop bad builds early!
Metrics Driven DevOps - Automate Scalability and Performance Into your PipelineAndreas Grabner
Continuous Delivery only works if you combine automation with automatic metrics driven quality gates focusing on architectural, scalabilty and performance metrics.
In this presentation I start with several dashboard examples explaining key metrics in production and explain how to automate these metrics into your delivery pipeline.
DevOps Pipelines and Metrics Driven Feedback LoopsAndreas Grabner
The goal behind devops is Faster Lead Times
What this really means for Software Delivery -> my Kodak/Smart Phone Analogy
How and Which Metrics to use along the Delivery Pipeline to make better decisions along the way.
Metrics-Driven Devops: Delivering High Quality Software Faster! Dynatrace
Becoming the next Uber is only possible if you can deliver your code updates faster to your end users. But for your organization, does delivering code faster present a higher likelihood of failing faster?
Discover four metrics you should be tracking starting from your workstation all the way through CI and into Ops.
Learn how companies like Facebook, CreditOne, and others apply metric-driven DevOps.
See use cases of crashed rapid deployments and how they used the metrics to detect the root cause.
Learn how to apply these metrics to steer your pipeline to build better code and deploy faster, without failing faster!
JavaOne - Performance Focused DevOps to Improve Cont DeliveryAndreas Grabner
These are the slides of my JavaOne presentation. The abstract goes like this:
How do companies developing business-critical Java enterprise Web applications increase releases from 40 to 300 per year and still remain confident about a spike of 1,800 percent in traffic during key events such as Super Bowl Sunday or Cyber Monday? It takes a fundamental change in culture. Although DevOps is often seen as a mechanism for taming the chaos, adopting an agile methodology across all teams is only the first step. This session explores best practices for continuous delivery with higher quality for improving collaboration between teams by consolidating tools and for reducing overhead to fix issues. It shows how to build a performance-focused culture with tools such as Hudson, Jenkins, Chef, Puppet, Selenium, and Compuware APM/dynaTrace
StarWest 2013 Performance is not an afterthought – make it a part of your Agi...Andreas Grabner
This presentation was given at StarWest 2013 in Anaheim, CA and also broadcasted through the Virtual Conference.
It shows how important it is to focus on performance throughout continuous delivery in order to avoid the most common performance problem patterns that still cause applications to crash and engineers spending their weekends and nights in a firefighting/war room situation
DevOps Days Toronto: From 6 Months Waterfall to 1 hour Code DeploysAndreas Grabner
Slides used for https://www.devopsdays.org/events/2017-toronto/program/andreas-grabner/
In 2011 we delivered 2 major releases of our on premise enterprise software. Market, technology and customer requirements forced us to change that in order to remain competitive.
Now – in 2017 - we are deploying and providing feature releases every 2 weeks for both our on premise and SaaS-based offering. We deploy 170 SaaS production changes per day and have a DevOps pipeline that allows us to deploy a code change within 1h if necessary.
To increase quality, we built and provide a DevOps pipeline that currently executes 31000 Unit & Integration Tests per Hour as well as 60h UI Tests per Build. Our application teams are responsible end-to-end for their features and use production monitoring to validate their deployments which allows them to find 93% of bugs in production before it impacts our end users.
In this session I explain how this transformation worked from both “Top Down” as well as “Bottom Up” in our organization. A key component was the 4 people strong DevOps Team who developed and “sell” their DevOps Pipeline to the globally distributed application teams. I will give insights into how our pipeline enables application teams to design, code, test and run a new feature for our user base.
I will also talk about the “dark moments” as change is never without friction. Both internally as well as with our customers who also had to get used to more rapid changes.
Presentation that explains the main concepts used with dynaTrace.
dynaTrace is a tool to perform browser performance analysis (for JS, images, CSS, network, caching, ..etc)
OOP 2016 - Building Software That Eats The WorldAndreas Grabner
According to VC and web pioneer Marc Andreessen software is eating the world. Evidence proves he is right. Uber, the biggest taxi company, has no cars, AirBnB, the biggest hotel service, has no rooms and there are many more examples. Looking at these success stories there is a clear blueprint how to build software that eats the world. Just a quick heads up: It is not about building your typical web application any more.
From Zero to Performance Hero in Minutes - Agile Testing Days 2014 PotsdamAndreas Grabner
As a Tester you need to level up. You can do more than functional verification or reporting Response Time
In my Performance Clinic Workshops I show you real life exampls on why Applications fail and what you can do to find these problems when you are testing these applications.
I am using Free Tools for all of these excercises - especially Dynatrace which gives full End-to-End Visibility (Browser to Database). You can test and download Dynatrace for Free @ http://bit.ly/atd2014challenge
Too many database queries, too much data loaded into memory, overloaded html pages, bad architectural decisions, ...
These are all reasons why Java Applications are slow. In this presentation - first given at Boston Java Meetup - shows 6 real life examples on why Java-based Applications failed - and you may even heard about this in the news.
All examples and the technical details were captured using Dynatrace which is available as a 30 Day Free Trial - http://bit.ly/dttrial - with an option to extend it for another 180 Days in case you share some of your results with us
I gave this presentation at the Sydney Continuous Delivery Meetup Group. The main goal was to talk about Performance Metrics that you should monitor along the pipeline. I examples in 4 different areas where deployments failed and how metrics would have helped preventing these problems
Top .NET, Java & Web Performance Mistakes - Meetup Jan 2015Andreas Grabner
Top .NET, Java & Web Performance Problems. Why these apps failed, how to avoid it and which metrics to look at, e.g: # of Busy vs. Idle Worker Threads, Connection Pool Acquisition Time, # Exceptions Thrown, ...
This presentation was given as part of a Dynatrace Lunch & Learn event. APM (=Application Performance Management) allows us to transform the way we develop, deploy and run software.
Here are some ideas how APM can be (r)evolutionized
Deploy Faster Without Failing Faster - Metrics-Driven - Dynatrace User Groups...Andreas Grabner
Do it like the "DevOps Unicorns" Etsy, Facebook and Co: Deploy more frequently. But how and why? Challenges?
Deploying Software Faster without Failing Faster is possible through Metrics driven Engineering. Identify problems early on using a "Shift-Left in Quality". This requires a Level-Up of Dev, Test, Ops, Biz
See some of the metrics that I think you need to look at and how to upgrade your engineering team to produce better quality right from the start
How to keep you out of the News: Web and End-to-End Performance TipsAndreas Grabner
Too many websites make it too the news when they fail to deliver, e.g: eCommerce when they go down on Cyber Monday, Tax Software on Tax Day or Online Banking when people want to check on their latest pay check.
In this presentation - presented at several Web Performance, Java, .NET, ... Meetups I walk through the most common performance mistakes people made in recent history. I explain in technical detail what the problem was and how to find these problems earlier as you dont want to wait until your site crashes and you end up in the news.
These are the slides used in my #devone (www.devone.at) keynote presentation:
DevOps is one of the most abused and overrated marketing terms in the last years! That’s not an alternative fact! It’s just Andi’s opinion! Yet - it is a very real thing that allowed many software companies to transform the way they think about software engineering. DevOps can mean something totally different thought depending on who you are and what type of business your company is doing. To clarify things, Andi gives us insights on how he explains the benefits to “DevOps Newbies” and how software companies around the world implement it in their own ways. Andi will answer: What does it really mean for developers, testers and operators? What will change? How does Facebook deploy twice a day without big issues? How does DevOps work in financial, government or healthcare where you have tight regulations? Does it mean Devs are responsible for Ops? Does it only work in the cloud? Or can we apply it to “old fashioned” on premise software as well? Learn for yourself and make up your own mind on whether DevOps is just a marketing term or something that can benefit you!
Boston DevOps Days 2016: Implementing Metrics Driven DevOps - Why and HowAndreas Grabner
How can we detect a bad deployment before it hits production? By automatically looking at the right architectural metrics in your CI/CD and stop a build before its too late. Lets hook up your test automation with app metrics and use them as quality gates to stop bad builds early!
Metrics Driven DevOps - Automate Scalability and Performance Into your PipelineAndreas Grabner
Continuous Delivery only works if you combine automation with automatic metrics driven quality gates focusing on architectural, scalabilty and performance metrics.
In this presentation I start with several dashboard examples explaining key metrics in production and explain how to automate these metrics into your delivery pipeline.
DevOps Pipelines and Metrics Driven Feedback LoopsAndreas Grabner
The goal behind devops is Faster Lead Times
What this really means for Software Delivery -> my Kodak/Smart Phone Analogy
How and Which Metrics to use along the Delivery Pipeline to make better decisions along the way.
Metrics-Driven Devops: Delivering High Quality Software Faster! Dynatrace
Becoming the next Uber is only possible if you can deliver your code updates faster to your end users. But for your organization, does delivering code faster present a higher likelihood of failing faster?
Discover four metrics you should be tracking starting from your workstation all the way through CI and into Ops.
Learn how companies like Facebook, CreditOne, and others apply metric-driven DevOps.
See use cases of crashed rapid deployments and how they used the metrics to detect the root cause.
Learn how to apply these metrics to steer your pipeline to build better code and deploy faster, without failing faster!
JavaOne - Performance Focused DevOps to Improve Cont DeliveryAndreas Grabner
These are the slides of my JavaOne presentation. The abstract goes like this:
How do companies developing business-critical Java enterprise Web applications increase releases from 40 to 300 per year and still remain confident about a spike of 1,800 percent in traffic during key events such as Super Bowl Sunday or Cyber Monday? It takes a fundamental change in culture. Although DevOps is often seen as a mechanism for taming the chaos, adopting an agile methodology across all teams is only the first step. This session explores best practices for continuous delivery with higher quality for improving collaboration between teams by consolidating tools and for reducing overhead to fix issues. It shows how to build a performance-focused culture with tools such as Hudson, Jenkins, Chef, Puppet, Selenium, and Compuware APM/dynaTrace
StarWest 2013 Performance is not an afterthought – make it a part of your Agi...Andreas Grabner
This presentation was given at StarWest 2013 in Anaheim, CA and also broadcasted through the Virtual Conference.
It shows how important it is to focus on performance throughout continuous delivery in order to avoid the most common performance problem patterns that still cause applications to crash and engineers spending their weekends and nights in a firefighting/war room situation
DevOps Days Toronto: From 6 Months Waterfall to 1 hour Code DeploysAndreas Grabner
Slides used for https://www.devopsdays.org/events/2017-toronto/program/andreas-grabner/
In 2011 we delivered 2 major releases of our on premise enterprise software. Market, technology and customer requirements forced us to change that in order to remain competitive.
Now – in 2017 - we are deploying and providing feature releases every 2 weeks for both our on premise and SaaS-based offering. We deploy 170 SaaS production changes per day and have a DevOps pipeline that allows us to deploy a code change within 1h if necessary.
To increase quality, we built and provide a DevOps pipeline that currently executes 31000 Unit & Integration Tests per Hour as well as 60h UI Tests per Build. Our application teams are responsible end-to-end for their features and use production monitoring to validate their deployments which allows them to find 93% of bugs in production before it impacts our end users.
In this session I explain how this transformation worked from both “Top Down” as well as “Bottom Up” in our organization. A key component was the 4 people strong DevOps Team who developed and “sell” their DevOps Pipeline to the globally distributed application teams. I will give insights into how our pipeline enables application teams to design, code, test and run a new feature for our user base.
I will also talk about the “dark moments” as change is never without friction. Both internally as well as with our customers who also had to get used to more rapid changes.
Presentation that explains the main concepts used with dynaTrace.
dynaTrace is a tool to perform browser performance analysis (for JS, images, CSS, network, caching, ..etc)
My Application is Slow | Best Practices for Troubleshooting and PreventioneG Innovations
IT performance management isn’t about monitoring CPU, memory or disk space any more. One of the toughest application performance challenges for any IT administrator is when a user says "my application is slow". You have to be able to quickly determine what the real cause of the problem is - is it in the network, the database, the application, storage? The fact that applications are using multi-tier architectures and being deployed in cloud and virtualized infrastructures only adds to the challenge.
View these slides from our webinar where Frank Ohlhorst, Enterprise IT Analyst & Consultant and Srinivas Ramanathan, CEO of eG Innovations, discussed the best practices for troubleshooting and prevention so even before a user complains their application is slow, you can pinpoint exactly where the cause of a problem lies – ensuring quick resolution and a positive user experience.
Performance monitoring and call tracing in microservice environmentsMartin Gutenbrunner
Performance analysis can easily be done with on-board tools of nearly any programming language. In microservice environments, the real challenge is not in single, high-performing services, but in resiliently running a complex ecosystem of many services.This talk will introduce open-source tools for analysis and call tracing. Concluding, we will briefly get to know Dynatrace Ruxit - a commercial alternative. After this session, the audience will know about how to get started in performance analysis and call-tracing and some according tools.
Managing and Monitoring Application PerformanceSebastian Marek
Writing your application is one thing. Making the application to perform well is another. We usually forget there is somebody else on the other side of the screen, that becomes very frustrated and upset when he needs to wait until this one page finally loads. It requires a lot of experience to predict specific behaviour and to know what kind of things to avoid. And even with that there is so many different factors that can affect the end user experience. During this talk I will talk about tools and techniques you can use to measure and monitor your application performance.
Taking AppSec to 11 - BSides Austin 2016Matt Tesauro
Curious how DevOps, Agile and CI/CD ideas can speed up your AppSec program? Here's how it can be done and an example where it lead to a 5x speed/flow improvement.
Taking AppSec to 11: AppSec Pipeline, DevOps and Making Things BetterMatt Tesauro
Slide deck from AppSec California 2016 + some additional slides.
Abstract:
How many applications are in your company’s portfolio? What’s the headcount for your AppSec team? Whatever your situation is, I am sure the numbers are not in your favor. Its not time to find a new career, it's time to up your game. This talk will cover how to take your small merry band of AppSec professionals and scale it up to a virtual army. By taking the best of DevOps, Agile and CI/CD, you can iteratively up your AppSec game over time and begin your ascent out of the security hole you are in.
The talk covers real world experiences running AppSec groups at two different companies. Rackspace with approximately 4,000+ employees and Pearson with 40,000+. Both have an international presence and far more apps and developers that AppSec staff. The talk covers the key principles to speed and scale up AppSec programs using an AppSec Pipeline as well as practical examples of these practices put into use. Start early and begin to buy down the technical security dept which feels inevitable with more traditional AppSec program thinking.
Learn how Site24x7 gives you end-to-end application performance visibility for your Java, .NET and Ruby web transactions with metrics of all components starting from URLs to SQL queries.
Our DevOps Journey
Transforming 6 Month Waterfalls to 1 Hour Code Deploys
https://info.dynatrace.com/17q3_wc_from_agile_to_cloudy_devops_na_registration.html
In the 2nd part of our webinar series, Anita Engleder, DevOps Lead at Dynatrace reviews and dissects lessons learned during the transformational journey moving Dynatrace from an on-prem culture to one that is cloud native. She will lend her perspective as a key member of the team that executed on the original vision: to “implement a new cloud native offering and deploy a new feature release every 2 weeks. Additionally, be able to support a 1-hour lead time from Code Change to Production”.
On November 17th at 1pm/10am PT Anita will present the challenges she and her team faced transforming 6 Months Waterfall to 1 Hour Code Deploys.
In this webinar Anita will discuss:
How to enable a complete cultural shift across multiple teams, in terms of thought process AND execution
What the specific role of her DevOps team is and how it played into the transformation
The role of Feature teams and why continuous feedback is critical for them
How to successfully influence key stakeholders for complete alignment
Today Anita’s team runs 170 production changes every day, running across several AWS Data Centers as well as On-Premise – something that would have been thought impossible only a few years prior.
Application Performance Management (APM) provides a 360° view that keeps your business healthy by monitoring end user experience and applications across traditional, mobile, virtual and cloud environments. It provides insight into every transaction for quick resolution of application issues and helps reduce costs by using a common tool in pre-production and production. Detailed diagnostics and real-time topology-based analytics improve application quality.
Trace real user transactions across application tiers to speed resolution times
Measure end user experience using repeatable transactions from multiple locations
Gain deep application insights for fast problem isolation and resolution
Use shared scripting across testing and operations for higher quality applications
AWS and Dynatrace: Moving your Cloud Strategy to the Next LevelDynatrace
AWS and Dynatrace: Moving your Cloud Strategy to the Next Level
On-Demand Webcast
AWS re:Invent was an exciting time for Dynatrace and we received a lot of “Wows” on our capabilities. We got to demonstrate the only AI-based, full-stack monitoring solution to thousands of AWS prospects and users. We announced our AWS Certified DevOps Competency partnership, and we introduced DAVIS, our natural-language voice interface, to thousands of attendees.
We know that many of you couldn’t attend the event in Las Vegas, so we wanted to share some of the key highlights from the show. And for those of you who were there, you may not have seen all of the benefits Dynatrace provides in the AWS ecosystem due to time constraints of sessions and the large tradeshow floor.
Listen to this 30 Minute webcast where Alois Reitbauer and Franz Karlsberger recap some of the highlights of the event, including:
How Dynatrace, as an AWS certified Migration Competency partner, uniquely supports enterprise migrations to AWS
How to achieve faster feedback and improved lead times with AWS CodePipeline and Dynatrace
An overview of the first ever VoiceOps and ChatOps interface via DAVIS, based on our AI approach to full-stack monitoring
Performance Metrics for your Build Pipeline - presented at Vienna WebPerf Oct...Andreas Grabner
Software Performance Metrics that you should look at throughout your Build Pipeline and not just when your app crashes in productiong.
Find performance and scalability problems as soon as executing your first Unit Test. Simply focus on metrics such as #SQLs, #LogMessages, #Objects on Heap, ...
The promise of DevOps is that we can push new ideas out to market faster while avoiding delivering serious defects into production. Andreas Grabner explains that testers are no longer measured by the number of defect reports they enter, nor are developers measured by the lines of code they write. As a team, you are measured by how fast you can deploy high quality functionality to the end user. Achieving this goal requires testers to increase their skills. It’s all about finding solutions—not just problems. Testers must transition from reporting “app crashes” to providing details such as “memory leak caused by bad cache implementation.” Instead of reporting “it’s slow,” testers must discover “wrong hibernate configuration causes too much traffic from the database.” Using three real-life examples, Andreas illustrates what it takes for testing teams to become part of the DevOps transformation—bringing more value to the entire organization.
Andreas Grabner maintains that most performance and scalability problems don’t need a large or long running performance test or the expertise of a performance engineering guru. Don’t let anybody tell you that performance is too hard to practice because it actually is not. You can take the initiative and find these often serious defects. Andreas analyzed and spotted the performance and scalability issues in more than 200 applications last year. He shares his performance testing approaches and explores the top problem patterns that you can learn to spot in your apps. By looking at key metrics found in log files and performance monitoring data, you will learn to identify most problems with a single functional test and a simple five-user load test. The problem patterns Andreas explains are applicable to any type of technology and platform. Try out your new skills in your current testing project and take the first step toward becoming a performance diagnostic hero.
STP 2014 - Lets Learn from the Top Performance Mistakes in 2013Andreas Grabner
Presentation given at STPCon 2014. It highlights the top performance problems seen in 2013 and how we can identify these problems in dev & test instead of waiting until the app crashes in production
Performance Quality Metrics for Mobile Web and Mobile Native - Agile Testing ...Andreas Grabner
5 Real Life Examples on why Mobile Web and Mobile Native Apps failed and Which Metrics would have shown the problem early on.
Using these metrics along your delivery chain allows you go get closer to full automated deployment pipeline but also making sure performance criteria is met
How to Build a Metrics-optimized Software Delivery PipelineDynatrace
Every company is under increased pressure to deliver software faster and better. The question is: “How do I get started?” Continuous firefighting is definitely not the answer!
XebiaLabs and Dynatrace share a practical step-by-step approach to optimizing your delivery process so you can deploy better quality software faster!
Learn:
• Why you should move to a metric-driven pipeline!
• Which key quality metrics to measure and how to integrate them to catch problems earlier
• How to use, measure and report on these metrics
• How finding architectural/quality issues earlier reduces cost spent investigating them
Did you know that just a handful of root causes cause the majority of application issues like crashes, slow performance or incorrect application behavior? Non-optimized database access, deployment mistakes, memory leaks, or inefficient coding are just some examples. Companies that think Continuous Delivery and DevOps will solve all their problems typically fail as they just run into these problems faster. In this session we take a closer look at the most common problems, how to detect them and how to incorporate performance into your DevOps culture by automatically detecting these top problems.
Monitoring and Scaling Redis at DataDog - Ilan Rabinovitch, DataDogRedis Labs
Think you have big data? What about high availability
requirements? At DataDog we process billions of data points every day including metrics and events, as we help the world
monitor the their applications and infrastructure. Being the world’s monitoring system is a big responsibility, and thanks to
Redis we are up to the task. Join us as we discuss how the DataDog team monitors and scales Redis to power our SaaS based monitoring offering. We will discuss our usage and deployment patterns, as well as dive into monitoring best practices for production Redis workloads
Operations: Production Readiness Review – How to stop bad things from HappeningAmazon Web Services
There is more to deploying code than pushing the deploy button. A good practice that many companies follow is a Production Readiness Review (PRR) which is essentially a pre-flight check list before a service launches. This helps ensure new services are properly architected, monitored, secured, and more. We’ll walk through an example PRR and discuss the value of ensuring each of these is properly taken care of before your service launches.
AWS re:Invent 2016: Amazon CloudFront Flash Talks: Best Practices on Configur...Amazon Web Services
In this series of 15-minute technical flash talks you will learn directly from Amazon CloudFront engineers and their best practices on debugging caching issues, measuring performance using Real User Monitoring (RUM), and stopping malicious viewers using CloudFront and AWS WAF.
SharePoint Performance: Physical to Virtual to Microsoft Azure Cloud and Offi...Joel Oleson
SharePoint has been on the move first from physical to virtual and then from Virtual to Azure and the move to Office 365. How to achieve good SharePoint Performance needs to be taken from a holistic approach. Can we find SharePoint Zen?
This webinar was a joint webinar with Joel Oleson of Konica Minolta and Andi Grabner from Dynatrace
Similar to Docker/DevOps Meetup: Metrics-Driven Continuous Performance and Scalabilty (20)
KCD Munich - Cloud Native Platform Dilemma - Turning it into an OpportunityAndreas Grabner
This talk was given at KCD Munich - July 17 2023
Abstract
“Kubernetes is a platform for building platforms. It’s a better place to start: not the endgame”, tweeted by Kelsey Hightower in November 2017. 6 years later the Cloud Native Community is faced with 159 different CNCF projects to choose from. Entering CNCF can be overwhelming!
Cloud Native Platform Engineering with white papers, best practices and reference architectures are here to convert this dilemma into an opportunity. Internal Developer Platforms (IDP) are being built as we speak enabling organizations to harness the power of Kubernetes as a self-service platform.
Join this talk with Andreas Grabner, CNCF Ambassador, and get some insights on tooling, use cases and best practices so we can all fulfill the idea that Kelsey put out years ago.
OpenTelemetry For GitOps: Tracing Deployments from Git Commit to ProductionAndreas Grabner
GitOps, with tools like Argo and Flux, are preferred platform tools managing configuration in cloud native environments. But it is hard to troubleshoot a failed deployment of a complex application as there is no built-in deployment lifecycle observability, standardized hooks nor the concept of an application vs individual workloads.
The CNCF project Keptn addresses those challenges by extending the Kubernetes Pod scheduler to provide OpenTelemetry Traces and Prometheus metrics for end-2-end deployment observability. Keptn introduces automated application-aware pre- and post-deployment lifecycle hooks to enforce dependency checks, send notifications or evaluates SLOs that otherwise need a custom K8s operator.
Join this talk and learn how the Keptn Lifecycle Toolkit (KLT) Operator extends observability into GitOps deployments and how it enables declarative deployment lifecycle orchestration!
Don't Deploy Into the Dark: DORA Metrics for your K8s GitOps DeploymentsAndreas Grabner
This talk was given at Boston Cloud Native Meetup on Feb 9th 2023
DORA’s Four Key DevOps have gained much attention as they provide critical insights into an organization’s maturity in automating the delivery of high-quality software. Google provides a blueprint implementation which requires extending your existing delivery pipelines (Jenkins, Argo, Flux, GitHub, GitLab …) to push those metrics to an external database. While doable, many platform engineers we spoke to are seeking an alternative solution and more cloud-native approach.
The CNCF project Keptn saw this as an opportunity to provide a K8s- & Cloud-Native solution that provides 100% coverage, WITHOUT changing pipelines and using OpenTelemetry as standard collection framework.
Join this talk where Andi (Andreas) Grabner, DevRel at Keptn, will show you how you can use Keptn’s Lifecyle Toolkit to get your DORA metrics within 5 minutes. Andi also covers how the Lifecycle Toolkit brings application-awareness into your deployments and allows you to execute pre- and post-deployment checks as serverless functions – all declaratively as part of your existing K8s CRDs.
Observability and Orchestration of your GitOps Deployments with KeptnAndreas Grabner
GitOps has become the default way to manage configuration in cloud native environments with tools like Argo or Flux keeping Git and K8s in sync.
But GitOps lacks end-2-end traceability when GitOps operators make changes on the target environments. And as k8s lacks application awareness its hard to enforce pre- and post-deployment orchestration task such as sending notifications upon successful app delivery or validating all SLOs are healthy for a new version.
The CNCF project Keptn is addressing those challenges by automatically providing End-2-End Observability through OpenTelemetry as well as introducing an application deployment lifecycle events enabling pre- and post-deployment checks natively on k8s.
Keptn therefore extends your GitOps approach with the missing observability and orchestration needed for successful cloud native development.
Release Readiness Validation with Keptn for Austrian Online Banking SoftwareAndreas Grabner
Marco and Andreas work at Raiffeisen Software who provides banking software for many Austrian financial institutions. In this session they show us how Keptn is used to automate the validation of key SLOs as part of their release process.
Adding Security to your SLO-based Release Validation with KeptnAndreas Grabner
This talk was given at DevSecOps Days Boston and DevOps & Security Meetup Vienna in 2021
Automatic Release Validation, aka Quality Gates, is not a new concept but often only covers functional or performance metrics. Keptn’s open SLO-based evaluation allows DevSecOps to have their favorite security tool report SLOs such as number of detected vulnerabilities as part of delivery automation
This talk was given at the Online Kubernetes Meetup July 2020 as well as DevOps Fusion 2020. The talk discusses 3 major problems in current delivery and operations: too much time spent in delivery, hard to maintain monolithic delivery pipelines and a lack of auto-remediation of production problems
The talk focuses on new approaches to solve these problems inspired by SRE practices and event-driven architectures.
As an implementation for a new approach we use Keptn (www.keptn.sh) - a CNCF Open Source project.
Continuous Delivery and Automated Operations on k8s with keptnAndreas Grabner
Slidedeck from Vienna DevOps & Security Meetup. This talk is keptn - an open source event driven control plane for continuous delivery and automated operations for kubernetes
Keptn - Automated Operations & Continuous Delivery for k8sAndreas Grabner
Keptn is a new OpenSource Framework for Automated Operations & Continuous Delivery for cloud native applications running on k8s, OpenShift, CloudFoundry ...
This presentation was used at Meetups to explain WHY we build keptn and which problems it solves in which way!
Applying AI to Performance Engineering: Shift-Left, Shift-Right, Self-HealingAndreas Grabner
AI might be an overused marketing buzzword but the advances in fullstack monitoring and data analytics are clearly an advantage for the modern performance engineer.
In this presentation given at different events, e.g: CMG Image, Neotys PAC, ... I discuss how monitoring has evolved and how we can leverage AI to implement concepts such as Shift-Left, Shift-Right and Self-Healing
Monitoring as a Self-Service in Atlassian DevOps ToolchainAndreas Grabner
As devs, testers and ops we must deal with monitoring data when analyzing test results, debugging problems or reporting on usage. But why stepping out of our Atlassian Tool Comfort Zone to get this data? We found new use cases on how to fully integrate monitoring as a self-service into Jira, Hipchat, Bamboo, Bitbucket & Confluence. This saves time in learning yet another tool and gives you the data when and where you need it: in your most favorite Atlassian Tool. Key Use Cases we discuss: Continuous Performance Analysis in Jira, Shift-Left in Bamboo / Bitbucket, ChatOps in Hipchat.
AWS Summit - Trends in Advanced Monitoring for AWS environmentsAndreas Grabner
Why you have to rethink your monitoring strategy when moving or building apps for new stack cloud based environments:
#1: Why "the old way" of monitoring doesnt work any longer!
#2: How the Cloud and New Stack has transformed Dynatrace!
#3: How Dynatrace Redefined Monitoring for Cloud Applications
DevOps Transformation at Dynatrace and with DynatraceAndreas Grabner
Presentation given at CMG Boston - April 20th 2017
#1: How to explain DevOps Transformation?
#2: How Dynatrace transformed from 6months waterfall to 1h code deploy
#3: The role of Monitoring in DevOps / CI/CD
#4: Using Dynatrace for your DevOps Transformation
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf
Docker/DevOps Meetup: Metrics-Driven Continuous Performance and Scalabilty
1. 1 @Dynatrace
Application Quality Metrics for your Pipeline
(and why Docker is not the solution to all of your problems)
Andreas (Andi) Grabner - @grabnerandi
Metrics-Driven DevOps
2.
3.
4. 700 deployments / year
10 + deployments / day
50 – 60 deployments / day
Every 11.6 seconds
5. Example #1: Online Casino 282! Objects
on that page9.68MB Page Size
8.8s Page
Load Time
Most objects are images
delivered from your main
domain
Very long Connect time
(1.8s) to your CDN
6. 879! SQL Queries8!Missing CSS & JS
Files
340!Calls to GetItemById
Example #2: Lawyer Website based on SharePoint
11s!To load
Landing Page
7.
8.
9.
10. • Waterfall Agile: 3 years
• 220 Apps - 1 deployment per month
“EVERYONE can do Continuous Delivery”
“Every manual tester does AUTOMATION”
“WE DON’T LOG BUGS – WE FIX THEM!”
Measures Built-In, Visible to Everyone
Promote your Wins, Educate your Peers
34. • Symptoms
• HTML takes between 60 and 120s to render
• High GC Time
• Developer Assumptions
• Bad GC Tuning
• Probably bad Database Performance as rendering was simple
• Result: 2 Years of Finger pointing between Dev and DBA
Project: Online Room Reservation System
35. Developers built own monitoring
void roomreservationReport(int officeId)
{
long startTime = System.currentTimeMillis();
Object data = loadDataForOffice(officeId);
long dataLoadTime = System.currentTimeMillis() - startTime;
generateReport(data, officeId);
}
Result:
Avg. Data Load Time: 45s!
DB Tool says:
Avg. SQL Query: <1ms!
36. #1: Loading too much data
24889! Calls to the Database
API!
High CPU and High Memory Usage
to keep all data in Memory
37. #2: On individual connections 12444!
individual
connections
Classical N+1 Query
Problem
Individual SQL
really <1ms
38. #3: Putting all data in temp Hashtable
Lots of time spent in
Hashtable.get
Called from their Entity
Objects
39. • … you know what code is doing you inherited!!
• … you are not making mistakes like this
• Explore the Right Tools
• Built-In Database Analysis Tools
• “Logging” options of Frameworks such as Hibernate, …
• JMX, Perf Counters, … of your Application Servers
• Performance Tracing Tools: Dynatrace, Ruxit, NewRelic,
AppDynamics, Your Profiler of Choice …
Lessons Learned – Don’t Assume …
40. Key Metrics
# of SQL Calls
# of same SQL Execs (1+N)
# of Connections
Rows/Data Transferred
43. 43 @Dynatrace
26.7s
Execution Time 33! Calls to the
same Web
Service
171! SQL Queries through LINQ
by this Web Service – request
similar data for each call
Architecture Violation: Direct access
to DB instead from frontend logic
44. 44 @Dynatrace
Key Metrics
# Service Calls, # Containers
# of Threads, Sync and Wait
# SQL executions
# of SAME SQL’s
Payload (kB) of Service Calls
47. Distance calculation issues
480km biking
in 1 hour!
Solution: Unit Test in
Live App reports Geo
Calc Problems
Finding: Only
happens on certain
Android versions
60. 60
What you currently measure
What you should measure
Quality Metrics
in your pipeline
# Test Failures
Overall Duration
Execution Time per test
# calls to API
# executed SQL statements
# Web Service Calls
# JMS Messages
# Objects Allocated
# Exceptions
# Log Messages
# HTTP 4xx/5xx
Request/Response Size
Page Load/Rendering Time
…
61. Extend your Continuous Integration
12 0 120ms
3 1 68ms
Build 20 testPurchase OK
testSearch OK
Build 17 testPurchase OK
testSearch OK
Build 18 testPurchase FAILED
testSearch OK
Build 19 testPurchase OK
testSearch OK
Build # Test Case Status # SQL # Excep CPU
12 0 120ms
3 1 68ms
12 5 60ms
3 1 68ms
75 0 230ms
3 1 68ms
Test & Monitoring Framework Results Architectural Data
We identified a regresesion
Problem solved
Exceptions probably reason for
failed tests
Problem fixed but now we have an
architectural regression
Problem fixed but now we have an
architectural regressionNow we have the functional and
architectural confidence
Let’s look behind the scenes
62. #1: Analyzing every Unit
& Integration test
#2: Metrics for each test
#3: Detecting regression
based on measure
Unit/Integration Tests are auto baselined! Regressions auto-detected!
69. #1: Pick your App Metrics
# of Service Calls Bytes Sent & Received
# of Worker
Threads
# of Worker
Threads
# of SQL Calls, # of
Same SQLs # of DB
Connections
# of SQL Calls, # of
Same SQLs # of DB
Connections
70. #2: Figure out how to monitor them
http://bit.ly/dtpersonal
Get Dynatrace Free Trial at http://bit.ly/dtpersonal
Video Tutorials on YouTube Channel: http://bit.ly/dttutorials
Online Webinars every other week: http://bit.ly/onlineperfclinic
Share Your PurePath with me: http://bit.ly/sharepurepath
More blogs on http://blog.dynatrace.com
If you are new to DevOps and Continuous Delivery check out these two books: Continuous Delivery from Jez Humble, David Farley and The Phoenix Project from Gene Kim, Kevin Behr, and George Spafford
Many companies that have a „DevOps Strategy“ too often just follow the Unicorns
Several companies changed their way they develop and deploy software over the years. Here are some examples (numbers from 2011 – 2014)
Cars: from 2 deployments to 700
Flicks: 10+ per Day
Etsy: lets every new employee on their first day of employment make a code change and push it through the pipeline in production: THAT’S the right approach towards required culture change
Amazon: every 11.6s
Remember: these are very small changes – which is also a key goal of continuous delivery. The smaller the change the easier it is to deploy, the less risk it has, the easier it is to test and the easier is it to take it out in case it has a problem.
If „Being DevOps“ just means you just increase the number of deployments then you are bound to fail. Here is an example of a bad web application. When deploying this more frequently you will end up in more war rooms
Another example from a SharePoint app that allows production deployments by SharePoint Admins. A simply change directly in production can have very negative impacts, e.g: deploying a new WebPart with a Data-Driven Performance Hotspot
Don‘t just copy the Unicorns – dont be just driven the number of deployments.
The problem is though – when you blindly copy what you read you may end up with a very ugly copy of a Unicorn. Its not about copying everything or thinking that you have to release as frequently as the Unicorns. It is about changing and adapting a lot of their best practices but doing it in a way that makes sense to you. For you it might be enough to release once a month or once week.
Listen to the next generation Unicorns, e.g: those talking at Velocity or other conferences: Target, CapitalOne, IG, ...
These are the highlights of these talks for me this year:
http://apmblog.dynatrace.com/2015/05/27/velocity-2015-our-conference-highlights/
http://apmblog.dynatrace.com/2015/05/28/velocity-2015-highlights-from-day-2/
http://apmblog.dynatrace.com/2015/05/29/velocity-2015-highlights-from-last-day/
Despite all these stories the main Challenge remains ...
Don’t’ just try to deploy faster …
… as you may just ending up failing faster and more often!
Don’t become the next headline on the news as United in the summer of 2015
Or the Fifa World Cup App that crashed for 80% of their Android Users caused by a memory leak in an outdated UI Library one week before the WorldCup
I love metrics – and I think we should make decisions on deployments based on key metrics. But also monitor deployments in production to learn whether the deployment was really good
The BASIC Metric EVERYONE has to have: Synthetic Availability Monitoring -> Clearly something went wrong
Even if the deployment seemed good because all features work and response time is the same as before. If your resource consumption goes up like this the deployment is NOT GOOD. As you are now paying a lot of money for that extra compute power:
http://apmblog.dynatrace.com/2015/06/30/fighting-technical-debt-memory-leak-detection-in-production/
http://apmblog.dynatrace.com/2014/10/28/hands-tutorial-5-steps-identify-java-net-memory-leaks/
Layer Breakdown perfectly shows which layer of your app is not scaling:
http://apmblog.dynatrace.com/2015/01/22/key-performance-metrics-load-tests-beyond-response-time-part/
Got a marketing campaign? If you roll it out do it smart: Start with a small number – monitor user behavior – fix errors if there are any before rolling out the rest of the campaign:
http://apmblog.dynatrace.com/2015/02/26/omni-channel-monitoring-in-real-life/
A lot of people dont look at these metrics and just add new code on an ever growing big pile of technical debt
Based on a recent study:
80% of Dev Team overall is spent in Bugfixing instead of building new cool features
$60B annual costs of bad software instead of investing it in new cool features to spearhead competition
Yes – we are focusing on quality TOO LATE
When its too late we end up here
We need to leave that status quo. And there are two numbers that tell us that it is not as hard to do as it may seem
Based on my experience
80% of the problems are only caused by 20% problem patterns. And focusing on 20% of potential problems that take away 80% of the pain is a very good starting point
Sounds super nice on paper – so – how do we get there?
This story is from Joe – a DB guy from a very large telco arguing with his developers over performance problems of an online room reservation system which has evolved from a small project implemented by an intern to an application that is now used in their entire organization
Devs buillt custom monitoring to proof their point! Contradicting what Joe‘s DB Tools had to say
Reading this Transaction Flow showed what the real problem was: Loading Too Much Data from the Database causing High Memory Usage and therefore high CPU to cleanup the garbage
Every SQL was executed on its on Connection
The intern back then implemented its own OR Mapper by loading the full database content into a HashTable using individual queries
This was a monolithic app for searching sports club websites. The executed sample search brought 33 sports club. Before this app was „migrated“ to Microservices everything was in a single monolith taking about 1s to execute. After the „migration“ to (micro)services the same call takes 26.7s including 33 calls to the new microservice and 171 roundtrips to the database
A Mobile App with a GPS Distance Calculation Problem. Couldnt be found in test – so they moved the Test to Production to find out which devices actually have the problem
http://apmblog.dynatrace.com/2013/07/23/too-fast-for-the-user/
As many mobile apps – you might rely on 3rd party services for your users to login. Make sure you monitor the response time and success of these calls and how it impacts your end users
Overloaded Kia website brings it down during superbowl:
http://apmblog.dynatrace.com/2014/03/05/bloated-web-pages-can-make-or-break-the-day-lessons-learned-from-super-bowl-advertisers/
GoDaddy is doing something different: they have a special „bare minimum static optimized“ website for the spike period -> thats smart:
http://apmblog.dynatrace.com/2014/02/19/dns-tcp-and-size-application-performance-best-practices-of-super-bowl-advertisers/
So – we have seen a lot of metrics. The goal now is that you start with one metric. Pick a single metric and take it back to your engineering team (Dev, Test, Ops and Business). Sit down and agree on what this metric means for everyone, how to measure it and also how to report it
Also remember that for most of these use cases discussed and metrics derived from it we only need a single user test. Even though we can identify performance, scalability and architectural issues – in most cases we don’t need a load test. Single user tests or unit tests are good enough
If you are already executing tests than that is great – BUT – you are only testing functionality. It is time to look „underneath“ the hood and automaitcally find all these other problems we just talked about by looking at the right metrics
Here is how we do this. In addition to looking at functional and unit test results which only tell us how functionality is we also look into these backed metrics for every test. With that we can immediately identify whether code changes result in any performance, scalability or architectural regressions. Knowing this allows us to stop that build early
This is how this can look like in a real life example. Analyzing Key Performance, Scalability and Architectural Metrics for every single test
Dynatrace can either show the data in our own dashboards or you can integrate this data through our REST APIs with your Build Server such as Jenkins, Bamboo, .... And even „BREAK THE BUILD“ if something is bad!
Make sure you do not end in Pre-Production. Once you deploy your application you also want to monitor how your application is doing in the wild. Same technical metrics are important to monitor but also correlate them with the business metrics such as Conversion Rates, Bounce Rates, Revenue, ...
Docker Fans: Make sure you monitor your Docker Enviornments to identify any bottlenecks – whether caused by Docker or by your app making inefficient use of Docker/Container resources!
http://apmblog.dynatrace.com/2015/07/21/how-to-get-visibility-into-docker-clusters-running-kubernetes/
More screenshots and tips and tricks on docker/container monitoring
http://apmblog.dynatrace.com/2015/07/21/how-to-get-visibility-into-docker-clusters-running-kubernetes/
A „dockerized“ app monitored with Dynatrace
http://apmblog.dynatrace.com/2015/07/21/how-to-get-visibility-into-docker-clusters-running-kubernetes/
So – our goal is to deploy new features faster to get it in front of our paying end users or employees