• Save
How DevOps and Agile Change the Game
Upcoming SlideShare
Loading in...5
×
 

How DevOps and Agile Change the Game

on

  • 883 views

Agile Development allows us to react to business needs faster. It requires us to change our traditional thinking of operations giving rise to the idea of DevOps. But how to we ensure performance in an ...

Agile Development allows us to react to business needs faster. It requires us to change our traditional thinking of operations giving rise to the idea of DevOps. But how to we ensure performance in an environment with daily releases and ever changing requirements? Learn how to optimize your testing processes and the 5 things you need to do to do successful continuous performance management.

But how do we ensure performance in an environment where releases are done daily and maintaining and executing load tests takes longer than the actual feature development.
When your target hardware goal is based on demand and highly dynamic or the expected load too high, you need a new strategy for performance management. It's the end of load testing as we know it, the dawn of true continuous APM.

Presented by Michael Kopp, Product Evangelist, dynaTrace-Compuware Center of Excellence, at the ceCMG Conference in Munich March 2011

Statistics

Views

Total Views
883
Slideshare-icon Views on SlideShare
883
Embed Views
0

Actions

Likes
1
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Agile developmentallowsReducestheriskofworkingtoolong in thewrongdirectionProvides a „Potential ReleasableProduct“ after every Sprint/IterationIncludesTesting Practices intothe Development Lifecycletoreact on changedrequirements (technicalandbusiness) fasterAutomatesmanytasks such asBuild, FunctionalTestingandprogressreporting (Burn-Down Chart, …)In short, betterquality, lessrisk, fasterreaction time
  • Estimatesgettrackedandverified on an ongoingbasis. Aber was dannAgile developmentallowsReducestheriskofworkingtoolong in thewrongdirectionProvides a „Potential ReleasableProduct“ after every Sprint/IterationIncludesTesting Practices intothe Development Lifecycletoreact on changedrequirements (technicalandbusiness) fasterAutomatesmanytasks such asBuild, FunctionalTestingandprogressreporting (Burn-Down Chart, …)In short, betterquality, lessrisk, fasterreaction time
  • LoadTestingandProduction.Often a separate organization.Firstlythismeanswe do not leveragetheadvantageof agile fully. We do not deliverthespeedofchangetothebusinessandthustothecustomer.In a normal shelfproductthismightbefine, ifyouoperate a SAAS oreComercesiteitreducesyourabilitytocompete.Apart fromthatthereare also severalproblemswiththehardsplitbetweenthetwosides. Twooftenmore separate organizationsleadto…Vorteile werden negiert. Reactiontobusiness, requirementsgetoftrack, becausenobodyreallyusesthesprintreleases
  • Functionality vs. StabilitySeparate teams, different mindset, different incentives. Operationsispaidtokeepthignsstableandrunningwhiledevelopmentistoaddnewfeatures. The funny thingis, thatoperationsisdrivenbycustomerdemands but does not wanttoaddbusinessfunctionalitytobetercompete.Dev on theothersidewantstoaddnewfunctionality all the time, but they lack theexperiencewiththecustomer. They lack the real worldproblems.
  • Distributed Teams makeithardtocoordinateMakescommunication all thathearder. Timezones, langugae different culture in additionto an already different mindset
  • Even worse, ifwewanttocommunicatesomething, thetechnologyis not compatible. Different monitoring, analysistools
  • All ofthisleadsto …I am done – nowyoudeployit
  • Not a surprise – isit?
  • holistic approach to software deliveryTatsaechlich ist das risiko kleiner je kleiner die releases sind. Der grund ist einfach, kleine aenderungen weniger moeglichkeit auf bugs. Sollte ausserdemueber automatisierte tests abgedeckt sein.Passiert mal ein fehler, ist das EntwicklungsTeam auch das Ops Team und kann schnell reagieren. PromoteColocationofteamsinsteadofspreadingthemaroundtheworldimaginetheguyrunninganddeployingyourappsitsnexttoyou. Thatwouldmake a lotstuffeasierOneToolsetReduces a lotofoverheadconvertingdata. Reducesmaintaince time. Andyouwouldmakesuretoreuseconfigurationofsaymonitoringfromautomatictest in prod, andfromautomaticbuildandtesttodeploythestufffor real.Agile DevlieryandDeploymentDeliverythe agile promisedirectlytothebusinessandcustomer. React in dayswithnewfeatures. Therearebigonesthat do that. (google, facebooktonametwo)ONE TEAM thatisresponsiblefor a Product (FromDevtoOps)Ifthereis a problem,theguyresponsibleisyou
  • Solution Proposals
  • Testdrivendevelopment. (usecase!)Automatic, Detect Change, AnalyzeArchitectureWeshouldhave a trendingcharthere.Perfearly, insidethecycle. This wayitispartofthesprint not outside. Itisautomted so itdoes not introduce additional effort.Be in Control!
  • Be In Control, earlyandautomatic.
  • Toomucheffortforscriptmaintenance, settingupthetestenvironmentCostsfortestingtoolsandtestenvironmentTest cycletakeslongerthanreserved in thesprint. Feedback todevscomeswhentheyarealready in thenextsprint
  • classical loadtestingtest environment that tries to be as close to production as possible (size/users/data vs. reality)test scripts with transactions that we expect (but we know we shouldexpecttheunexpected. Never happens)TRIES TO REBUILD TO REAL WORLD -> HUGE EFFORT!!Result: no real guarantee that what has been tested really works in prodProblems:Testing on not frozen Code Base -> Results of Testing are available not before mid or end of NEXT Sprint -> Results are outdated!HArd to get real life Production Load BehaviorMore changes in application lead to more effort in testing (script maintenance, regression analysis, ...)Size ofenvironmentNumberof UsersTransaction Mix
  • Rerun, additional dataAdding log statemetnsandstatistics in code, errorprone.Scalabilityandregressiontesting!First, automatethestuff.Second, captureenoughfromthebeginning. (traces, BTM)Dynamic Capture all the time.Overhead discussion 5-10Benefits of Agile and DevOpsShorter iterations with fewer changes leads to reduced effort in testingfewer script maintenance changes, higher quality smoke test, easier regression analysis,more focused testing on the areas that have changed
  • Real environment, real validityLesscost, lessmaintaince,lesseffort.Yoursystem will bemoststableandyou will learnperformancemanagement.Test during off-hours/off-daysorduringmaintainencecyclesOff peaktimesnewversiontestloadtestingPart ofproductionwithnewversionShadow production, trafficduplication
  • Kosten, vorteil ist wenn die software ok ist, ist der deployment schritt fuer einen teil der servers bereits erfuellt.
  • Eine variante ist das sogenannte shadowing. Das setup ist wie zuvor nur jetzt wird traffic dupliziert. Wichtig hierbei ist wieder wie vorhin im last test das ich die dementsprechenden monitoringtools habe um probleme zu erkennen ohne reproduzieren zu muessen.
  • Als letzte variante kann ich einen patch auf einem teil der machinenedeployen. Vorteil dabei ist das nicht mein gesammtestsystem beeinflusst ist. Ich kann also gut reagieren. Weiters sehen wir den impact sofort. Da wir gut vergleichen koennen.
  • Testing real numberofusers, with real loadand on yourproductionwith real contentSimulatenumberofusersasyoucouldn‘tyourselfReal end userperformanceIfyouknowwhereyouruserbaseis, testfromthereSee localdifferencesAt a fractionofthecost, ist on demand.Letthemmaintaintheirscripts, concentrate on yourproductLeverageserviceslikekeynoteandgomes. Simulate real userload, and not like in loadtesting just transactionload.provisioning in the cloud -> dynamic resources -> cater for millions of users
  • Knowwhat‘sreallygoing onReal problemsNoreproductionOptimzebased on what‘s realNoguess – just factsDirectfeedbacktodevUseProductiontoanalyzetheperformance. Reactto real problems. Noreproduction.Tune andoptimzebased on real worldtransactionanduserload.We solve the "Black Box View" of the test departmentWe dont know the insight of the app -> so what shall we look for?We dont know how the real prod evnrionment looks like -> so how shall we know if it works in prod or notAPM in Produktion --> fast feedback to dev (same team anyway?), agile cycles allow formsquick fixesless change, smaller errors, faster solution
  • Kreis mit pfeilen, mitte ein tool, communicationbarrier, objectivness, lessmaintaincePerformance analysisPerformance testingPerformance monitoringvalidation
  • Agile + DevOps + CAPMChanged Testing Mindest -> WE ARE NEVER DONE -> WE CONSTANTLY IMPROVE

How DevOps and Agile Change the Game How DevOps and Agile Change the Game Presentation Transcript

  • How DevOps and Agilechange the GameWhy Traditional Performance Managementis no longer State-of-the-Art
  • Why Agile Development took off
  • You are in controlStory Points Developme Testing Estimate nt Sprint Timeline Production Remaining Team Velocity
  • What is the Status Quo beyond Dev?Source: http://dev2ops.org/blog/2010/2/22/what-is-devops.html
  • Problem #1: Different MindsetSource: http://dev2ops.org/blog/2010/2/22/what-is-devops.html
  • Problem #2: Dislocated TeamsSource: http://dev2ops.org/blog/2010/2/22/what-is-devops.html
  • Problem #2: Different ToolsSource: http://dev2ops.org/blog/2010/2/22/what-is-devops.html
  • Problem #4: Over the Fence AttitudeSource: http://dev2ops.org/blog/2010/2/22/what-is-devops.html
  • These Problems lead to …Source: http://dev2ops.org/blog/2010/2/22/what-is-devops.html
  • A potential SolutionSource: http://dev2ops.org/blog/2010/2/22/what-is-devops.html
  • Real World Perf Test in Feedback CI ONECloud based Toolset Architecture ValidationTesting Test in Production Traditional Load Testing
  • Performance in Continuous Integration
  • Identify Performance Regressions Early
  • Architecture Validation Analyze your existing Unit Tests Validate against your Arch Rules
  • Traditional Load Testing takes too long time Sprint 1 Starts with Dev and dedicated Time for Test at the End Update Test Setup Run Tests Scripts Environment Adapt Scripts Re Run Tests Feedback to Dev Testing extends sprints Consumes too many Dev Resources Sprint 2 Re Run Tests Feedback to Dev Ready for Ops Either postponed or impacted because Dev Resources are still occopuied with testing of previous sprint
  • Load Testing needs to be „real“ Real Size and Content Real Third Party CodeReal Infrastructure Setup No Guarantees! Real Load Real Number of Users
  • Minimize and automate real Load TestsDeveloping Test Run Reproduction Refine Capturing Re Run Tests Reproduction Refine Capturing Multiple Test Iterations needed to analyze Root-cause Re Run Tests Reproduction Problem Analysis Problem Solving timeDeveloping Test Run Reproduction Refine Capturing Re Run Tests Reproduction Refine Capturing •Eliminates Test Iterations •Go directly to problem analysis •Frees up resources for other proje Re Run Tests Reproduction Problem Analysis Problem Solving time
  • Test in your Production Environment Real Users Production Environment Virtual Users LB/FW Web Server App Server DB Route Incoming Use some machines Traffic during off hours for testing
  • Scenario 2: Duplicate Traffic Real Users Production Environment LB/FW Web Server App Server DB Send incoming traffic to Prod and Testing Machines
  • Scenario 3: Partial Update Real Users Production Environment LB/FW Web Server App Server DB Update some machines to the version that should be tested
  • Use Cloud based Testing Services Real End User TimeCDN Proxies High ControlVolume Costs Last Mile Load Balancers
  • APM in Production
  • Understand your production environment
  • Focus on areas with biggest impact160140120100 80 Number of Sales 60 No of Users 40 20 0 Response Time Transaction Load
  • ONE set of Tools Communication Collaboration
  • Engineering PerformanceWE ARE NEVER DONE… …WE CONSTANTLY IMPROVE
  • Michael Koppmichael.kopp@dynaTrace.comhttp://blog.dynatrace.com@mikopp
  • Download the latest Application Performance almanac:Web | Cloud | DevOps | Mobile | Automation | Tuningwww.dynatrace.com/almanac
  • Tip: Stable Environment
  • Tip: Exclude Garbage Collection
  • Learn how foreign code works• Get insight into whats going on „under the hood“ • Understand internals of the frameworks you use • Choose the right usage for your use case
  • Tip: Measure more than Response Time
  • Tip: Identify Scalabilty Problems Response time Presentation Layer Presentation Layer Business Logic Presentation Layer Business Logic Business Logic Web Service/Remoting Web Service/Remoting Web Service/Remoting Data Access Data Access Data Access Minimum Load Medium Load Maximum Load
  • Tip: Identify Regressions on Code Level
  • Tip: Get as granular data as possible
  • Monitoring in Agile DevelopmentStory Points Developme Testing Estimate nt Sprint Timeline Production Remaining Team Velocity
  • What happened?Story Points Developme Testing Estimate nt Sprint Timeline Production Remaining Team Velocity
  • Missed Goals and EstimatesStory Points Estimates Missed Developme Testing Estimate nt Production Missed Remaining Goal Team Velocity