David Rico, profile picture
Uploaded byDavid Rico
PDF, PPTX5,327 views

Business Value of Agile Testing: Using TDD, CI, CD, & DevOps

The document discusses the business value of agile testing using methodologies such as Test-Driven Development (TDD), Continuous Integration (CI), Continuous Delivery (CD), and DevOps. It emphasizes the significance of agility in software development to address challenges like overruns, inefficiencies, and high failure rates in traditional projects. It further explores various agile methods and frameworks, highlighting their effectiveness in enhancing productivity and reducing costs in IT projects.

Embed presentation

Download as PDF, PPTX
Business Value of Agile Testing Using TDD, CI, CD & DevOps Dr. David F. Rico, PMP, CSEP, FCP, FCT, ACP, CSM, SAFe Twitter: @dr_david_f_rico Website: http://www.davidfrico.com LinkedIn: http://www.linkedin.com/in/davidfrico Agile Capabilities: http://davidfrico.com/rico-capability-agile.pdf Agile Resources: http://www.davidfrico.com/daves-agile-resources.htm Agile Cheat Sheet: http://davidfrico.com/key-agile-theories-ideas-and-principles.pdf
Author Background  Gov’t contractor with 32+ years of IT experience  B.S. Comp. Sci., M.S. Soft. Eng., & D.M. Info. Sys.  Large gov’t projects in U.S., Far/Mid-East, & Europe 2   Career systems & software engineering methodologist  Lean-Agile, Six Sigma, CMMI, ISO 9001, DoD 5000  NASA, USAF, Navy, Army, DISA, & DARPA projects  Published seven books & numerous journal articles  Intn’l keynote speaker, 130 talks to 12,000+ people  Specializes in metrics, models, & cost engineering  Cloud Computing, SOA, Web Services, FOSS, etc.  Adjunct at five Washington, DC-area universities
Today’s Whirlwind Environment 3 Overruns Attrition Escalation Runaways Cancellation Global Competition Demanding Customers Organization Downsizing System Complexity Technology Change Vague Requirements Work Life Imbalance Inefficiency High O&M Lower DoQ Vulnerable N-M Breach Reduced IT Budgets 81 Month Cycle Times Redundant Data Centers Lack of Interoperability Poor IT Security Overburdening Legacy Systems Obsolete Technology & Skills Pine, B. J. (1993). Mass customization: The new frontier in business competition. Boston, MA: Harvard Business School Press. Pontius, R. W. (2012). Acquisition of IT: Improving efficiency and effectiveness in IT acquisition in the DoD. Second Annual AFEI/NDIA Conference on Agile in DoD, Springfield, VA, USA.
Software in U.S. DoD Systems Kennedy, M. P., & Umphress, D. A. (2011). An agile systems engineering process: The missing link. Crosstalk, 24(3), 16-20.  No. of software-intensive systems is growing  80% of US DoD functions performed in software  Major driver of cost, schedule, & tech. performance 4 
Software in U.S. DoD Avionics Blackburn, M. R. (2014). Transforming systems engineering through a holistic approach to model centric engineering. Washington, DC: Stevens Institute of Technology.  Software in U.S. DoD avionics growing exponentially  10x growth from F-16 to F-22 (& another 10x to F-35)  Productivity must grow by 10x for next gen systems 5 
Traditional Projects 6  Big projects result in poor quality and scope changes  Productivity declines with long queues/wait times  Large projects are unsuccessful or canceled Jones, C. (1991). Applied software measurement: Assuring productivity and quality. New York, NY: McGraw-Hill. Size vs. Quality DEFECTS 0.00 3.20 6.40 9.60 12.80 16.00 0 2 6 25 100 400 SIZE Size vs. Productivity PRODUCTIVITY 0.00 1.00 2.00 3.00 4.00 5.00 0 2 6 25 100 400 SIZE Size vs. Change CHANGE 0% 8% 16% 24% 32% 40% 0 2 6 25 100 400 SIZE Size vs. Success SUCCESS 0% 12% 24% 36% 48% 60% 0 2 6 25 100 400 SIZE 
Global Project Failures 7 Standish Group. (2015). Chaos summary 2015. Boston, MA: Author. Sessions, R. (2009). The IT complexity crisis: Danger and opportunity. Houston, TX: Object Watch.  Challenged and failed projects hover at 67%  Big projects fail more often, which is 5% to 10%  Of $1.7T spent on IT projects, over $858B were lost $0.0 $0.4 $0.7 $1.1 $1.4 $1.8 2002 2003 2004 2005 2006 2007 2008 2009 2010 Trillions(USDollars) Expenditures Failed Investments  0% 20% 40% 60% 80% 100% 28% 34% 29% 35% 32% 33% 27% 28% 29% 49% 51% 53% 46% 44% 41% 56% 55% 52% 23% 15% 18% 19% 24% 26% 17% 17% 19% 2000 2002 2004 2006 2008 2010 2012 2014 2015 Year Successful Challenged Failed
Requirements Defects & Waste 8 Sheldon, F. T. et al. (1992). Reliability measurement: From theory to practice. IEEE Software, 9(4), 13-20 Johnson, J. (2002). ROI: It's your job. Extreme Programming 2002 Conference, Alghero, Sardinia, Italy.  Requirements defects are #1 reason projects fail  Traditional projects specify too many requirements  More than 65% of requirements are never used at all Other 7% Requirements 47% Design 28% Implementation 18% Defects Always 7% Often 13% Sometimes 16% Rarely 19% Never 45% Waste 
What is Agility?  A-gil-i-ty (ә-'ji-lә-tē) Property consisting of quickness, lightness, and ease of movement; To be very nimble  The ability to create and respond to change in order to profit in a turbulent global business environment  The ability to quickly reprioritize use of resources when requirements, technology, and knowledge shift  A very fast response to sudden market changes and emerging threats by intensive customer interaction  Use of evolutionary, incremental, and iterative delivery to converge on an optimal customer solution  Maximizing BUSINESS VALUE with right sized, just- enough, and just-in-time processes and documentation Highsmith, J. A. (2002). Agile software development ecosystems. Boston, MA: Addison-Wesley. 9  
What are Agile Methods? 10  People-centric way to create innovative solutions  Product-centric alternative to documents/process  Market-centric model to maximize business value Agile Manifesto. (2001). Manifesto for agile software development. Retrieved September 3, 2008, from http://www.agilemanifesto.org Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing. Rico, D. F. (2012). Agile conceptual model. Retrieved February 6, 2012, from http://davidfrico.com/agile-concept-model-1.pdf Customer Collaboration Working Systems & Software Individuals & Interactions Responding to Change valued more than valued more than valued more than valued more than Contracts Documentation Processes Project Plans  Frequent comm.  Close proximity  Regular meetings  Multiple comm. channels  Frequent feedback  Relationship strength  Leadership  Boundaries  Empowerment  Competence  Structure  Manageability/Motivation  Clear objectives  Small/feasible scope  Acceptance criteria  Timeboxed iterations  Valid operational results  Regular cadence/intervals  Org. flexibility  Mgt. flexibility  Process flexibility  System flexibility  Technology flexibility  Infrastructure flexibility  Contract compliance  Contract deliverables  Contract change orders  Lifecycle compliance  Process Maturity Level  Regulatory compliance  Document deliveries  Document comments  Document compliance  Cost Compliance  Scope Compliance  Schedule Compliance     Courage 
Agile World View  “Agility” has many dimensions other than IT  It ranges from leadership to technological agility  Today’s focus is on organizational & enterprise agility   Agile Leaders Agile Organization Change Agile Acquisition & Contracting Agile Strategic Planning Agile Capability Analysis Agile Program Management Agile Tech. Agile Information Systems Agile Tools Agile Processes & Practices Agile Systems Development Agile Project Management 11 
Network Computer Operating System Middleware Applications APIs GUI How Agile Works  Agile requirements implemented in slices vs. layers  User needs with higher business value are done first  Reduces cost & risk while increasing business success 12Shore, J. (2011). Evolutionary design illustrated. Norwegian Developers Conference, Oslo, Norway. Agile Traditional 1 2 3 Faster  Early ROI  Lower Costs  Fewer Defects  Manageable Risk  Better Performance  Smaller Attack Surface Late  No Value  Cost Overruns  Very Poor Quality  Uncontrollable Risk  Slowest Performance  More Security Incidents Seven Wastes 1. Rework 2. Motion 3. Waiting 4. Inventory 5. Transportation 6. Overprocessing 7. Overproduction MINIMIZES MAXIMIZES  JIT, Just-enough architecture  Early, in-process system V&V  Fast continuous improvement  Scalable to systems of systems  Maximizes successful outcomes  Myth of perfect architecture  Late big-bang integration tests  Year long improvement cycles  Breaks down on large projects  Undermines business success 
Thousands of Tests Continuously Executed No More Late Big Bang Integration  User needs designed & developed one-at-a-time  Changes automatically detected, built, and tested  System fully tested and deployed as changes occur 13Humble, J., & Farley, D. (2011). Continuous delivery. Boston, MA: Pearson Education. Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration. Boston, MA: Addison-Wesley. Build Integration Server Version Control Server Build Scripts UsesWatches Build Status ProvidesDeveloper A Developer B Developer C Commits Changes Commits Changes Commits Changes Builds Database Analysis Testing Reporting Documentation Deployment Early, Automated, Fast, Efficient, & Repeatable Constant Readiness State & CM Control Lean, Waste Free, Low WIP, No Deadlocked Test Queues Rapidly & Successfully Dev. Complex Systems  Basic Agile Mechanics
14 Capability/MMF #1 ● Feature 1 ● Feature 2 ● Feature 3 ● Feature 4 ● Feature 5 ● Feature 6 ● Feature 7 Capability/MMF #2 ● Feature 8 ● Feature 9 ● Feature 10 ● Feature 11 ● Feature 12 ● Feature 13 ● Feature 14 Capability/MMF #3 ● Feature 15 ● Feature 16 ● Feature 17 ● Feature 18 ● Feature 19 ● Feature 20 ● Feature 21 Capability/MMF #4 ● Feature 22 ● Feature 23 ● Feature 24 ● Feature 25 ● Feature 26 ● Feature 27 ● Feature 28 Capability/MMF #5 ● Feature 29 ● Feature 30 ● Feature 31 ● Feature 32 ● Feature 33 ● Feature 34 ● Feature 35 Capability/MMF #6 ● Feature 36 ● Feature 37 ● Feature 38 ● Feature 39 ● Feature 40 ● Feature 41 ● Feature 42 Capability/MMF #7 ● Feature 43 ● Feature 44 ● Feature 45 ● Feature 46 ● Feature 47 ● Feature 48 ● Feature 49 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Evolving “Unified/Integrated” Enterprise Data Model “Disparate” LEGACY SYSTEM DATABASES (AND DATA MODELS) ETL A A B C D E F G H I J K A B C D E F A B C D E A B C D A B C A B “Legacy” MS SQL Server Stovepipes “Inter-Departmental” Linux Blade/Oracle/Java/WebSphere Server “Leased” DWA/HPC/Cloud Services Sprint 1 Sprint 2 Sprint 3 Sprint 4 Sprint 5 Sprint 6 Sprint 7 Release Release Release Release ETL ETL ETL ETL ETL ETL Bente, S., Bombosch, U., & Langade, S. (2012). Collaborative enterprise architecture: Enriching EA with lean, agile, and enterprise 2.0 practices. Waltham, MA: Elsevier. (for example, assume 25 user stories per feature, 175 user stories per capability/MMF, and 1,225 user stories total)  Organize needs into capabilities, features, and stories  Prioritize features, group releases, and initiate sprints  Develop minimum set of features with highest value Agile Systems Development
Models of AGILE DEVELOPMENT 15  Agile methods spunoff flexible manufacturing 1990s  Extreme Programming (XP) swept the globe by 2002  Today, over 90% of IT projects use Scrum/XP hybrid Use Cases Domain Model Object Oriented Iterative Dev. Risk Planning Info. Radiators Planning Poker Product Backlog Sprint Backlog 2-4 Week Spring Daily Standup Sprint Demo Feasibility Business Study Func. Iteration Design Iteration Implementation Testing Domain Model Feature List Object Oriented Iterative Dev. Code Inspection Testing Release Plans User Stories Pair Programmer Iterative Dev. Test First Dev. Onsite Customer Cockburn, A. (2002). Agile software development. Boston, MA: Addison-Wesley. Schwaber, K., & Beedle, M. (2001). Agile software development with scrum. Upper Saddle River, NJ: Prentice-Hall. Stapleton, J. (1997). DSDM: A framework for business centered development. Harlow, England: Addison-Wesley. Palmer, S. R., & Felsing, J. M. (2002). A practical guide to feature driven development. Upper Saddle River, NJ: Prentice-Hall. Beck, K. (2000). Extreme programming explained: Embrace change. Reading, MA: Addison-Wesley. CRYSTAL METHODS - 1991 - SCRUM - 1993 - DSDM - 1993 - FDD - 1997 - XP - 1998 - Reflection W/S Retrospective Quality Control Quality Control Continuous Del. 
Basic SCRUM Framework Schwaber, K., & Beedle, M. (2001). Agile software development with scrum. Upper Saddle River, NJ: Prentice-Hall.  Created by Jeff Sutherland at Easel in 1993  Product backlog comprised of prioritized features  Iterative sprint-to-sprint, adaptive & emergent model 16
Models of AGILE PROJECT MGT. 17  Dozens of Agile project management models emerged  Many stem from principles of Extreme Programming  Vision, releases, & iterative development common Prioritization Feasibility Planning Tracking Reporting Review Visionate Speculate Innovate Re-Evaluate Disseminate Terminate Scoping Planning Feasibility Cyclical Dev. Checkpoint Review Envision Speculate Explore Iterate Launch Close Vision Roadmap Release Plan Sprint Plan Daily Scrum Retrospective Thomsett, R. (2002). Radical project management. Upper Saddle River, NJ: Prentice-Hall. DeCarlo, D. (2004). Extreme project management: Using leadership, principles, and tools to deliver value in the face of volatility. San Francisco, CA: Jossey-Bass. Wysocki, R.F. (2010). Adaptive project framework: Managing complexity in the face of uncertainty. Boston, MA: Pearson Education. Highsmith, J. A. (2010). Agile project management: Creating innovative products. Boston, MA: Pearson Education. Layton, M. C., & Maurer, R. (2011). Agile project management for dummies. Hoboken, NJ: Wiley Publishing. RADICAL - 2002 - EXTREME - 2004 - ADAPTIVE - 2010 - AGILE - 2010- SIMPLIFIED - 2011 - 
Layton, M. C., & Maurer, R. (2011). Agile project management for dummies. Hoboken, NJ: Wiley Publishing.  Created by Mark Layton at PlatinumEdge in 2012  Mix of new product development, XP, and Scrum  Simplified codification of XP and Scrum hybrid 18 Simplified AGILE PROJECT MGT.
19  Numerous models of agile portfolio mgt. emerging  Based on lean-kanban, release planning, and Scrum  Include organization, program, & project management Schwaber, K. (2007). The enterprise and scrum. Redmond, WA: Microsoft Press. Leffingwell, D. (2007). Scaling software agility: Best practices for large enterprises. Boston, MA: Pearson Education. Larman, C., & Vodde, B. (2008). Scaling lean and agile development: Thinking and organizational tools for large-scale scrum. Boston, MA: Addison-Wesley. Ambler, S. W., & Lines, M. (2012). Disciplined agile delivery: A practitioner's guide to agile software delivery in the enterprise. Boston, MA: Pearson Education. Thompson, K. (2013). cPrime’s R.A.G.E. is unleashed: Agile leaders rejoice! Retrieved March 28, 2014, from http://www.cprime.com/tag/agile-governance Schwaber, K. (2015). The definitive guide to nexus: The exoskeleton of scaled scrum development. Lexington, MA: Scrum.Org  Models of AGILE PORTFOLIO MGT. ESCRUM - 2007 - SAFe - 2007 - LESS - 2007 - DAD - 2012 - RAGE - 2013 - SPS - 2015 - Product Mgt Program Mgt Project Mgt Process Mgt Business Mgt Market Mgt Strategic Mgt Portfolio Mgt Program Mgt Team Mgt Quality Mgt Delivery Mgt Business Mgt Portfolio Mgt Product Mgt Area Mgt Sprint Mgt Release Mgt Business Mgt Portfolio Mgt Inception Construction Iterations Transition Business Governance Portfolio Program Project Delivery Product Mgt Program Mgt Sprint Mgt Team Mgt. Integ Mgt. Release Mgt
Scaled Agile Framework (SAFE)  Created by Dean Leffingwell of Rally in 2007  Knowledge to scale agile practices to enterprise  Hybrid of Kanban, XP release planning, and Scrum 20 Leffingwell, D. (2007). Scaling software agility: Best practices for large enterprises. Boston, MA: Pearson Education. 
21 Agile Performance MeasurementWork(Story,Point,Task)orEffort(Week,Day,Hour) Time Unit (Roadmap, Release, Iteration, Month, Week, Day, Hour, etc.) Burndown Work(Story,Point,Task)orEffort(Week,Day,Hour) Time Unit (Roadmap, Release, Iteration, Month, Week, Day, Hour, etc.) Cumulative Flow Work(Story,Point,Task)orEffort(Week,Day,Hour) Time Unit (Roadmap, Release, Iteration, Month, Week, Day, Hour, etc.) Earned Value Management - EVM CPI SPI PPC APC Work(Story,Point,Task)orEffort(Week,Day,Hour) Time Unit (Roadmap, Release, Iteration, Month, Week, Day, Hour, etc.) Earned Business Value - EBV
What is Agile Testing?  Traditional testing is a late, manual process  Agile testing is an early and automated process  Goal to deliver early & often and V&V components 22 Rico, D. F. (2012). Agile testing resources. Retrieved Sep. 9, 2012, from http://davidfrico.com/agile-testing-resources.txt Crispin, L., & Gregory, J. (2009). Agile testing: A practical guide for testers and agile teams. Boston, MA: Addison-Wesley. Grant, T. (2005). Continuous integration using cruise control. Northern Virginia Java Users Group (Novajug), Reston, Virginia, USA.  AGILE TESTING - Early Incremental Testing - TRADITIONAL TESTING - Late Big Bang Integration Testing - Test Criteria Accompany Stories Automated Tests Written First Units Coded-Tested One at Time Code is Frequently Checked In Code Automatically Retrieved Code Automatically Compiled Tests Automatically Executed Instant Feedback & Test Reports Test Criteria Written After Fact Manual Tests Written Much Later Units Coded Late All at One Time Code Checked In Late in Project Code Manually Submitted to Test Code Manually Compiled & Built Tests Manually Executed Late Late Project Feedback & Reports  Code Automatically DeployedLate Defects Freeze Projects
BASIC—Test Driven Development  Term coined by Kent Beck in 2003  Consists of writing all tests before design  Ensures all components are verified and validated 23Beck, K. (2003). Test-driven development: By example. Boston, MA: Addison-Wesley.
ADVANCED—Continuous Integration  Term coined by Martin Fowler in 1998  Process of automated build/regression testing  Evaluates impact of changes against entire system 24Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration: Improving software quality and reducing risk. Boston, MA: Addison-Wesley.  ALL DEVELOPERS RUN PRIVATE BUILDS  DEVELOPERS COMMIT CODE TO VERSION CONTROL  INTEGRATION BUILDS OCCUR SEVERAL TIMES PER DAY  100% OF SYSTEM TESTS MUST PASS FOR EVERY BUILD  A SHIPPABLE PRODUCT RESULTS FROM EVERY BUILD  FIXING BROKEN BUILDS IS OF THE HIGHEST PRIORITY  REPORTS AUTOMATICALLY GENERATED & REVIEWED
 Agile testing consists of seven broad practices  Automated build, database, inspection, tests, etc.  Include reporting, documentation, deployment, etc. 25 Practice Building Database Inspections Testing Feedback Documentation Deployment Description Frequently assembling products and services to ensure delivery readiness Frequently generating/analyzing database schemas, queries, and forms Frequently performing automated static analysis of product/service quality Frequently performing automated dynamic product and service evaluation Frequently generating automated status reports/messages for all stakeholders Frequently performing automated technical/customer document generation Frequently performing automated delivery of products/services to end users Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration: Improving software quality and reducing risk. Boston, MA: Addison-Wesley. Humble, J., & Farley, D. (2011). Continuous delivery. Boston, MA: Pearson Education.  PRACTICES—Continuous Integration
 Created by Jez Humble of ThoughtWorks in 2011  Includes CM, build, testing, integration, release, etc.  Goal is one-touch automation of deployment pipeline 26 Humble, J., & Farley, D. (2011). Continuous delivery. Boston, MA: Pearson Education. Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration. Boston, MA: Addison-Wesley. Ohara, D. (2012). Continuous delivery and the world of devops. San Francisco, CA: GigaOM Pro.   CoQ • 80% MS Tst • 8/10 No Val • $24B in 90s • Rep by CD • Not Add MLK ENTERPRISE—Continuous Delivery
 Created by Patrick Debois of Jedi BVBA in 2007  Collaboration of developers & infrastructure people  Goal to automate the deployment to end-user devices 27 Bass, L., Weber, I., & Zhu, L. (2015). Devops: A software architect's perspective. Old Tappan, NJ: Pearson Education. Gruver, G., & Mouser, T. (2015). Leading the transformation: Applying agile and devops at scale. Portland, OR: IT Revolution Press. Humble, J., Molesky, J., & O'Reilly, B. (2015). Lean enterprise: How high performance organizations innovate at scale. Sebastopol, CA: O'Reilly Media.  GLOBAL—Development Operations 
 Agile methods are based on traditional measures  Story points, velocity, and burndown basic metrics  Experts use Agile EVM, test, ROI & portfolio metrics 28Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing. AGILE METRICS 1. Agile CODE Metrics 2. Agile PROJECT Metrics 3. Agile TRACKING Metrics 4. Agile TESTING Metrics 5. Agile VALUE Metrics 6. Agile HEALTH Metrics 7. Agile PORTFOLIO Metrics 1. Agile CODE Metrics  Code Size  Code Complexity  Object Oriented  Code Coverage  Code Defects  Relational Design 2. Agile PROJECT Metrics  Software Size  Software Productivity  Software Effort  Software Quality  Software Schedule  Software Success 3. Agile TRACKING Metrics  Story Points  Sprint Burndown  Release Burndown  Velocity  Feature Progress  Agile Earned Value 4. Agile TESTING Metrics  Test Coverage  Test Automation  Integration Builds  Running Tested Features  DevOps Automation  Deployment Frequency 7. Agile PORTFOLIO Metrics  Portfolio Kanban  Epic Progress  Portfolio Radar  Release Train Radar  Lean Portfolio Metrics  Enterprise Scorecard 6. Agile HEALTH Metrics  Teamwork Quality  Collaboration Quality  Agile Process Maturity  Agile Adoption Rate  Degree of Agility  Product Flexibility 5. Agile VALUE Metrics  Total Lifecycle Costs  Total Lifecycle Benefits  Benefit to Cost Ratio  Return on Investment  Net Present Value  Real Options Analysis Agile Testing Metrics—Taxonomy
29 METRIC DESCRIPTION TEST COVERAGE Percent or degree to which software source code is tested TEST AUTOMATION Ratio or degree to which software tests are automated INTEGRATION BUILDS Frequency of automated software builds and integrations RUNNING TESTED FEATURES Number of completed and tested features or user stories DEVOPS AUTOMATION Ratio or degree to which deployments are automated DEPLOYMENT FREQUENCY Frequency of automated software deployments or deliveries  Software test automation emerged during the 1970s  Reached their height in personal computer (PC) era  Most are FOSS and used by successful agile teams Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration: Improving software quality and reducing risk. Boston, MA: Addison-Wesley. Agile Testing Metrics—Definitions
Agile Testing Metrics—Example 30Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration: Improving software quality and reducing risk. Boston, MA: Addison-Wesley.
31 Traditional vs. Agile Cumulative Flow Work(Story,Point,Task)orEffort(Week,Day,Hour) Time Unit (Roadmap, Release, Iteration, Month, Week, Day, Hour, etc.) Work(Story,Point,Task)orEffort(Week,Day,Hour) Time Unit (Roadmap, Release, Iteration, Month, Week, Day, Hour, etc.) Traditional Cumulative Flow Agile Cumulative Flow  Late big bang integration increases WIP backlog  Agile testing early and often reduces WIP backlog  Improves workflow and reduces WIP & lead times Anderson, D. J. (2004). Agile management for software engineering. Upper Saddle River, NJ: Pearson Education. Anderson, D. J. (2010). Kanban: Successful evolutionary change for your technology business. Sequim, WA: Blue Hole Press.  Agile Testing—Workflow
 Fewer integrations leave in higher bug counts  Frequent, early integrations eliminate most defects  Goal is to have as many early integrations as possible 32 Lacoste, F. J. (2009). Killing the gatekeeper: Introducing a continuous integration system. Proceedings of the Agile 2009 Conference, Chicago, Illinois, USA, 387-392.   Number of Integrations Less Defects •More Integrations •Early IntegrationsMore Defects •Few Integrations •Late Integrations  Agile Testing—Economic Drivers
 Traditional testing finds a defect in about 10 hours  Manual code inspections find a defect in 1 hour  Agile testing finds a defect every 6 minutes 33 Rico, D. F. (2012). The Cost of Quality (CoQ) for Agile vs. Traditional Project Management. Fairfax, VA: Gantthead.Com.  Agile Testing—Economics
 Agile testing is 10x better than code inspections  Agile testing is 100x better than traditional testing  Agile testing is done earlier “and” 1,000x more often 34 Rico, D. F. (2012). The Cost of Quality (CoQ) for Agile vs. Traditional Project Management. Fairfax, VA: Gantthead.Com.  Agile Testing—Cost of Quality
Agile Cost & Benefit Analysis  Costs based on avg. productivity and quality  Productivity ranged from 4.7 to 5.9 LOC an hour  Costs were $588,202 and benefits were $3,930,631 35 Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods: Maximizing ROI with just-in-time processes and documentation. Ft. Lauderdale, FL: J. Ross Publishing. d1 = [ln(Benefits  Costs) + (Rate + 0.5  Risk2)  Years]  Risk   Years, d2 = d1  Risk   Years   5 1i 
Benefits of Agile Methods  Analysis of 23 agile vs. 7,500 traditional projects  Agile projects are 54% better than traditional ones  Agile has lower costs (61%) and fewer defects (93%) Mah, M. (2008). Measuring agile in the enterprise: Proceedings of the Agile 2008 Conference, Toronto, Canada. Project Cost in Millions $ 0.75 1.50 2.25 3.00 2.8 1.1 Before Agile After Agile 61% Lower Cost Total Staffing 18 11 Before Agile After Agile 39% Less Staff 5 10 15 20 Delivery Time in Months 5 10 15 20 18 13.5 Before Agile After Agile 24% Faster Cumulative Defects 625 1250 1875 2500 2270 381 Before Agile After Agile 93% Less Defects 36     
Agile vs. Traditional Success  Traditional projects succeed at 50% industry avg.  Traditional projects are challenged 20% more often  Agile projects succeed 3x more and fail 3x less often Standish Group. (2012). Chaos manifesto. Boston, MA: Author. 37 Agile Traditional Success 42% Failed 9% Challenged 49% Success 14% Failed 29% Challenged 57% 
Grant, T. (2005). Continuous integration using cruise control. Northern Virginia Java Users Group (Novajug), Reston, Virginia, USA. Fredrick, J. (2008). Accelerate software delivery with continuous integration and testing. Japanese Symposium on Software Testing, Tokyo, Japan.  Most agile testing tools are “free” open source  Build server costs no more than a commodity PC  10x more efficient/effective than traditional testing 38  Agile Testing—CI Statistics
39  Hewlett-Packard is a major user of CI, CD, & DevOps  400 engineers developed 10 million LOC in 4 years  Major gains in testing, deployment, & innovation Gruver, G., Young, M. & Fulghum, P. (2013). A practical approach to large-scale agile development. Upper Saddle River, NJ: Pearson Education.  TYPE METRIC MANUAL DEVOPS MAJOR GAINS CYCLE TIME IMPROVEMENTS Build Time 40 Hours 3 Hours 13 x No. Builds 1-2 per Day 10-15 per Day 8 x Feedback 1 per Day 100 per Day 100 x Regression Testing 240 Hours 24 Hours 10 x DEVELOPMENT COST EFFORT DISTRIBUTION Integration 10% 2% 5 x Planning 20% 5% 4 x Porting 25% 15% 2 x Support 25% 5% 5 x Testing 15% 5% 3 x Innovation 5% 40% 8 x Agile Testing—CD Statistics
 Assembla went from 2 to 45 releases every month  15K Google developers run 120 million tests per day  30K+ Amazon developers deliver 8,600 releases a day 40Singleton, A. (2014). Unblock: A guide to the new continuous agile. Needham, MA: Assembla, Inc.  62x Faster U.S. DoD IT Project 3,645x Faster U.S. DoD IT Project  Agile Testing—DevOps Statistics
 Google early adopter of agile methods and Scrum  Google also uses agile testing at enterprise scale  15,000 developers run 120 million tests per day 41 Micco, J. (2013). Continuous integration at google scale. Eclipse Con, Boston, MA. Whittaker, J., Arbon, J., & Carollo, J. (2012). How google tests software. Upper Saddle River, NJ: Pearson Education.  440 billion unique users run 37 trillion searches each year  Single monolithic code tree with mixed language code  Submissions at head – One branch – All from source  20+ code changes/minute – 50% code change/month  5,500+ submissions/day – 120 million tests per day  80,000 builds per day – 20 million builds per year  Auto code inspections – For low defect density  10X programming productivity improvement  $150 million in annual labor savings (ROI as a result)    Agile Testing—Google Statistics
 Amazon adopted agile in 1999 and Scrum in 2004  Using enterprise-scale continuous delivery by 2010  30,000+ developers deploy over 8,600 releases a day 42 Atlas, A. (2009). Accidental adoption: The story of scrum at amazon.com. Proceedings of the Agile 2009 Conference, Chicago, Illinois, USA, 135-140. Jenkins, J. (2011). Velocity culture at amazon.com. Proceedings of the Velocity 2011 Conference, Santa Clara, California, USA. Elisha, S. (2013). Continuous deployment with amazon web services. Proceedings of the AWS Summit 2013, Sydney, New South Wales, Australia.  Software deployment every 11.6 seconds (as of 2011)  24,828 to 86,320 releases per Iteration  161,379 to 561,080 releases per Quarter  645,517 to 2,244,320 releases per Year  Automatic, split-second roll-forward & backward  75-90% reduction in release-caused outages (0.001%)  Millions of times faster (than traditional methods)  4,357,241 to 15,149,160 per traditional release  Thousands of times faster (than manual agility)  161,379 to 561,080 per Scrum/SAFe release  Used agile methods long before U.S. government (1999)      Agile Testing—Amazon Statistics
 Enables enterprises to be flexible but disciplined  Allows enterprises to distribute project work teams  Ensures distributed project teams are collaborating 43 Agile Tools “Across the Life Cycle” Project Management Requirements DOORS Requisite Pro SLATE Design Rhapsody Telelogic System Architect Rational System Architect Coding Eclipse Visual Studio Sun Studio Testing JUnit NUnit Xunit CPPUnit Gtest Fit Fitnesse Selenium Quality Assurance CheckStyle PMD EMMA Jdepend Cobertura Gcov Configuration Mgt Subversion (SVN) Concurrent Versions Sys. ClearCase Build Automation Ant NAnt Maven Make Continuous Integ. Cruise Control Hudson BuildBot Collaboration WebEx Skype MeetMe Wimba Wiki MediaWiki TracWiki PhpWiki Documentation NDoc Javadoc Doxygen iText Version One Rally Scrum Works VSTS Agile Team Agile Enterprise Scope Manager Story Studio XP Plan It Iterate XP Tracker Agilo XP CGI XP Web Xplanner Ice Scrum Project Cards Target Process Xtreme Planner Team System Community Enterprise Mingle Hansoft 
 There are literally hundreds of agile testing tools  There are tools for building, testing, and deployment  Integration tools monitor repositories and initiate tests 44 Agile Tools “In-Depth Test Automation” Smart, J. (2009). Automated deployment with maven and friends: Going the whole nine yards. Proceedings of the Agile 2009 Conference, Chicago, Illinois, USA. 
 Simple example of a DevOps reference architecture  Includes CM, continuous integration, & deployment  Code automatically built/tested/deployed to users 45 Agile Tools “Simple DevOps Automation” Morris, B., & Cassatt, C. (2015). Devops for the rest of us. Proceedings of the Agile DC Conference, Washington, DC, USA. Weeks, D. E. (2014). Devops and continuous delivery reference architectures (volume 1 & 2). Fulton, MD: Sonatype. 
46 Agile Tools “Periodic Table of DevOps Automation” XeniaLabs. (2016). Periodic table of devops tools. Retrieved April 11, 2016, from https://xebialabs.com/periodic-table-of-devops-tools.
47 Holler, R. (2015). Ninth annual state of agile survey: State of agile development. Atlanta, GA: VersionOne.  VersionOne found 94% using agile methods today  Most are using Scrum with several key XP practices  Lean-Kanban is a rising practice with a 31% adoption   Continuous Integration ● ● ● ● ● ● ● ● ● ● ●  ●  ● Agile Testing—Adoption Statistics
 Agile test use is low in spite of its age, i.e., 15 years  Many do not understand its utter simplicity and power  Failure to use agile testing undermines project success 48 Kim, D. (2013). The state of scrum: Benchmarks and guidelines. Indianapolis, IN: Scrum Alliance. Agile Practices Retrospectives Refactoring Done Definition Test Tools Test Driven Dev. CM Tools Simplicity Pair Programming Technical Debt Agile Testing 13% Continuous Integrations Weekly Daily 2-3 Times Per Day Never 2-3 Times Per Iteration    Agile Testing—Usage Statistics
 Agile teams don’t often use TDD, CI, CD & DevOps  Implement independent test teams after Sprints done  Sprint Waterfalling, Scrummerfalling, & Wagile result 49 Heusser, M. (2015). 12 years of agile testing: What do we know now. Proceedings of the Agile Gathering, Grand Rapids, Michigan, USA.     Incorrect • Phased Testing • Separate Teams • Delayed Testing Correct • Integrated Testing • Integrated Teams • Continuous Testing Agile Testing—Anti-Patterns
 Agile testing slows down with very large systems  Slow testing slows integration and increases bugs  Agile testing can speed back up with more attention 50 Kokko, H. (2009). Increase productivity with large scale continuous integration. Proceedings of the Agile 2009 Conference, Chicago, Illinois, USA.  MICRO ADJUSTMENTS - Focused Impact Tuning- MACRO ADJUSTMENTS - Wide Impact Tuning- Add More CPUs & Memory Parallelize System Builds Replace 3rd Party Test Libraries Reduce or Remove Test Timeouts Select Different Tests Refactor Code & Components Tune Network & Software Tune Database & Middleware In-Memory Compilation Parallelize Test Runs Pre-Install Test Libraries Remove Process Randomness Use Faster Code & Test Tools Incremental vs. Big Bang Tests Parallelize Build & Install Tune & Optimize Build Process  Agile Testing—Scaling Practices
 Industry very slow in adopting agile testing model  Cost, difficulty, and territorialism are common issues  Developers must take initiative for disciplined testing 51 Technical BarriersOrganizational Barriers Developers don’t want to test · Infrequently committing code · Committing broken code · Failing to immediately fix builds · Not writing automated tests · Not ensuring 100% of tests pass · Not running private builds · Resorting to traditional testing Resistance to change · Fear of investment costs · Fear of learning new skills · Test group territorialism · Organizational policy conflicts · Overhead of maintaining CI · Complexity and scaling · Not developing a quality culture ·· Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration: Improving software quality and reducing risk. Boston, MA: Addison-Wesley.       Agile Testing—Common Barriers
 Eliminates big-bang integration in the 11th hour  Creates a repeatable and reliable testing process  Evaluates system-wide changes throughout project 52Maeda, M. K. (2009). Agile testing: Early, often, and smart. Arlington, MA: Cutter Consortium. What’s the Bottom Line? “Agile Testing Done Early & Often” Agile TestingTraditional Testing Dramatically reduces risks · Automates manual processes · Instant verification & validation · High project visibility · Greater confidence and morale · Incremental business value · 24x7 deployability to users · Highly quality and reliability Late defect discovery · Low quality software · Poor project visibility · Lack of deployability · Late big-bang integration · Testing is a bottleneck · Poor customer satisfaction · Outright project failure ·· 
Conclusion  Agile methods DON’T mean deliver it now & fix it later  Lightweight, yet disciplined approach to development  Reduced cost, risk, & waste while improving quality 53 Rico, D. F. (2012). What’s really happening in agile methods: Its principles revisited? Retrieved June 6, 2012, from http://davidfrico.com/agile-principles.pdf Rico, D. F. (2012). The promises and pitfalls of agile methods. Retrieved February 6, 2013 from, http://davidfrico.com/agile-pros-cons.pdf Rico, D. F. (2012). How do lean & agile intersect? Retrieved February 6, 2013, from http://davidfrico.com/agile-concept-model-3.pdf What How Result Flexibility Use lightweight, yet disciplined processes and artifacts Low work-in-process Customer Involve customers early and often throughout development Early feedback Prioritize Identify highest-priority, value-adding business needs Focus resources Descope Descope complex programs by an order of magnitude Simplify problem Decompose Divide the remaining scope into smaller batches Manageable pieces Iterate Implement pieces one at a time over long periods of time Diffuse risk Leanness Architect and design the system one iteration at a time JIT waste-free design Swarm Implement each component in small cross-functional teams Knowledge transfer Collaborate Use frequent informal communications as often as possible Efficient data transfer Test Early Incrementally test each component as it is developed Early verification Test Often Perform system-level regression testing every few minutes Early validation Adapt Frequently identify optimal process and product solutions Improve performance                 
Dave’s PROFESSIONAL CAPABILITIES 54 Software Quality Mgt. Technical Project Mgt. Software Development Methods Organization Change Systems Engineering Cost Estimating Government Contracting Government Acquisitions Lean Kanban Big Data, Cloud, NoSQL Workflow Automation Metrics, Models, & SPC Six Sigma BPR, IDEF0, & DoDAF DoD 5000, TRA, & SRA PSP, TSP, & Code Reviews CMMI & ISO 9001 Innovation Management Statistics, CFA, EFA, & SEM Research Methods Evolutionary Design Valuation — Cost-Benefit Analysis, B/CR, ROI, NPV, BEP, Real Options, etc. Lean-Agile — Scrum, SAFe, Continuous Integration & Delivery, DevOps, etc. STRENGTHS – Data Mining  Gathering & Reporting Performance Data  Strategic Planning  Executive & Manage- ment Briefs  Brownbags & Webinars  White Papers  Tiger-Teams  Short-Fuse Tasking  Audits & Reviews  Etc. ● Data mining. Metrics, benchmarks, & performance. ● Simplification. Refactoring, refinement, & streamlining. ● Assessments. Audits, reviews, appraisals, & risk analysis. ● Coaching. Diagnosing, debugging, & restarting stalled projects. ● Business cases. Cost, benefit, & return-on-investment (ROI) analysis. ● Communications. Executive summaries, white papers, & lightning talks. ● Strategy & tactics. Program, project, task, & activity scoping, charters, & plans. PMP, CSEP, FCP, FCT ACP, CSM, & SAFE 32 YEARS IN IT INDUSTRY
Books on Agile Testing  Thousands of textbooks on agile methods  Include requirements, design, coding, test, etc.  Continuous Integration, Delivery, & DevOps best 55  Beck, K. (2003). Test-driven development: By example. Boston, MA: Addison-Wesley. Crispin, L., & Gregory, J. (2009). Agile testing: A practical guide for testers and agile teams. Boston, MA: Addison-Wesley. Gregory, J., & Crispin, L. (2015). More agile testing: Learning journeys for the whole team. Upper Saddle River, NJ: Pearson Education. Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration: Improving software quality and reducing risk. Boston, MA: Addison-Wesley. Humble, J., & Farley, D. (2011). Continuous delivery: Reliable software releases through build, test, and deployment automation. Boston, MA: Pearson Education.
Books on ROI of SW Methods  Guides to software methods for business leaders  Communicates the business value of IT approaches  Rosetta stones to unlocking ROI of software methods  http://davidfrico.com/agile-book.htm (Description)  http://davidfrico.com/roi-book.htm (Description) 56 

More Related Content

PDF
The Product Owner Role
byRoman Pichler
 
PPT
Agile Scrum Presentation-Detailed
byPrashaanth T R
 
PDF
Introducing SAFe 5.0 the operating system for Business Agility
byLeanwisdom
 
PDF
The Agile Product Owner: Beyond the Books and Classrooms
byEnterprise Knowledge
 
PDF
Agile Process Introduction
byNguyen Hai
 
PDF
12 principles for Agile Development
byJulien Henzelin
 
PPTX
The Next Wave of Reliability Engineering
byMichael Kehoe
 
PDF
What is Agile Methodology?
byQA InfoTech
 
The Product Owner Role
byRoman Pichler
 
Agile Scrum Presentation-Detailed
byPrashaanth T R
 
Introducing SAFe 5.0 the operating system for Business Agility
byLeanwisdom
 
The Agile Product Owner: Beyond the Books and Classrooms
byEnterprise Knowledge
 
Agile Process Introduction
byNguyen Hai
 
12 principles for Agile Development
byJulien Henzelin
 
The Next Wave of Reliability Engineering
byMichael Kehoe
 
What is Agile Methodology?
byQA InfoTech
 

What's hot

PDF
Getting started with Site Reliability Engineering (SRE)
byAbeer R
 
PDF
Agile & Scrum Training
byConscires Agile Practices
 
PDF
Scrum - Agile Methodology
byNiel Deckx
 
PPTX
Scrum best practices
byShuchi Singla AKT,SPC4,PMI-ACP,ITIL(F),CP-AAT
 
PPT
Agile presentation
byinfolock
 
PDF
10 steps to a successsful enterprise agile transformation global scrum 2018
byAgile Velocity
 
PDF
Cost of Delay: An Economic Approach to Decision Making
byRoger Turnau
 
PDF
Waterfall to Agile
byMighty Guides, Inc.
 
PDF
Introduction agile scrum methodology
byAmit Verma
 
PDF
Prioritization Techniques for Agile Teams
byTarang Baxi
 
PPTX
What is Agile Project Management? | Agile Project Management | Invensis Learn...
byInvensis Learning
 
PDF
Product Owner and Strategy
byRoman Pichler
 
PDF
Agile mindset
byMustafa Savaşcı
 
PDF
SRE From Scratch
byGrier Johnson
 
PDF
mÉTODO CRYSTAL CLEAR.pdf
byKaduFarias1
 
PPTX
Agile Mindset For Executives
byMichael Tarnowski
 
PDF
Professional Scrum Product Owner I (PSPO-I)
byMirosław Dąbrowski C-level IT manager, CEO, Agile, ICF Coach, Speaker
 
PPTX
Project Management With Scrum
byTommy Norman
 
PPTX
Agile - Scrum Presentation
bygihanlsw
 
PPTX
Strategies for Large Scale Agile Transformation
byNishanth K Hydru
 
Getting started with Site Reliability Engineering (SRE)
byAbeer R
 
Agile & Scrum Training
byConscires Agile Practices
 
Scrum - Agile Methodology
byNiel Deckx
 
Scrum best practices
byShuchi Singla AKT,SPC4,PMI-ACP,ITIL(F),CP-AAT
 
Agile presentation
byinfolock
 
10 steps to a successsful enterprise agile transformation global scrum 2018
byAgile Velocity
 
Cost of Delay: An Economic Approach to Decision Making
byRoger Turnau
 
Waterfall to Agile
byMighty Guides, Inc.
 
Introduction agile scrum methodology
byAmit Verma
 
Prioritization Techniques for Agile Teams
byTarang Baxi
 
What is Agile Project Management? | Agile Project Management | Invensis Learn...
byInvensis Learning
 
Product Owner and Strategy
byRoman Pichler
 
Agile mindset
byMustafa Savaşcı
 
SRE From Scratch
byGrier Johnson
 
mÉTODO CRYSTAL CLEAR.pdf
byKaduFarias1
 
Agile Mindset For Executives
byMichael Tarnowski
 
Professional Scrum Product Owner I (PSPO-I)
byMirosław Dąbrowski C-level IT manager, CEO, Agile, ICF Coach, Speaker
 
Project Management With Scrum
byTommy Norman
 
Agile - Scrum Presentation
bygihanlsw
 
Strategies for Large Scale Agile Transformation
byNishanth K Hydru
 

Viewers also liked

PDF
Business Value of Agile Organizations: Strategies, Models, & Principles for E...
byDavid Rico
 
PDF
Lean & Agile Project Manaagement: Its Leadership Considerations
byDavid Rico
 
PPTX
The Business Value of Test-Driven Development
byRob Myers
 
PDF
Lean & Agile Organizational Leadership: History, Theory, Models, & Popular Ideas
byDavid Rico
 
PDF
Business Value of CI, CD, & DevOpsSec: Scaling to Billion User Systems Using ...
byDavid Rico
 
PPTX
The (financial) Return of Agile
byFrank Vogelezang
 
PDF
(v2.0) Better Functional Design Through Test-Driven Development
byPhil Calçado
 
PDF
Hadoop MapReduce - I'm Sold, Now What?
byikanow
 
PDF
Mobile Apps development best practices. TDD, CI, CD
byGlobalLogic Ukraine
 
PDF
Toronto Agile Tour - Pikachu Talk
byJason Little
 
ODP
Pair Programming Talk
byjlangr
 
PDF
Business Value of Agile Methods: Its Leadership Considerations
byDavid Rico
 
PDF
Xtreme Programming
byNoretSarted
 
PDF
A 3-Day Introduction for Sr. Engineers and Tech. Support Staff
byDavid Rico
 
PDF
ODD: Extending a Specification 1.2
byJonathan Herring
 
PPTX
Standish Chaos Report - Cercle de Discussion - PMI Branche Midi-Pyrénées (Vi...
byVincent Dumain
 
PDF
Business Value of Agile Methods: Using ROI and REal Options
byDavid Rico
 
PDF
Introduction to Project Management
byGhent University
 
PPS
Bdd: Tdd and beyond the infinite
byGiordano Scalzo
 
PDF
XP In 10 slides
byRobert Burrell Donkin
 
Business Value of Agile Organizations: Strategies, Models, & Principles for E...
byDavid Rico
 
Lean & Agile Project Manaagement: Its Leadership Considerations
byDavid Rico
 
The Business Value of Test-Driven Development
byRob Myers
 
Lean & Agile Organizational Leadership: History, Theory, Models, & Popular Ideas
byDavid Rico
 
Business Value of CI, CD, & DevOpsSec: Scaling to Billion User Systems Using ...
byDavid Rico
 
The (financial) Return of Agile
byFrank Vogelezang
 
(v2.0) Better Functional Design Through Test-Driven Development
byPhil Calçado
 
Hadoop MapReduce - I'm Sold, Now What?
byikanow
 
Mobile Apps development best practices. TDD, CI, CD
byGlobalLogic Ukraine
 
Toronto Agile Tour - Pikachu Talk
byJason Little
 
Pair Programming Talk
byjlangr
 
Business Value of Agile Methods: Its Leadership Considerations
byDavid Rico
 
Xtreme Programming
byNoretSarted
 
A 3-Day Introduction for Sr. Engineers and Tech. Support Staff
byDavid Rico
 
ODD: Extending a Specification 1.2
byJonathan Herring
 
Standish Chaos Report - Cercle de Discussion - PMI Branche Midi-Pyrénées (Vi...
byVincent Dumain
 
Business Value of Agile Methods: Using ROI and REal Options
byDavid Rico
 
Introduction to Project Management
byGhent University
 
Bdd: Tdd and beyond the infinite
byGiordano Scalzo
 
XP In 10 slides
byRobert Burrell Donkin
 

Similar to Business Value of Agile Testing: Using TDD, CI, CD, & DevOps

PDF
Business Value of Agile Methods: Benefits of Testing Early & Often
byDavid Rico
 
PDF
Business Value of Agile Methods: Using ROI & Real Options
byDavid Rico
 
PDF
Lean & Agile Methods & Frameworks: Perspectives on Kanban for IT
byDavid Rico
 
PDF
Intro to Agile Methods for Execs, Leaders, and Managers
byDavid Rico
 
PDF
David Rico Business Value Agile Methods19 Feb 14
byLiana Underwood
 
PPTX
Business Case for Agile - Time for ROI Check
byDr. Tathagat Varma
 
PDF
Agile Methods Cost of Quality: Benefits of Testing Early & Often
byDavid Rico
 
PDF
Agile methods cost of quality
byCristiano Caetano
 
PDF
What is agile
byOrange and Bronze Software Labs
 
PDF
Why Agile?
byDaniel van den Hoven
 
PDF
Impact of Agile in Software Development Organizations
byAlvaro Ruiz de Mendarozqueta
 
PDF
Return on Investment (ROI) of Lean & Agile Methods
byDavid Rico
 
PPT
Agile Executive Briefing - Situational Assessment + 50k Ft View
byMichael Sahota
 
PDF
Hans Eckman: 7 Agile and DevOps Insights I Wish I Knew Earlier
byEdunomica
 
PDF
Business Value of Lean Thinking
byDavid Rico
 
PDF
The Rationale for Continuous Delivery
byPerforce
 
PDF
Lean & Agile Performance Measurement: Metrics, Models, & Measures
byDavid Rico
 
PPTX
Agile metrics - Agile KC Meeting 9/26/13
bymolsonkc
 
PPTX
Introducing Agile to the Enterprise
byGibraltar Software
 
PDF
Agile Product Management
bySVPMA
 
Business Value of Agile Methods: Benefits of Testing Early & Often
byDavid Rico
 
Business Value of Agile Methods: Using ROI & Real Options
byDavid Rico
 
Lean & Agile Methods & Frameworks: Perspectives on Kanban for IT
byDavid Rico
 
Intro to Agile Methods for Execs, Leaders, and Managers
byDavid Rico
 
David Rico Business Value Agile Methods19 Feb 14
byLiana Underwood
 
Business Case for Agile - Time for ROI Check
byDr. Tathagat Varma
 
Agile Methods Cost of Quality: Benefits of Testing Early & Often
byDavid Rico
 
Agile methods cost of quality
byCristiano Caetano
 
What is agile
byOrange and Bronze Software Labs
 
Why Agile?
byDaniel van den Hoven
 
Impact of Agile in Software Development Organizations
byAlvaro Ruiz de Mendarozqueta
 
Return on Investment (ROI) of Lean & Agile Methods
byDavid Rico
 
Agile Executive Briefing - Situational Assessment + 50k Ft View
byMichael Sahota
 
Hans Eckman: 7 Agile and DevOps Insights I Wish I Knew Earlier
byEdunomica
 
Business Value of Lean Thinking
byDavid Rico
 
The Rationale for Continuous Delivery
byPerforce
 
Lean & Agile Performance Measurement: Metrics, Models, & Measures
byDavid Rico
 
Agile metrics - Agile KC Meeting 9/26/13
bymolsonkc
 
Introducing Agile to the Enterprise
byGibraltar Software
 
Agile Product Management
bySVPMA
 

More from David Rico

PDF
Business Value of Agile Product Management
byDavid Rico
 
PDF
Business Value of Agile Human Resources (AHR)
byDavid Rico
 
PDF
ROI of Organizational Agility for Transforming 21st Century Enterprises
byDavid Rico
 
PDF
ROI of Evolutionary Design to Rapidly Create Innovatively New Products & Serv...
byDavid Rico
 
PDF
Lean & Agile Thinking Principles for Leaders
byDavid Rico
 
PDF
ROI & Business Value of CI, CD, DevOps, DevSecOps, & Microservices
byDavid Rico
 
PDF
Scaled Agile Framework (SAFe) 4.6 in U.S. Government
byDavid Rico
 
PDF
Scaled Agile Framework (SAFe) 4.5 Tutorial ...
byDavid Rico
 
PDF
Scaled Agile Framework (SAFe) 4.5 Metrics
byDavid Rico
 
PDF
Lean & Agile Organizational Leadership
byDavid Rico
 
PDF
Business, Enterprise, & Organizational Agility
byDavid Rico
 
PDF
Lean & Agile Organizational Change
byDavid Rico
 
PDF
Using SAFe to Manage U.S. Government Agencies, Portfolios, & Acquisition Prog...
byDavid Rico
 
PDF
Growth of SAFe in Government Acquisitions, Contracts, & Portfolios
byDavid Rico
 
PDF
Lean & Agile Project Management
byDavid Rico
 
PDF
Business Value of CI, CD, & DevOps(Sec)
byDavid Rico
 
PDF
Lean & Agile Enterprise Frameworks: For Managing Large U.S. Government Cloud ...
byDavid Rico
 
Business Value of Agile Product Management
byDavid Rico
 
Business Value of Agile Human Resources (AHR)
byDavid Rico
 
ROI of Organizational Agility for Transforming 21st Century Enterprises
byDavid Rico
 
ROI of Evolutionary Design to Rapidly Create Innovatively New Products & Serv...
byDavid Rico
 
Lean & Agile Thinking Principles for Leaders
byDavid Rico
 
ROI & Business Value of CI, CD, DevOps, DevSecOps, & Microservices
byDavid Rico
 
Scaled Agile Framework (SAFe) 4.6 in U.S. Government
byDavid Rico
 
Scaled Agile Framework (SAFe) 4.5 Tutorial ...
byDavid Rico
 
Scaled Agile Framework (SAFe) 4.5 Metrics
byDavid Rico
 
Lean & Agile Organizational Leadership
byDavid Rico
 
Business, Enterprise, & Organizational Agility
byDavid Rico
 
Lean & Agile Organizational Change
byDavid Rico
 
Using SAFe to Manage U.S. Government Agencies, Portfolios, & Acquisition Prog...
byDavid Rico
 
Growth of SAFe in Government Acquisitions, Contracts, & Portfolios
byDavid Rico
 
Lean & Agile Project Management
byDavid Rico
 
Business Value of CI, CD, & DevOps(Sec)
byDavid Rico
 
Lean & Agile Enterprise Frameworks: For Managing Large U.S. Government Cloud ...
byDavid Rico
 

Recently uploaded

PPTX
Theory to Practice of Unsaturated Soil Mechanics-1.pptx
byMahmoodKhalid11
 
PDF
Basic Thumb rule For switchgear Selection
byVimal Kr
 
PDF
Computer Graphics Fundamentals (v0p1) - DannyJiang
byDanny Jiang
 
PPTX
We-Optimized-Everything-Except-Decision-Quality-Maintenance-MaintWiz.pptx
byMaintWiz Technologies Private Limited
 
PPT
L9_Statistical Evaluation of Measurement Data.ppt
bykohila15
 
DOCX
yearly Management report for Engineering and maintenance
bylamoushalhaj
 
PDF
Model QP 2025 scheme Q &A- Module 1 and 2.pdf
bySURESHA V
 
PDF
Infinite Sequence and Series: It Includes basic Sequence and Series
byDynamicDomain1
 
PDF
Introduction to C++.pdf it is best material for beginner student and above that
byaron63chala
 
PDF
Shear Strength of Soil/Mohr Coulomb Failure Criteria-1.pdf
byMahmoodKhalid11
 
PDF
Chemical Hazards at Workplace – Types, Properties & Exposure Routes, CORE-EHS
byCORE EHS
 
PPTX
Origin of Soil and Grain Size with Introduction and explanation
byMahmoodKhalid11
 
PDF
Module 4 python programming-1BPLCK105B-2025 by Dr.SV.pdf
bySURESHA V
 
PPT
Momentum and collisions in physics or engineering
byazizrahmanhakimi
 
PDF
Decision-Support-Systems-and-Decision-Making-Processes.pdf
byPatankarNikhil
 
PDF
Manual de instalacion de notifire NFS320
byMarcoPolitoPerez
 
PDF
3.2 Log & antilog amplifiers.pdf analog electronics
byanu200770
 
PPTX
Basin Design Service, LLC Introduction Presentation
byHubie Fix
 
PPTX
The Most Controversial TPM Debate in Maintenance Today Checklist TPM vs Outco...
byMaintWiz Technologies Private Limited
 
PDF
CME397 SURFACE ENGINEERING UNIT 1 FULL NOTES
bykarthi keyan
 
Theory to Practice of Unsaturated Soil Mechanics-1.pptx
byMahmoodKhalid11
 
Basic Thumb rule For switchgear Selection
byVimal Kr
 
Computer Graphics Fundamentals (v0p1) - DannyJiang
byDanny Jiang
 
We-Optimized-Everything-Except-Decision-Quality-Maintenance-MaintWiz.pptx
byMaintWiz Technologies Private Limited
 
L9_Statistical Evaluation of Measurement Data.ppt
bykohila15
 
yearly Management report for Engineering and maintenance
bylamoushalhaj
 
Model QP 2025 scheme Q &A- Module 1 and 2.pdf
bySURESHA V
 
Infinite Sequence and Series: It Includes basic Sequence and Series
byDynamicDomain1
 
Introduction to C++.pdf it is best material for beginner student and above that
byaron63chala
 
Shear Strength of Soil/Mohr Coulomb Failure Criteria-1.pdf
byMahmoodKhalid11
 
Chemical Hazards at Workplace – Types, Properties & Exposure Routes, CORE-EHS
byCORE EHS
 
Origin of Soil and Grain Size with Introduction and explanation
byMahmoodKhalid11
 
Module 4 python programming-1BPLCK105B-2025 by Dr.SV.pdf
bySURESHA V
 
Momentum and collisions in physics or engineering
byazizrahmanhakimi
 
Decision-Support-Systems-and-Decision-Making-Processes.pdf
byPatankarNikhil
 
Manual de instalacion de notifire NFS320
byMarcoPolitoPerez
 
3.2 Log & antilog amplifiers.pdf analog electronics
byanu200770
 
Basin Design Service, LLC Introduction Presentation
byHubie Fix
 
The Most Controversial TPM Debate in Maintenance Today Checklist TPM vs Outco...
byMaintWiz Technologies Private Limited
 
CME397 SURFACE ENGINEERING UNIT 1 FULL NOTES
bykarthi keyan
 

Business Value of Agile Testing: Using TDD, CI, CD, & DevOps

  • 1.
    Business Value of AgileTesting Using TDD, CI, CD & DevOps Dr. David F. Rico, PMP, CSEP, FCP, FCT, ACP, CSM, SAFe Twitter: @dr_david_f_rico Website: http://www.davidfrico.com LinkedIn: http://www.linkedin.com/in/davidfrico Agile Capabilities: http://davidfrico.com/rico-capability-agile.pdf Agile Resources: http://www.davidfrico.com/daves-agile-resources.htm Agile Cheat Sheet: http://davidfrico.com/key-agile-theories-ideas-and-principles.pdf
  • 2.
    Author Background  Gov’tcontractor with 32+ years of IT experience  B.S. Comp. Sci., M.S. Soft. Eng., & D.M. Info. Sys.  Large gov’t projects in U.S., Far/Mid-East, & Europe 2   Career systems & software engineering methodologist  Lean-Agile, Six Sigma, CMMI, ISO 9001, DoD 5000  NASA, USAF, Navy, Army, DISA, & DARPA projects  Published seven books & numerous journal articles  Intn’l keynote speaker, 130 talks to 12,000+ people  Specializes in metrics, models, & cost engineering  Cloud Computing, SOA, Web Services, FOSS, etc.  Adjunct at five Washington, DC-area universities
  • 3.
    Today’s Whirlwind Environment 3 Overruns Attrition Escalation Runaways Cancellation Global Competition Demanding Customers Organization Downsizing System Complexity Technology Change Vague Requirements WorkLife Imbalance Inefficiency High O&M Lower DoQ Vulnerable N-M Breach Reduced IT Budgets 81 Month Cycle Times Redundant Data Centers Lack of Interoperability Poor IT Security Overburdening Legacy Systems Obsolete Technology & Skills Pine, B. J. (1993). Mass customization: The new frontier in business competition. Boston, MA: Harvard Business School Press. Pontius, R. W. (2012). Acquisition of IT: Improving efficiency and effectiveness in IT acquisition in the DoD. Second Annual AFEI/NDIA Conference on Agile in DoD, Springfield, VA, USA.
  • 4.
    Software in U.S.DoD Systems Kennedy, M. P., & Umphress, D. A. (2011). An agile systems engineering process: The missing link. Crosstalk, 24(3), 16-20.  No. of software-intensive systems is growing  80% of US DoD functions performed in software  Major driver of cost, schedule, & tech. performance 4 
  • 5.
    Software in U.S.DoD Avionics Blackburn, M. R. (2014). Transforming systems engineering through a holistic approach to model centric engineering. Washington, DC: Stevens Institute of Technology.  Software in U.S. DoD avionics growing exponentially  10x growth from F-16 to F-22 (& another 10x to F-35)  Productivity must grow by 10x for next gen systems 5 
  • 6.
    Traditional Projects 6  Bigprojects result in poor quality and scope changes  Productivity declines with long queues/wait times  Large projects are unsuccessful or canceled Jones, C. (1991). Applied software measurement: Assuring productivity and quality. New York, NY: McGraw-Hill. Size vs. Quality DEFECTS 0.00 3.20 6.40 9.60 12.80 16.00 0 2 6 25 100 400 SIZE Size vs. Productivity PRODUCTIVITY 0.00 1.00 2.00 3.00 4.00 5.00 0 2 6 25 100 400 SIZE Size vs. Change CHANGE 0% 8% 16% 24% 32% 40% 0 2 6 25 100 400 SIZE Size vs. Success SUCCESS 0% 12% 24% 36% 48% 60% 0 2 6 25 100 400 SIZE 
  • 7.
    Global Project Failures 7 StandishGroup. (2015). Chaos summary 2015. Boston, MA: Author. Sessions, R. (2009). The IT complexity crisis: Danger and opportunity. Houston, TX: Object Watch.  Challenged and failed projects hover at 67%  Big projects fail more often, which is 5% to 10%  Of $1.7T spent on IT projects, over $858B were lost $0.0 $0.4 $0.7 $1.1 $1.4 $1.8 2002 2003 2004 2005 2006 2007 2008 2009 2010 Trillions(USDollars) Expenditures Failed Investments  0% 20% 40% 60% 80% 100% 28% 34% 29% 35% 32% 33% 27% 28% 29% 49% 51% 53% 46% 44% 41% 56% 55% 52% 23% 15% 18% 19% 24% 26% 17% 17% 19% 2000 2002 2004 2006 2008 2010 2012 2014 2015 Year Successful Challenged Failed
  • 8.
    Requirements Defects &Waste 8 Sheldon, F. T. et al. (1992). Reliability measurement: From theory to practice. IEEE Software, 9(4), 13-20 Johnson, J. (2002). ROI: It's your job. Extreme Programming 2002 Conference, Alghero, Sardinia, Italy.  Requirements defects are #1 reason projects fail  Traditional projects specify too many requirements  More than 65% of requirements are never used at all Other 7% Requirements 47% Design 28% Implementation 18% Defects Always 7% Often 13% Sometimes 16% Rarely 19% Never 45% Waste 
  • 9.
    What is Agility? A-gil-i-ty (ә-'ji-lә-tē) Property consisting of quickness, lightness, and ease of movement; To be very nimble  The ability to create and respond to change in order to profit in a turbulent global business environment  The ability to quickly reprioritize use of resources when requirements, technology, and knowledge shift  A very fast response to sudden market changes and emerging threats by intensive customer interaction  Use of evolutionary, incremental, and iterative delivery to converge on an optimal customer solution  Maximizing BUSINESS VALUE with right sized, just- enough, and just-in-time processes and documentation Highsmith, J. A. (2002). Agile software development ecosystems. Boston, MA: Addison-Wesley. 9  
  • 10.
    What are AgileMethods? 10  People-centric way to create innovative solutions  Product-centric alternative to documents/process  Market-centric model to maximize business value Agile Manifesto. (2001). Manifesto for agile software development. Retrieved September 3, 2008, from http://www.agilemanifesto.org Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing. Rico, D. F. (2012). Agile conceptual model. Retrieved February 6, 2012, from http://davidfrico.com/agile-concept-model-1.pdf Customer Collaboration Working Systems & Software Individuals & Interactions Responding to Change valued more than valued more than valued more than valued more than Contracts Documentation Processes Project Plans  Frequent comm.  Close proximity  Regular meetings  Multiple comm. channels  Frequent feedback  Relationship strength  Leadership  Boundaries  Empowerment  Competence  Structure  Manageability/Motivation  Clear objectives  Small/feasible scope  Acceptance criteria  Timeboxed iterations  Valid operational results  Regular cadence/intervals  Org. flexibility  Mgt. flexibility  Process flexibility  System flexibility  Technology flexibility  Infrastructure flexibility  Contract compliance  Contract deliverables  Contract change orders  Lifecycle compliance  Process Maturity Level  Regulatory compliance  Document deliveries  Document comments  Document compliance  Cost Compliance  Scope Compliance  Schedule Compliance     Courage 
  • 11.
    Agile World View “Agility” has many dimensions other than IT  It ranges from leadership to technological agility  Today’s focus is on organizational & enterprise agility   Agile Leaders Agile Organization Change Agile Acquisition & Contracting Agile Strategic Planning Agile Capability Analysis Agile Program Management Agile Tech. Agile Information Systems Agile Tools Agile Processes & Practices Agile Systems Development Agile Project Management 11 
  • 12.
    Network Computer Operating System Middleware Applications APIs GUI How AgileWorks  Agile requirements implemented in slices vs. layers  User needs with higher business value are done first  Reduces cost & risk while increasing business success 12Shore, J. (2011). Evolutionary design illustrated. Norwegian Developers Conference, Oslo, Norway. Agile Traditional 1 2 3 Faster  Early ROI  Lower Costs  Fewer Defects  Manageable Risk  Better Performance  Smaller Attack Surface Late  No Value  Cost Overruns  Very Poor Quality  Uncontrollable Risk  Slowest Performance  More Security Incidents Seven Wastes 1. Rework 2. Motion 3. Waiting 4. Inventory 5. Transportation 6. Overprocessing 7. Overproduction MINIMIZES MAXIMIZES  JIT, Just-enough architecture  Early, in-process system V&V  Fast continuous improvement  Scalable to systems of systems  Maximizes successful outcomes  Myth of perfect architecture  Late big-bang integration tests  Year long improvement cycles  Breaks down on large projects  Undermines business success 
  • 13.
    Thousands of Tests ContinuouslyExecuted No More Late Big Bang Integration  User needs designed & developed one-at-a-time  Changes automatically detected, built, and tested  System fully tested and deployed as changes occur 13Humble, J., & Farley, D. (2011). Continuous delivery. Boston, MA: Pearson Education. Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration. Boston, MA: Addison-Wesley. Build Integration Server Version Control Server Build Scripts UsesWatches Build Status ProvidesDeveloper A Developer B Developer C Commits Changes Commits Changes Commits Changes Builds Database Analysis Testing Reporting Documentation Deployment Early, Automated, Fast, Efficient, & Repeatable Constant Readiness State & CM Control Lean, Waste Free, Low WIP, No Deadlocked Test Queues Rapidly & Successfully Dev. Complex Systems  Basic Agile Mechanics
  • 14.
    14 Capability/MMF #1 ● Feature1 ● Feature 2 ● Feature 3 ● Feature 4 ● Feature 5 ● Feature 6 ● Feature 7 Capability/MMF #2 ● Feature 8 ● Feature 9 ● Feature 10 ● Feature 11 ● Feature 12 ● Feature 13 ● Feature 14 Capability/MMF #3 ● Feature 15 ● Feature 16 ● Feature 17 ● Feature 18 ● Feature 19 ● Feature 20 ● Feature 21 Capability/MMF #4 ● Feature 22 ● Feature 23 ● Feature 24 ● Feature 25 ● Feature 26 ● Feature 27 ● Feature 28 Capability/MMF #5 ● Feature 29 ● Feature 30 ● Feature 31 ● Feature 32 ● Feature 33 ● Feature 34 ● Feature 35 Capability/MMF #6 ● Feature 36 ● Feature 37 ● Feature 38 ● Feature 39 ● Feature 40 ● Feature 41 ● Feature 42 Capability/MMF #7 ● Feature 43 ● Feature 44 ● Feature 45 ● Feature 46 ● Feature 47 ● Feature 48 ● Feature 49 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Evolving “Unified/Integrated” Enterprise Data Model “Disparate” LEGACY SYSTEM DATABASES (AND DATA MODELS) ETL A A B C D E F G H I J K A B C D E F A B C D E A B C D A B C A B “Legacy” MS SQL Server Stovepipes “Inter-Departmental” Linux Blade/Oracle/Java/WebSphere Server “Leased” DWA/HPC/Cloud Services Sprint 1 Sprint 2 Sprint 3 Sprint 4 Sprint 5 Sprint 6 Sprint 7 Release Release Release Release ETL ETL ETL ETL ETL ETL Bente, S., Bombosch, U., & Langade, S. (2012). Collaborative enterprise architecture: Enriching EA with lean, agile, and enterprise 2.0 practices. Waltham, MA: Elsevier. (for example, assume 25 user stories per feature, 175 user stories per capability/MMF, and 1,225 user stories total)  Organize needs into capabilities, features, and stories  Prioritize features, group releases, and initiate sprints  Develop minimum set of features with highest value Agile Systems Development
  • 15.
    Models of AGILEDEVELOPMENT 15  Agile methods spunoff flexible manufacturing 1990s  Extreme Programming (XP) swept the globe by 2002  Today, over 90% of IT projects use Scrum/XP hybrid Use Cases Domain Model Object Oriented Iterative Dev. Risk Planning Info. Radiators Planning Poker Product Backlog Sprint Backlog 2-4 Week Spring Daily Standup Sprint Demo Feasibility Business Study Func. Iteration Design Iteration Implementation Testing Domain Model Feature List Object Oriented Iterative Dev. Code Inspection Testing Release Plans User Stories Pair Programmer Iterative Dev. Test First Dev. Onsite Customer Cockburn, A. (2002). Agile software development. Boston, MA: Addison-Wesley. Schwaber, K., & Beedle, M. (2001). Agile software development with scrum. Upper Saddle River, NJ: Prentice-Hall. Stapleton, J. (1997). DSDM: A framework for business centered development. Harlow, England: Addison-Wesley. Palmer, S. R., & Felsing, J. M. (2002). A practical guide to feature driven development. Upper Saddle River, NJ: Prentice-Hall. Beck, K. (2000). Extreme programming explained: Embrace change. Reading, MA: Addison-Wesley. CRYSTAL METHODS - 1991 - SCRUM - 1993 - DSDM - 1993 - FDD - 1997 - XP - 1998 - Reflection W/S Retrospective Quality Control Quality Control Continuous Del. 
  • 16.
    Basic SCRUM Framework Schwaber,K., & Beedle, M. (2001). Agile software development with scrum. Upper Saddle River, NJ: Prentice-Hall.  Created by Jeff Sutherland at Easel in 1993  Product backlog comprised of prioritized features  Iterative sprint-to-sprint, adaptive & emergent model 16
  • 17.
    Models of AGILEPROJECT MGT. 17  Dozens of Agile project management models emerged  Many stem from principles of Extreme Programming  Vision, releases, & iterative development common Prioritization Feasibility Planning Tracking Reporting Review Visionate Speculate Innovate Re-Evaluate Disseminate Terminate Scoping Planning Feasibility Cyclical Dev. Checkpoint Review Envision Speculate Explore Iterate Launch Close Vision Roadmap Release Plan Sprint Plan Daily Scrum Retrospective Thomsett, R. (2002). Radical project management. Upper Saddle River, NJ: Prentice-Hall. DeCarlo, D. (2004). Extreme project management: Using leadership, principles, and tools to deliver value in the face of volatility. San Francisco, CA: Jossey-Bass. Wysocki, R.F. (2010). Adaptive project framework: Managing complexity in the face of uncertainty. Boston, MA: Pearson Education. Highsmith, J. A. (2010). Agile project management: Creating innovative products. Boston, MA: Pearson Education. Layton, M. C., & Maurer, R. (2011). Agile project management for dummies. Hoboken, NJ: Wiley Publishing. RADICAL - 2002 - EXTREME - 2004 - ADAPTIVE - 2010 - AGILE - 2010- SIMPLIFIED - 2011 - 
  • 18.
    Layton, M. C.,& Maurer, R. (2011). Agile project management for dummies. Hoboken, NJ: Wiley Publishing.  Created by Mark Layton at PlatinumEdge in 2012  Mix of new product development, XP, and Scrum  Simplified codification of XP and Scrum hybrid 18 Simplified AGILE PROJECT MGT.
  • 19.
    19  Numerous modelsof agile portfolio mgt. emerging  Based on lean-kanban, release planning, and Scrum  Include organization, program, & project management Schwaber, K. (2007). The enterprise and scrum. Redmond, WA: Microsoft Press. Leffingwell, D. (2007). Scaling software agility: Best practices for large enterprises. Boston, MA: Pearson Education. Larman, C., & Vodde, B. (2008). Scaling lean and agile development: Thinking and organizational tools for large-scale scrum. Boston, MA: Addison-Wesley. Ambler, S. W., & Lines, M. (2012). Disciplined agile delivery: A practitioner's guide to agile software delivery in the enterprise. Boston, MA: Pearson Education. Thompson, K. (2013). cPrime’s R.A.G.E. is unleashed: Agile leaders rejoice! Retrieved March 28, 2014, from http://www.cprime.com/tag/agile-governance Schwaber, K. (2015). The definitive guide to nexus: The exoskeleton of scaled scrum development. Lexington, MA: Scrum.Org  Models of AGILE PORTFOLIO MGT. ESCRUM - 2007 - SAFe - 2007 - LESS - 2007 - DAD - 2012 - RAGE - 2013 - SPS - 2015 - Product Mgt Program Mgt Project Mgt Process Mgt Business Mgt Market Mgt Strategic Mgt Portfolio Mgt Program Mgt Team Mgt Quality Mgt Delivery Mgt Business Mgt Portfolio Mgt Product Mgt Area Mgt Sprint Mgt Release Mgt Business Mgt Portfolio Mgt Inception Construction Iterations Transition Business Governance Portfolio Program Project Delivery Product Mgt Program Mgt Sprint Mgt Team Mgt. Integ Mgt. Release Mgt
  • 20.
    Scaled Agile Framework(SAFE)  Created by Dean Leffingwell of Rally in 2007  Knowledge to scale agile practices to enterprise  Hybrid of Kanban, XP release planning, and Scrum 20 Leffingwell, D. (2007). Scaling software agility: Best practices for large enterprises. Boston, MA: Pearson Education. 
  • 21.
    21 Agile Performance MeasurementWork(Story,Point,Task)orEffort(Week,Day,Hour) TimeUnit (Roadmap, Release, Iteration, Month, Week, Day, Hour, etc.) Burndown Work(Story,Point,Task)orEffort(Week,Day,Hour) Time Unit (Roadmap, Release, Iteration, Month, Week, Day, Hour, etc.) Cumulative Flow Work(Story,Point,Task)orEffort(Week,Day,Hour) Time Unit (Roadmap, Release, Iteration, Month, Week, Day, Hour, etc.) Earned Value Management - EVM CPI SPI PPC APC Work(Story,Point,Task)orEffort(Week,Day,Hour) Time Unit (Roadmap, Release, Iteration, Month, Week, Day, Hour, etc.) Earned Business Value - EBV
  • 22.
    What is AgileTesting?  Traditional testing is a late, manual process  Agile testing is an early and automated process  Goal to deliver early & often and V&V components 22 Rico, D. F. (2012). Agile testing resources. Retrieved Sep. 9, 2012, from http://davidfrico.com/agile-testing-resources.txt Crispin, L., & Gregory, J. (2009). Agile testing: A practical guide for testers and agile teams. Boston, MA: Addison-Wesley. Grant, T. (2005). Continuous integration using cruise control. Northern Virginia Java Users Group (Novajug), Reston, Virginia, USA.  AGILE TESTING - Early Incremental Testing - TRADITIONAL TESTING - Late Big Bang Integration Testing - Test Criteria Accompany Stories Automated Tests Written First Units Coded-Tested One at Time Code is Frequently Checked In Code Automatically Retrieved Code Automatically Compiled Tests Automatically Executed Instant Feedback & Test Reports Test Criteria Written After Fact Manual Tests Written Much Later Units Coded Late All at One Time Code Checked In Late in Project Code Manually Submitted to Test Code Manually Compiled & Built Tests Manually Executed Late Late Project Feedback & Reports  Code Automatically DeployedLate Defects Freeze Projects
  • 23.
    BASIC—Test Driven Development Term coined by Kent Beck in 2003  Consists of writing all tests before design  Ensures all components are verified and validated 23Beck, K. (2003). Test-driven development: By example. Boston, MA: Addison-Wesley.
  • 24.
    ADVANCED—Continuous Integration  Termcoined by Martin Fowler in 1998  Process of automated build/regression testing  Evaluates impact of changes against entire system 24Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration: Improving software quality and reducing risk. Boston, MA: Addison-Wesley.  ALL DEVELOPERS RUN PRIVATE BUILDS  DEVELOPERS COMMIT CODE TO VERSION CONTROL  INTEGRATION BUILDS OCCUR SEVERAL TIMES PER DAY  100% OF SYSTEM TESTS MUST PASS FOR EVERY BUILD  A SHIPPABLE PRODUCT RESULTS FROM EVERY BUILD  FIXING BROKEN BUILDS IS OF THE HIGHEST PRIORITY  REPORTS AUTOMATICALLY GENERATED & REVIEWED
  • 25.
     Agile testingconsists of seven broad practices  Automated build, database, inspection, tests, etc.  Include reporting, documentation, deployment, etc. 25 Practice Building Database Inspections Testing Feedback Documentation Deployment Description Frequently assembling products and services to ensure delivery readiness Frequently generating/analyzing database schemas, queries, and forms Frequently performing automated static analysis of product/service quality Frequently performing automated dynamic product and service evaluation Frequently generating automated status reports/messages for all stakeholders Frequently performing automated technical/customer document generation Frequently performing automated delivery of products/services to end users Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration: Improving software quality and reducing risk. Boston, MA: Addison-Wesley. Humble, J., & Farley, D. (2011). Continuous delivery. Boston, MA: Pearson Education.  PRACTICES—Continuous Integration
  • 26.
     Created byJez Humble of ThoughtWorks in 2011  Includes CM, build, testing, integration, release, etc.  Goal is one-touch automation of deployment pipeline 26 Humble, J., & Farley, D. (2011). Continuous delivery. Boston, MA: Pearson Education. Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration. Boston, MA: Addison-Wesley. Ohara, D. (2012). Continuous delivery and the world of devops. San Francisco, CA: GigaOM Pro.   CoQ • 80% MS Tst • 8/10 No Val • $24B in 90s • Rep by CD • Not Add MLK ENTERPRISE—Continuous Delivery
  • 27.
     Created byPatrick Debois of Jedi BVBA in 2007  Collaboration of developers & infrastructure people  Goal to automate the deployment to end-user devices 27 Bass, L., Weber, I., & Zhu, L. (2015). Devops: A software architect's perspective. Old Tappan, NJ: Pearson Education. Gruver, G., & Mouser, T. (2015). Leading the transformation: Applying agile and devops at scale. Portland, OR: IT Revolution Press. Humble, J., Molesky, J., & O'Reilly, B. (2015). Lean enterprise: How high performance organizations innovate at scale. Sebastopol, CA: O'Reilly Media.  GLOBAL—Development Operations 
  • 28.
     Agile methodsare based on traditional measures  Story points, velocity, and burndown basic metrics  Experts use Agile EVM, test, ROI & portfolio metrics 28Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods. Ft. Lauderdale, FL: J. Ross Publishing. AGILE METRICS 1. Agile CODE Metrics 2. Agile PROJECT Metrics 3. Agile TRACKING Metrics 4. Agile TESTING Metrics 5. Agile VALUE Metrics 6. Agile HEALTH Metrics 7. Agile PORTFOLIO Metrics 1. Agile CODE Metrics  Code Size  Code Complexity  Object Oriented  Code Coverage  Code Defects  Relational Design 2. Agile PROJECT Metrics  Software Size  Software Productivity  Software Effort  Software Quality  Software Schedule  Software Success 3. Agile TRACKING Metrics  Story Points  Sprint Burndown  Release Burndown  Velocity  Feature Progress  Agile Earned Value 4. Agile TESTING Metrics  Test Coverage  Test Automation  Integration Builds  Running Tested Features  DevOps Automation  Deployment Frequency 7. Agile PORTFOLIO Metrics  Portfolio Kanban  Epic Progress  Portfolio Radar  Release Train Radar  Lean Portfolio Metrics  Enterprise Scorecard 6. Agile HEALTH Metrics  Teamwork Quality  Collaboration Quality  Agile Process Maturity  Agile Adoption Rate  Degree of Agility  Product Flexibility 5. Agile VALUE Metrics  Total Lifecycle Costs  Total Lifecycle Benefits  Benefit to Cost Ratio  Return on Investment  Net Present Value  Real Options Analysis Agile Testing Metrics—Taxonomy
  • 29.
    29 METRIC DESCRIPTION TEST COVERAGEPercent or degree to which software source code is tested TEST AUTOMATION Ratio or degree to which software tests are automated INTEGRATION BUILDS Frequency of automated software builds and integrations RUNNING TESTED FEATURES Number of completed and tested features or user stories DEVOPS AUTOMATION Ratio or degree to which deployments are automated DEPLOYMENT FREQUENCY Frequency of automated software deployments or deliveries  Software test automation emerged during the 1970s  Reached their height in personal computer (PC) era  Most are FOSS and used by successful agile teams Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration: Improving software quality and reducing risk. Boston, MA: Addison-Wesley. Agile Testing Metrics—Definitions
  • 30.
    Agile Testing Metrics—Example 30Duvall,P., Matyas, S., & Glover, A. (2006). Continuous integration: Improving software quality and reducing risk. Boston, MA: Addison-Wesley.
  • 31.
    31 Traditional vs. AgileCumulative Flow Work(Story,Point,Task)orEffort(Week,Day,Hour) Time Unit (Roadmap, Release, Iteration, Month, Week, Day, Hour, etc.) Work(Story,Point,Task)orEffort(Week,Day,Hour) Time Unit (Roadmap, Release, Iteration, Month, Week, Day, Hour, etc.) Traditional Cumulative Flow Agile Cumulative Flow  Late big bang integration increases WIP backlog  Agile testing early and often reduces WIP backlog  Improves workflow and reduces WIP & lead times Anderson, D. J. (2004). Agile management for software engineering. Upper Saddle River, NJ: Pearson Education. Anderson, D. J. (2010). Kanban: Successful evolutionary change for your technology business. Sequim, WA: Blue Hole Press.  Agile Testing—Workflow
  • 32.
     Fewer integrationsleave in higher bug counts  Frequent, early integrations eliminate most defects  Goal is to have as many early integrations as possible 32 Lacoste, F. J. (2009). Killing the gatekeeper: Introducing a continuous integration system. Proceedings of the Agile 2009 Conference, Chicago, Illinois, USA, 387-392.   Number of Integrations Less Defects •More Integrations •Early IntegrationsMore Defects •Few Integrations •Late Integrations  Agile Testing—Economic Drivers
  • 33.
     Traditional testingfinds a defect in about 10 hours  Manual code inspections find a defect in 1 hour  Agile testing finds a defect every 6 minutes 33 Rico, D. F. (2012). The Cost of Quality (CoQ) for Agile vs. Traditional Project Management. Fairfax, VA: Gantthead.Com.  Agile Testing—Economics
  • 34.
     Agile testingis 10x better than code inspections  Agile testing is 100x better than traditional testing  Agile testing is done earlier “and” 1,000x more often 34 Rico, D. F. (2012). The Cost of Quality (CoQ) for Agile vs. Traditional Project Management. Fairfax, VA: Gantthead.Com.  Agile Testing—Cost of Quality
  • 35.
    Agile Cost &Benefit Analysis  Costs based on avg. productivity and quality  Productivity ranged from 4.7 to 5.9 LOC an hour  Costs were $588,202 and benefits were $3,930,631 35 Rico, D. F., Sayani, H. H., & Sone, S. (2009). The business value of agile software methods: Maximizing ROI with just-in-time processes and documentation. Ft. Lauderdale, FL: J. Ross Publishing. d1 = [ln(Benefits  Costs) + (Rate + 0.5  Risk2)  Years]  Risk   Years, d2 = d1  Risk   Years   5 1i 
  • 36.
    Benefits of AgileMethods  Analysis of 23 agile vs. 7,500 traditional projects  Agile projects are 54% better than traditional ones  Agile has lower costs (61%) and fewer defects (93%) Mah, M. (2008). Measuring agile in the enterprise: Proceedings of the Agile 2008 Conference, Toronto, Canada. Project Cost in Millions $ 0.75 1.50 2.25 3.00 2.8 1.1 Before Agile After Agile 61% Lower Cost Total Staffing 18 11 Before Agile After Agile 39% Less Staff 5 10 15 20 Delivery Time in Months 5 10 15 20 18 13.5 Before Agile After Agile 24% Faster Cumulative Defects 625 1250 1875 2500 2270 381 Before Agile After Agile 93% Less Defects 36     
  • 37.
    Agile vs. TraditionalSuccess  Traditional projects succeed at 50% industry avg.  Traditional projects are challenged 20% more often  Agile projects succeed 3x more and fail 3x less often Standish Group. (2012). Chaos manifesto. Boston, MA: Author. 37 Agile Traditional Success 42% Failed 9% Challenged 49% Success 14% Failed 29% Challenged 57% 
  • 38.
    Grant, T. (2005).Continuous integration using cruise control. Northern Virginia Java Users Group (Novajug), Reston, Virginia, USA. Fredrick, J. (2008). Accelerate software delivery with continuous integration and testing. Japanese Symposium on Software Testing, Tokyo, Japan.  Most agile testing tools are “free” open source  Build server costs no more than a commodity PC  10x more efficient/effective than traditional testing 38  Agile Testing—CI Statistics
  • 39.
    39  Hewlett-Packard isa major user of CI, CD, & DevOps  400 engineers developed 10 million LOC in 4 years  Major gains in testing, deployment, & innovation Gruver, G., Young, M. & Fulghum, P. (2013). A practical approach to large-scale agile development. Upper Saddle River, NJ: Pearson Education.  TYPE METRIC MANUAL DEVOPS MAJOR GAINS CYCLE TIME IMPROVEMENTS Build Time 40 Hours 3 Hours 13 x No. Builds 1-2 per Day 10-15 per Day 8 x Feedback 1 per Day 100 per Day 100 x Regression Testing 240 Hours 24 Hours 10 x DEVELOPMENT COST EFFORT DISTRIBUTION Integration 10% 2% 5 x Planning 20% 5% 4 x Porting 25% 15% 2 x Support 25% 5% 5 x Testing 15% 5% 3 x Innovation 5% 40% 8 x Agile Testing—CD Statistics
  • 40.
     Assembla wentfrom 2 to 45 releases every month  15K Google developers run 120 million tests per day  30K+ Amazon developers deliver 8,600 releases a day 40Singleton, A. (2014). Unblock: A guide to the new continuous agile. Needham, MA: Assembla, Inc.  62x Faster U.S. DoD IT Project 3,645x Faster U.S. DoD IT Project  Agile Testing—DevOps Statistics
  • 41.
     Google earlyadopter of agile methods and Scrum  Google also uses agile testing at enterprise scale  15,000 developers run 120 million tests per day 41 Micco, J. (2013). Continuous integration at google scale. Eclipse Con, Boston, MA. Whittaker, J., Arbon, J., & Carollo, J. (2012). How google tests software. Upper Saddle River, NJ: Pearson Education.  440 billion unique users run 37 trillion searches each year  Single monolithic code tree with mixed language code  Submissions at head – One branch – All from source  20+ code changes/minute – 50% code change/month  5,500+ submissions/day – 120 million tests per day  80,000 builds per day – 20 million builds per year  Auto code inspections – For low defect density  10X programming productivity improvement  $150 million in annual labor savings (ROI as a result)    Agile Testing—Google Statistics
  • 42.
     Amazon adoptedagile in 1999 and Scrum in 2004  Using enterprise-scale continuous delivery by 2010  30,000+ developers deploy over 8,600 releases a day 42 Atlas, A. (2009). Accidental adoption: The story of scrum at amazon.com. Proceedings of the Agile 2009 Conference, Chicago, Illinois, USA, 135-140. Jenkins, J. (2011). Velocity culture at amazon.com. Proceedings of the Velocity 2011 Conference, Santa Clara, California, USA. Elisha, S. (2013). Continuous deployment with amazon web services. Proceedings of the AWS Summit 2013, Sydney, New South Wales, Australia.  Software deployment every 11.6 seconds (as of 2011)  24,828 to 86,320 releases per Iteration  161,379 to 561,080 releases per Quarter  645,517 to 2,244,320 releases per Year  Automatic, split-second roll-forward & backward  75-90% reduction in release-caused outages (0.001%)  Millions of times faster (than traditional methods)  4,357,241 to 15,149,160 per traditional release  Thousands of times faster (than manual agility)  161,379 to 561,080 per Scrum/SAFe release  Used agile methods long before U.S. government (1999)      Agile Testing—Amazon Statistics
  • 43.
     Enables enterprisesto be flexible but disciplined  Allows enterprises to distribute project work teams  Ensures distributed project teams are collaborating 43 Agile Tools “Across the Life Cycle” Project Management Requirements DOORS Requisite Pro SLATE Design Rhapsody Telelogic System Architect Rational System Architect Coding Eclipse Visual Studio Sun Studio Testing JUnit NUnit Xunit CPPUnit Gtest Fit Fitnesse Selenium Quality Assurance CheckStyle PMD EMMA Jdepend Cobertura Gcov Configuration Mgt Subversion (SVN) Concurrent Versions Sys. ClearCase Build Automation Ant NAnt Maven Make Continuous Integ. Cruise Control Hudson BuildBot Collaboration WebEx Skype MeetMe Wimba Wiki MediaWiki TracWiki PhpWiki Documentation NDoc Javadoc Doxygen iText Version One Rally Scrum Works VSTS Agile Team Agile Enterprise Scope Manager Story Studio XP Plan It Iterate XP Tracker Agilo XP CGI XP Web Xplanner Ice Scrum Project Cards Target Process Xtreme Planner Team System Community Enterprise Mingle Hansoft 
  • 44.
     There areliterally hundreds of agile testing tools  There are tools for building, testing, and deployment  Integration tools monitor repositories and initiate tests 44 Agile Tools “In-Depth Test Automation” Smart, J. (2009). Automated deployment with maven and friends: Going the whole nine yards. Proceedings of the Agile 2009 Conference, Chicago, Illinois, USA. 
  • 45.
     Simple exampleof a DevOps reference architecture  Includes CM, continuous integration, & deployment  Code automatically built/tested/deployed to users 45 Agile Tools “Simple DevOps Automation” Morris, B., & Cassatt, C. (2015). Devops for the rest of us. Proceedings of the Agile DC Conference, Washington, DC, USA. Weeks, D. E. (2014). Devops and continuous delivery reference architectures (volume 1 & 2). Fulton, MD: Sonatype. 
  • 46.
    46 Agile Tools “Periodic Tableof DevOps Automation” XeniaLabs. (2016). Periodic table of devops tools. Retrieved April 11, 2016, from https://xebialabs.com/periodic-table-of-devops-tools.
  • 47.
    47 Holler, R. (2015).Ninth annual state of agile survey: State of agile development. Atlanta, GA: VersionOne.  VersionOne found 94% using agile methods today  Most are using Scrum with several key XP practices  Lean-Kanban is a rising practice with a 31% adoption   Continuous Integration ● ● ● ● ● ● ● ● ● ● ●  ●  ● Agile Testing—Adoption Statistics
  • 48.
     Agile testuse is low in spite of its age, i.e., 15 years  Many do not understand its utter simplicity and power  Failure to use agile testing undermines project success 48 Kim, D. (2013). The state of scrum: Benchmarks and guidelines. Indianapolis, IN: Scrum Alliance. Agile Practices Retrospectives Refactoring Done Definition Test Tools Test Driven Dev. CM Tools Simplicity Pair Programming Technical Debt Agile Testing 13% Continuous Integrations Weekly Daily 2-3 Times Per Day Never 2-3 Times Per Iteration    Agile Testing—Usage Statistics
  • 49.
     Agile teamsdon’t often use TDD, CI, CD & DevOps  Implement independent test teams after Sprints done  Sprint Waterfalling, Scrummerfalling, & Wagile result 49 Heusser, M. (2015). 12 years of agile testing: What do we know now. Proceedings of the Agile Gathering, Grand Rapids, Michigan, USA.     Incorrect • Phased Testing • Separate Teams • Delayed Testing Correct • Integrated Testing • Integrated Teams • Continuous Testing Agile Testing—Anti-Patterns
  • 50.
     Agile testingslows down with very large systems  Slow testing slows integration and increases bugs  Agile testing can speed back up with more attention 50 Kokko, H. (2009). Increase productivity with large scale continuous integration. Proceedings of the Agile 2009 Conference, Chicago, Illinois, USA.  MICRO ADJUSTMENTS - Focused Impact Tuning- MACRO ADJUSTMENTS - Wide Impact Tuning- Add More CPUs & Memory Parallelize System Builds Replace 3rd Party Test Libraries Reduce or Remove Test Timeouts Select Different Tests Refactor Code & Components Tune Network & Software Tune Database & Middleware In-Memory Compilation Parallelize Test Runs Pre-Install Test Libraries Remove Process Randomness Use Faster Code & Test Tools Incremental vs. Big Bang Tests Parallelize Build & Install Tune & Optimize Build Process  Agile Testing—Scaling Practices
  • 51.
     Industry veryslow in adopting agile testing model  Cost, difficulty, and territorialism are common issues  Developers must take initiative for disciplined testing 51 Technical BarriersOrganizational Barriers Developers don’t want to test · Infrequently committing code · Committing broken code · Failing to immediately fix builds · Not writing automated tests · Not ensuring 100% of tests pass · Not running private builds · Resorting to traditional testing Resistance to change · Fear of investment costs · Fear of learning new skills · Test group territorialism · Organizational policy conflicts · Overhead of maintaining CI · Complexity and scaling · Not developing a quality culture ·· Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration: Improving software quality and reducing risk. Boston, MA: Addison-Wesley.       Agile Testing—Common Barriers
  • 52.
     Eliminates big-bangintegration in the 11th hour  Creates a repeatable and reliable testing process  Evaluates system-wide changes throughout project 52Maeda, M. K. (2009). Agile testing: Early, often, and smart. Arlington, MA: Cutter Consortium. What’s the Bottom Line? “Agile Testing Done Early & Often” Agile TestingTraditional Testing Dramatically reduces risks · Automates manual processes · Instant verification & validation · High project visibility · Greater confidence and morale · Incremental business value · 24x7 deployability to users · Highly quality and reliability Late defect discovery · Low quality software · Poor project visibility · Lack of deployability · Late big-bang integration · Testing is a bottleneck · Poor customer satisfaction · Outright project failure ·· 
  • 53.
    Conclusion  Agile methodsDON’T mean deliver it now & fix it later  Lightweight, yet disciplined approach to development  Reduced cost, risk, & waste while improving quality 53 Rico, D. F. (2012). What’s really happening in agile methods: Its principles revisited? Retrieved June 6, 2012, from http://davidfrico.com/agile-principles.pdf Rico, D. F. (2012). The promises and pitfalls of agile methods. Retrieved February 6, 2013 from, http://davidfrico.com/agile-pros-cons.pdf Rico, D. F. (2012). How do lean & agile intersect? Retrieved February 6, 2013, from http://davidfrico.com/agile-concept-model-3.pdf What How Result Flexibility Use lightweight, yet disciplined processes and artifacts Low work-in-process Customer Involve customers early and often throughout development Early feedback Prioritize Identify highest-priority, value-adding business needs Focus resources Descope Descope complex programs by an order of magnitude Simplify problem Decompose Divide the remaining scope into smaller batches Manageable pieces Iterate Implement pieces one at a time over long periods of time Diffuse risk Leanness Architect and design the system one iteration at a time JIT waste-free design Swarm Implement each component in small cross-functional teams Knowledge transfer Collaborate Use frequent informal communications as often as possible Efficient data transfer Test Early Incrementally test each component as it is developed Early verification Test Often Perform system-level regression testing every few minutes Early validation Adapt Frequently identify optimal process and product solutions Improve performance                 
  • 54.
    Dave’s PROFESSIONAL CAPABILITIES 54 Software Quality Mgt. Technical Project Mgt. Software Development Methods Organization Change Systems Engineering Cost Estimating Government Contracting Government Acquisitions Lean Kanban BigData, Cloud, NoSQL Workflow Automation Metrics, Models, & SPC Six Sigma BPR, IDEF0, & DoDAF DoD 5000, TRA, & SRA PSP, TSP, & Code Reviews CMMI & ISO 9001 Innovation Management Statistics, CFA, EFA, & SEM Research Methods Evolutionary Design Valuation — Cost-Benefit Analysis, B/CR, ROI, NPV, BEP, Real Options, etc. Lean-Agile — Scrum, SAFe, Continuous Integration & Delivery, DevOps, etc. STRENGTHS – Data Mining  Gathering & Reporting Performance Data  Strategic Planning  Executive & Manage- ment Briefs  Brownbags & Webinars  White Papers  Tiger-Teams  Short-Fuse Tasking  Audits & Reviews  Etc. ● Data mining. Metrics, benchmarks, & performance. ● Simplification. Refactoring, refinement, & streamlining. ● Assessments. Audits, reviews, appraisals, & risk analysis. ● Coaching. Diagnosing, debugging, & restarting stalled projects. ● Business cases. Cost, benefit, & return-on-investment (ROI) analysis. ● Communications. Executive summaries, white papers, & lightning talks. ● Strategy & tactics. Program, project, task, & activity scoping, charters, & plans. PMP, CSEP, FCP, FCT ACP, CSM, & SAFE 32 YEARS IN IT INDUSTRY
  • 55.
    Books on AgileTesting  Thousands of textbooks on agile methods  Include requirements, design, coding, test, etc.  Continuous Integration, Delivery, & DevOps best 55  Beck, K. (2003). Test-driven development: By example. Boston, MA: Addison-Wesley. Crispin, L., & Gregory, J. (2009). Agile testing: A practical guide for testers and agile teams. Boston, MA: Addison-Wesley. Gregory, J., & Crispin, L. (2015). More agile testing: Learning journeys for the whole team. Upper Saddle River, NJ: Pearson Education. Duvall, P., Matyas, S., & Glover, A. (2006). Continuous integration: Improving software quality and reducing risk. Boston, MA: Addison-Wesley. Humble, J., & Farley, D. (2011). Continuous delivery: Reliable software releases through build, test, and deployment automation. Boston, MA: Pearson Education.
  • 56.
    Books on ROIof SW Methods  Guides to software methods for business leaders  Communicates the business value of IT approaches  Rosetta stones to unlocking ROI of software methods  http://davidfrico.com/agile-book.htm (Description)  http://davidfrico.com/roi-book.htm (Description) 56 