Your SlideShare is downloading. ×
An Automation Culture: The Key to Agile Success
An Automation Culture: The Key to Agile Success
An Automation Culture: The Key to Agile Success
An Automation Culture: The Key to Agile Success
An Automation Culture: The Key to Agile Success
An Automation Culture: The Key to Agile Success
An Automation Culture: The Key to Agile Success
An Automation Culture: The Key to Agile Success
An Automation Culture: The Key to Agile Success
An Automation Culture: The Key to Agile Success
An Automation Culture: The Key to Agile Success
An Automation Culture: The Key to Agile Success
An Automation Culture: The Key to Agile Success
An Automation Culture: The Key to Agile Success
An Automation Culture: The Key to Agile Success
An Automation Culture: The Key to Agile Success
An Automation Culture: The Key to Agile Success
An Automation Culture: The Key to Agile Success
An Automation Culture: The Key to Agile Success
An Automation Culture: The Key to Agile Success
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

An Automation Culture: The Key to Agile Success

176

Published on

For organizations developing large-scale applications, transitioning to agile is challenging enough. But if your organization has not yet adopted an automation culture, brace yourself for a big …

For organizations developing large-scale applications, transitioning to agile is challenging enough. But if your organization has not yet adopted an automation culture, brace yourself for a big surprise because automation is essential to agile success. From the safety nets provided by automated unit and acceptance tests to the automation of build, build verification, and deployment processes, the iterative nature of agile demands a culture of automation across your engineering organization. Geoff Meyer shares lessons learned in adopting a test automation culture as the Dell Enterprise Systems Group simultaneously adopted Scrum and agile processes across its entire software product portfolio. Learn to address the practical challenges of establishing an automation culture at the outset by ensuring that your organizational makeover incorporates changes to your hiring, staffing, and training practices. Find out how you can apply automation beyond the Scrum team in areas of continuous integration, scale and stress testing, and performance testing.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
176
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
23
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. AW4 Concurrent Session 11/13/2013 10:15 AM "An Automation Culture: The Key to Agile Success" Presented by: Geoff Meyer Dell, Inc. Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888 268 8770 904 278 0524 sqeinfo@sqe.com www.sqe.com
  • 2. Geoff Meyer Dell, Inc. A test architect in the Dell Enterprise Solutions Group, Geoff Meyer has more than twenty-seven years of experience as a software developer, manager, test architect, and business analyst. Geoff co-chairs the Agile Steering committee within Dell Enterprise Solutions Group which guides the software development practices of more than 600 development, test, and UX engineers across three Global Design Centers. He is an active member of the Agile Austin community.
  • 3. 9/3/13% Automation Culture: Essential to Agile Success Agile East 2013 Geoff Meyer, geoffrey_meyer@dell.com November 2013 Last updated: August, 28, 2013 Session Objectives •  Challenges to expect when adopting test automation during your Agile transition IS •  Practical solutions to what & when to automate •  Real-world examples − from a large organization perspective •  Challenges automating in a HW-dependent environment •  For organizational leaders driving strategy IS •  A Tools Discussion NOT •  Focused on how to develop Test Automation 1%
  • 4. 9/3/13% Introductions 3 Geoff Meyer •  Dell Inc, 1998 – present –  Test Architect ›  Agile Test & Automation Strategy –  Agile Steering Committee Co-Chair –  Global Projects ›  15 Scrum teams (2 regions) ›  9 scrum teams (2 regions) ›  7 Scrum teams (4 regions) •  NCR Corp. 1984 – 1998 –  SW developer, Project Lead, SW Manager •  B.S. Computer Science, San Diego State University •  Masters Engineering Management - NTU 4 2%
  • 5. 9/3/13% Agenda •  Why Automation is essential to Agile •  The Dell Landscape and Agile @ Dell –  Organizational Context Matters •  Challenges and Common Pitfalls •  The Automation Landscape •  Foundations of an Automation Culture •  Care and Feeding of the Automation Culture 5 The Need, The Challenge, The Pitfalls 6 Ø 3%
  • 6. 9/3/13% Why is Automation So Important in Agile? •  Near-term –  Ensures that you don’t break what you just built –  Provides safety net for developers & rapid feedback to new changes –  Continuous Integration and use of Build Verification Test (BVT) •  Long-term –  Maximizes velocity of Scrum team –  Creates capacity for Exploratory and ad-hoc Testing –  Enables activities that can’t be done cost-effectively by humans And if you don’t… Quality is at risk from an unmanageable regression suite Differences with Automation in Waterfall? In Waterfall… •  Automated tests are derived from the backlog of completed features –  In Agile, Automation can be incorporated in the requirement •  Testing and automation is performed after Development is “complete” •  Focus is on first-time discovery of defects and optimizing your test coverage vs. Agile… where automation development is ongoing and provides immediate feedback throughout development 8 4%
  • 7. 9/3/13% The Context @ Dell “Apply Test Automation in the context of your organizational automation needs” http://amazngwallpapers.blogspot.com --- Bob Galen, Agile Coach 9 Dell Enterprise Solutions Group Global Design and Development using Agile Scrum Silicon Valley Design Center Austin Design Center Noida Design Center Bangalore Design Center 10 5%
  • 8. 9/3/13% Dell Enterprise Solutions Software Products •  Server Systems Management •  Converged Infrastructure Systems Management •  Private Cloud Server Management •  Console Plug-ins (i.e. for SCCM, vCenter…) Common Product Characteristics: •  Large hardware support test matrix •  Software is installed in the Data Center •  Enterprise update cycles ~6 months •  Products must function even as underlying HW, FW, BIOS, and Drivers are upgraded - SUSTAINING 11 Agile @ Dell Roles/Responsibilities Product Owner •  Small teams Product Owner Proxy 1 per Scrum •  Co-located Scrum Master 1 per Scrum •  Distributed Projects Development 4-5 per Scrum •  Teams formed from functional silos Test 2:1 Tech Pubs 1 per 3 scrum team UI 1 per Scrum Scrum teams 12 6%
  • 9. 9/3/13% Agile @ Dell with Adaptations Feature Complete Release Plan Release Exit Code Freeze Sprints 1 1 2 2 1 2 3 3 … 3 N-1 N-1 N N N-1 … N Hardening Stability Extended Sprint Test Software System Test OLP Define Plan Develop Launch 13 Pitfalls encountered at •  Development didn’t historically automate unit tests •  Build teams were staffed with non-Build practitioners •  Minimal guidance to Test beyond “Go forth and automate” •  Architecture(s) not optimized for Automatability •  Automation was interpreted by many as “Automate the UI” •  Insufficient SW engineering background across Test teams 14 7%
  • 10. 9/3/13% Which Resulted In… •  Over-emphasis on UI automation •  Test Automation not keeping up within the sprint •  Feature Devotion1 •  Multiple automation tools & licenses •  Test scripts not designed for re-use 1 – “A nasty condition where people start valuing ticking off features more than tracking the real outcome of the project.”--- Martin Fowler 15 The Automation Landscape http://amazngwallpapers.blogspot.com 16 8%
  • 11. 9/3/13% Opportunities for Automation •  It’s not just about automating test cases •  Can also provide efficiencies to: – Test Preparation, Setup and Configuration – Non-Functional Testing (“ility”) ›  Longevity, Scale and Performance Characterization – Compatibility Testing (Solution and Device) 17 Test Preparation •  Bare-metal Deployment •  Setup & Configuration –  OS Provisioning and Configuration •  Test case staging •  Environment Cleanup/Baseline •  Virtualization-based test environments 18 9%
  • 12. 9/3/13% Test Content Development Whole team ownership of QA & Automation •  UI automation on Customer Usage workflows •  Automated CI (UT, Build, BVT) •  Web-services (or CLI) Test Automation for Functional Testing •  In-Sprint, Automated Acceptance Tests •  Unit Test Automation 2 CAUTION: Application Architecture can be an enabler or inhibitor 2 - Agile Testing: A Practical Guide for Testers and Agile teams 19 Application Architecture Matters •  Does it facilitate testing at the API/Services level? •  Does the Business/Error Logic reside below the services level? •  Does the UI architecture support test automation other than record-playback? For Large organizations: –  Standardize UI architectures across the product portfolio ›  HTML5, Flex, Silverlight… –  Standardize Service architectures across the product portfolio ›  SOAP, REST, CLI, API 20 10%
  • 13. 9/3/13% Prioritize and Identify what NOT to Automate Prioritize High ROI 1.  BVT Candidate Acceptance Test –  Test of core functionality that executes in a short duration 2.  Core Functionality to be run in Nightly Regression 3.  Sustaining Test Candidate Test –  Functional test that verifies the application can withstand subsystem changes What NOT to Automate •  Everything •  GUI – Except for High-value customer usage flows after the UI is stable •  Tests that would only find low severity bugs –  Where the product is unchanging 21 Non-Functional Testing •  Performance Characterization •  Longevity 3 •  Stress •  Scale •  Concurrency Often times analysis of these areas are simply too cost-prohibitive to be done manually 3- http://lisacrispin.com/wordpress/2011/11/08/using-the-agile-testing-quadrants/ 22 11%
  • 14. 9/3/13% Foundations of an Automation Culture Images.yahoo.com 23 Keys To A Culture Transition •  Development vs. Test •  A different “School” of Test4 •  Evolve from Functional Responsibility …to Whole Product Ownership 4 – Scott Barber “Approaches to Software Testing: An Introduction “ 24 12%
  • 15. 9/3/13% Establishing an Automation Culture •  Identify •  Near and long-term automation focus areas •  Inventory the culture and skillset of organization •  Establish –  Tooling and Infrastructure Standardization –  Your ‘community’ •  Develop –  Workforce transition plan •  Organize –  Embedded vs. Specialized Automation team –  Re-align project staffing •  Operationalize 25 Identify Focus Areas for Automation 4 4 - http://lisacrispin.com/wordpress/2011/11/08/using-the-agile-testing-quadrants/ 26 13%
  • 16. 9/3/13% Establish •  Tooling and Infrastructure – Tooling Standards and Frameworks – Team to develop/manage home-grown tools – Lead tool evaluations to meet new architectures & technologies Multiple automation Tools/FW’s may be needed to automate at layers of testing: •  •  •  Unit level Middle tier UI-driven •  Community: Internal and External 27 Develop & Organize •  Adjust Hiring Practices •  Re-align Project staffing •  Embedded Automation vs. Specialized Team •  Training – New-hire – Ongoing 28 Confidential 14%
  • 17. 9/3/13% Operationalize •  Whole team commitment in words and actions •  Include Test Automation in Acceptance Criteria – Unit – Acceptance •  Continuous Integration and BVT •  Establish Metrics, Governance & Rewards 29 Automation @ PG Enterprise Solutions Group How did we go about it? Process/Project Non-Functional Scale, Longevity Automated BVT UI Automation Service-level automation (CLI & Web-services) Automation Day Setup & Config 2012 2013 Automation Content Libraries Automation Architects Culture/Organization IVT Device IVT Networking & Servers 2011 2010 Established ALT In-Sprint Automation Internal Training Programs “Automate-First” Accelerate Automation Skillset 2014 Maximize Utilization TDD & Automated UT 30 15%
  • 18. 9/3/13% Care and Feeding of the Automation Culture 31 Maintaining the Automation Culture •  Operationalize Automation –  Include Test Automation in the Acceptance Criteria •  Encourage Community Practice and Participation –  External - Industry User Groups & Conferences –  Internal - Brown-bag sessions, Showcases, Mini-conference •  For Large organizations: –  Overcome skillset deficiencies by adjusting staffing strategy –  Automation Leadership Team, Automation Architect(s) –  Continually monitor the alignment of Architecture, Dev and Test –  Develop reward systems 32 16%
  • 19. 9/3/13% Recognizing a Successful Automation Culture •  Automation is a shared responsibility •  Automatability is a key architectural consideration •  Continuous Integration & BVT •  Teams are staffed to include an automation skillset •  Automation is operationalized Resources •  Agile Manifesto - http://agilemanifesto.org/ •  Articles: –  http://support.smartbear.com/articles/testcomplete/automated-testing-agile-environment/ SmartBear –  http://lisacrispin.com/wordpress/2011/11/08/using-the-agile-testing-quadrants/ - Lisa Crispin –  http://www.stickyminds.com/sitewide.asp? Function=edetail&ObjectType=COL&ObjectId=17793&tth=DYN&tt=siteemail&iDyn=2 – Rajini Padmanaban –  Beyond Agile Testing: http://www.utest.com/int-v1/a/beyond-agile-testing - uTest •  Books: –  Scaling Software Agility: Best Practices for Large Enterprises – Dean Leffingwell –  How Google Tests Software – James Whittaker, Jason Arbon, Jeff Carollo –  Agile Testing: A Practical Guide for Testers and Agile teams – Lisa Crispin, Janet Gregory –  http://www.dorothygraham.co.uk/automationExperiences/index.html - Dorothy Graham •  Presentations: –  Approaches to Software Testing: An Introduction – Scott Barber –  Agile Testing: Challenges Beyond the Easy Contexts – Bob Galen 34 17%
  • 20. 9/3/13% Questions? 35 18%

×