SlideShare a Scribd company logo
1 of 34
Download to read offline
TH
AM Tutorial
10/14/2014 8:30:00 AM
"Planning, Architecting, Implementing,
and Measuring Automation"
Presented by:
Michael Sowers
Software Quality Engineering
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
Mike Sowers
Software Quality Engineering
Mike Sowers has more than twenty-five years of practical experience as a global quality and
test leader of internationally distributed test teams across multiple industries. Mike is a senior
consultant, skilled in working with both large and small organizations to improve their software
development, testing, and delivery approaches. He has worked with companies including
Fidelity Investments, PepsiCo, FedEx, Southwest Airlines, Wells Fargo, ADP, and Lockheed to
improve software quality, reduce time to market, and decrease costs. With his passion for
helping teams deliver software faster, better, and cheaper, Mike has mentored and coached
senior software leaders, small teams, and direct contributors worldwide.
Me and My Contemporaries 1
Michael D. Sowers, CIO
Software Quality Engineering
Planning, Architecting,
Implementing, and Measuring
Automation
Copyright M.D. Sowers
2
Administrivia
Messages Meals
Tutorial timing Tutorial materials
Breaks
Pagers and
Cell phones Facilities
Smoking
Me and My Contemporaries 2
Copyright M.D. Sowers
A Heap of Tools
Copyright M.D. Sowers
Tutorial Goals
At the end of this tutorial you will be able to:
• Define your integrated automation team
• Document your current state
• Layout a future state and roadmap
• Create your pilot and implementation plan
• Identify initial key measures
Me and My Contemporaries 3
Copyright M.D. Sowers
Outline
Introduction
The Compelling Business Need(s)
Automation Challenges
Automation Requirements
Life Cycle Automation
Overarching Approach
Define the Team, Assess Current State
Develop the Architecture
Plan, Pilot & Implement
Measure and Report Benefits
Lessons Learned
Copyright M.D. Sowers
Outline
Introduction
The Compelling Business Need(s)
Automation Challenges
Automation Requirements
Life Cycle Automation
Overarching Approach
Define the Team, Assess Current State
Develop the Architecture
Plan, Pilot & Implement
Measure and Report Benefits
Lessons Learned
Me and My Contemporaries 4
Copyright M.D. Sowers
Compelling Business Need(s)
• Faster…..........
– Development
– Testing
– Deployment
– And more…
• Better…………………
– Features
– Performance
– Usability
– And more…
• Cheaper……………………
– Elaboration
– Implementation
– Delivery
– And more…
Copyright M.D. Sowers
Compelling Business Need(s)
• Faster………………………………… Speed to Market
– Development
– Testing
– Deployment
– And more…
• Better……………………………….. Quality Product(s)
– Features
– Performance
– Usability
– And more…
• Cheaper……………………………. Competitive Innovation
– Elaboration
– Implementation
– Delivery
– And more…
Me and My Contemporaries 5
Copyright M.D. Sowers
Automation Challenges
• An abundant array of disconnected tools…
with assembly usually required…
focused on execution vs. design…
demanding a high-level of technical expertise..
and significant maintenance…
• Culminating in an investment not always
understood and an array of tools not used…
Copyright M.D. Sowers
Automation Key Requirements
• A strategy, a vision and a plan
• An architecture and integration (across the
lifecycle and between technology/vendors)
• Early lifecycle centric
• Roles based with associated training
• Aligned and integrated with processes
• SUT Focused
• With business oriented measures and reporting
Me and My Contemporaries 6
Copyright M.D. Sowers
Life Cycle Automation
Copyright M.D. Sowers
alm Generalized (video)
Me and My Contemporaries 7
Copyright M.D. Sowers
Outline
Introduction
The Compelling Business Need(s)
Automation Challenges
Automation Requirements
Life Cycle Automation
Overarching Approach
Define the Team, Assess Current State
Develop the Architecture
Plan, Pilot & Implement
Measure and Report Benefits
Lessons Learned
Copyright M.D. Sowers
Define the Team
• Cross Functional with representation from:
– Product Office
– Technical Architecture (and standards)
– Software Development (and maintenance team)
– Quality Assurance and Testing
– IT
– Business Units
• Global representation
• Key Opinion Leaders
Me and My Contemporaries 8
Copyright M.D. Sowers
Define the Team – Exercise
• Draft answers to the questions in the Appendix
Copyright M.D. Sowers
Assess the Current State
• Inventory current tools & processes
– Tool Category
– Purpose (single function/multi-function)
– Value
– At what lifecycle stage does the tool apply
– What process(es) does the tool support?
– Existing Linkage to other tools
– License and Maintenance expenses
– How many users (departments, BU’s, teams)
– Current issues/deficiencies
– Platforms Supported
– Others?
Me and My Contemporaries 9
Copyright M.D. Sowers
Assess – Current State
Copyright M.D. Sowers
Assess – Current State
Me and My Contemporaries 10
Copyright M.D. Sowers
Assess the Current State – Exercise
• Fill in a draft of the spreadsheet in the Appendix
Copyright M.D. Sowers
Plan - Vision, Strategy, Approach
• Vision: End in mind/automation dream state?
– 1-3 year time horizon
– Aligned with critical business needs
– A quantified “end in mind”
• Strategy: The “what” of achieving your automation vision
– Engage the key opinion leaders
– Address existing pain points
– Link to business objectives
• Approach: The “how” of achieving your automation strategy
– A blueprint (of current and future state)
– Aligned with the Technical Architecture
– An agile plan – responsive & flexible to change
Me and My Contemporaries 11
Copyright M.D. Sowers
Establish an End in Mind (Vision-1)
Copyright M.D. Sowers
Establish an End in Mind (Vision-2)
Me and My Contemporaries 12
Copyright M.D. Sowers
Establish an End in Mind (Vision-3)
Copyright M.D. Sowers
Architectural Considerations
Operations
Access &
Presentation
Business
Security
Infrastructure
Application
Data
Integration
Me and My Contemporaries 13
Copyright M.D. Sowers
Process Considerations
Copyright M.D. Sowers
Process Considerations
Me and My Contemporaries 14
Copyright M.D. Sowers
Establish a Vision – Exercise
• Draft (draw) a vision, an end state of your tools
architecture (big boxes are fine to start)
Copyright M.D. Sowers
Establish a Plan
• Partial Blueprint
• Integrated Tools Architecture
• Tools Roadmap
• Tools Mapped to the lifecycle
• Roles Mapped to the lifecycle
Me and My Contemporaries 15
Copyright M.D. Sowers
It’s like building an Airplane in the Air!
Copyright M.D. Sowers
Partial Blueprint
Technology Area 2008 2009 2010 2011
Requirements Management Borland -- CaliberRM 2006--
(U)
IBM -- Rational
RequisitePro 8.X-- (P)
IBM -- Rational
RequisitePro 8.X-- (P)
IBM -- Rational
RequisitePro 8.X-- (P)
HP/Mercury -- Test Director
(Quality Center) for
Requirements Management
9.x-- (E)
HP/Mercury -- Test
Director (Quality
Center) for
Requirements
Management 9.x-- (E)
HP/Mercury -- Test
Director (Quality Center)
for Requirements
Management 9.x-- (P)
HP/Mercury -- Test
Director (Quality Center)
for Requirements
Management 10.x-- (P)
Developer Test CxxTest (R) Parasoft – C++ Test
(P)
Parasoft – C++ Test (P) Parasoft – C++ Test (P)
Test Management IBM -- Rational Test Manager--
(R)
IBM -- Rational Test
Manager-- (R)
HP/Mercury -- TestDirector:
Quality Center 10.X-- (P)
HP/Mercury --
TestDirector: Quality
Center 10.X-- (P)
HP/Mercury --
TestDirector: Quality
Center 10.X-- (P)
HP/Mercury --
TestDirector: Quality
Center 10.X-- (P)
Functional Automation IBM -- Rational Functional
Tester-- (R)
IBM -- Rational
Functional Tester-- (R)
IBM -- Rational
Functional Tester-- (R)
IBM -- Rational Robot 7.x-- (R) IBM -- Rational Robot
7.x-- (R)
Borland -- Silk Test-- (R) Borland -- Silk Test--
(R)
HP/Mercury -- WinRunner 9.x-
- (R)
HP/Mercury --
WinRunner 9.x– (R)
HP/Mercury – QTP (P) HP/Mercury – QTP (P)
Me and My Contemporaries 16
Copyright M.D. Sowers
Integrated Tools Architecture
Copyright M.D. Sowers
Tools Roadmap (1)
Me and My Contemporaries 17
Copyright M.D. Sowers
Tools Roadmap (2)
Copyright M.D. Sowers
Tools Mapped to Lifecycle
Me and My Contemporaries 18
Copyright M.D. Sowers
Methods, Tools and Measures Engineers
Functional Test Engineers
Test Automators
Test Coordinators
Test Architects
Performance Test Engineers
Test Designers
Roles Based & Life Cycle Integrated
Production Excellence Manager
SIT PRODUATDITTSDSDSSRAPPI
AIP Reviews Function Test Performance Test Operational
Readiness Test
Unit Test
VerificationVerification
ValidationValidation
Production
Excellence
Production
Excellence
Systems Test Engineers
Copyright M.D. Sowers
Establish a Plan – Exercise
• Using the examples presented, begin to think
about (draft) the planning questions in the
appendix
Me and My Contemporaries 19
Copyright M.D. Sowers
Select, Pilot & Implement
Tool selection/purchase
Review of tools
Evaluate/pilot a select set of tools
Demonstrate tools
Market analysis
Tool requirements specification
Copyright M.D. Sowers
Select, Pilot & Implement – Exercise
• Begin to think about (draft) the select, pilot &
implement questions in the appendix
Me and My Contemporaries 20
Copyright M.D. Sowers
Measure & Report Benefits
• Tool Measures in Technical Terms
– Comprehensiveness of Tool Functionality
– Number of issues/defects in or with the tool
– Satisfaction/response time with Tool Support
– Ease of Use
– Ease of Integration (api)
– And more…
• Measures in Business Terms
–The MOST important!
Copyright M.D. Sowers
Business Oriented Measures
2008 2009 2010
Expansion
• Applications covered by QA&T Team 90 160 230
• Number of Applications Automated 22 46 78
Effectiveness
• Pre-Release defects 162 200 274
• Post-Release defects 28 22 18
• Requirements coverage (%) 53 71 87
Efficiency
• QA & Test staff to developer ratio 1/5 1/5.5 1/6.5
• Time for regression test runs (days) 8 5 4
Business Results
• QA & Test Total Cycle Time (days) 40 36 31
• Cost per Total Test Cycle (K dollars) 320 288 248
• Post release defect containment (%) 85.3 90.1 94
Me and My Contemporaries 21
Copyright M.D. Sowers
Measure/Metrics – Exercise
• Begin to think about (draft) the
measures/metrics questions in the appendix
Copyright M.D. Sowers
Outline
Introduction
The Compelling Business Need(s)
Automation Challenges
Automation Requirements
Life Cycle Automation
Overarching Approach
Define the Team, Assess Current State
Develop the Architecture
Plan, Pilot & Implement
Measure and Report Benefits
Lessons Learned
Me and My Contemporaries 22
Copyright M.D. Sowers
Key Learning's
• Do:
– Engage Sr. Technical Staff
– Secure Stakeholder Alignment
– Have a larger vision, but focus on one area at
a time
– Allow for flexibility and customization
– Monitor the “scope” of standardization
• Don’t:
– Permit too much separation between roles
– Ignore existing processes & dependencies
– Limit your thinking to the current technology
Copyright M.D. Sowers
(A Few) Keys to Success
Automation as a
Technology Issue
Automation as a Business
Imperative
Technical Measures Business Measures
No Vision, Strategy,
Architecture
Complete Automation
Blueprint
Buckets of Automation Integrated Workflow
Everyone does
everything
Roles Based
FROM TO
Me and My Contemporaries 23
Copyright M.D. Sowers
A Less Than Functional Home
Copyright M.D. Sowers
Remember - We build Airplanes in
the Air
• Doing so with isolated tools
• Without LEADERSHIP and a PLAN
• Results in an interesting – but not so useful
automation mansion
• Wasting the lives of many skilled crafts men
and women (our developers & testers)
• And no value contributed to the business
Me and My Contemporaries 24
Copyright M.D. Sowers
Tutorial Goals Achieved
You are now able to:
• Define your integrated automation team
• Document your current state
• Layout a future state and roadmap
• Create your pilot and implementation plan
• Identify initial key measures
Copyright M.D. Sowers
Wrap Up
• Current Challenges?
• ?
• ?
• ?
• ?
Me and My Contemporaries 25
Copyright M.D. Sowers
49
How did we do?
* Great *
Wonderful
* Fantastic *
Terrific
* Superior *
Excellent
Tutorial Evaluation
Tutorial Evaluations
Copyright M.D. Sowers
50
Thank You
• On behalf of Software Quality Engineering,
thank you for attending this tutorial.
• If you have further needs for training or
consulting, please think first of SQE.
• If I can be of further assistance, please let me
know. My email is msowers@sqe.com
Me and My Contemporaries 26
Copyright M.D. Sowers
Appendix
Appendix A
Copyright M. D. Sowers, Software Quality Engineering 1
Integrated Tool Architecture
Team Selection
1. Who are the key stakeholders for software lifecycle automation? Why are they key
stakeholders?
2. Of the key stakeholders, which are critical to include? Is there balanced participation globally?
3. What's the approach for engaging and securing a commitment to participate from the critical
stakeholders?
Appendix A
Copyright M. D. Sowers, Software Quality Engineering 2
Tool(s) Inventory (Example)
Appendix A
Copyright M. D. Sowers, Software Quality Engineering 3
Tool(s) Inventory (blank)
Appendix A
Copyright M. D. Sowers, Software Quality Engineering 4
Tool(s) Implementation Plan
1. What’s the first area of focus? (requirements, dev, test, deployment)
2. For the initial area of focus, map those tools to the life cycle steps/stages
3. Tools to retire, tools to add?
4. Where would tool connections be beneficial?
Appendix A
Copyright M. D. Sowers, Software Quality Engineering 5
Select, Pilot & Implement
1. For the specific area of focus or tool selected what are the requirements?
2. What will be the focus of your market research? (Commercial, Open Source, Custom
Development)
3. Who are the key stakeholders that need to be engaged in the demo's?
4. What (1 or 2) teams might be good candidates to pilot the tool?
5. What changes to existing methodology or standards may be needed to adopt this tool into our
existing environment?
Appendix A
Copyright M. D. Sowers, Software Quality Engineering 6
Measures/Metrics
1. What specific technical measures will be used to assess the success and contribution of this
tool?
2. What specific business measures will be used to assess the success and contribution of this tool?
3. What information do I need to plan to collect to track these measures?

More Related Content

What's hot

20140121 cisec-safety criticalsoftwaredevelopment
20140121 cisec-safety criticalsoftwaredevelopment20140121 cisec-safety criticalsoftwaredevelopment
20140121 cisec-safety criticalsoftwaredevelopmentCISEC
 
V&V Lessons Learnt under multiple Standards
V&V Lessons Learnt under multiple StandardsV&V Lessons Learnt under multiple Standards
V&V Lessons Learnt under multiple StandardsOak Systems
 
Agile & Secure SDLC
Agile & Secure SDLCAgile & Secure SDLC
Agile & Secure SDLCPaul Yang
 
How to Achieve Functional Safety in Safety-Citical Embedded Systems
How to Achieve Functional Safety in Safety-Citical Embedded SystemsHow to Achieve Functional Safety in Safety-Citical Embedded Systems
How to Achieve Functional Safety in Safety-Citical Embedded Systemsevatjohnson
 
Top Ten things that have been proven to effect software reliability
Top Ten things that have been proven to effect software reliabilityTop Ten things that have been proven to effect software reliability
Top Ten things that have been proven to effect software reliabilityAnn Marie Neufelder
 
Software FMEA and Software FTA – An Effective Tool for Embedded Software Qual...
Software FMEA and Software FTA – An Effective Tool for Embedded Software Qual...Software FMEA and Software FTA – An Effective Tool for Embedded Software Qual...
Software FMEA and Software FTA – An Effective Tool for Embedded Software Qual...Mahindra Satyam
 
Requirements Analysis and Management using Innoslate
Requirements Analysis and Management using InnoslateRequirements Analysis and Management using Innoslate
Requirements Analysis and Management using InnoslateElizabeth Steiner
 
Four things that are almost guaranteed to reduce the reliability of a softwa...
Four things that are almost guaranteed to reduce the reliability of a softwa...Four things that are almost guaranteed to reduce the reliability of a softwa...
Four things that are almost guaranteed to reduce the reliability of a softwa...Ann Marie Neufelder
 
The Top Ten things that have been proven to effect software reliability
The Top Ten things that have been proven to effect software reliabilityThe Top Ten things that have been proven to effect software reliability
The Top Ten things that have been proven to effect software reliabilityAnn Marie Neufelder
 
Offshore CAP technical Architect
Offshore CAP technical ArchitectOffshore CAP technical Architect
Offshore CAP technical ArchitectVimal Vijayan
 
Predict Software Reliability Before the Code is Written
Predict Software Reliability Before the Code is WrittenPredict Software Reliability Before the Code is Written
Predict Software Reliability Before the Code is WrittenAnn Marie Neufelder
 
Systems Modeling Language (SysML) and Model-Based Systems Engineering (MBSE) ...
Systems Modeling Language (SysML) and Model-Based Systems Engineering (MBSE) ...Systems Modeling Language (SysML) and Model-Based Systems Engineering (MBSE) ...
Systems Modeling Language (SysML) and Model-Based Systems Engineering (MBSE) ...Tonex
 

What's hot (16)

20140121 cisec-safety criticalsoftwaredevelopment
20140121 cisec-safety criticalsoftwaredevelopment20140121 cisec-safety criticalsoftwaredevelopment
20140121 cisec-safety criticalsoftwaredevelopment
 
V&V Lessons Learnt under multiple Standards
V&V Lessons Learnt under multiple StandardsV&V Lessons Learnt under multiple Standards
V&V Lessons Learnt under multiple Standards
 
SRE vs DevOps
SRE vs DevOpsSRE vs DevOps
SRE vs DevOps
 
Agile & Secure SDLC
Agile & Secure SDLCAgile & Secure SDLC
Agile & Secure SDLC
 
How to Achieve Functional Safety in Safety-Citical Embedded Systems
How to Achieve Functional Safety in Safety-Citical Embedded SystemsHow to Achieve Functional Safety in Safety-Citical Embedded Systems
How to Achieve Functional Safety in Safety-Citical Embedded Systems
 
Top Ten things that have been proven to effect software reliability
Top Ten things that have been proven to effect software reliabilityTop Ten things that have been proven to effect software reliability
Top Ten things that have been proven to effect software reliability
 
Software FMEA and Software FTA – An Effective Tool for Embedded Software Qual...
Software FMEA and Software FTA – An Effective Tool for Embedded Software Qual...Software FMEA and Software FTA – An Effective Tool for Embedded Software Qual...
Software FMEA and Software FTA – An Effective Tool for Embedded Software Qual...
 
Requirements Analysis and Management using Innoslate
Requirements Analysis and Management using InnoslateRequirements Analysis and Management using Innoslate
Requirements Analysis and Management using Innoslate
 
Four things that are almost guaranteed to reduce the reliability of a softwa...
Four things that are almost guaranteed to reduce the reliability of a softwa...Four things that are almost guaranteed to reduce the reliability of a softwa...
Four things that are almost guaranteed to reduce the reliability of a softwa...
 
The Top Ten things that have been proven to effect software reliability
The Top Ten things that have been proven to effect software reliabilityThe Top Ten things that have been proven to effect software reliability
The Top Ten things that have been proven to effect software reliability
 
Offshore CAP technical Architect
Offshore CAP technical ArchitectOffshore CAP technical Architect
Offshore CAP technical Architect
 
Predict Software Reliability Before the Code is Written
Predict Software Reliability Before the Code is WrittenPredict Software Reliability Before the Code is Written
Predict Software Reliability Before the Code is Written
 
RAMS for new projects
RAMS for new projectsRAMS for new projects
RAMS for new projects
 
ESS Software and Firmware
ESS Software and FirmwareESS Software and Firmware
ESS Software and Firmware
 
Systems Modeling Language (SysML) and Model-Based Systems Engineering (MBSE) ...
Systems Modeling Language (SysML) and Model-Based Systems Engineering (MBSE) ...Systems Modeling Language (SysML) and Model-Based Systems Engineering (MBSE) ...
Systems Modeling Language (SysML) and Model-Based Systems Engineering (MBSE) ...
 
Embedded world 2017
Embedded world 2017Embedded world 2017
Embedded world 2017
 

Similar to Planning, Architecting, Implementing, and Measuring Automation

Capstone Presentation 2015 - Quality+
Capstone Presentation 2015 - Quality+Capstone Presentation 2015 - Quality+
Capstone Presentation 2015 - Quality+Eric M. Pastore
 
Uncovering Emerging Information Trends in Information Technology
Uncovering Emerging Information Trends in Information TechnologyUncovering Emerging Information Trends in Information Technology
Uncovering Emerging Information Trends in Information TechnologyEric M. Pastore
 
Critical steps in Determining Your Value Stream Management Solution
Critical steps in Determining Your Value Stream Management SolutionCritical steps in Determining Your Value Stream Management Solution
Critical steps in Determining Your Value Stream Management SolutionDevOps.com
 
Primer on application_performance_testing_v0.2
Primer on application_performance_testing_v0.2Primer on application_performance_testing_v0.2
Primer on application_performance_testing_v0.2Trevor Warren
 
Microsoft interview questions Microsoft sde sdet jobs Microsoft Careers
Microsoft interview questions Microsoft sde sdet jobs Microsoft CareersMicrosoft interview questions Microsoft sde sdet jobs Microsoft Careers
Microsoft interview questions Microsoft sde sdet jobs Microsoft CareersSumit Arora
 
T&D Systems View Assessment Workshop
T&D Systems View Assessment WorkshopT&D Systems View Assessment Workshop
T&D Systems View Assessment WorkshopEPPIC Inc.
 
ITeLearn-Manual Testing Day 05.pptx
ITeLearn-Manual Testing Day 05.pptxITeLearn-Manual Testing Day 05.pptx
ITeLearn-Manual Testing Day 05.pptxRoopa slideshare
 
ITeLearn-Manual Testing Day 05.pptx
ITeLearn-Manual Testing Day 05.pptxITeLearn-Manual Testing Day 05.pptx
ITeLearn-Manual Testing Day 05.pptxGaurav Nigam
 
ITeLearn-Manual Testing Day 05.pptx
ITeLearn-Manual Testing Day 05.pptxITeLearn-Manual Testing Day 05.pptx
ITeLearn-Manual Testing Day 05.pptxRoopa slideshare
 
ITeLearn-Manual Testing Day 05.pptx
ITeLearn-Manual Testing Day 05.pptxITeLearn-Manual Testing Day 05.pptx
ITeLearn-Manual Testing Day 05.pptxGaurav Nigam
 
Testing Imp Document
Testing Imp DocumentTesting Imp Document
Testing Imp DocumentHappy2501
 
Risk Driven Testing
Risk Driven TestingRisk Driven Testing
Risk Driven TestingJorge Boria
 
Be Part Of Software Development
Be Part Of Software DevelopmentBe Part Of Software Development
Be Part Of Software DevelopmentBahaa Farouk
 
Software Development Methodologies.pptx
Software Development Methodologies.pptxSoftware Development Methodologies.pptx
Software Development Methodologies.pptxMohamedElshaikh10
 
Downloads abc 2006 presentation downloads-ramesh_babu
Downloads abc 2006   presentation downloads-ramesh_babuDownloads abc 2006   presentation downloads-ramesh_babu
Downloads abc 2006 presentation downloads-ramesh_babuHem Rana
 
Salesforce Application Lifecycle Management presented to EA Forum by Sam Garf...
Salesforce Application Lifecycle Management presented to EA Forum by Sam Garf...Salesforce Application Lifecycle Management presented to EA Forum by Sam Garf...
Salesforce Application Lifecycle Management presented to EA Forum by Sam Garf...Sam Garforth
 

Similar to Planning, Architecting, Implementing, and Measuring Automation (20)

Capstone Presentation 2015 - Quality+
Capstone Presentation 2015 - Quality+Capstone Presentation 2015 - Quality+
Capstone Presentation 2015 - Quality+
 
Uncovering Emerging Information Trends in Information Technology
Uncovering Emerging Information Trends in Information TechnologyUncovering Emerging Information Trends in Information Technology
Uncovering Emerging Information Trends in Information Technology
 
TestOps and Shift Left
TestOps and Shift LeftTestOps and Shift Left
TestOps and Shift Left
 
Critical steps in Determining Your Value Stream Management Solution
Critical steps in Determining Your Value Stream Management SolutionCritical steps in Determining Your Value Stream Management Solution
Critical steps in Determining Your Value Stream Management Solution
 
Primer on application_performance_testing_v0.2
Primer on application_performance_testing_v0.2Primer on application_performance_testing_v0.2
Primer on application_performance_testing_v0.2
 
Microsoft interview questions Microsoft sde sdet jobs Microsoft Careers
Microsoft interview questions Microsoft sde sdet jobs Microsoft CareersMicrosoft interview questions Microsoft sde sdet jobs Microsoft Careers
Microsoft interview questions Microsoft sde sdet jobs Microsoft Careers
 
T&D Systems View Assessment Workshop
T&D Systems View Assessment WorkshopT&D Systems View Assessment Workshop
T&D Systems View Assessment Workshop
 
Manual Testing
Manual TestingManual Testing
Manual Testing
 
ITeLearn-Manual Testing Day 05.pptx
ITeLearn-Manual Testing Day 05.pptxITeLearn-Manual Testing Day 05.pptx
ITeLearn-Manual Testing Day 05.pptx
 
ITeLearn-Manual Testing Day 05.pptx
ITeLearn-Manual Testing Day 05.pptxITeLearn-Manual Testing Day 05.pptx
ITeLearn-Manual Testing Day 05.pptx
 
ITeLearn-Manual Testing Day 05.pptx
ITeLearn-Manual Testing Day 05.pptxITeLearn-Manual Testing Day 05.pptx
ITeLearn-Manual Testing Day 05.pptx
 
ITeLearn-Manual Testing Day 05.pptx
ITeLearn-Manual Testing Day 05.pptxITeLearn-Manual Testing Day 05.pptx
ITeLearn-Manual Testing Day 05.pptx
 
Testing Imp Document
Testing Imp DocumentTesting Imp Document
Testing Imp Document
 
SoftServe Overview
SoftServe  OverviewSoftServe  Overview
SoftServe Overview
 
Risk Driven Testing
Risk Driven TestingRisk Driven Testing
Risk Driven Testing
 
Be Part Of Software Development
Be Part Of Software DevelopmentBe Part Of Software Development
Be Part Of Software Development
 
Software Development Methodologies.pptx
Software Development Methodologies.pptxSoftware Development Methodologies.pptx
Software Development Methodologies.pptx
 
Downloads abc 2006 presentation downloads-ramesh_babu
Downloads abc 2006   presentation downloads-ramesh_babuDownloads abc 2006   presentation downloads-ramesh_babu
Downloads abc 2006 presentation downloads-ramesh_babu
 
Outsourcing With Agile
Outsourcing With AgileOutsourcing With Agile
Outsourcing With Agile
 
Salesforce Application Lifecycle Management presented to EA Forum by Sam Garf...
Salesforce Application Lifecycle Management presented to EA Forum by Sam Garf...Salesforce Application Lifecycle Management presented to EA Forum by Sam Garf...
Salesforce Application Lifecycle Management presented to EA Forum by Sam Garf...
 

More from TechWell

Failing and Recovering
Failing and RecoveringFailing and Recovering
Failing and RecoveringTechWell
 
Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization TechWell
 
Test Design for Fully Automated Build Architecture
Test Design for Fully Automated Build ArchitectureTest Design for Fully Automated Build Architecture
Test Design for Fully Automated Build ArchitectureTechWell
 
System-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good StartSystem-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good StartTechWell
 
Build Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test StrategyBuild Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test StrategyTechWell
 
Testing Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for SuccessTesting Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for SuccessTechWell
 
Implement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlowImplement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlowTechWell
 
Develop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your SanityDevelop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your SanityTechWell
 
Eliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps StrategyEliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps StrategyTechWell
 
Transform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOpsTransform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOpsTechWell
 
The Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—LeadershipThe Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—LeadershipTechWell
 
Resolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile TeamsResolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile TeamsTechWell
 
Pin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile GamePin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile GameTechWell
 
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile TeamsAgile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile TeamsTechWell
 
A Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps ImplementationA Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps ImplementationTechWell
 
Databases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery ProcessDatabases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery ProcessTechWell
 
Mobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to AutomateMobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to AutomateTechWell
 
Cultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for SuccessCultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for SuccessTechWell
 
Turn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile TransformationTurn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile TransformationTechWell
 

More from TechWell (20)

Failing and Recovering
Failing and RecoveringFailing and Recovering
Failing and Recovering
 
Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization
 
Test Design for Fully Automated Build Architecture
Test Design for Fully Automated Build ArchitectureTest Design for Fully Automated Build Architecture
Test Design for Fully Automated Build Architecture
 
System-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good StartSystem-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good Start
 
Build Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test StrategyBuild Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test Strategy
 
Testing Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for SuccessTesting Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for Success
 
Implement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlowImplement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlow
 
Develop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your SanityDevelop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your Sanity
 
Ma 15
Ma 15Ma 15
Ma 15
 
Eliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps StrategyEliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps Strategy
 
Transform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOpsTransform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOps
 
The Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—LeadershipThe Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—Leadership
 
Resolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile TeamsResolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile Teams
 
Pin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile GamePin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile Game
 
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile TeamsAgile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
 
A Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps ImplementationA Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps Implementation
 
Databases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery ProcessDatabases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery Process
 
Mobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to AutomateMobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to Automate
 
Cultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for SuccessCultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for Success
 
Turn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile TransformationTurn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile Transformation
 

Recently uploaded

Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...Aggregage
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 

Recently uploaded (20)

Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
The Data Metaverse: Unpacking the Roles, Use Cases, and Tech Trends in Data a...
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 

Planning, Architecting, Implementing, and Measuring Automation

  • 1. TH AM Tutorial 10/14/2014 8:30:00 AM "Planning, Architecting, Implementing, and Measuring Automation" Presented by: Michael Sowers Software Quality Engineering Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
  • 2. Mike Sowers Software Quality Engineering Mike Sowers has more than twenty-five years of practical experience as a global quality and test leader of internationally distributed test teams across multiple industries. Mike is a senior consultant, skilled in working with both large and small organizations to improve their software development, testing, and delivery approaches. He has worked with companies including Fidelity Investments, PepsiCo, FedEx, Southwest Airlines, Wells Fargo, ADP, and Lockheed to improve software quality, reduce time to market, and decrease costs. With his passion for helping teams deliver software faster, better, and cheaper, Mike has mentored and coached senior software leaders, small teams, and direct contributors worldwide.
  • 3. Me and My Contemporaries 1 Michael D. Sowers, CIO Software Quality Engineering Planning, Architecting, Implementing, and Measuring Automation Copyright M.D. Sowers 2 Administrivia Messages Meals Tutorial timing Tutorial materials Breaks Pagers and Cell phones Facilities Smoking
  • 4. Me and My Contemporaries 2 Copyright M.D. Sowers A Heap of Tools Copyright M.D. Sowers Tutorial Goals At the end of this tutorial you will be able to: • Define your integrated automation team • Document your current state • Layout a future state and roadmap • Create your pilot and implementation plan • Identify initial key measures
  • 5. Me and My Contemporaries 3 Copyright M.D. Sowers Outline Introduction The Compelling Business Need(s) Automation Challenges Automation Requirements Life Cycle Automation Overarching Approach Define the Team, Assess Current State Develop the Architecture Plan, Pilot & Implement Measure and Report Benefits Lessons Learned Copyright M.D. Sowers Outline Introduction The Compelling Business Need(s) Automation Challenges Automation Requirements Life Cycle Automation Overarching Approach Define the Team, Assess Current State Develop the Architecture Plan, Pilot & Implement Measure and Report Benefits Lessons Learned
  • 6. Me and My Contemporaries 4 Copyright M.D. Sowers Compelling Business Need(s) • Faster….......... – Development – Testing – Deployment – And more… • Better………………… – Features – Performance – Usability – And more… • Cheaper…………………… – Elaboration – Implementation – Delivery – And more… Copyright M.D. Sowers Compelling Business Need(s) • Faster………………………………… Speed to Market – Development – Testing – Deployment – And more… • Better……………………………….. Quality Product(s) – Features – Performance – Usability – And more… • Cheaper……………………………. Competitive Innovation – Elaboration – Implementation – Delivery – And more…
  • 7. Me and My Contemporaries 5 Copyright M.D. Sowers Automation Challenges • An abundant array of disconnected tools… with assembly usually required… focused on execution vs. design… demanding a high-level of technical expertise.. and significant maintenance… • Culminating in an investment not always understood and an array of tools not used… Copyright M.D. Sowers Automation Key Requirements • A strategy, a vision and a plan • An architecture and integration (across the lifecycle and between technology/vendors) • Early lifecycle centric • Roles based with associated training • Aligned and integrated with processes • SUT Focused • With business oriented measures and reporting
  • 8. Me and My Contemporaries 6 Copyright M.D. Sowers Life Cycle Automation Copyright M.D. Sowers alm Generalized (video)
  • 9. Me and My Contemporaries 7 Copyright M.D. Sowers Outline Introduction The Compelling Business Need(s) Automation Challenges Automation Requirements Life Cycle Automation Overarching Approach Define the Team, Assess Current State Develop the Architecture Plan, Pilot & Implement Measure and Report Benefits Lessons Learned Copyright M.D. Sowers Define the Team • Cross Functional with representation from: – Product Office – Technical Architecture (and standards) – Software Development (and maintenance team) – Quality Assurance and Testing – IT – Business Units • Global representation • Key Opinion Leaders
  • 10. Me and My Contemporaries 8 Copyright M.D. Sowers Define the Team – Exercise • Draft answers to the questions in the Appendix Copyright M.D. Sowers Assess the Current State • Inventory current tools & processes – Tool Category – Purpose (single function/multi-function) – Value – At what lifecycle stage does the tool apply – What process(es) does the tool support? – Existing Linkage to other tools – License and Maintenance expenses – How many users (departments, BU’s, teams) – Current issues/deficiencies – Platforms Supported – Others?
  • 11. Me and My Contemporaries 9 Copyright M.D. Sowers Assess – Current State Copyright M.D. Sowers Assess – Current State
  • 12. Me and My Contemporaries 10 Copyright M.D. Sowers Assess the Current State – Exercise • Fill in a draft of the spreadsheet in the Appendix Copyright M.D. Sowers Plan - Vision, Strategy, Approach • Vision: End in mind/automation dream state? – 1-3 year time horizon – Aligned with critical business needs – A quantified “end in mind” • Strategy: The “what” of achieving your automation vision – Engage the key opinion leaders – Address existing pain points – Link to business objectives • Approach: The “how” of achieving your automation strategy – A blueprint (of current and future state) – Aligned with the Technical Architecture – An agile plan – responsive & flexible to change
  • 13. Me and My Contemporaries 11 Copyright M.D. Sowers Establish an End in Mind (Vision-1) Copyright M.D. Sowers Establish an End in Mind (Vision-2)
  • 14. Me and My Contemporaries 12 Copyright M.D. Sowers Establish an End in Mind (Vision-3) Copyright M.D. Sowers Architectural Considerations Operations Access & Presentation Business Security Infrastructure Application Data Integration
  • 15. Me and My Contemporaries 13 Copyright M.D. Sowers Process Considerations Copyright M.D. Sowers Process Considerations
  • 16. Me and My Contemporaries 14 Copyright M.D. Sowers Establish a Vision – Exercise • Draft (draw) a vision, an end state of your tools architecture (big boxes are fine to start) Copyright M.D. Sowers Establish a Plan • Partial Blueprint • Integrated Tools Architecture • Tools Roadmap • Tools Mapped to the lifecycle • Roles Mapped to the lifecycle
  • 17. Me and My Contemporaries 15 Copyright M.D. Sowers It’s like building an Airplane in the Air! Copyright M.D. Sowers Partial Blueprint Technology Area 2008 2009 2010 2011 Requirements Management Borland -- CaliberRM 2006-- (U) IBM -- Rational RequisitePro 8.X-- (P) IBM -- Rational RequisitePro 8.X-- (P) IBM -- Rational RequisitePro 8.X-- (P) HP/Mercury -- Test Director (Quality Center) for Requirements Management 9.x-- (E) HP/Mercury -- Test Director (Quality Center) for Requirements Management 9.x-- (E) HP/Mercury -- Test Director (Quality Center) for Requirements Management 9.x-- (P) HP/Mercury -- Test Director (Quality Center) for Requirements Management 10.x-- (P) Developer Test CxxTest (R) Parasoft – C++ Test (P) Parasoft – C++ Test (P) Parasoft – C++ Test (P) Test Management IBM -- Rational Test Manager-- (R) IBM -- Rational Test Manager-- (R) HP/Mercury -- TestDirector: Quality Center 10.X-- (P) HP/Mercury -- TestDirector: Quality Center 10.X-- (P) HP/Mercury -- TestDirector: Quality Center 10.X-- (P) HP/Mercury -- TestDirector: Quality Center 10.X-- (P) Functional Automation IBM -- Rational Functional Tester-- (R) IBM -- Rational Functional Tester-- (R) IBM -- Rational Functional Tester-- (R) IBM -- Rational Robot 7.x-- (R) IBM -- Rational Robot 7.x-- (R) Borland -- Silk Test-- (R) Borland -- Silk Test-- (R) HP/Mercury -- WinRunner 9.x- - (R) HP/Mercury -- WinRunner 9.x– (R) HP/Mercury – QTP (P) HP/Mercury – QTP (P)
  • 18. Me and My Contemporaries 16 Copyright M.D. Sowers Integrated Tools Architecture Copyright M.D. Sowers Tools Roadmap (1)
  • 19. Me and My Contemporaries 17 Copyright M.D. Sowers Tools Roadmap (2) Copyright M.D. Sowers Tools Mapped to Lifecycle
  • 20. Me and My Contemporaries 18 Copyright M.D. Sowers Methods, Tools and Measures Engineers Functional Test Engineers Test Automators Test Coordinators Test Architects Performance Test Engineers Test Designers Roles Based & Life Cycle Integrated Production Excellence Manager SIT PRODUATDITTSDSDSSRAPPI AIP Reviews Function Test Performance Test Operational Readiness Test Unit Test VerificationVerification ValidationValidation Production Excellence Production Excellence Systems Test Engineers Copyright M.D. Sowers Establish a Plan – Exercise • Using the examples presented, begin to think about (draft) the planning questions in the appendix
  • 21. Me and My Contemporaries 19 Copyright M.D. Sowers Select, Pilot & Implement Tool selection/purchase Review of tools Evaluate/pilot a select set of tools Demonstrate tools Market analysis Tool requirements specification Copyright M.D. Sowers Select, Pilot & Implement – Exercise • Begin to think about (draft) the select, pilot & implement questions in the appendix
  • 22. Me and My Contemporaries 20 Copyright M.D. Sowers Measure & Report Benefits • Tool Measures in Technical Terms – Comprehensiveness of Tool Functionality – Number of issues/defects in or with the tool – Satisfaction/response time with Tool Support – Ease of Use – Ease of Integration (api) – And more… • Measures in Business Terms –The MOST important! Copyright M.D. Sowers Business Oriented Measures 2008 2009 2010 Expansion • Applications covered by QA&T Team 90 160 230 • Number of Applications Automated 22 46 78 Effectiveness • Pre-Release defects 162 200 274 • Post-Release defects 28 22 18 • Requirements coverage (%) 53 71 87 Efficiency • QA & Test staff to developer ratio 1/5 1/5.5 1/6.5 • Time for regression test runs (days) 8 5 4 Business Results • QA & Test Total Cycle Time (days) 40 36 31 • Cost per Total Test Cycle (K dollars) 320 288 248 • Post release defect containment (%) 85.3 90.1 94
  • 23. Me and My Contemporaries 21 Copyright M.D. Sowers Measure/Metrics – Exercise • Begin to think about (draft) the measures/metrics questions in the appendix Copyright M.D. Sowers Outline Introduction The Compelling Business Need(s) Automation Challenges Automation Requirements Life Cycle Automation Overarching Approach Define the Team, Assess Current State Develop the Architecture Plan, Pilot & Implement Measure and Report Benefits Lessons Learned
  • 24. Me and My Contemporaries 22 Copyright M.D. Sowers Key Learning's • Do: – Engage Sr. Technical Staff – Secure Stakeholder Alignment – Have a larger vision, but focus on one area at a time – Allow for flexibility and customization – Monitor the “scope” of standardization • Don’t: – Permit too much separation between roles – Ignore existing processes & dependencies – Limit your thinking to the current technology Copyright M.D. Sowers (A Few) Keys to Success Automation as a Technology Issue Automation as a Business Imperative Technical Measures Business Measures No Vision, Strategy, Architecture Complete Automation Blueprint Buckets of Automation Integrated Workflow Everyone does everything Roles Based FROM TO
  • 25. Me and My Contemporaries 23 Copyright M.D. Sowers A Less Than Functional Home Copyright M.D. Sowers Remember - We build Airplanes in the Air • Doing so with isolated tools • Without LEADERSHIP and a PLAN • Results in an interesting – but not so useful automation mansion • Wasting the lives of many skilled crafts men and women (our developers & testers) • And no value contributed to the business
  • 26. Me and My Contemporaries 24 Copyright M.D. Sowers Tutorial Goals Achieved You are now able to: • Define your integrated automation team • Document your current state • Layout a future state and roadmap • Create your pilot and implementation plan • Identify initial key measures Copyright M.D. Sowers Wrap Up • Current Challenges? • ? • ? • ? • ?
  • 27. Me and My Contemporaries 25 Copyright M.D. Sowers 49 How did we do? * Great * Wonderful * Fantastic * Terrific * Superior * Excellent Tutorial Evaluation Tutorial Evaluations Copyright M.D. Sowers 50 Thank You • On behalf of Software Quality Engineering, thank you for attending this tutorial. • If you have further needs for training or consulting, please think first of SQE. • If I can be of further assistance, please let me know. My email is msowers@sqe.com
  • 28. Me and My Contemporaries 26 Copyright M.D. Sowers Appendix
  • 29. Appendix A Copyright M. D. Sowers, Software Quality Engineering 1 Integrated Tool Architecture Team Selection 1. Who are the key stakeholders for software lifecycle automation? Why are they key stakeholders? 2. Of the key stakeholders, which are critical to include? Is there balanced participation globally? 3. What's the approach for engaging and securing a commitment to participate from the critical stakeholders?
  • 30. Appendix A Copyright M. D. Sowers, Software Quality Engineering 2 Tool(s) Inventory (Example)
  • 31. Appendix A Copyright M. D. Sowers, Software Quality Engineering 3 Tool(s) Inventory (blank)
  • 32. Appendix A Copyright M. D. Sowers, Software Quality Engineering 4 Tool(s) Implementation Plan 1. What’s the first area of focus? (requirements, dev, test, deployment) 2. For the initial area of focus, map those tools to the life cycle steps/stages 3. Tools to retire, tools to add? 4. Where would tool connections be beneficial?
  • 33. Appendix A Copyright M. D. Sowers, Software Quality Engineering 5 Select, Pilot & Implement 1. For the specific area of focus or tool selected what are the requirements? 2. What will be the focus of your market research? (Commercial, Open Source, Custom Development) 3. Who are the key stakeholders that need to be engaged in the demo's? 4. What (1 or 2) teams might be good candidates to pilot the tool? 5. What changes to existing methodology or standards may be needed to adopt this tool into our existing environment?
  • 34. Appendix A Copyright M. D. Sowers, Software Quality Engineering 6 Measures/Metrics 1. What specific technical measures will be used to assess the success and contribution of this tool? 2. What specific business measures will be used to assess the success and contribution of this tool? 3. What information do I need to plan to collect to track these measures?