QA Should be led from top down and implemented from bottom up. A presentation by Moti Demri, QA consultant and manager, experienced in building QA teams from the ground up, establishing market level standards (ISO 9000, CMMI) , processes, and testing methodologies for both manual and automated testing. Presented November 2013 to the JAX Chamber IT Council.

1. QA Center Of Excellence (TCoE)
for JITC
Should be led from top down
and
implemented from bottom up.

2. Agenda
•
•
•
•
•
•
What is the QA Center of Excellence (TCoE)
Why implementing TCoE
Benefits of TCoE
Key Strategies/ Tactics/ Approach
Challenges and Opportunities
Business case
• Q&A
• Name – Moti Demri
• Email – mdofqa@gmail.com
• Phone - 904- 386 -6620
2

3. What is QA Center of Excellence (TCoE)?
• TCoE
is a centralized solution that brings into a shared services
function, people, processes, tools and infrastructure. Standardizing processes
and optimizing resource utilization is to bring in maturity.
• The Test Center of Excellence (TCoE) solution is targeted towards customer
who are challenged with decentralized and under-performing testing functions,
under utilization of testing people, inconsistent test processes and are looking
to move up the maturity ladder towards a standardized testing function, and at
the same time improve efficiency, optimize people and tool utilization and
lower testing costs
• A TCoE may define a best or commonly accepted practice and encourage
adoption of that practice
• A TCoE may govern or enforce standards
3

4. This is NOT a one person job…it is the entire organization’s effort
4

5. Why implementing a TCoE?
Reduce Reputational Risk
• Reduce incidence of live issues
• Avoid “customer being our best testers”
• Avoid Emergency releases
Improve Quality
• Reduce number of defects occurring in later stages of
testing
• Increase the independence of testing
• Apply rigor to test process
• Focused collaboration between business and test
• Promote the understanding of test requirements
across the organization
Cost efficiency
• Reduce the need for long test phases
• Increase the usage of automation in Dev and UAT
• Enable earlier detection of defects
• Enable faster more reliable test cycles
• Ensure correct resources (number and skills are being
utilized
5

6. TCoE – Evolution (A gradual approach– from sharing processes to Services Utility)
Phase 1– introduction to standards, governance and processes
• Begin establishing fundamental policies around application testing
• Decide on measurable quality standards
• Define basic reporting procedures
• Implement initial governance process
Phase 2 - TCoE functions as a shared service
• Unified testing manual and automation tool across the company
• Training other groups using tools , scripts writing
• Resources Pool
• Share Regression tests and scripts
Phase 3 - Service utility
• TcoE acts as a central source of services and expertise for the entire origination
• included test management office, domain-specific expertise team, shared services
teams for specialized testing activities
Phase 4 - Innovation authority and flexible resources pool
• TcoE acts as a central source of services and expertise for the entire origination
• included test management office, domain-specific expertise team, shared services
teams for specialized testing activities
6

7. Why do we need TCoE (Benefits)
TCoE can accrue many benefits to an organization
•
Define and promote testing standards
•
Improve quality
•
Define testing boundaries and provide specialized testing services
•
Faster time to market
•
Reduce testing costs
•
Bring together different testing organizations to align their test processes and technical disciplines
•
Increase the testing value to external groups
•
Insure best practices are in place
•
Better ways to control training and knowledge transfer
•
Provide a practical approach to testing and QA functions
•
Consolidation and test process improvement
•
TCoE are becoming increasingly common within large and medium scale businesses
•
TCoE is a centralized solution that brings together people, processes, tools and infrastructure into a
shared services function
•
Standardize processes and optimizing resource utilization
•
Everyone on the same page!!!!
7

8. Why do we need TCoE (Benefits)
• Optimized use of existing resources:
• QA budgets, tools, environments and people
•Faster time-to-market:
• Test times reduce manual by 30%
• and increase test automation levels of 50-70%
•Cost efficiency:
• Typical resource cost reduction is 35% over a 2-year time frame
•Optimized application quality and performance:
• Reduce risk of failure and leakage of high-severity defects to under 2% and improved
customer experience
•Tighter alignment:
• Align quality effort more tightly with business needs by defining and measuring KPIs (Key
performance indicator)
8

9. Quality Process Reduce Cost of Error
Cost of Error Correction
Quality
Process
$2000
$1
Time
9

10. Key Strategies and Tactics
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Senior management involvement
Find a key champions that care about quality and process
Chose doer NOT talker
Start working as team not as individual
Develop very clear vision
Define clear and achievable goals
Define tactics and strategies
Full integration between all departments and business
Work and think on the enterprise level, not local
Scalability, usability and
Semi- integrated model / approach
Pilot it first on a smaller scale project
Establishing a TCoE is a phased approach
Market the TCoE concepts
Training across the company
Define measurements and matrixes
10

11. TCoE - Why others failed to implement?
•
•
•
•
•
•
•
•
•
•
No senior management support , need to gain the buy-in
Need commitment from management and individuals
No clear vision or goals
Implementation going too slowly
No implementation plan
No agreeable timeframe
No upfront planning
Big steps approach implementation
No dedicated staff
Not sufficient tools
11

12. TCoE
-
Stability and Reliability (Vision)
12

13. Governance and Collaboration Aspect
(between IT and Business)
Alignment
Business
Coordination
Company strategy
And
Operation
Business unit
strategy and
Operation
IT
Companywide
IT Governance
Linking Mechanism
Project Plan
Project Management
Enterprise
Architecture
Business unit
architecture
Project IT
Architecture
13
Company
Level
Business
Unit level
Project team
level

14. Testing department assessment Framework
14

15. TCoE Maturity Levels
15

16. TCoE - Metrics program is essential to measure QA effectiveness
The standard definition of test metrics allows an organization to provide visibility into test effectiveness, demonstrate
delivery excellence, track key goals, and enable efficient management and control through business analytics and
predictive management.
Value Delivered
Benefit Type
Best Practice Metrics
Calculation
Effort / Cost
Test Effort Rate (Cost of Quality)
(Total Test Effort / Total Effort) * 100
Effort / Cost
Cost of Poor Quality
(Total Rework Effort / Total Effort) * 100
Quality
Defect Rate
(Total # of Development Defects / Effort Total)
Quality
Test Effectiveness
(Total # of Development Defects / (Total # of Development Defects
+ Total # of Delivery Defects)) * 100
Quality
Defect Ratio
(Total # of Development Defects CLOSED / Total # of Development
Defects) * 100
Quality
Pass Rate
(# of test scripts PASSED / total # of test scripts)
Quality
Defect Arrival Rate
# of defects OPENED / day
Productivity
Defect Closure Rate
# of defects CLOSED / day
Productivity
Retest Backlog
# of defects FIXED (ready for retest) / day
Productivity
Defect Turnaround Time
# of total CLOSED (fixed + retested) defects / Total Rework Effort
Productivity
Cycle Time
Test Script Development Rate
# of scripts APPROVED / day
Productivity
Cycle Time
Test Script Execution Rate
# of scripts EXECUTED / day
Cycle Time
Test Duration
# of days / cycle
Indicates whether
strategy and execution
are contributing to the
bottom line and meeting
financial expectations
16
Service
Excellence
Indicates how well the
organization meets client
and customer needs and
their expectations
Process
Excellence
Indicates whether the
organization has the
process capability,
maturity and quality to
deliver business value
Organizational
Excellence
Indicates how well the
organization meets
employee needs and
their expectations