Worked on this project with others in my team and also heavily on platform metrics. Glad to share it! More power to #Dataanalytics #BusinessIntelligence

1. WEBINAR: QA ANALYTICS
Customer Story:
TIAA-CREF's QA Director explains
how TotalView analytics
revolutionized QA And Testing

2. This webinar will be ~60 minutes
with time for Q&A
Ask questions in the Questions pane
of GoToMeeting
As you exit the webinar, please fill in
the short survey form that opens.
We appreciate it.
You’ll receive a recording and the
slides after the webinar.
WELCOME

3. Assure
Founded in 2006
Leader in Analytics for Application Lifecycle
HP Partner – working with the largest HP ALM
customers
TotalView
Analytics platform
End-to-end visibility for Enterprise Application
Delivery
On-demand dashboards for all levels of stakeholders
Consists of automatic data collection and Datamart
Presentation layer powered by Qlik
ABOUT ASSURE AND TOTALVIEW
Shir Goldberg
VP Biz Dev

4. ABOUT THE SPEAKER
Andrea Gonsalves
Director QA, TIAA-CREF
Andrea currently leads the Online and Horizontal Quality
Assurance teams for the Retirement & Individual Technology
organization at TIAA-CREF. She has over 20 years of IT and
Financial Services experience with 15 years in the field of
Quality Assurance, across multiple platforms, technologies and
lines of business. She has expertise in Quality Metrics and has
led numerous transformation efforts. Previously, Andrea worked
at Fidelity Investments, leading their Online Retirement testing
team and heading up the Workplace & Personal Investing
Center of Excellence, focused on developing quality
engineering practices.

5. From Teacher Retirement Services to Full-Service Financial Services
ABOUT TIAA CREF

6. BusinessIntelligence/
Communication
RITQAServiceOfferings
Systems Integration
Testing (SIT) *
Quality Assurance
Assess & Measure SDLC
Quality
Test Data Mining
Freeze Exception
Management
The Retirement & Individual Technology
(RIT) QA Team
RITMethodology
Monthly Releases
Agile & Waterfall
30 – 50 Projects Per Month
Multiple Test Environments

7. From a ‘Mom & Pop Shop’ to a ‘Fortune 100 Company’,
with BI as a critical component
 Easily Monitor & Predict Outcomes
 Increased Transparency
 Improved Credibility & Relationships
 Faster and More Informed Decisions
 One Version of the Truth
 Improved Delivery Quality
Using
Business
Intelligence
 Conflict & Discrepancies on Status
 Costly Manual Reporting
 No Release Level View
 Lots of Drama to Complete Testing
 Many Different Measures
 Low Trust

8. The Release View allows us to see a quick snapshot of how we’re doing against
our plan for a given monthly release and pin point potential risk areas and trends
We can view the detailed project stats within the Project View to assess key
indicators and determine the project’s release readiness health
Business Intelligence: Assessing Release Readiness
ReleaseView

9. The Release View allows us to see a quick snapshot of how we’re doing against
our plan for a given monthly release and pin point potential risk areas and trends
We can view the detailed project stats within the Project View to assess key
indicators and determine the project’s release readiness health
Business Intelligence: Assessing Release Readiness
ReleaseViewProjectView

10. Business Intelligence: Predicting Test Completion
We monitor test execution rates, and predict if testing will be complete on time.

11. Business Intelligence: Predicting Defect Completion
We monitor defect detection and closure rates, forecast new defects to
be detected and predict defect closure by code freeze to ensure release
readiness.

12. Business Intelligence: Defect Analysis
We have drill down capability to
help pin point potential issues
or risk areas.

13. Dashboard Implementation
13
Phase1
Dedicated Metrics Expert
But not engaged in
Projects
Ended up with siloed
views of Test execution &
Defects
Focused only on needs of
QA (one view/one role)
Phase2
Started with a prototype
focused on usability and
simplification
Interviewed actual users
Examined Decisions
Examined Roles
Training and Education;
socialized usage for key
decisions
Phase3
Focused on interpreting
data and recommending
actions
Utilize advanced analytics
and trending
Information from
additional systems for
holistic view
A Dashboard that
few people used
A powerful
Dashboard, broadly
used
(Expected) A
Prescriptive
Dashboard

14. 0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
>90% (Green) <90% (Red)
2013
2014
Outcome: Increase in Test Execution
Before Code Freeze
% Projects completing test
execution by Code Freeze (YOY)
From 35% of projects executing 90% before code freeze to over 70%

15. Outcome: IT saw a decrease in average days late and
an increase in number of ‘healthy projects’.

16.  Hit #1 in Dalbar for our Plan Sponsor, Mutual Funds & Insurance websites!
 Our Participant website moved up to #5 in Dalbar
 Continued our top position as one of the fastest websites in the industry
 Named ‘Best Insurance Website,’ by Web Marketing Association
Outcome: Created a Competitive Advantage

17. Summary
Key Changes
Fact based culture, taking the emotion out
Consistent view, providing improved communication
One version of the truth
Less drama
Ability to implement changes and measure the impact
Eliminated manual reports – focus on testing instead 

18. 5 Key Benefits from Enterprise Views of
Quality Assurance
Better Release Quality
Increased Customer Satisfaction
Improved Productivity and Decreased Costs
Earlier Risk Mitigation with Informed Decision Making
Improved Defect Containment

19. Q & A
Contact:
Andrea: agonsalves@tiaa-cref.org
Shir: shir.goldberg@assure.net
Assure Corporation | www.assure.net | Shir Goldberg | 954-643-7622

20. Assure TotalView videos
http://assure.net/what-is-totalview/
http://assure.net/video-assure-totalview/
Assure HP ALM Project Harmonizer and Standardizer
http://assure.net/video-harmonizer/
http://assure.net/datastandardizer/
Assure blog:
http://assure.net/blog/
MORE MATERIAL IS
ON OUR WEB SITE

21. THE TRUSTED CHOICE
FOR ALM ANALYTICS
ASSURE