Are you running automated tests during development yet not providing automated feedback to the project stakeholders? Vikas Bhupalam approached this problem by leveraging and integrating monitoring, logging, and defect tracking systems to provide automatic feedback to stakeholders. Tests are executed using a Java-based framework, and the results are sent to a monitoring tool that shows up as traffic lights on a dashboard. The dashboard links to logs on the server that provide insights into failing tests and root causes of problems. Alerts can be triggered for specific conditions. Change requests are then automatically filed in the defect tracking system with the appropriate severity and priority set. The QA sign off in all environments is provided to DevOps and all other stakeholders in this automated process. Learn about the framework and the integration involved in bringing all these pieces together.

1. T21
Concurrent Class
10/3/2013 3:00:00 PM
"End-to-End Automation:
Providing Stakeholders
Feedback on Quality"
Presented by:
Vikas Bhupalam
Intuit, Inc.
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

2. Vikas Bhupalam
Intuit, Inc.
A leader in quality engineering at Intuit, Vikas Bhupalam encourages the adoption of new
processes and tools in test automation. Vikas’ innovative approach to solving complex problems
across various business units has helped products scale to optimal levels of quality,
performance, and security. His expertise across both functional and performance test
automation, including a development background, has helped him build various frameworks
using a variety of tools and technologies. A passionate engineer, Vikas attends to details that
help test SaaS-based applications.

3. 9/19/2013
END-TO-END AUTOMATION
PROVIDING STAKEHOLDERS FEEDBACK ON QUALITY
Vikas Bhupalam
Lead Engineer
Intuit, Inc.
Email: Vikas_bhupalam@intuit.com
Monitoring Dashboard
TestNG Results
View Errors in
Splunk
Errors Sent to
DTS
WebDriver:TestFramework
Defect
Tracking
System
1

4. 9/19/2013
Test Automation: Overview
 Tests Created in Java using Webdriver
 TestNG Results parsed by Monitoring
framework
 Monitoring framework links to Logging
Framework
 Defect filed in tracking system based on
exceptions
TestNG/JUnit Parser
JUnit Output
 Choosing the right framework
 Values to Parse- Name, ClassName, Failures
2

5. 9/19/2013
TestNG/JUnit Parser
TestNG Output
Fields: Status, Name
Test Results
HTML Results
3

6. 9/19/2013
Test Results
Details
Monitoring Framework
 Dashboard for each environment
 One View Per Cluster
 Agent Deployed on Admin Server that
connects to all Environments
 Framework parses XML files from
TestNG/Junit
 Each test that has a traffic light view
4

7. 9/19/2013
Monitoring Framework
 Traffic light for test turns Red on Failure and
Green on Success
 Redundancy in tests executed
 Type of tests
 Services
 Front-end (UI Based)
 Type of tests
Monitoring Framework
Green/Red
5

8. 9/19/2013
Logging Framework
 Indexing all the required log files
 Test Failures Cause Exceptions in Server Logs
 LF/W tails log files every 5 mins
 Saved Searches and Dashboards @ Server
 Multiple “sourcetypes” to help identify each
environment
 Scheduled Searches have actions like running
a script/emails to stakeholders
Monitoring Framework
 Exceptions/Error redundancy
6

9. 9/19/2013
Monitoring/Logging Framework
 Monitoring stakeholders get access to all
Errors with a single view
Connect frameworks
7

10. 9/19/2013
Defect Tracking Framework
 Create bugs on command line
 Admin access to Defect tracking system
 Check Defect database to search for existing
defects to tackle duplicates
 Customizable fields with a mandatory list of
fields to be included with each bug
Defect Tracking Framework
 Scheduled search that automatically creates
a bug in DTF
8

11. 9/19/2013
Defect Tracking Framework
 Sample Script with mandatory/customizable
fields
Thank You!!!!
Questions
9