As the software development lifecycle shifts toward agile and lean methodologies, quality in every build becomes critical. Continuous integration allows development teams to receive immediate feedback on their code, creating more efficiency and higher quality. After exploring the differences in continuous integration, continuous delivery, and continuous deployment, Jennifer Bonine and Michael Faulise discuss what is needed for their successful implementation, including the technologies and resources required at each stage of the process. Jennifer and Mike share models that show where your organization is on the continuous integration/continuous delivery path, the required technical skills needed to implement them, and how to decide if this strategy is right for you. They describe the inevitable “shifting left” of testing, and what your projects will need to optimize quality and increase velocity. Jennifer an Mike share a perspective of what has been successful and what has not worked in companies from start-ups to Fortune 100.

1. AW3
Agile Development Concurrent Session
11/12/2014 11:30 AM
"Shifting Left: The Evolution of
Test Automation"
Presented by:
Jennifer Bonine and Michael Faulise
tap | QA
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

2. Jennifer Bonine is a VP of global delivery and solutions for tap|QA,
Inc., a global company that specializes in strategic solutions for
businesses. Jennifer began her career in consulting, implementing
large ERP solutions. She has held executive level positions leading
development, quality assurance and testing, organizational
development, and process improvement teams for Fortune 500
companies in several domains. In a recent engagement for one of
the world’s largest technology companies, Jennifer served as a
strategy executive and in corporate marketing for the C-Suite. In her
career, she has had several opportunities to build global teams from
the ground up and has been fortunate to see how many of the
world’s top companies operate from the C-Suite viewpoint.
Michael Faulise is the founder and managing partner at tap|QA,
LLC, a global company that specializes in quality solutions for
businesses. Mike focuses on sales and delivery where he consults
with clients in the areas of leading development, quality assurance
and testing, technology and process training, and process
improvement. He has seen software development evolve along the
multiple paths of various methodologies but has found quality has
remained essentially constant.

3. Continuous Integration
April 2014
tap|QA Presentation

4. 2
Agenda
Overview
What is involved?
Mobile
Skill Sets
Solutions Focus Pays
Copyright © 2014 TAP/QA, LLC – Confidential & Proprietary. For more information: www.tapQA.com
Continuous Integration

5. 3
Solutions Focus Pays
Success StoriesOur Mission
Copyright © 2014 TAP/QA, LLC – Confidential & Proprietary. For more information: www.tapQA.com
tap|OVERVIEW
Focus Pays
Client Examples
 Our mission is to provide our clients with focused solutions
for delivering Strategic and Tactical Quality Initiatives
 We pride ourselves on hiring highly talented people that
deliver high quality work for our Clients
Why tap|QA?
 Our investment in our employees has allowed us to meet the
growing demand or QA roles. Including the new Technologies and
Business Requirements every QA now has along with the desire
and drive in Quality.
 Our innovation has created new approaches in competing with Off-
shore models with our tap|Lakeshore. Our tap|Technology team is
advancing leading edge ideas in Continuous Integration, Delivery
and Deployment.

6. 4
Continuous Integration – The practice of
everyone involved with code, merging their
local copies multiple times a day.
Attempting to prevent Deployment Delays
Solutions Focus Pays
Copyright © 2014 TAP/QA, LLC – Confidential & Proprietary. For more information: www.tapQA.com
Continuous Integration

7. 5
Continuous Delivery – The automated process of
implementing, validating and reporting code from
development to production. But only moving into
Production after passing a manual quality gate.
Continuous Deployment – The automated process of
implementing, validating and reporting code from
development to production.
Solutions Focus Pays
Copyright © 2014 TAP/QA, LLC – Confidential & Proprietary. For more information: www.tapQA.com
Delivery Vs. Deployment

8. 6
Solutions Focus Pays
Copyright © 2014 TAP/QA, LLC – Confidential & Proprietary. For more information: www.tapQA.com
Continuous Integration Cont.

9. 7
Solutions Focus Pays
Copyright © 2014 TAP/QA, LLC – Confidential & Proprietary. For more information: www.tapQA.com
Traditional Vs. CI Automation
UI
Automation:
QTP
Selenui
m
Load Testing:
LoadRunn
er
Jmeter
UI
Automation:
QTP
Selenui
m
Load Testing:
Loadrunne
r
Jmeter
Expanded
Harness:
Jmeter
CodedUI
Test Harness:
Junit
SoapUI

10. 8
Local Developer Level:
Unit tests (TDD)
Execute in < 30 seconds
Solutions Focus Pays
Copyright © 2014 TAP/QA, LLC – Confidential & Proprietary. For more information: www.tapQA.com
What is Involved ?

11. 9
QA Takes ownership of Test Harness:
Enhance and Expand Test Harness
Adding Code
Additional Test Scenarios
Solutions Focus Pays
Copyright © 2014 TAP/QA, LLC – Confidential & Proprietary. For more information: www.tapQA.com
Where QA is Involved ?

12. 10
CI Implementation at Development Master:
Unit, Smoke, and Core Functionality tests
Adding Code
Additional Test Scenarios
Jmeter, SoapUI, Visual Studio, TFS, etc.
Execute in < 2 minutes
Solutions Focus Pays
Copyright © 2014 TAP/QA, LLC – Confidential & Proprietary. For more information: www.tapQA.com
Where QA is Involved Cont.

13. 11
Solutions Focus Pays
Copyright © 2014 TAP/QA, LLC – Confidential & Proprietary. For more information: www.tapQA.com
Deployment Management

14. 12
QA Level
Smoke, Expanded Core Functionality, and UI tests
QTP, Selenium, Jmeter, SoapUI, TFS, etc.
Execute in < 2 – 4 hours
Solutions Focus Pays
Copyright © 2014 TAP/QA, LLC – Confidential & Proprietary. For more information: www.tapQA.com
Where QA is Involved Cont.

15. 13
Solutions Focus Pays
Copyright © 2014 TAP/QA, LLC – Confidential & Proprietary. For more information: www.tapQA.com
Where QA is Involved Cont.
Stage Level
Smoke, Complete Core functionality, Full UI, and Load
tests
LoadRunner, Jmeter, LoadUI, etc.
Execute in < 6 – 12 hours

16. 14
Solutions Focus Pays
Copyright © 2014 TAP/QA, LLC – Confidential & Proprietary. For more information: www.tapQA.com
Mobile
Mobile
Replicate all harnesses across all supported platforms
Windows 8 (C#)
Android (Java)
IOS (objectiveC)

17. 15
Technical skills:
Architect level – Work with systems architect
designing CI automated processes, and implement of
initial automation framework POC.
Automation Engineer – Expansion of automation
coverage and maintenance of framework.
Non-Technical:
QA Analyst– Requirement decomposition, writing of
test scenarios, reviewing of failed executions.
Solutions Focus Pays
Copyright © 2014 TAP/QA, LLC – Confidential & Proprietary. For more information: www.tapQA.com
Skill Sets

18. 16
Trend:
Current Requirements – Since CI is heavily integrated
within technology the teams are attempted to be
built with QA Analysts that have Architect and
Automation Skills.
Solutions Focus Pays
Copyright © 2014 TAP/QA, LLC – Confidential & Proprietary. For more information: www.tapQA.com
Observations
Quality
Assurance
Business
Industry
Technology

19. Questions?
Continuous Integration