For those of you that could not attend the South Florida Agile Summit last week due to some other event, here's one of the presentations that you missed.

1. May 28, 2015
South Florida Agile Association – Fort Lauderdale, FL
“It’s not called Continuous Integration for Nothing!”
Dan Boutin – Senior Product Evangelist
dboutin@soasta.com
Mobile (404) 304-9529
@DanBoutinSOASTA

2. © 2014 SOASTA. All rights reserved. 2
In This Discussion Today
Agenda:
• SOASTA Introduction
• The need for “Continuous”
• The “Big 3”
• CI with mobile and performance testing

3. © 2014 SOASTA. All rights reserved. 3
SOASTA – Performance Analytics
100 Billion
User Experiences
Tested
10
Million
Tests
Performed
5 Million
Failures
Avoided
Actual CloudTest
view

4. © 2014 SOASTA. All rights reserved. 4
In This Discussion Today
Agenda:
• SOASTA Introduction
• The need for “Continuous”
• The “Big 3”
• CI with mobile and performance testing

5. © 2014 SOASTA. All rights reserved. 5
CI Offers Speed of Delivery
What’s not to like?

6. © 2014 SOASTA. All rights reserved. 6
CI Offers Speed of Delivery
o Small Batch Sizes
What’s not to like?

7. © 2014 SOASTA. All rights reserved. 7
CI Offers Speed of Delivery
o Small Batch Sizes
o Comprehensive Version Control
What’s not to like?

8. © 2014 SOASTA. All rights reserved. 8
CI Offers Speed of Delivery
o Small Batch Sizes
o Comprehensive Version Control
o Simple Branch Strategies
What’s not to like?

9. © 2014 SOASTA. All rights reserved. 9
CI Offers Speed of Delivery
o Small Batch Sizes
o Comprehensive Version Control
o Simple Branch Strategies
o Automation
What’s not to like?

10. © 2014 SOASTA. All rights reserved. 10
CI Offers Speed of Delivery
o Small Batch Sizes
o Comprehensive Version Control
o Simple Branch Strategies
o Automation
o Continuous Feedback
What’s not to like?

11. © 2014 SOASTA. All rights reserved. 11
CI Offers Speed of Delivery
o Small Batch Sizes
o Comprehensive Version Control
o Simple Branch Strategies
o Automation
o Continuous Feedback
o Emphasis on working builds
What’s not to like?

12. © 2014 SOASTA. All rights reserved. 12
CI Offers Speed of Delivery
o Small Batch Sizes
o Comprehensive Version Control
o Simple Branch Strategies
o Automation
o Continuous Feedback
o Emphasis on working builds
o Consistent environments
What’s not to like?

13. © 2014 SOASTA. All rights reserved. 13
CI Offers Speed of Delivery
o Small Batch Sizes
o Comprehensive Version Control
o Simple Branch Strategies
o Automation
o Continuous Feedback
o Emphasis on working builds
o Consistent environments
o Developer – Tester Collaboration
What’s not to like?

14. © 2014 SOASTA. All rights reserved. 14
In This Discussion Today
Agenda:
• SOASTA Introduction
• The need for “Continuous”
• The “Big 3”
• CI with mobile and performance testing

15. © 2014 SOASTA. All rights reserved. 15
What’s Been Missing?

16. © 2014 SOASTA. All rights reserved. 16
So, let’s break down DevOps – What is it, really?
o It’s not what’s within the silos
o It’s about the relationships

17. A typical silo’d organization lacks end-to-end integration
Performance Engineering & Continuous Integration-Full Lifecycle
Operational Configuration
Continuous Integration touchpoints
ADLC
QA/Test Pre-
Production
Production
QA/Test
Environment
Operational
Production
Environment
Application
Development
Environment

18. Development Test
Configuration management
Release management
Change management
Performance management
CI Focus Areas – The assembly line Hand-offs
Key Processes

19. © 2014 SOASTA. All rights reserved. 19
Software is simple: It’s Manufacturing & BOM
o Bill of Materials (BOM!)
o The “Big 3” => Change, Configuration & Release
• That is your software assembly line conveyor belt

20. © 2014 SOASTA. All rights reserved. 20
So, what?
• We can only test as fast as the conveyor belt allows.
• If the conveyor belt is moving slow, testing is delayed.
• The Big 3 drives testing cycle.
o Why is CI critical?
• Speed
• New features, functions, etc. need to get to the customer, ASAP!

21. © 2014 SOASTA. All rights reserved. 21
So, let’s break that down…
o CI is a key component of Performance Engineering
o Performance Engineering spans the silos
BIG 3
Testin
g
Applicatio
nDevelopmen
t Life
Cycle
IT
Operation
s
Capacit
yManagemen
t
Performanc
eManagemen
t
Performanc
eEngineering

22. © 2014 SOASTA. All rights reserved. 22
Laying the Foundation
A factory streamlines for quality and speed
Does it work as
designed?
Does it perform
under extreme
circumstances?
Approve materials Real Drivers
What happens
in actual
conditions?
Assembly Line

23. © 2014 SOASTA. All rights reserved. 23
Laying the Foundation
CI streamlines for continuous quality
Functional validation
Performance,
reliability,
scalability tests
Real Users
Build
What happens
in actual
conditions?
Unit-tested code

24. © 2014 SOASTA. All rights reserved. 24
In This Discussion Today
Agenda:
• SOASTA Introduction
• The need for “Continuous”
• The “Big 3”
• CI with mobile and performance testing

25. © 2014 SOASTA. All rights reserved. 25
Continuous Quality
SOASTA’s Lifecycle

26. © 2014 SOASTA. All rights reserved. 26
Continuous Quality
SOASTA’s LifeCycle
I JUST MADE A
CODE CHANGE
Jenkins runs any
unit tests as usual

27. © 2014 SOASTA. All rights reserved. 27
Continuous Quality
SOASTA’s Lifecycle
Run iterative load
tests via Jenkins
Spin up suitable, consistent
test environments in the cloud
or the lab
I JUST MADE A
CODE CHANGE
Jenkins runs any
unit tests as usual

28. © 2014 SOASTA. All rights reserved. 28
Continuous Quality
SOASTA’s Lifecycle
Run iterative load
tests via Jenkins
Mobile? Run functional
automation on real devices
for functional & performance
validation
Spin up suitable, consistent
test environments in the cloud
or the lab
I JUST MADE A
CODE CHANGE
Jenkins runs any
unit tests as usual

29. © 2014 SOASTA. All rights reserved. 29
Continuous Quality
SOASTA’s Lifecycle
Run iterative load
tests via Jenkins
Daily (or more frequent) updates
keep teams quality focused
Embedded failure
details & bottleneck
data isolate issues
Mobile? Run functional
automation on real devices
for functional & performance
validation
Spin up suitable, consistent
test environments in the cloud
or the lab
I JUST MADE A
CODE CHANGE
Jenkins runs any
unit tests as usual

30. © 2014 SOASTA. All rights reserved. 30
Continuous Quality
SOASTA’s Lifecycle
Run iterative load
tests via Jenkins
Daily (or more frequent) updates
keep teams quality focused
Embedded failure
details & bottleneck
data isolate issues
Manage to a
performance
baseline
Mobile? Run functional
automation on real devices
for functional & performance
validation
Spin up suitable, consistent
test environments in the cloud
or the lab
I JUST MADE A
CODE CHANGE
Jenkins runs any
unit tests as usual

31. © 2014 SOASTA. All rights reserved. 31
Continuous Quality
SOASTA’s Lifecycle
Use real user data to
iterate dev & test plans
Run iterative load
tests via Jenkins
Daily (or more frequent) updates
keep teams quality focused
Embedded failure
details & bottleneck
data isolate issues
Manage to a
performance
baseline
Mobile? Run functional
automation on real devices
for functional & performance
validation
Spin up suitable, consistent
test environments in the cloud
or the lab
I JUST MADE A
CODE CHANGE
Jenkins runs any
unit tests as usual

33. SOASTA – Branch Build from CloudTest

34. SOASTA – Test Results

35. SOASTA – Drill Down (Test Failures)

36. SOASTA – UI Testing Bundle Results

37. SOASTA – UI Testing Failure Drill down into
SOASTA CloudTest from Jenkins

38. SOASTA – Successful regression test results

39. SOASTA – Release Promotion

40. © 2014 SOASTA. All rights reserved. 40
Continuous Quality for Continuous Delivery
o SOASTA’s Process Metrics
• 3 Main Products
• (2) Builds per day of each product (Dev branch + Customer branch)
• 8,000 tests executed per build
• For Mobile (TouchTest), Over 300 fully-automated tests each for iOS & Android
SOASTA - Takeaways

41. © 2014 SOASTA. All rights reserved. 41
The Path to Continuous Testing
o Understand your requirements
o Identify the cloud advantages for you
o Automate the obvious and most critical
o Connect to a continuous process
o Align teams with actionable information
SOASTA - Takeaways

42. © 2014 SOASTA. All rights reserved. 42
The Path to Continuous Testing
o Understand your requirements
o Identify the cloud advantages for you
o Automate the obvious and most critical
o Connect to a continuous process
o Align teams with actionable information
SOASTA - Takeaways

43. © 2014 SOASTA. All rights reserved. 43
The Path to Continuous Testing
o Understand your requirements
o Identify the cloud advantages for you
o Automate the obvious and most critical
o Connect to a continuous process
o Align teams with actionable information
SOASTA - Takeaways

44. © 2014 SOASTA. All rights reserved. 44
The Path to Continuous Testing
o Understand your requirements
o Identify the cloud advantages for you
o Automate the obvious and most critical
o Connect to a continuous process
o Align teams with actionable information
SOASTA - Takeaways

45. © 2014 SOASTA. All rights reserved. 45
The Path to Continuous Testing
o Understand your requirements
o Identify the cloud advantages for you
o Automate the obvious and most critical
o Connect to a continuous process
o Align teams with actionable information
SOASTA - Takeaways

46. © 2014 SOASTA. All rights reserved. 46CONFIDENTIAL – Not for Distribution
Final Takeaway
Why is CI important?
Revenue
Brand
Competitive advantage

47. © 2014 SOASTA. All rights reserved. 47
We will close with some Mobile Trivia
@DanBoutinSOASTA

48. © 2014 SOASTA. All rights reserved. 48
Questions?
Dan Boutin – Senior Product Evangelist
dboutin@soasta.com
Mobile (404) 304-9529
@DanBoutinSOASTA
o SOASTA TouchTest
• FREE DownLoad: http://goo.gl/6CErMN
o Visit our Blog: http://goo.gl/TA2HKP
o Questions? @DanBoutinSOASTA

49. May 28, 2015
South Florida Agile Association – Fort Lauderdale, FL
“It’s not called Continuous Integration for Nothing!”
Dan Boutin – Senior Product Evangelist
dboutin@soasta.com
Mobile (404) 304-9529
@DanBoutinSOASTA