Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
World®
’16
Shift-Left	Performance	Testing	
for	Shift-Forward	Quality
Karl	Miller	- Technical	Account	Manager	- CA	Technolo...
2 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
©	2016	CA.	All	rights	reserved.	All	trademarks	referenced	herein	bel...
3 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Abstract
Traditional	performance	testing	typically	requires	that	all...
4 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Agenda
PERSPECTIVE
CHALLENGES	WITH	TRADITIONAL	APPROACHES
Q	AND	A
RE...
5 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
DevTestOps
idea
AGILE	DEVELOPMENT AGILE	OPERATIONS
Establish	a	flawl...
6 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Why	Do	Apps	Perform	Poorly	in	Production?
Not	all	Components	are	
Te...
7 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Reinventing	Performance	Testing
Virtualize	Dependencies	of	System	Un...
8 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Service	Virtualization	for	Performance	Testing	of	
Multiple	Layers
A...
9 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Benefits	to	Performance	Testing
Virtual	Services	+	current	testing	t...
10 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Decompose	Performance	Expectations	“Into	the	Lifecycle”	
Verify
0.3...
11 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Decomposing	an	SLA	into	Performance	Expectations	at	
a	Component	Le...
12 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Make	Test	Environments	as	“Life-Like”	as	Possible
What’s	the	Differ...
13 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Don’t	Skimp	on	the	Data.	Surround	and	Capture!	
Why	is	90%	of	the	I...
14 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Virtual	Data	is	Generated	on	the	Basis	of	a	Message	
Definition	and...
15 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
+	Linking	Performance	Monitoring	and	Testing
16 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Virtual	Services	Support	Performance	Model
Virtual	Service	Model	pr...
17 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Regulate	Virtual	Service	Response	Time
CA	Service	Virtualization	ca...
18 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Demo
Stay	connected	at	communities.ca.com
19 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Save	Millions
in	infrastructure	costs	and	
3rd	party	fees
Detect	De...
20 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Deliver	Innovations…	Faster!	
TIME
Week	4Week	2Week	1
x
Week	6Week	...
21 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
99% reduction in testing times
-------------------
25% reduction in...
22 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
CA	Continuous	Delivery	Portfolio
Development OperationsTestPlanning...
23 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Questions?
24 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Resources
§ Next	Generation	Performance	Testing	Executive	Brief	– h...
25 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Recommended	Sessions
SESSION	# TITLE DATE/TIME
DO5T01S
Keynote:	End...
26 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Must	See	Demos
Modernize	
App	Delivery
Integrated	CD
Theater	5	- DO...
27 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Thank	you.
Stay	connected	at	communities.ca.com
@CAWORLD					#CAWORLD ©	2016	CA.	All	RIGHTS	RESERVED.28 @CAWORLD					#CAWORLD
DevOps	– Continuous	Delivery
For	more	inform...
Upcoming SlideShare
Loading in …5
×

Pre-Con Education: Shift-Left Performance Testing for Shift-Forward Quality

482 views

Published on

Pre-Con Education: Shift-Left Performance Testing for Shift-Forward Quality

Published in: Technology
  • HOT MILFS LOOKING FOR SEX! F.U.C.K A MILF NEAR YOU TONIGHT!●●● http://t.cn/AiuWKDWR
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Have you ever heard of taking paid surveys on the internet before? We have one right now that pays $50, and takes less than 10 minutes! If you want to take it, here is your personal link ■■■ http://ishbv.com/surveys6/pdf
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • I made $2,600 with this. I already have 7 days with this... ★★★ http://ishbv.com/surveys6/pdf
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Pre-Con Education: Shift-Left Performance Testing for Shift-Forward Quality

  1. 1. World® ’16 Shift-Left Performance Testing for Shift-Forward Quality Karl Miller - Technical Account Manager - CA Technologies DO5X10E DEVOPS
  2. 2. 2 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD © 2016 CA. All rights reserved. All trademarks referenced herein belong to their respective companies. The content provided in this CA World 2016 presentation is intended for informational purposes only and does not form any type of warranty. The information provided by a CA partner and/or CA customer has not been reviewed for accuracy by CA. For Informational Purposes Only Terms of this Presentation
  3. 3. 3 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Abstract Traditional performance testing typically requires that all components of the application are “completed,” integrated and deployed into an appropriate environment. This results in testing not being done until late in the delivery cycle or sometimes skipped entirely. Which can then lead to a less then optimal user experience, expensive rework and potential loss of business. Many organizations have adopted service virtualization to overcome the key challenges associated with performance testing. During this session we’ll show you why and specifically how service virtualization enables you to do testing early in the dev cycle by simulating unavailable production systems and missing components, helps you control the inputs (like response times and 3rd party system responses) so you can do more negative and exploratory testing, and enables testing and development to work in parallel. Karl Miller CA Technologies Technical Account Manager, Continuous Delivery
  4. 4. 4 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Agenda PERSPECTIVE CHALLENGES WITH TRADITIONAL APPROACHES Q AND A REINVENTING PERFORMANCE TESTING WITH CA TECHNOL0GIES ADVANTAGES AND REAL WORLD EXAMPLES DEMO 1 2 3 4 5 6
  5. 5. 5 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD DevTestOps idea AGILE DEVELOPMENT AGILE OPERATIONS Establish a flawless customer experience with apps optimized for performance Agile aligned testing supported with advanced technology Eliminate constraints and dependencies across the SDLC CUSTOMER EXPERIENCE AGILE TESTING Agility across all 3 increases speed, reliability and efficiency
  6. 6. 6 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Why Do Apps Perform Poorly in Production? Not all Components are Tested1 Dev/Test Environments Don’t Mirror Production2 Systems and Data are not Available for Testing3
  7. 7. 7 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Reinventing Performance Testing Virtualize Dependencies of System Under Development/Test1 Leverage Service Virtualization and Drive Testing at the API Layer3 Move Functional Virtual Services to Performance Server4 Convert Functional Tests to Performance / Concurrency Tests5 Create Automated Functional Tests at the Component Level2 Performance Test System Under Development backed by Virtual Services beyond Peak Loads 6
  8. 8. 8 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Service Virtualization for Performance Testing of Multiple Layers APPLICATION SERVER System Under Test Load Testing Tools System Under Test Load Testing Tools All Operations Virtualized ESB LEGACY INTERFAC E ORACLE DB2 I-SERIES MAINFRAME JDBC Oracle JDBC DB2 TCP/IP with XML Respons e TCP/IP Fixed Length HTTPS Custom Data Protocol
  9. 9. 9 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Benefits to Performance Testing Virtual Services + current testing tools CHALLENGE Creating a lab capable of handling production capacity loads may not be realistically possible. APPLICATION “X” CURRENT TESTING TOOLS Thousands of transactions BillMeLater PayPal TNT Express 3rd Party Service Load Test at the Component Level Test for Volume and Capacity Independently Locate Bottlenecks early Virtualize 3rd Party Systems, Mainframe Data: Always Ready, Highly Scalable Immune to Availability and Constraint Issues Vary the response times from the virtual services! Run thousands of transactions through the actual application on the actual hardware with NO IMPACT on your existing systems!
  10. 10. 10 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Decompose Performance Expectations “Into the Lifecycle” Verify 0.3s Lookup 0.5s Response 1.2s Application SLA: 2.1 Seconds
  11. 11. 11 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Decomposing an SLA into Performance Expectations at a Component Level Test 1 Component Performance Testing Against VSE Response 1.2s Verify Lookup Application 0.9s Virtual Service Environment (VSE) Virtual Service
  12. 12. 12 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Make Test Environments as “Life-Like” as Possible What’s the Difference Between Stubs, Mocks, and Service Virtualization?
  13. 13. 13 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Don’t Skimp on the Data. Surround and Capture! Why is 90% of the IT Development & Test cost weighing down delivery? In-Scope System: Environments under your team’s control Out-Scope System: Data, services and apps you depend on = 1 = 10- 1000x Dev & Test Teams System under Dev In-Scope Data App 1 App 2 App 3 App 4 App 5 Saas Out-of-Scope Dependencies
  14. 14. 14 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Virtual Data is Generated on the Basis of a Message Definition and Injected into a Deployed Virtual Service
  15. 15. 15 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD + Linking Performance Monitoring and Testing
  16. 16. 16 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Virtual Services Support Performance Model Virtual Service Model provides companions that can be configured for Virtual Service Think Scale, Batch Response and Recurring Think time scale to vary the performance of the Virtual Service over a period. Companions for Performance model in Virtual Service Model
  17. 17. 17 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Regulate Virtual Service Response Time CA Service Virtualization can regulate request timing and simulate slow network. Think time spec, specifies the latency in ms Think time spec specifies the response latency. This can be scaled using Think time scale during deployment or using Companions. High Think time spec or Think time scale can simulate slow network behavior.
  18. 18. 18 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Demo Stay connected at communities.ca.com
  19. 19. 19 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Save Millions in infrastructure costs and 3rd party fees Detect Defects earlier in the SDLC Eliminate capacity constraints for performance testing Reduce delays with high-performance masking and subsetting, and automated data discovery Application “X” Test Cases Thousands of transactions Agent Performance Testing in a Box Innovate Faster while Delivering a High Quality Experience ü Use the same test cases created and optimized for functional testing ü Simulate backend load by running test thru an API ü Simulate life like test environments ü Generate purpose built test data ü Control outlier conditions ü Adjust think times
  20. 20. 20 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Deliver Innovations… Faster! TIME Week 4Week 2Week 1 x Week 6Week 3 DEV 1 DEV 2 DEV 3 DEV 1 DEV 2 DEV 3 DEV 1 DEV 2 DEV 3 DEV 1 DEV 2 DEV 3 DEV 1 DEV 2 DEV 3 DEV 1 DEV 2 DEV 3 App 3 App 2 App 1 Week 7Week 5 Sequential Parallel App 3 App 2 App 1 Accelerate Application Delivery Simulate needed systems … end-to-end Create virtual services … easily Increase developer productivity 1 2 3 Cut of Set up time for dev environments (from 42 days to 3) Reduced Time-to-Market by for key applications. http://bcove.me/4rx4wt6x 30 Days 15% http://bit.ly/SV-TelefonicaChile
  21. 21. 21 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD 99% reduction in testing times ------------------- 25% reduction in software defects Business Autotrader sells advertising that allows private and trade buyers and sellers to exchange vehicles. More than 18 million people use Auto- Trader.com each month. Challenge To meet customer demand for new online features and accessible across multiple devices, Autotrader needed to release software updates on a weekly basis. Testing, however, had become complex, costly and time- consuming. Solution CA Service Virtualization enables Autotrader to virtualize and test new services and apps on a multitude of devices, while simulating both apps working normally and while experiencing performance issues. CA Services completed implementation and training in just six weeks. Autotrader Avoids $300,000 in Testing Costs with CA Service Virtualization Autotrader is improving the customer experience with faster time-to-market. “The CA Technologies team blew the competition away with their knowledge and passion.”
  22. 22. 22 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD CA Continuous Delivery Portfolio Development OperationsTestPlanning Deploy CA Agile Requirements Designer CA Test Data Manager CA Continuous Application Insight CA Application Test CA Release Automation Agile Ops Agile Central CA Service Virtualization CA Release Automation Standard + Continuous Delivery Edition APIM APM
  23. 23. 23 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Questions?
  24. 24. 24 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Resources § Next Generation Performance Testing Executive Brief – http://bit.ly/2alTsrr § Forrester SV TEI Study – http://bit.ly/CASV-TEI § Service Virtualization Today – www.servicevirtualization.com § Gartner Market Guide for API Testing and Virtualization – http://bit.ly/Gartner_SV-Guide § Key Capabilities of CA Service Virtualization Whitepaper - http://bit.ly/CASV_WP § Real Customers. Real Results – Service Virtualization customer success stories www.ca.com/CDResults § IT Central Station – Real User Reviews and Comparison Guide https://www.itcentralstation.com/products/ca-service-virtualization § CA Service Virtualization Home Page - http://www.ca.com/us/products/ca- service-virtualization.html?intcmp=headernav
  25. 25. 25 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Recommended Sessions SESSION # TITLE DATE/TIME DO5T01S Keynote: End-to-End Integrated Continuous Delivery Ecosystem: How to Drive Innovation and Differentiation 11/16/2016 at 10:30 am DO5T02P Leadership Panel: DevOps Leaders Discuss the Journey to Continuous Delivery 11/16/2016 at 01:45 pm DO5T03P Leadership Panel: Continuous Delivery in the Financial Services Industry 11/16/2016 at 04:30 pm
  26. 26. 26 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Must See Demos Modernize App Delivery Integrated CD Theater 5 - DOV501P Deliver Test Data Faster Test Data Manager Theater 5 - DOV511P Deliver Better Apps Service Virtualization Theater 5 - DOV507P Orchestrate Your Release Release Automation Theater 5 - DOV513P
  27. 27. 27 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Thank you. Stay connected at communities.ca.com
  28. 28. @CAWORLD #CAWORLD © 2016 CA. All RIGHTS RESERVED.28 @CAWORLD #CAWORLD DevOps – Continuous Delivery For more information on DevOps – Continuous Delivery, please visit: http://cainc.to/PiTFpu

×