Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
World®
’16
Case	Study:	American	Family	Insurance	Best	
Practice	for	Automating	Guidewire	Testing
Sanket Shah	- Test	Automa...
2 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
©	2016	CA.	All	rights	reserved.	All	trademarks	referenced	herein	bel...
3 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Abstract
Sanket Shah	(Lead	Test	Automation	Engineer)	discusses	how	h...
4 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Agenda
CASE	BACKGROUND
CHALLENGES	FACED
Q	&	A
INITIAL	SOLUTION
DRAWB...
5 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Case	Background
6 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Billing	and	Policy	Themes	at	Amfam
Strategic	to	the	
Business
Optimi...
7 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Auto	and	Property	Casualty	Policy	and	Billing	Testing	Goals
POLICY
§...
8 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Current	Legacy	Systems
Auto+
Property+
Customer	Billing	System	(CBS)...
9 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Migrating	Legacy	Systems	to	Open	Systems
Auto+	and	Property+ Custome...
10 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Guidewire	Policy	Center
Issuance
Changes	(charges/credits)
Cancella...
11 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Guidewire	Billing	Center
§ Generate	and	bill	invoices
§ Scheduling	...
12 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
A	Policy’s	Lifespan
6	to	12	months
12	months
13 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
PolicyCenter-BillingCenter Integration	(Pre-Prod/Prod)
PolicyCenter...
14 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Challenges	Faced
15 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
…in	Testing	BillingCenter in	Pre-prod	Tiers
Different	teams	testing...
16 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Initial	Solution
17 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
…to	Test	Clock	Dependent	Scenarios	in	BillingCenter
BC	(deployed	on...
18 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Steps	Involved
19 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
…in	Setting	Up	BillingCenter on	Local	for	Testing	Each	
Scenario
St...
20 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Drawback
21 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
…for	Setting	Up	on	Local
Execution	time	for	
each	scenario:	
~9-10	...
22 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Solution	à Retester	Framework
23 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
What	is	Retester
§ In-house	developed	automation	regression	framewo...
24 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
…a	Comprehensive	BillingCenter Application	Testing	Tool
MS	Access	f...
25 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Retester	Framework	– High	level	workflow
START
Get	list	of	business...
26 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Retester	Framework	– Detail	Flow
START
Initialize	day	
counter
No
Y...
27 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
DevTest	from	CA	and	Retester	Framework
Uses	PolicyCenter/	
BillingC...
28 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Questions?
29 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Recommended	Sessions
SESSION	# TITLE DATE/TIME
DO5T02P
Leadership	P...
30 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Must	See	Demos
Modernize	
App	Delivery
Integrated	CD
Theater	5	- DO...
31 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
Stay	connected	at	communities.ca.com
Thank	you.
32 ©	2016	CA.	ALL	RIGHTS	RESERVED.@CAWORLD				#CAWORLD
DevOps	– Continuous	Delivery
For	more	information	on	DevOps	– Conti...
Upcoming SlideShare
Loading in …5
×

Case Study: American Family Insurance Best Practices for Automating Guidewire functional, regression and performance Testing.

778 views

Published on

Case Study: American Family Insurance Best Practices for Automating Guidewire functional, regression and performance Testing.

Published in: Technology
  • Be the first to comment

Case Study: American Family Insurance Best Practices for Automating Guidewire functional, regression and performance Testing.

  1. 1. World® ’16 Case Study: American Family Insurance Best Practice for Automating Guidewire Testing Sanket Shah - Test Automation Architect - American Family Insurance DO5X20S DEVOPS
  2. 2. 2 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD © 2016 CA. All rights reserved. All trademarks referenced herein belong to their respective companies. The content provided in this CA World 2016 presentation is intended for informational purposes only and does not form any type of warranty. The information provided by a CA partner and/or CA customer has not been reviewed for accuracy by CA. For Informational Purposes Only Terms of this Presentation
  3. 3. 3 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Abstract Sanket Shah (Lead Test Automation Engineer) discusses how his 30 member test automation team facilitates Guidewire functional and regression testing as well as best practices. Automation is an accelerator, which fuels consistency, robustness and reliability in an environment where tasks are to be repeated. One of the key advantages of it is in automated regression testing, which saves a lot of effort in carrying out the same testing activities repeatedly. Identifying any impacted integrated systems or the source system due to change(s) in features/functionality and/or addition of feature/functionality in a software product, would be a daunting task if carried out manually (especially when we want consistency and when the AUT is integrated with several services at the application service layer). Leveraging automated regression testing would take the building up of API to a whole new level, thus accelerating the user’s journey through their web application. Sanket Shah American Family Insurance Test Automation Architect
  4. 4. 4 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Agenda CASE BACKGROUND CHALLENGES FACED Q & A INITIAL SOLUTION DRAWBACK ULTIMATE SOLUTION 1 2 3 4 5 6
  5. 5. 5 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Case Background
  6. 6. 6 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Billing and Policy Themes at Amfam Strategic to the Business Optimizes Customer Experience Transformational to the business Supports a sophisticated pricing model Enhances Amfam's brand and market leadership
  7. 7. 7 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Auto and Property Casualty Policy and Billing Testing Goals POLICY § Reduce underwriting costs by optimizing test case efficiencies § Improve application quality / reduce risk and impact of defects BILLING § Optimize customer e-Commerce and Agent application experience (i.e., ensure a high quality user experience) § Reduce testing risk and duration: Build automated test cases to support the myriad of business processes (e.g., billing plans, exception handling, etc.)
  8. 8. 8 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Current Legacy Systems Auto+ Property+ Customer Billing System (CBS) POLICY MANAGEMENT BILLING
  9. 9. 9 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Migrating Legacy Systems to Open Systems Auto+ and Property+ Customer Billing System (CBS)
  10. 10. 10 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Guidewire Policy Center Issuance Changes (charges/credits) Cancellations (non-pay/insured) Reinstates Rewrites Renewals
  11. 11. 11 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Guidewire Billing Center § Generate and bill invoices § Scheduling invoices § Accepting and applying payments § Payment reversals § Cancellation initiation (non-pay) § Delinquencies § Third party interest billing management § Disbursements
  12. 12. 12 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD A Policy’s Lifespan 6 to 12 months 12 months
  13. 13. 13 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD PolicyCenter-BillingCenter Integration (Pre-Prod/Prod) PolicyCenter BillingCenter Database Integration 1Integration 2 Integration 3 Integration 4 Integration 5 Integration 6 Integration 7 Integration 8 Integration 9
  14. 14. 14 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Challenges Faced
  15. 15. 15 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD …in Testing BillingCenter in Pre-prod Tiers Different teams testing different application functionalities/ themes at the same time, which are clock dependent. Test scenarios to test a policy’s state at a given point in time. Clock dependency of BC in pre-prod
  16. 16. 16 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Initial Solution
  17. 17. 17 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD …to Test Clock Dependent Scenarios in BillingCenter BC (deployed on local)Simulator (simulating PC transactions using spreadsheets as data source and set of batch files to load them) L O C A L H2
  18. 18. 18 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Steps Involved
  19. 19. 19 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD …in Setting Up BillingCenter on Local for Testing Each Scenario Stop BC if started Drop DB Start BC Load initial data Execute test case (~2-3 sec) (~1 sec) (~5 min) (~2 min) (Repeat the flow for each subsequent test scenario)
  20. 20. 20 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Drawback
  21. 21. 21 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD …for Setting Up on Local Execution time for each scenario: ~9-10 min Execution time for 100 scenarios: ~15-16 min
  22. 22. 22 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Solution à Retester Framework
  23. 23. 23 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD What is Retester § In-house developed automation regression framework § In-depth verification of billing lifecycle of a policy § Simulation of policy transactions through APIs § Business friendly data entry forms § Production simulation of a real life business transaction § A layer of solidity before rolling out to larger audience § Robust, fast, consistent, repeatable
  24. 24. 24 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD …a Comprehensive BillingCenter Application Testing Tool MS Access forms to enter scenarios Oracle to store scenarios data and results Automation tool as master controller
  25. 25. 25 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Retester Framework – High level workflow START Get list of business scenarios to be executed Execute pre-batch events for current day- all scenarios Validate data for the events pre- batch and log result Advance clock and run batch Execute post- batch events for current day – all scenarios Validated data for the events executed post- batch and log results Log events result and validate result in a database and continue with next day End of Test? No Yes At end of the test, send completion email with passed/fail stats Log events result, validate result in database and continue with next day END
  26. 26. 26 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Retester Framework – Detail Flow START Initialize day counter No Yes Get new test ID for each execution Get max number of days to execute currDay = currDay + 1 Get total scenarios count for currDay Scenarios exist for currDay? End of Days? Initialize scenario counter currSce = currSce + 1 A A End of scenari os for curr Day? Get total pre- batch events for currDay and currSce Pre- batch events exist? Initialize event counter * * currEve = currEve + 1 A Advance clock and run EOD batch No Yes Scena rios Exist? No No Get total post batch events for currDay and currSce Yes Post- batch events exist? No Yes Yes B END
  27. 27. 27 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD DevTest from CA and Retester Framework Uses PolicyCenter/ BillingCenter APIs to initiate transactions Customized APIs to validate data on the BillingCenter application Creating new scenarios is as simple as filling up a form Cycles through 100s of scenarios in < 2 hrs. Gain of 85%
  28. 28. 28 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Questions?
  29. 29. 29 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Recommended Sessions SESSION # TITLE DATE/TIME DO5T02P Leadership Panel: DevOps Leaders Discuss the Journey to Continuous Delivery 11/16/2016 at 01:45 pm DO5T03P Leadership Panel: Continuous Delivery in the Financial Services Industry 11/16/2016 at 04:30 pm DO5T14S Continuous Delivery: Making DevOps Awesome Nicole Forsgren – Theater 5 11/17/2016 at 10:30 am
  30. 30. 30 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Must See Demos Modernize App Delivery Integrated CD Theater 5 - DOV501P Deliver Test Data Faster Test Data Manager Theater 5 - DOV511P Deliver Better Apps Service Virtualization Theater 5 - DOV507P Orchestrate Your Release Release Automation Theater 5 - DOV513P
  31. 31. 31 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD Stay connected at communities.ca.com Thank you.
  32. 32. 32 © 2016 CA. ALL RIGHTS RESERVED.@CAWORLD #CAWORLD DevOps – Continuous Delivery For more information on DevOps – Continuous Delivery, please visit: http://cainc.to/PiTFpu

×