This session presented by Oracle and a customer, provides an overview of how the quality assurance team tests Oracle E-Business Suite. The session covers main areas that you should consider during functional testing, approaches for new feature and regression testing, how to reduce the test script generation and execution time, experiences on capturing and presenting metrics to showcase the ROI of a testing investment, leveraging automation for testing Oracle E-Business Suite applications, and more.

2. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Testing Oracle E-Business Suite
Best Practices
Prasanti Madireddi
Senior Director
Oracle, E-Business Suite Quality Assurance
Jake Westphal
Senior IT Manager
First American, Enterprise Applications
October 26, 2015
Oracle Confidential – Internal/Restricted/Highly Restricted

3. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
3

4. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
Industry 4.0 & EBS
Testing Challenges
Recommendations & Tools
First American Customer Story
1
2
3
4
4

5. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Industry 4.0
1.0
Power
Generation
2.0
Mass
Production
3.0
Electronic
Automation
4.0
Smart
Automation
5
1782 1913 1954 2015

6. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 6
Business Transformation

7. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Industry Drivers
Business
Transformation
Mobile
Analytics
CloudSocial
IOT
7

8. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 8
EBS Investment Priorities

9. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
Industry 4.0 & EBS
Testing Challenges
Recommendations & Tools
First American Customer Story
1
2
3
4
9

10. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Testing Challenges
• Fast growing businesses require quick go-live turnaround
• Faster adoption rate of new technologies
• Upgrades are rich with new features and new technology
• Increasing complexity with different platforms/technologies/UI
• Testing to match consumer’s usage pattern
10

11. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Testing Challenges – Smartphone Applications
11
Functional
User Interface
Performance
Security
Translation
Installation
Accessibility
Usability
Operating Systems
OS Versions
Devices
Device Models
Networks

12. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Testing Challenges – In Memory Applications
12
Functional
User Interface
Performance
Translation
Security
Accessibility
Usability
Data Setup
ETL (EBS & Standalone)
Configurations
User Interface

13. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Testing Challenges – Information Discovery
13
Functional
User Interface
Performance
ETL
Translation
Security
Accessibility
Usability
Multi-layer validations
Synchronization
Data Intensive
Data Integrity
High Level of Precision

14. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Testing Challenges
Delivering Quality Product
Impacted
Area
14

15. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
Industry 4.0 & EBS
Testing Challenges
Recommendations & Tools
First American Customer Story
1
2
3
4
15

16. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
Analyze
Design
BuildTest
Release
Quality Focus in SDLC
Prioritize and freeze requirements
Incorporate feedback from previous
release
Analyze
Design
BuildTest
Release
Analyze
Design
BuildTest
Release
Analyze
Design
BuildTest
Release
Analyze
Design
BuildTest
Release
Document deliverable review
Approval by all stakeholders
Code review and inspection
Unit testing by developers
Manual testing
Automated testing
Perform internal audits
Post mortem analysis of release
16

17. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Analyze
Design
BuildTest
Release
Recommendations & Tools
Quality Focus in SDLC
Prioritize and freeze requirements
Incorporate PMA feedback from
previous release
Document deliverable review
Approval by all stakeholders
Code review and inspection
Unit testing by developers
Manual testing
Automated testing
Perform internal audits
Post mortem analysis of release
17

18. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
18
Organization Alignment

19. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
Formulate test
scenarios
Prioritize based
on usage
patterns
Optimize test
scenarios
Create test data
volume and
variety
Testing
workshops for
group and role
based testing
19

20. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
Formulate test
scenarios
Prioritize based
on usage
patterns
Optimize test
scenarios
Create test data
volume and
variety
Testing
workshops for
group and role
based testing
20
Release
Content
Document
Functional
Impact
Assessment
Document

21. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
Formulate test
scenarios
Prioritize based
on usage
patterns
Optimize test
scenarios
Create test data
volume and
variety
Testing
workshops for
group and role
based testing
21

22. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
Formulate test
scenarios
Prioritize based
on usage
patterns
Optimize test
scenarios
Create test data
volume and
variety
Testing
workshops for
group and role
based testing
22
Factors and Levels All Possible Combinations Pair Wise Combinations
5 factors at 3 levels each 243 11
6 factors at 4 levels each 4096 23
7 factors at 6 levels each 279,936 56
10 factors at 7 levels each 282,475,249 89

23. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
Formulate test
scenarios
Prioritize based
on usage
patterns
Optimize test
scenarios
Create test data
volume and
variety
Testing
workshops for
group and role
based testing
23
API/OI/WS

24. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
Formulate test
scenarios
Prioritize based
on usage
patterns
Optimize test
scenarios
Create test data
volume and
variety
Testing
workshops for
group and role
based testing
24

25. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
25
Automation
Who
• Everyone!
• Functional engineers
can assemble the
flows
• Automation engineers
can stabilize and add
validation points
What
• Critical sanity flows
• High priority flows
• Most repeatable
functional feature
flows
When
• Should be on the high
priority list
• Product features
should be stable
• Like a vacation,
anytime is good time
for automation

26. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Application Quality Management
Solutions to Manage Quality
26

27. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
• Load and Performance Style Testing
• Web and Oracle Application Testing Accelerators
• Smart Integration with EM Diagnostics
• Test Planning, Requirements and Test Cases
• Defects Tracking
• Reporting
• Functional and Regression Testing
• Web and Oracle Application Testing Accelerators
• Oracle Flow Builder and Starter Pack
Oracle Application Testing Suite
Overview
Oracle Functional Testing Suite for
Oracle Applications
Oracle Load Testing Suite
for Oracle Applications
Oracle Test Manager
27

28. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle Functional Testing Suite for Oracle Applications
Oracle Flow Builder and EBS Pre-built Test Content
• Use Flow Builder to Generate Test Scripts
• Start with EBS Pre-Built Test Content
• Create and Maintain Your Test Scripts
– Add Your Flows & Data
– Extend with More Components
– Support Customizations
– Non technical resources
Record
Manually
Generate
Test
Automation
Scripts
OpenScript
Record & Playback
+ Advanced Scripting
Flow Builder
Framework to combine
flows and data to generate
test scripts
EBS Pre-Built
Test Content
Your
Data
Your
Flows
OracleFunctionalTestingSuite
forOracleApplications
EBS Quality Assurance Team
Reduced Automation Time by 60%
28

29. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle Flow Builder: EBS Pre-Built Test Content
Starter Pack
• Commonly used base functionality
• For creating high level certification flows
• Broad coverage to jump start the testing
For Both 12.1.3 and 12.2
2000+ Components and 200+ Flows across EBS
ProcurementFinancials
ManufacturingLogistics Service
Master Data
Management
Order
Management
29
Asset
Lifecycle
Management
Projects HCM
3600+ components and 230+ flows across EBS
Logistics ManufacturingFinancials
ProjectsProcurement
Product
Information
Management Service
Human Capital
Management
Order
Management
Asset
Lifecycle
Management

30. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle Flow Builder: EBS Pre-Built Test Content
Advanced Pack
• Niche products which stand alone and deep
functionality
• For creating detailed and complex test flows
• Significant number of components above Starter Pack
(95% coverage)
For Both 12.1.3 and 12.2
30
10 product families / products across EBS
Incentive
Compensation
Discrete
Manufacturing
Channel
Revenue
Management
Project
ManagementProcurement
Process
Manufacturing
Property
Manager Service
Lease /Finance
Management
Asset
Lifecycle
Management

31. PROCESS • ANALYTICS • TECHNOLOGY 31© 2014 Copyright Genpact. All Rights Reserved.
“ Overall, Oracle EBS team is simply awesome to work with. Incredibly flexible, easy to communicate with, prompt responsive
and goes beyond the boundaries to recommend best solution. It was phenomenal partnership in sales and
implementation….” – Genpact Leadership
Oracle Flow Builder has been instrumental in building long term regression test suite for our customers.
It very well supports the Standard Oracle Forms and Web pages. It is also scalable to customizations which is incredible.
The standard Components and Flows that come along with the basic package have a very good coverage in terms of types
of transactions performed in E-Business Suite. Easily Scalable.
Very High Reusability as it offers simple Java based coding for developing new custom components or modify
pre-built component for specific business flows.
Open Script and Flow Builder supports iterative runs and helps perform complex and time consuming setups, reduces time
during cutover……..Less System Downtime.
Oracle Flow Builder and ORACLE EBS Team ---- Testimonial

32. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
Industry 4.0 & EBS
Testing Challenges
Recommendations & Tools
First AmericanCustomer Story
1
2
3
4
32

33. First American Title Insurance Company makes no express or implied warranty respecting the information presented and assumes no responsibility for errors or omissions.
First American, the eagle logo, and First American Title are registered trademarks or trademarks of First American Financial Corporation and/or its affiliates.
The following presentation is for informational purposes only and is not and may not be construed as legal advice. First American Title Insurance Company is not a law firm
and does not offer legal services of any kind. No third party entity may rely upon anything contained herein when making legal and/or other determinations regarding title
practices. You should consult with an attorney prior to embarking upon any specific course of action.
©2015 First American FinancialCorporation and/or its affiliates. All rights reserved. q NYSE: FAF
First American Customer Story
Jake Westphal

34. ©2015 First American FinancialCorporation and/or its affiliates. All rights reserved. q NYSE: FAF
Enterprise Application Services (EAS)
EAS is a part of the Corporate IT organization and specializes in managing Enterprise Applications.
Products
Oracle R12 Financials (EBS) Insbridge
Oracle Fusion Master Data Management (MDM) Oracle Application Testing Suite (OATS)
Oracle Fusion Procurement User Productivity Kit (UPK)
Oracle EPM (HFM/HPM) Oracle Identity Management
Oracle BI/Reporting solutions/OBIEE Oracle Identity Analytics

35. ©2015 First American FinancialCorporation and/or its affiliates. All rights reserved. q NYSE: FAF
Testing Challenges
Analyze
Design
Build
Test
Release
- Manual testing vs Automation
- Transforming the team
- Automation Impact/Benefit
- Who is involved in test plan review
- Who approves the scope
- How soon to start building test plan
- How to maintain the automation scripts
- How to find ways to grow/mature scripts
- Incorporate Framework/best practices
- How much do we test
- Have we tested this before
- Do I need full regression testing
- What’s the right team size
- What's the skill diversity
- What's the right shift/location

36. ©2015 First American FinancialCorporation and/or its affiliates. All rights reserved. q NYSE: FAF
Common Test Management Challenges
• No centralized repository for test cases - difficult to track/manage/update the test cases
• Lack of maintenance and accessibility of artifacts such as test cases & test results
• Loss of testing artifacts over time due to changes in test cases, process & employee turnover
• Difficulty in tracking the progress of test cases/scenarios during testing cycles
• Gaps in test plans, test some, test all
• No traceability between requirements to test cases to scenarios to defects
• Inconsistency in structure of defect reporting by various testers
• Testers assigning improper severity/priority to defects-duplicate defects opened
• Lack of adequate test result reporting, PMO dashboards,..

37. ©2015 First American FinancialCorporation and/or its affiliates. All rights reserved. q NYSE: FAF
R12/OATS Timeline
February 2015 June 2015 August 2015August 2014
Vendor supported
OATS Implementation
Performance
Test Cycles
R12 Project Test
Cycle Kickoff
Leveraging Automation
for Testing
Build Full Automation Regression Set
Designed Test
Management
Process
Determine
automation scope
R12 Go Live
December 2015
Future application
Automation consideration
May 2014
Designed/Implemented
test script mgmt process
Implemented Monthly
Regression Suite
Testing/Maintenance
Continued Maintenance of Automation Scripts
January 2016

38. ©2015 First American FinancialCorporation and/or its affiliates. All rights reserved. q NYSE: FAF
Lessons Learned
• Identify types of tests to be performed/requirement traceability
• Gather details about testing priorities, focus, scope, impact analysis,….
• Building/documenting/applying QA process to support the testing approach/framework
• Testing tool - Automation and Manual
• Identify regression test cases early for automation and performance testing
• Management of different testing cycles (Integration, User Acceptance, Performance…)
• Audit value in traceability of requirements, Test scripts, Test Scenarios & Defects
• Managing the testing activities
(time spent in test design, test execution, whether we are on schedule or on budget)

39. ©2015 First American FinancialCorporation and/or its affiliates. All rights reserved. q NYSE: FAF
QA Test Automation Values
• Automation eliminated the need for module specific testers
• Full regression execution time is reduced 40% with Oracle Application Testing Suite
• Robust automation scripts = less script maintenance
• Auto creation of test scenarios from Flow Builder saved time & brought consistency
• Load testing simulated 500 concurrent users /eliminated need for human intervention
• Established Automation framework

40. ©2015 First American FinancialCorporation and/or its affiliates. All rights reserved. q NYSE: FAF
Oracle Test Manager
Brings structure, repeatability and visibility to the testing efforts
• 5 integrated modules for total
test process management
• Test execution scheduling
• Common repository for all assets
• Web-based interface
• Manual and automated tests
• User-defined roles and fields
• Increased visibility of test readiness

41. ©2015 First American FinancialCorporation and/or its affiliates. All rights reserved. q NYSE: FAF
Oracle Test Manager Key Values
• Tool is re-usable from a test management perspective
• Central repository of test material/artifacts can be ported for new projects/ test cycles
• Tool is scalable for rollout across multiple user groups & products
• Ease of user administration functionality
• Custom fields provide flexibility to capture additional data - customizable to specific project
• Very easy to user/tester friendly
• Import feature for loading data helps fast track data entry
• Supports both Oracle and non-Oracle web based applications
• Tool is configurable to track requirements, test cases, test scripts & defect management
• Facilitates integration with 3rd party test tools
• Tool has multiple modules to store and maintain respective testing artifacts

42. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Related Sessions
Strategy & Roadmap
Monday
Oct 26
4:00 p.m.
CON6413 - Oracle E-Business Suite Technology: Latest Features and Roadmap
Lisa Parekh, Vice President, Oracle
Moscone
West—3002
Tuesday
Oct 27
11:00 a.m.
GEN6409 - General Session: Oracle E-Business Suite Update, Strategy, and Roadmap
Clifford Godwin, Senior Vice President, Oracle
Moscone
West—2008
Tuesday
Oct 27
5:15 p.m.
CON6410 - Oracle E-Business Suite: Mobile Update, Strategy, and Roadmap
Jeanne Lowell, Vice President, EBS Product Strategy, Oracle
Moscone
West—3022
Thursday
Oct 29
12:00 p.m.
CON8136 - Oracle E-Business Suite Technology Certification Primer and Roadmap
Steven Chan, Senior Director, Oracle
Moscone
West—3022
42

43. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Visit Us
• Demo Pod – WCL-002 - Automated Oracle E-Business Suite Tests Using
Oracle Flow Builder
• Moscone West – Exhibition Hall
Mon, Oct 26: 10:15 a.m. – 6:00 p.m.
Tue, Oct 27: 10:15 a.m. – 6:00 p.m.
Wed, Oct 28: 10:15 a.m. – 4:15 p.m.
43

44. Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Questions
• Prasanti Madireddi – prashanti.madireddi@oracle.com
• Jake Westphal – awestphal@firstam.com
44