OOW16 - Testing Oracle E-Business Suite Best Practices [CON6713]

1. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Testing Oracle E-Business Suite Best Practices Keeping Up with Change… Confidential – Oracle Internal/Restricted/Highly Restricted Presented with Mazen Arar Senior Director, Oracle Sarvesh Tiwari Vice President, Genpact September 21, 2016

2. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Confidential – Oracle Internal/Restricted/Highly Restricted 3

3. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Program Agenda EBS Evolution Testing Readiness Recommendations & Tools Customer Story 1 2 3 4 4Confidential – Oracle Internal/Restricted/Highly Restricted

4. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | EBS Evolution Change is Constant • Change happens every day • In today’s business world, it is the only thing that is consistent • Long-term plans may become outdated and possibly obsolete within months • Business plans must be reviewed, updated, and revised constantly 5Confidential – Oracle Internal/Restricted/Highly Restricted

5. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | EBS Evolution Oracle E-Business Suite in the Digital World Cloud Social Mobile IoTBig Data Talent Mgmt Revenue Mgmt Recruiting Innovation MgtCommerce Financial Reporting Global Trade Mgmt Enterprise Planning Service Transportation Mgmt Sales Marketing 6Confidential – Oracle Internal/Restricted/Highly Restricted

7. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Testing Readiness Challenging Facts • Continuous development calls for continuous testing • With ongoing EBS innovation, all teams need to be aligned • Different types of activities require different strategy for testing • QA teams have to be pro-active not re-active • Focusing on the present and planning for the future 8Confidential – Oracle Internal/Restricted/Highly Restricted

8. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Testing Readiness Ongoing EBS Innovation • New features / Products – EBS 12.1 – EBS 12.2 – Modern / Simplified UI • Mobile • In Memory Applications • Big Data - Information Discovery • Cloud • IoT 9Confidential – Oracle Internal/Restricted/Highly Restricted

9. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Testing Readiness Strategy 10 Smartphone Applications In Memory Applications Information Discovery Cloud IoT Confidential – Oracle Internal/Restricted/Highly Restricted

13. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Recommendations & Tools • What is the task on hand • Available resources – Release Content Document • Information about new or changed functionality • List of bugs included – Functional Impact Assessment document • Functional changes • Functional impacts • Testing recommendations 14 Plan Confidential – Oracle Internal/Restricted/Highly Restricted

14. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Recommendations & Tools 15 Evaluate • Task on hand Review • Documentation Scope • Key business flows • Impacted Flows • Usage patterns Plan Confidential – Oracle Internal/Restricted/Highly Restricted

16. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Recommendations & Tools Basics … 17 Execute Entry / Exit Criteria Follow A Testing Methodology Based on Task Theme Based Exploratory Role based End to end flow based Mark Test Flows With A Priority Basic Sanity check Detailed Confidential – Oracle Internal/Restricted/Highly Restricted

17. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 18 Shorter Release Cycle Automation coverage Rapid Addition of Features Faster validation Security Testing data security and data leaks Integration Test integrations for Cloud & On Premise Multi Tenancy Simulated customers and roles Testing Recommendations & Tools Cloud Testing … Execute Accelerate testing with automation tools Test in short sprints Authentication, Authorization, Input Validation, PWD, etc .. End to end flows between the cloud apps and integrating apps including on premise ERP Ensure isolation of individual customer applications and data

18. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 19 Complex Architecture Component specific & integration testing Complex Data Controlled statistical data with pre- defined patterns Security Data security, leaks, and authentication Validation Insights & predictions VS. Pre-validated data Performance and Scalability Performance and stress testing Recommendations & Tools IoT Testing … Execute Test different services separately and run end to end application flows Feed data into the IOT apps to see if system can decode the time series data Authentication, Authorization, Input Validation, PWD, etc .. Feed same data to IoT apps & data mining tool to compare results Test with volume data & multiple users

19. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Recommendations & Tools Good Practice • Be creative / Innovative – Review results – Learn from results • Hot spots • Safe spots – Plan next cycle with a new flavor • Shift focus to hot spots • Certify safe spots – Improve scoping 20 Execute Confidential – Oracle Internal/Restricted/Highly Restricted

21. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Recommendations & Tools • Manual Vs. Automation • Reduce testing time & effort • Repeatable! 22 Automate Confidential – Oracle Internal/Restricted/Highly Restricted

22. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Recommendations & Tools 23 Anyone Critical + Repeatable Flows It is 2016 Automate Confidential – Oracle Internal/Restricted/Highly Restricted

23. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | • Load and Performance Style Testing • Web and Oracle Application Testing Accelerators • Smart Integration with EM Diagnostics • Test Planning, Requirements and Test Cases • Defects Tracking • Reporting • Functional and Regression Testing • Web and Oracle Application Testing Accelerators • Oracle Flow Builder and Starter Pack Oracle Application Testing Suite Overview Oracle Functional Testing Suite for Oracle Applications Oracle Load Testing Suite for Oracle Applications Oracle Test Manager 25 Automate Confidential – Oracle Internal/Restricted/Highly Restricted

24. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Functional Testing Suite for Oracle Applications Oracle Flow Builder and EBS Pre-built Test Content • Use Flow Builder to Generate Test Scripts • Start with EBS Pre-Built Test Content • Create and Maintain Your Test Scripts – Add Your Flows & Data – Extend with More Components – Support Customizations – Non technical resources Record Manually Generate Test Automation Scripts OpenScript Record & Playback + Advanced Scripting Flow Builder Framework to combine flows and data to generate test scripts EBS Pre-Built Test Content Your Data Your Flows OracleFunctionalTestingSuite forOracleApplications EBS Quality Assurance Team Reduced Automation Time by 60% 26 Automate Confidential – Oracle Internal/Restricted/Highly Restricted

25. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Flow Builder: EBS Pre-Built Test Content Starter Pack • Commonly used base functionality • For creating high level certification flows • Broad coverage to jump start the testing For Both 12.1.3 and 12.2 2000+ Components and 200+ Flows across EBS ProcurementFinancials ManufacturingLogistics Service Master Data Management Order Management 27 Asset Lifecycle Management Projects HCM 3600+ components and 230+ flows across EBS Logistics ManufacturingFinancials ProjectsProcurement Product Information Management Service Human Capital Management Order Management Asset Lifecycle Management Automate

26. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Flow Builder: EBS Pre-Built Test Content Advanced Pack • Niche products which stand alone and deep functionality • For creating detailed and complex test flows • Significant number of components above Starter Pack (95% coverage) For Both 12.1.3 and 12.2 28 10 product families / products across EBS Incentive Compensation Discrete Manufacturing Channel Revenue Management Project ManagementProcurement Process Manufacturing Property Manager Service Lease /Finance Management Asset Lifecycle Management Automate

27. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Flow Builder: EBS Web Services & APIs Support • What are Web Services & APIs? – Stored procedures that enable you to insert and update data into Oracle Applications • Automate Web Services & APIs using OFB – No need for an in-depth knowledge of PL/SQL • Advantage – Automated mass upload into the application – Scripts created are more stable and robust than UI 29 Automate Confidential – Oracle Internal/Restricted/Highly Restricted

28. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Automate Oracle Flow Builder: EBS Pre-Built Test Content Advanced Pack includes content for API/Web Services • Drastic reduction in test execution time • Hybrid flows, where you can execute a flow with API and UI scenarios • Verification of data in UI, Database, or output data after interface execution Enhanced Content for 12.1.3 30 875 components and 48 flows across EBS Logistics ManufacturingFinancials ProjectsProcurement Product Information Management Service Trade Management Order Management Asset Lifecycle Management API Content for EBS 12.1.3

29. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Flow Builder: EBS Web Services & APIs Support • UI VS. API – Value added through execution • In Oracle Project Management: – Creating a Project takes 1 - 1.5 hrs using UI mode VS. 26 seconds using OFB API – Create Task Assignments takes 15 min using UI mode VS. 8 seconds using OFB API 31 Automate Confidential – Oracle Internal/Restricted/Highly Restricted

31. 33Qualcomm Proprietary and Confidential Testimonial from Qualcomm • Created ~115 custom components in Oracle Flow Builder which saved huge effort during design phase. • Created ~160 automated scripts which cover ~1470 business process scenarios in past ~2years • Metrics against manual testing that is qualified for automation: ◦ Time Savings with Automation ~50% ◦ Automation Script Coverage ~50% • Automated scripts used in below activities which reduced a significant manual effort during major projects: ◦ Regression cycle ◦ Quick Build test ◦ Month-end activity close ◦ Pre-production checkout ◦ User Acceptance test ◦ Go-live checkout ◦ Emergency test in PRD Support

32. 34Qualcomm Proprietary and Confidential Testimonial from Qualcomm • Custom Solutions: ◦ We use QC for test management. Created a plug-in in order to interface OATS scripts directly to update results in QC. ◦ For accelerated test execution created customized framework in order to execute same OATS script in parallel for different business scenarios

34. Testing Oracle E-Business Suite Best Practices & Lessons Learned Sarvesh Tiwari Lead Solutions Architect Sarvesh.Tiwari@genpact.com @ideating

35. 37© 2015 Copyright Genpact. All Rights Reserved. 68,000 + employees 800+ Clients (One Third of Fortune 500) 30 languages 17 delivery countries 70 delivery centers 50 + Oracle Go Lives / year BPMAnalyticsDigital Technolog y ReengineerConsulting Outcome driven, Process- led ERP Analytic Solutions & Insights to support executive decisions Using Lean Six Sigma for process improvements & Transformations Robotic and Digital technologies While running some of the most complex business processes for our clients Deep domain consulting for business blueprints AT-A-GLANCE Founded by GE in 1997 NYSE (“G”) $2.5 Billion (FY2015)

36. 38© 2015 Copyright Genpact. All Rights Reserved. FUTURE IS HERE, NOW Computing Everywhere … EBS on wearables? Internet of Things … sensor to boardroom Advanced Pervasive Invisible Analytics … In-memory Context Rich Applications UI … tried Endeca? Cloud-Client Architecture … mobile native Infinitely Elastic Infra … cloud compute…

37. 39© 2015 Copyright Genpact. All Rights Reserved. IT Practices Business Technology 17% CIOs say they are fast enough for the pace of their business “IT Speed: The Crisis and the Savior of the Enterprise” Forrester Research CRISIS

39. 41© 2015 Copyright Genpact. All Rights Reserved. HOWEVER, MOST FACE SPEED BUMPS IN TESTING 45% of effort is spent in creation of prerequisites and documenting, rather than testing the outcomes 53% ERP Programs execute EBS testing as “part time” function of the business end users 62% report having no automation in EBS Functional Testing 43% CRITICAL PATH OF A TYPICAL EBS PROJECT IS IN TESTING

40. 42© 2015 Copyright Genpact. All Rights Reserved. LEARNING FROM A JOURNEY US headquartered global leader in data center solutions, cloud management, performance management, services management, workload automation and virtualization lifecycle management products & services • Complex global EBS footprint (Fin, SCM, CRM) • 58 SOBs, 40 countries, 13 localizations • 20+ modules, 30+ integrated systems • Heavily customized (3,500+ CEMLI) • 5 custom-built bespoke bolt-on modules • Products shifting to SaaS • Failed attempts to upgrade to R12.2 • Heavily manual-dependent testing effort • Painful release cycle time: want to establish monthly releases CONTEXT CHALLENGE CLIENT

41. 43© 2015 Copyright Genpact. All Rights Reserved. 5 TOP CHALLENGES THEY FACED IN AUTOMATING EBS TESTING • Domain Knowledge and Testing knowledge mutually exclusive impacting speed, quality and effectiveness of testing • Too much cycle time it takes to develop automated test scripts • Always Obsolete: Scripting based test cases /data become easily obsolete therefore require huge ongoing effort in keeping scripts up to date. • Difficult to discover / reuse the test data, scripts, scenarios from release to release, project to project • Difficult to scope and tailor (Test some / Test whole) for individual releases

42. 44© 2015 Copyright Genpact. All Rights Reserved. JOURNEY Oct 2014 Dec 2014 Mar 2015 June 2015 Sep 2015 Nov 2015 Jan 2016 R12 Upgrade Kick Off OATS Pilot 1st OATS MVP Ready – Used in Unit Test Progressive Additions in OATS R122.2 Gone Live Monthly Regression Pack Test Iterations Ongoing Optimization

43. 45© 2015 Copyright Genpact. All Rights Reserved. RESULTS •50% faster test iterations •40% Less business effort in test cycles: Focus on value-added test •60+% reduction in regression cycle times •Production releases: 8 /year (from earlier 4 /year) •Adoption of Agile and Test-driven development methods •100% Sanity, 80% Regression, 60% “projects” EBS testing automated

44. 46© 2015 Copyright Genpact. All Rights Reserved. LESSONS LEARNED: BALANCE OF MANUAL & AUTOMATED Low Testing Effort Repeatable / Rule Based High Testing Effort High Intuitive 1 2 • Automate the pre-requisites, support SMEs to focus on “real” validation • “Auto-gathering” of test evidence / progress: End to End scenarios • Role Based Testing 3 Limited value: diminishing returns of automation Low Hanging Fruits Automate to sustainable extent X

46. 48© 2015 Copyright Genpact. All Rights Reserved. PRE-BUILT TEST CONTENT: AGILE JUMP START Business Scenarios • End to end test scenarios covering detailed critical business processes, use cases and industry nuances • Context specific functional scenarios • COA change • R12.2 upgrade

47. 49© 2015 Copyright Genpact. All Rights Reserved. DESIGN FOR MAINTAINABILITY: MINIMIZE OBSOLESCENCE •EBS Configuration Half Life: •Data / Set Ups: 3 - 6 months •Functionality/Logic: 6 - 12 months •UI / major patch: 9 – 18 months •Continuously updated library of test content •Not “Screen Placement” but Object-Action driven •“Keywords > Component > Flow” | upward bubbling of changes •Changes in scripts for EBS customizations

48. 50© 2015 Copyright Genpact. All Rights Reserved. HOT SPOTS VS SAFE SPOTS •Test Clustering: L1 … L5 end to end scripts •Iterative Learning: Progressive adaptability – Hot Spots vs Safe Spots •Impact Traceability to Test Scenarios •Test Driven Enhancement Management •Continuous Optimization: “Regression Packs”, “Sanity Packs” •Test Data •Role based Focal Data Points (PO data vs Accounting data) •Data Randomization, Variability •Purpose-driven Data Generator scripts

49. Thank You Sarvesh Tiwari Sarvesh.Tiwari@genpact.com | @ideating

50. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Questions 52Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted

OOW16 - Testing Oracle E-Business Suite Best Practices [CON6713]

vasuballa

OOW16 - Testing Oracle E-Business Suite Best Practices [CON6713]