Data Con LA 2022 - Why Data Quality vigilance requires an End-to-End, Automated and Continuous Data

Editor's Notes

• #2 Hello - Introduction
• #4 We understand the need for enterprise data governance and can help in the process. This is the umbrella I will talk to with the “pillar” of data quality our focus today…
• #5 Changes are constantly happening throughout your IT landscape One end-to-end business process within that landscape could touch several interconnected systems and technologies typical enterprise portfolio contains thousands of applications, (2000 – 3000) applications in production MuleSoft research - Single transaction touches an average of 83 different technologies from mainframes, legacy customs apps to microservices ad cloud native apps. Next slide: New and real challenge is not just dealing with application changes, but dealing with environment changes and data changes Examples of environment could be – as simple as a Microsoft update, or a deployment of a new browser version Due to the move to the cloud, we now have an extremely short cycle to keep up with those environment and data changes Three dimensions are happening in parallel: application, environment and data changes = complexity
• #6 New and real challenge is not just dealing with application changes, but dealing with environment changes and data changes Examples of environment could be – as simple as a Microsoft update, or a deployment of a new browser version Due to the move to the cloud, we now have an extremely short cycle to keep up with those environment and data changes Three dimensions are happening in parallel: application, environment and data changes = complexity
• #7 The global pandemic has accelerated change across the digital landscape (remote work, remote sales, remote everything) Companies need to go digital Mayfield CXO survey: 85% of organizations are planning to move to the cloud twice as fast Application Modernization – related to cloud transformation but could mean many different things: Migrating your applications to more modern ones in order to deliver more business value or enhance customer/employee experience (for example, shifting Namely and Expensify to Workday – for more scalability) Application modernization could also mean migrating your legacy applications to scalable, cloud-native app environments – 67% plan to migrate on-premise applications. Example: according to the ASUG Tricentis 2021 survey Future of SAP delivery, 91% plan to move to S4HANA. But SAP ECC is more popular. Organizations say cloud is great, but need to justify the cost and risk involved Challenges include rapid and regular updates: keeping up with the pace of updates release by SAP, Salesforce, ServiceNow Other challenges include data: 43% say data migration in cloud migration projects is a huge problem (for SAP – ASUG survey) It could also mean Digital optimization – delivering new business/digital functionality: 50% of companies said they are planning on moving their eCommerce, marketing and sales to digital platforms in order to improve customer self-service and experience
• #8 if you don’t deal with this inevitable change, you will experience consequences Example 1: Large multinational bank (Barclays – not our customer)– has online banking down, leaving customers locked out of accounts as payments also delayed: The average cost of downtime is $5,600 per minute (according to Gartner research), So it is estimated this company suffered 7 million dollars in loss, loss of customers, reputational damage Barclays online banking goes down: https://www.thesun.co.uk/money/12788031/barclays-online-banking-down-2/ Average cost of downtime: https://www.atlassian.com/incident-management/kpis/cost-of-downtime Why the cost of network downtime is so high in the banking industry: https://www.garlandtechnology.com/blog/why-is-the-cost-of-network-downtime-so-high-in-the-banking-industry Barclays tops list of banks with most IT shutdowns: https://www.bbc.com/news/business-49412055 Example 2- One of the largest telecommunications providers in Europe (A1 – our customer, before scenario). They were doing manual testing which resulted in more than 10,000 tests performed. Business process was distributed across 60 critical systems and enterprise applications. As a result, they were getting only 3 to 4 major releases a year, delayed innovation. Additionally, if a program is late, it costs money to get people to deliver that program on time. Example 3: inefficient resource management. World’s most established oil and gas providers – Exxon Mobil (our customer, before scenario). They needed highly specialized people to develop and maintain their testing. This lead to high costs not only in technical IT staff but also high maintenance to keep up with those changes
• #9 if you don’t deal with this inevitable change, you will experience consequences
• #10 If you're like most enterprise organizations, you've already invested in lots of tools to get and keep your data in good shape. Fantastic, they do a great job at what they do — keep on using them. The issue is that they're designed with a particular task in mind — like governing your master data or profiling data at specific points in time, like after a transformation event. In complex enterprise landscapes where you're dealing with masses of structured and unstructured data from many different systems, these point solutions leave dangerous data management gaps.
• #11 These gaps are risk in your data process. I listed some examples of the gaps they often don’t fill.
• #12 We must find the problem ad fix it, but where in the process was the failure.
• #13 At the end of the day all the testing of the gaps in the data process is a manual effort, we call stare and compare Points to make: Manual “stare and compare” methods that involve people spot checking data in source systems and the final data destination This doesn’t scale or get them the coverage they need It’s also a real waste of expensive human resources it’s estimated that all knowledge workers in an organization spend 50% of their day, fixing data, when automating this task makes much more sense
• #14 TO be clear, to mitigate the risk of changes to the processes you must test ALL the processes End to END BEFORE they are in production. Not just a single point like an ETL change but preform true REGRESSION that ensures the process is golden end to end after any change. Also, you can’t rely on production testing or profiling or monitoring they only check a part of the process and if you are catching problems in production, it is TOO LATE! Point “testing” solution like Informatica’s IQA, their testing solution, only have coverage on their intersection points. You must look at all the integration points, you can now test all the intersection of the data, and much more that just a “sample” of the data. All the data at all the intersection points. .
• #15 []
• #16 []
• #17 [Jeanette] - Talk Track – To solve this business problem every enterprise has you need an automated, end to end solution, that can run continuously and integrate with a company’s DevOps or DataOps CI/CD structure.
• #18 [Jeanette] - Our major differentiator, is that SAP EDIT (Tosca DI) has unique ability to test across 160 UI and API technologies as well as across the over 250 different data technologies enterprises have deployed across the enterprise. An we do this testing in an automated, scriptless  framework we call Model Based Test Automation. Truly our secret sauce for accomplishing the automation at scale.
• #19 [Jeanette – handover to Curtis to walk through B of A examples] - Talk Track – Once we implement it we can catch the problem at the root cause.  Finding issues early, identifying the root cause and remediating them is the technical purpose behind all testing.  Doing it with automation ensures the coverage needed, and in the end you get higher data quality for better business decisions.
• #21 Use this slide to show the value of model-based test automation (and test case design – turn one test model into dozens of tests, easy update them in this one model)