1.
Building Data Quality Audit
Framework using Delta at Cerner
Madhav Agni
www.linkedin.com/in/madhav-agni
Manager | Lead Software Engineer

2.
Agenda
Intro for Beacon
Beacon Architecture
Need for Data Quality Audit
Delta Lake
Demo

3.
What is Beacon?
▪ Inventory of configuration items (IT "thing“) and assets
▪ Relationships and attributes
▪ Data and the tools used to collect the data
▪ Cerner’s Configuration Management System is called Beacon

4.
Why do we need Beacon?
▪ Multiple tools with asset and config information
▪ One single accountable team
▪ Cerner has over 1 billion dollars in assets / CIs
▪ Manage security risks
▪ Service costing and licensing
▪ Capacity management
▪ Service Operations
▪ One solution to rule them all

5.
Beacon Technical Architecture

6.
What is Data Quality

7.
Why Data Quality is Important?
▪ No trust in Data
▪ False facts
▪ Different view on Data
▪ Flexibility
▪ Optimize business process
▪ Stricter compliance

8.
Delta Lake Features
▪ ACID Transactions
▪ Time Travel (data versioning)
▪ Schema Enforcement
▪ Schema Evolution
▪ Audit History
▪ Updates and Deletes

9.
Audit Framework Flow
Delta LakeAzure Data Factory Power BI Teams Alert

10.
Impact of the framework
▪ Increase in pre-prod defects
▪ 20% decrease in SIE test execution
▪ Reduction in CRQs
▪ 25% decrease in Prod defects
▪ Ability to perform data rollbacks

11.
Demo

12.
Feedback
Your feedback is important to us.
Don’t forget to rate and
review the sessions.