Successfully reported this slideshow.
Your SlideShare is downloading. ×

Building Data Quality Audit Framework using Delta Lake at Cerner

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Loading in …3
×

Check these out next

1 of 16 Ad

Building Data Quality Audit Framework using Delta Lake at Cerner

Download to read offline

Cerner needs to know what assets it owns, where they are located, and the status of those assets. A configuration management system is an inventory of IT assets and IT things like servers, network devices, storage arrays, and software licenses.

Cerner needs to know what assets it owns, where they are located, and the status of those assets. A configuration management system is an inventory of IT assets and IT things like servers, network devices, storage arrays, and software licenses.

Advertisement
Advertisement

More Related Content

Slideshows for you (20)

Similar to Building Data Quality Audit Framework using Delta Lake at Cerner (20)

Advertisement

More from Databricks (20)

Advertisement

Building Data Quality Audit Framework using Delta Lake at Cerner

  1. 1. Building Data Quality Audit Framework using Delta at Cerner Madhav Agni www.linkedin.com/in/madhav-agni Manager | Lead Software Engineer
  2. 2. Agenda Intro for Beacon Beacon Architecture Need for Data Quality Audit Delta Lake Demo
  3. 3. What is Beacon? ▪ Inventory of configuration items (IT "thing“) and assets ▪ Relationships and attributes ▪ Data and the tools used to collect the data ▪ Cerner’s Configuration Management System is called Beacon
  4. 4. Why do we need Beacon? ▪ Multiple tools with asset and config information ▪ One single accountable team ▪ Cerner has over 1 billion dollars in assets / CIs ▪ Manage security risks ▪ Service costing and licensing ▪ Capacity management ▪ Service Operations ▪ One solution to rule them all
  5. 5. Beacon Technical Architecture
  6. 6. What is Data Quality
  7. 7. Why Data Quality is Important? ▪ No trust in Data ▪ False facts ▪ Different view on Data ▪ Flexibility ▪ Optimize business process ▪ Stricter compliance
  8. 8. Delta Lake Features ▪ ACID Transactions ▪ Time Travel (data versioning) ▪ Schema Enforcement ▪ Schema Evolution ▪ Audit History ▪ Updates and Deletes
  9. 9. Audit Framework Flow Delta LakeAzure Data Factory Power BI Teams Alert
  10. 10. Impact of the framework ▪ Increase in pre-prod defects ▪ 20% decrease in SIE test execution ▪ Reduction in CRQs ▪ 25% decrease in Prod defects ▪ Ability to perform data rollbacks
  11. 11. Demo
  12. 12. Feedback Your feedback is important to us. Don’t forget to rate and review the sessions.

×