Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

0

Share

Building Data Quality Audit Framework using Delta Lake at Cerner

Download to read offline

Cerner needs to know what assets it owns, where they are located, and the status of those assets. A configuration management system is an inventory of IT assets and IT things like servers, network devices, storage arrays, and software licenses.

  • Be the first to like this

Building Data Quality Audit Framework using Delta Lake at Cerner

  1. 1. Building Data Quality Audit Framework using Delta at Cerner Madhav Agni www.linkedin.com/in/madhav-agni Manager | Lead Software Engineer
  2. 2. Agenda Intro for Beacon Beacon Architecture Need for Data Quality Audit Delta Lake Demo
  3. 3. What is Beacon? ▪ Inventory of configuration items (IT "thing“) and assets ▪ Relationships and attributes ▪ Data and the tools used to collect the data ▪ Cerner’s Configuration Management System is called Beacon
  4. 4. Why do we need Beacon? ▪ Multiple tools with asset and config information ▪ One single accountable team ▪ Cerner has over 1 billion dollars in assets / CIs ▪ Manage security risks ▪ Service costing and licensing ▪ Capacity management ▪ Service Operations ▪ One solution to rule them all
  5. 5. Beacon Technical Architecture
  6. 6. What is Data Quality
  7. 7. Why Data Quality is Important? ▪ No trust in Data ▪ False facts ▪ Different view on Data ▪ Flexibility ▪ Optimize business process ▪ Stricter compliance
  8. 8. Delta Lake Features ▪ ACID Transactions ▪ Time Travel (data versioning) ▪ Schema Enforcement ▪ Schema Evolution ▪ Audit History ▪ Updates and Deletes
  9. 9. Audit Framework Flow Delta LakeAzure Data Factory Power BI Teams Alert
  10. 10. Impact of the framework ▪ Increase in pre-prod defects ▪ 20% decrease in SIE test execution ▪ Reduction in CRQs ▪ 25% decrease in Prod defects ▪ Ability to perform data rollbacks
  11. 11. Demo
  12. 12. Feedback Your feedback is important to us. Don’t forget to rate and review the sessions.

Cerner needs to know what assets it owns, where they are located, and the status of those assets. A configuration management system is an inventory of IT assets and IT things like servers, network devices, storage arrays, and software licenses.

Views

Total views

363

On Slideshare

0

From embeds

0

Number of embeds

0

Actions

Downloads

24

Shares

0

Comments

0

Likes

0

×