SlideShare a Scribd company logo
DISCOVERING FIRE
OR WHY YOUR DATA QUALITY PROBLEM IS NOT THE USER’S FAULT
www.suade.org/fire
HOW DO WE PREVENT THE NEXT FINANCIAL CRISIS?
REGTECH...
R(x1, x2, ..., xn) = y
where,
y = metrics, reports, analytics, insights
R = rules, regulations
x = data
THE REGULATORY EQUATION
 We think garbage is acceptable
 Garbage input is not our problem
 Garbage output also not our problem
TECHNOLOGY SAYS GARBAGE IN, GARBAGE OUT
Leverage your data model to prevent garbage in!
Objectives:
 Quality, control and governance for
Financial Institutions
 Comparability and supervision of
underlying data for Regulators
 Seamless integration for FinTechs and
third parties
FINANCIAL REGULATION (FIRE) DATA FORMAT
Github:
 https://github.com/suadelabs/fire
WHAT IS BAD DATA?
Leverage your data model to prevent garbage in!
Bad data is data with:
 No definitions
 No validations
 Missing values
 Wrong values
 Flags
 Derived values
https://xkcd.com/1838/
BAD DATA: NO DEFINITIONS
Amount in USD
Total Deposits ?
ACT_DEP_BAL R10_USD_AMT GTEE_AMT LIQ14_TOT_BAL R12_AMT
90 60 60 30 120
Rule 1: No attribute without definition
Data:
Output:
BAD DATA: NO VALIDATIONS
Rule 2: No attribute without limitation
Data:
CCY START_DT CATEGORY BALANCE USDRATE USD_BAL
GBP 2019-03-08 Retail 30 1.20 36
ALL 18/11/20 International -20 0.01 0
USD 18/11/19 Govt 15 100 15
BAD DATA: MISSING/WRONG VALUES
OK. Missing values tell you what you don’t know
Data:
CCY START_DT CATEGORY BALANCE USDRATE USD_BAL
GBP 2019-03-08 Retail 30 1.20 36
ALL International -20 0.01
USD Govt 15 15
Total USD
Deposits from SMEs ?
Deposits from Govts ?
Deposits from other
customers
15
O_REP R10_TYPE CATEGORY USD_AMT SME? GOVT?
GB Central Bank Retail 30 True True
UK Small Business International 20 False True
DE Financial Govt 15 False False
BAD DATA: FLAGS
Rule 3: Flags are not your friend
Output:Data:
BAD DATA: DERIVED VALUES
Amount
in EUR
Total Deposits ?
EURUSD EURGBP GBP_AMT TYPE USD_AMT
1.00 1.16 100 DEP 120
Rule 4: Avoid attributes that can easily be deduced from others
ADHERE TO PRINCIPLES, NOT YOUR BOSS
Good data has:
 Definitions
 Validations
 No flags
 No derivable values
FIRE GUIDING PRINCIPLES
1. Data attributes should always be true:
 “...every pull request requires a corresponding, documented, legal reference, preferably to a
currently in-force financial regulation.”
2. Data attributes should be atomic
 “One property should not be derivable from other properties.“
3. Data attributes should be consistent
 “Schema properties should try to avoid logical inconsistencies.”
TO CONCLUDE
Death List
OOOO----RENRENRENREN ISHIIISHIIISHIIISHII(added columns without defintions)
 VERNITAVERNITAVERNITAVERNITA GREENGREENGREENGREEN (added columns without validations)
BUDDBUDDBUDDBUDD(added a whole bunch of flags)
 ELLEELLEELLEELLE DRIVERDRIVERDRIVERDRIVER (added redundant information)
BILLBILLBILLBILL (allowed this to happen)
GET INVOLVED! www.suade.org/FIRE

More Related Content

Similar to Shift Money 2019 - Why your data quality problem is not your user's fault - Murat Abur (Suade)

Live ramp analystdaypresotemplate-fina_lv3
Live ramp analystdaypresotemplate-fina_lv3Live ramp analystdaypresotemplate-fina_lv3
Live ramp analystdaypresotemplate-fina_lv3
acxiom2016ir
 
A Vision for Quantitative Investing in the Data Economy by Michael Beal at Qu...
A Vision for Quantitative Investing in the Data Economy by Michael Beal at Qu...A Vision for Quantitative Investing in the Data Economy by Michael Beal at Qu...
A Vision for Quantitative Investing in the Data Economy by Michael Beal at Qu...
Quantopian
 
Insurance stream - Marc DonFrancesco slides
Insurance stream - Marc DonFrancesco slidesInsurance stream - Marc DonFrancesco slides
Insurance stream - Marc DonFrancesco slides
Callcredit123
 
Feeding the Beast-How Fraud Tools Bring Context into Authentication (Gartner ...
Feeding the Beast-How Fraud Tools Bring Context into Authentication (Gartner ...Feeding the Beast-How Fraud Tools Bring Context into Authentication (Gartner ...
Feeding the Beast-How Fraud Tools Bring Context into Authentication (Gartner ...
TransUnion
 
Auditing and fraud detection using Picalo
Auditing and fraud detection using PicaloAuditing and fraud detection using Picalo
Auditing and fraud detection using Picalo
Sii Quist
 
Audit,fraud detection Using Picalo
Audit,fraud detection Using PicaloAudit,fraud detection Using Picalo
Audit,fraud detection Using Picalo
guest4ea866f
 
Acc 675 control audit final project
Acc 675 control audit final projectAcc 675 control audit final project
Acc 675 control audit final project
Kelly Giambra
 
Chapter 6 corporate lending
Chapter 6   corporate lendingChapter 6   corporate lending
Chapter 6 corporate lending
Quan Risk
 
Business Intelligence For Anti-Money Laundering
Business Intelligence For Anti-Money LaunderingBusiness Intelligence For Anti-Money Laundering
Business Intelligence For Anti-Money Laundering
Kartik Mehta
 
AXA x DSSG Meetup Sharing (Feb 2016)
AXA x DSSG Meetup Sharing (Feb 2016)AXA x DSSG Meetup Sharing (Feb 2016)
AXA x DSSG Meetup Sharing (Feb 2016)
Eugene Yan Ziyou
 
Fintech summit 2016 thomson reuters tim baker_presentation final
Fintech summit 2016 thomson reuters tim baker_presentation finalFintech summit 2016 thomson reuters tim baker_presentation final
Fintech summit 2016 thomson reuters tim baker_presentation final
Glen Frost
 
2011 Intro Bio Lock
2011 Intro Bio Lock2011 Intro Bio Lock
2011 Intro Bio Lock
realtime Mexico & LA
 
FinTech Belgium Summit 2018 - Vadis Technologies - Jean-François Heering
FinTech Belgium Summit 2018 - Vadis Technologies - Jean-François HeeringFinTech Belgium Summit 2018 - Vadis Technologies - Jean-François Heering
FinTech Belgium Summit 2018 - Vadis Technologies - Jean-François Heering
FinTech Belgium
 
FinTech Belgium - MeetUp on The Ideal RegTech for Banks and FinTechs - Jean-F...
FinTech Belgium - MeetUp on The Ideal RegTech for Banks and FinTechs - Jean-F...FinTech Belgium - MeetUp on The Ideal RegTech for Banks and FinTechs - Jean-F...
FinTech Belgium - MeetUp on The Ideal RegTech for Banks and FinTechs - Jean-F...
FinTech Belgium
 
Big data-analytics-ebook
Big data-analytics-ebookBig data-analytics-ebook
Big data-analytics-ebook
Shubhashish Biswas
 
Forces and Threats in a Data Warehouse (and why metadata and architecture is ...
Forces and Threats in a Data Warehouse (and why metadata and architecture is ...Forces and Threats in a Data Warehouse (and why metadata and architecture is ...
Forces and Threats in a Data Warehouse (and why metadata and architecture is ...
Stefan Urbanek
 
The Hidden Gems: Optimizing your DNB Credit Reports
The Hidden Gems: Optimizing your DNB Credit ReportsThe Hidden Gems: Optimizing your DNB Credit Reports
The Hidden Gems: Optimizing your DNB Credit Reports
Dun & Bradstreet
 
Business Intelligence For Aml
Business Intelligence For AmlBusiness Intelligence For Aml
Business Intelligence For Aml
Kartik Mehta
 
Predictive analytics retention
Predictive analytics retentionPredictive analytics retention
Predictive analytics retention
Qubit
 
Driving customer retention using predictive analytics
Driving customer retention using predictive analyticsDriving customer retention using predictive analytics
Driving customer retention using predictive analytics
Vanessa Beeswanger
 

Similar to Shift Money 2019 - Why your data quality problem is not your user's fault - Murat Abur (Suade) (20)

Live ramp analystdaypresotemplate-fina_lv3
Live ramp analystdaypresotemplate-fina_lv3Live ramp analystdaypresotemplate-fina_lv3
Live ramp analystdaypresotemplate-fina_lv3
 
A Vision for Quantitative Investing in the Data Economy by Michael Beal at Qu...
A Vision for Quantitative Investing in the Data Economy by Michael Beal at Qu...A Vision for Quantitative Investing in the Data Economy by Michael Beal at Qu...
A Vision for Quantitative Investing in the Data Economy by Michael Beal at Qu...
 
Insurance stream - Marc DonFrancesco slides
Insurance stream - Marc DonFrancesco slidesInsurance stream - Marc DonFrancesco slides
Insurance stream - Marc DonFrancesco slides
 
Feeding the Beast-How Fraud Tools Bring Context into Authentication (Gartner ...
Feeding the Beast-How Fraud Tools Bring Context into Authentication (Gartner ...Feeding the Beast-How Fraud Tools Bring Context into Authentication (Gartner ...
Feeding the Beast-How Fraud Tools Bring Context into Authentication (Gartner ...
 
Auditing and fraud detection using Picalo
Auditing and fraud detection using PicaloAuditing and fraud detection using Picalo
Auditing and fraud detection using Picalo
 
Audit,fraud detection Using Picalo
Audit,fraud detection Using PicaloAudit,fraud detection Using Picalo
Audit,fraud detection Using Picalo
 
Acc 675 control audit final project
Acc 675 control audit final projectAcc 675 control audit final project
Acc 675 control audit final project
 
Chapter 6 corporate lending
Chapter 6   corporate lendingChapter 6   corporate lending
Chapter 6 corporate lending
 
Business Intelligence For Anti-Money Laundering
Business Intelligence For Anti-Money LaunderingBusiness Intelligence For Anti-Money Laundering
Business Intelligence For Anti-Money Laundering
 
AXA x DSSG Meetup Sharing (Feb 2016)
AXA x DSSG Meetup Sharing (Feb 2016)AXA x DSSG Meetup Sharing (Feb 2016)
AXA x DSSG Meetup Sharing (Feb 2016)
 
Fintech summit 2016 thomson reuters tim baker_presentation final
Fintech summit 2016 thomson reuters tim baker_presentation finalFintech summit 2016 thomson reuters tim baker_presentation final
Fintech summit 2016 thomson reuters tim baker_presentation final
 
2011 Intro Bio Lock
2011 Intro Bio Lock2011 Intro Bio Lock
2011 Intro Bio Lock
 
FinTech Belgium Summit 2018 - Vadis Technologies - Jean-François Heering
FinTech Belgium Summit 2018 - Vadis Technologies - Jean-François HeeringFinTech Belgium Summit 2018 - Vadis Technologies - Jean-François Heering
FinTech Belgium Summit 2018 - Vadis Technologies - Jean-François Heering
 
FinTech Belgium - MeetUp on The Ideal RegTech for Banks and FinTechs - Jean-F...
FinTech Belgium - MeetUp on The Ideal RegTech for Banks and FinTechs - Jean-F...FinTech Belgium - MeetUp on The Ideal RegTech for Banks and FinTechs - Jean-F...
FinTech Belgium - MeetUp on The Ideal RegTech for Banks and FinTechs - Jean-F...
 
Big data-analytics-ebook
Big data-analytics-ebookBig data-analytics-ebook
Big data-analytics-ebook
 
Forces and Threats in a Data Warehouse (and why metadata and architecture is ...
Forces and Threats in a Data Warehouse (and why metadata and architecture is ...Forces and Threats in a Data Warehouse (and why metadata and architecture is ...
Forces and Threats in a Data Warehouse (and why metadata and architecture is ...
 
The Hidden Gems: Optimizing your DNB Credit Reports
The Hidden Gems: Optimizing your DNB Credit ReportsThe Hidden Gems: Optimizing your DNB Credit Reports
The Hidden Gems: Optimizing your DNB Credit Reports
 
Business Intelligence For Aml
Business Intelligence For AmlBusiness Intelligence For Aml
Business Intelligence For Aml
 
Predictive analytics retention
Predictive analytics retentionPredictive analytics retention
Predictive analytics retention
 
Driving customer retention using predictive analytics
Driving customer retention using predictive analyticsDriving customer retention using predictive analytics
Driving customer retention using predictive analytics
 

More from Shift Conference

Shift Remote: AI: How Does Face Recognition Work (ars futura)
Shift Remote: AI: How Does Face Recognition Work  (ars futura)Shift Remote: AI: How Does Face Recognition Work  (ars futura)
Shift Remote: AI: How Does Face Recognition Work (ars futura)
Shift Conference
 
Shift Remote: AI: Behind the scenes development in an AI company - Matija Ili...
Shift Remote: AI: Behind the scenes development in an AI company - Matija Ili...Shift Remote: AI: Behind the scenes development in an AI company - Matija Ili...
Shift Remote: AI: Behind the scenes development in an AI company - Matija Ili...
Shift Conference
 
Shift Remote: AI: Smarter AI with analytical graph databases - Victor Lee (Ti...
Shift Remote: AI: Smarter AI with analytical graph databases - Victor Lee (Ti...Shift Remote: AI: Smarter AI with analytical graph databases - Victor Lee (Ti...
Shift Remote: AI: Smarter AI with analytical graph databases - Victor Lee (Ti...
Shift Conference
 
Shift Remote: DevOps: Devops with Azure Devops and Github - Juarez Junior (Mi...
Shift Remote: DevOps: Devops with Azure Devops and Github - Juarez Junior (Mi...Shift Remote: DevOps: Devops with Azure Devops and Github - Juarez Junior (Mi...
Shift Remote: DevOps: Devops with Azure Devops and Github - Juarez Junior (Mi...
Shift Conference
 
Shift Remote: DevOps: Autodesks research into digital twins for AEC - Kean W...
Shift Remote: DevOps: Autodesks research into digital twins for AEC -  Kean W...Shift Remote: DevOps: Autodesks research into digital twins for AEC -  Kean W...
Shift Remote: DevOps: Autodesks research into digital twins for AEC - Kean W...
Shift Conference
 
Shift Remote: DevOps: When metrics are not enough, and everyone is on-call - ...
Shift Remote: DevOps: When metrics are not enough, and everyone is on-call - ...Shift Remote: DevOps: When metrics are not enough, and everyone is on-call - ...
Shift Remote: DevOps: When metrics are not enough, and everyone is on-call - ...
Shift Conference
 
Shift Remote: DevOps: Modern incident management with opsgenie - Kristijan L...
Shift Remote: DevOps: Modern incident management with opsgenie -  Kristijan L...Shift Remote: DevOps: Modern incident management with opsgenie -  Kristijan L...
Shift Remote: DevOps: Modern incident management with opsgenie - Kristijan L...
Shift Conference
 
Shift Remote: DevOps: Gitlab ci hands-on experience - Ivan Rimac (Barrage)
Shift Remote: DevOps: Gitlab ci hands-on experience - Ivan Rimac (Barrage)Shift Remote: DevOps: Gitlab ci hands-on experience - Ivan Rimac (Barrage)
Shift Remote: DevOps: Gitlab ci hands-on experience - Ivan Rimac (Barrage)
Shift Conference
 
Shift Remote: DevOps: DevOps Heroes - Adding Advanced Automation to your Tool...
Shift Remote: DevOps: DevOps Heroes - Adding Advanced Automation to your Tool...Shift Remote: DevOps: DevOps Heroes - Adding Advanced Automation to your Tool...
Shift Remote: DevOps: DevOps Heroes - Adding Advanced Automation to your Tool...
Shift Conference
 
Shift Remote: DevOps: An (Un)expected Journey - Zeljko Margeta (RBA)
Shift Remote: DevOps: An (Un)expected Journey - Zeljko Margeta (RBA)Shift Remote: DevOps: An (Un)expected Journey - Zeljko Margeta (RBA)
Shift Remote: DevOps: An (Un)expected Journey - Zeljko Margeta (RBA)
Shift Conference
 
Shift Remote: Game Dev - Localising Mobile Games - Marta Kunic (Nanobit)
Shift Remote: Game Dev - Localising Mobile Games - Marta Kunic (Nanobit)Shift Remote: Game Dev - Localising Mobile Games - Marta Kunic (Nanobit)
Shift Remote: Game Dev - Localising Mobile Games - Marta Kunic (Nanobit)
Shift Conference
 
Shift Remote: Game Dev - Challenges Introducing Open Source to the Games Indu...
Shift Remote: Game Dev - Challenges Introducing Open Source to the Games Indu...Shift Remote: Game Dev - Challenges Introducing Open Source to the Games Indu...
Shift Remote: Game Dev - Challenges Introducing Open Source to the Games Indu...
Shift Conference
 
Shift Remote: Game Dev - Ghost in the Machine: Authorial Voice in System Desi...
Shift Remote: Game Dev - Ghost in the Machine: Authorial Voice in System Desi...Shift Remote: Game Dev - Ghost in the Machine: Authorial Voice in System Desi...
Shift Remote: Game Dev - Ghost in the Machine: Authorial Voice in System Desi...
Shift Conference
 
Shift Remote: Game Dev - Building Better Worlds with Game Culturalization - K...
Shift Remote: Game Dev - Building Better Worlds with Game Culturalization - K...Shift Remote: Game Dev - Building Better Worlds with Game Culturalization - K...
Shift Remote: Game Dev - Building Better Worlds with Game Culturalization - K...
Shift Conference
 
Shift Remote: Game Dev - Open Match: An Open Source Matchmaking Framework - J...
Shift Remote: Game Dev - Open Match: An Open Source Matchmaking Framework - J...Shift Remote: Game Dev - Open Match: An Open Source Matchmaking Framework - J...
Shift Remote: Game Dev - Open Match: An Open Source Matchmaking Framework - J...
Shift Conference
 
Shift Remote: Game Dev - Designing Inside the Box - Fernando Reyes Medina (34...
Shift Remote: Game Dev - Designing Inside the Box - Fernando Reyes Medina (34...Shift Remote: Game Dev - Designing Inside the Box - Fernando Reyes Medina (34...
Shift Remote: Game Dev - Designing Inside the Box - Fernando Reyes Medina (34...
Shift Conference
 
Shift Remote: Mobile - Efficiently Building Native Frameworks for Multiple Pl...
Shift Remote: Mobile - Efficiently Building Native Frameworks for Multiple Pl...Shift Remote: Mobile - Efficiently Building Native Frameworks for Multiple Pl...
Shift Remote: Mobile - Efficiently Building Native Frameworks for Multiple Pl...
Shift Conference
 
Shift Remote: Mobile - Introduction to MotionLayout on Android - Denis Fodor ...
Shift Remote: Mobile - Introduction to MotionLayout on Android - Denis Fodor ...Shift Remote: Mobile - Introduction to MotionLayout on Android - Denis Fodor ...
Shift Remote: Mobile - Introduction to MotionLayout on Android - Denis Fodor ...
Shift Conference
 
Shift Remote: Mobile - Devops-ify your life with Github Actions - Nicola Cort...
Shift Remote: Mobile - Devops-ify your life with Github Actions - Nicola Cort...Shift Remote: Mobile - Devops-ify your life with Github Actions - Nicola Cort...
Shift Remote: Mobile - Devops-ify your life with Github Actions - Nicola Cort...
Shift Conference
 
Shift Remote: WEB - GraphQL and React – Quick Start - Dubravko Bogovic (Infobip)
Shift Remote: WEB - GraphQL and React – Quick Start - Dubravko Bogovic (Infobip)Shift Remote: WEB - GraphQL and React – Quick Start - Dubravko Bogovic (Infobip)
Shift Remote: WEB - GraphQL and React – Quick Start - Dubravko Bogovic (Infobip)
Shift Conference
 

More from Shift Conference (20)

Shift Remote: AI: How Does Face Recognition Work (ars futura)
Shift Remote: AI: How Does Face Recognition Work  (ars futura)Shift Remote: AI: How Does Face Recognition Work  (ars futura)
Shift Remote: AI: How Does Face Recognition Work (ars futura)
 
Shift Remote: AI: Behind the scenes development in an AI company - Matija Ili...
Shift Remote: AI: Behind the scenes development in an AI company - Matija Ili...Shift Remote: AI: Behind the scenes development in an AI company - Matija Ili...
Shift Remote: AI: Behind the scenes development in an AI company - Matija Ili...
 
Shift Remote: AI: Smarter AI with analytical graph databases - Victor Lee (Ti...
Shift Remote: AI: Smarter AI with analytical graph databases - Victor Lee (Ti...Shift Remote: AI: Smarter AI with analytical graph databases - Victor Lee (Ti...
Shift Remote: AI: Smarter AI with analytical graph databases - Victor Lee (Ti...
 
Shift Remote: DevOps: Devops with Azure Devops and Github - Juarez Junior (Mi...
Shift Remote: DevOps: Devops with Azure Devops and Github - Juarez Junior (Mi...Shift Remote: DevOps: Devops with Azure Devops and Github - Juarez Junior (Mi...
Shift Remote: DevOps: Devops with Azure Devops and Github - Juarez Junior (Mi...
 
Shift Remote: DevOps: Autodesks research into digital twins for AEC - Kean W...
Shift Remote: DevOps: Autodesks research into digital twins for AEC -  Kean W...Shift Remote: DevOps: Autodesks research into digital twins for AEC -  Kean W...
Shift Remote: DevOps: Autodesks research into digital twins for AEC - Kean W...
 
Shift Remote: DevOps: When metrics are not enough, and everyone is on-call - ...
Shift Remote: DevOps: When metrics are not enough, and everyone is on-call - ...Shift Remote: DevOps: When metrics are not enough, and everyone is on-call - ...
Shift Remote: DevOps: When metrics are not enough, and everyone is on-call - ...
 
Shift Remote: DevOps: Modern incident management with opsgenie - Kristijan L...
Shift Remote: DevOps: Modern incident management with opsgenie -  Kristijan L...Shift Remote: DevOps: Modern incident management with opsgenie -  Kristijan L...
Shift Remote: DevOps: Modern incident management with opsgenie - Kristijan L...
 
Shift Remote: DevOps: Gitlab ci hands-on experience - Ivan Rimac (Barrage)
Shift Remote: DevOps: Gitlab ci hands-on experience - Ivan Rimac (Barrage)Shift Remote: DevOps: Gitlab ci hands-on experience - Ivan Rimac (Barrage)
Shift Remote: DevOps: Gitlab ci hands-on experience - Ivan Rimac (Barrage)
 
Shift Remote: DevOps: DevOps Heroes - Adding Advanced Automation to your Tool...
Shift Remote: DevOps: DevOps Heroes - Adding Advanced Automation to your Tool...Shift Remote: DevOps: DevOps Heroes - Adding Advanced Automation to your Tool...
Shift Remote: DevOps: DevOps Heroes - Adding Advanced Automation to your Tool...
 
Shift Remote: DevOps: An (Un)expected Journey - Zeljko Margeta (RBA)
Shift Remote: DevOps: An (Un)expected Journey - Zeljko Margeta (RBA)Shift Remote: DevOps: An (Un)expected Journey - Zeljko Margeta (RBA)
Shift Remote: DevOps: An (Un)expected Journey - Zeljko Margeta (RBA)
 
Shift Remote: Game Dev - Localising Mobile Games - Marta Kunic (Nanobit)
Shift Remote: Game Dev - Localising Mobile Games - Marta Kunic (Nanobit)Shift Remote: Game Dev - Localising Mobile Games - Marta Kunic (Nanobit)
Shift Remote: Game Dev - Localising Mobile Games - Marta Kunic (Nanobit)
 
Shift Remote: Game Dev - Challenges Introducing Open Source to the Games Indu...
Shift Remote: Game Dev - Challenges Introducing Open Source to the Games Indu...Shift Remote: Game Dev - Challenges Introducing Open Source to the Games Indu...
Shift Remote: Game Dev - Challenges Introducing Open Source to the Games Indu...
 
Shift Remote: Game Dev - Ghost in the Machine: Authorial Voice in System Desi...
Shift Remote: Game Dev - Ghost in the Machine: Authorial Voice in System Desi...Shift Remote: Game Dev - Ghost in the Machine: Authorial Voice in System Desi...
Shift Remote: Game Dev - Ghost in the Machine: Authorial Voice in System Desi...
 
Shift Remote: Game Dev - Building Better Worlds with Game Culturalization - K...
Shift Remote: Game Dev - Building Better Worlds with Game Culturalization - K...Shift Remote: Game Dev - Building Better Worlds with Game Culturalization - K...
Shift Remote: Game Dev - Building Better Worlds with Game Culturalization - K...
 
Shift Remote: Game Dev - Open Match: An Open Source Matchmaking Framework - J...
Shift Remote: Game Dev - Open Match: An Open Source Matchmaking Framework - J...Shift Remote: Game Dev - Open Match: An Open Source Matchmaking Framework - J...
Shift Remote: Game Dev - Open Match: An Open Source Matchmaking Framework - J...
 
Shift Remote: Game Dev - Designing Inside the Box - Fernando Reyes Medina (34...
Shift Remote: Game Dev - Designing Inside the Box - Fernando Reyes Medina (34...Shift Remote: Game Dev - Designing Inside the Box - Fernando Reyes Medina (34...
Shift Remote: Game Dev - Designing Inside the Box - Fernando Reyes Medina (34...
 
Shift Remote: Mobile - Efficiently Building Native Frameworks for Multiple Pl...
Shift Remote: Mobile - Efficiently Building Native Frameworks for Multiple Pl...Shift Remote: Mobile - Efficiently Building Native Frameworks for Multiple Pl...
Shift Remote: Mobile - Efficiently Building Native Frameworks for Multiple Pl...
 
Shift Remote: Mobile - Introduction to MotionLayout on Android - Denis Fodor ...
Shift Remote: Mobile - Introduction to MotionLayout on Android - Denis Fodor ...Shift Remote: Mobile - Introduction to MotionLayout on Android - Denis Fodor ...
Shift Remote: Mobile - Introduction to MotionLayout on Android - Denis Fodor ...
 
Shift Remote: Mobile - Devops-ify your life with Github Actions - Nicola Cort...
Shift Remote: Mobile - Devops-ify your life with Github Actions - Nicola Cort...Shift Remote: Mobile - Devops-ify your life with Github Actions - Nicola Cort...
Shift Remote: Mobile - Devops-ify your life with Github Actions - Nicola Cort...
 
Shift Remote: WEB - GraphQL and React – Quick Start - Dubravko Bogovic (Infobip)
Shift Remote: WEB - GraphQL and React – Quick Start - Dubravko Bogovic (Infobip)Shift Remote: WEB - GraphQL and React – Quick Start - Dubravko Bogovic (Infobip)
Shift Remote: WEB - GraphQL and React – Quick Start - Dubravko Bogovic (Infobip)
 

Recently uploaded

Draya Michele’s Son – Kniko Howard’s Rise to Fame.pptx
Draya Michele’s Son – Kniko Howard’s Rise to Fame.pptxDraya Michele’s Son – Kniko Howard’s Rise to Fame.pptx
Draya Michele’s Son – Kniko Howard’s Rise to Fame.pptx
ashishkumarrana9
 
Effective Tips for Creating the Best Rich Media Ads .pptx
Effective Tips for Creating the Best Rich Media Ads .pptxEffective Tips for Creating the Best Rich Media Ads .pptx
Effective Tips for Creating the Best Rich Media Ads .pptx
AirtoryInc
 
High Profile Girls Call ServiCe Chennai XX00XXX00X Tanisha Best High Class Ch...
High Profile Girls Call ServiCe Chennai XX00XXX00X Tanisha Best High Class Ch...High Profile Girls Call ServiCe Chennai XX00XXX00X Tanisha Best High Class Ch...
High Profile Girls Call ServiCe Chennai XX00XXX00X Tanisha Best High Class Ch...
shamrisumri
 
Network Layer and its protocols mod .pptx
Network Layer and its protocols mod .pptxNetwork Layer and its protocols mod .pptx
Network Layer and its protocols mod .pptx
cossykin19
 
Iot-Internet-of-Things_Industrial revolution 4.0-ppt.pptx
Iot-Internet-of-Things_Industrial revolution 4.0-ppt.pptxIot-Internet-of-Things_Industrial revolution 4.0-ppt.pptx
Iot-Internet-of-Things_Industrial revolution 4.0-ppt.pptx
DeepakKumar862274
 
Chennai Girls Call ServiCe X00XXX00XX Tanisha Best High Class Chennai Available
Chennai Girls Call ServiCe X00XXX00XX Tanisha Best High Class Chennai AvailableChennai Girls Call ServiCe X00XXX00XX Tanisha Best High Class Chennai Available
Chennai Girls Call ServiCe X00XXX00XX Tanisha Best High Class Chennai Available
shamrisumri
 
Jarren Duran Fuck EM T shirts Jarren Duran Fuck EM T shirts
Jarren Duran Fuck EM T shirts Jarren Duran Fuck EM T shirtsJarren Duran Fuck EM T shirts Jarren Duran Fuck EM T shirts
Jarren Duran Fuck EM T shirts Jarren Duran Fuck EM T shirts
exgf28
 
Female Service Girls Call Delhi 9873940964 Provide Best And Top Girl Service ...
Female Service Girls Call Delhi 9873940964 Provide Best And Top Girl Service ...Female Service Girls Call Delhi 9873940964 Provide Best And Top Girl Service ...
Female Service Girls Call Delhi 9873940964 Provide Best And Top Girl Service ...
elbertablack
 
Bitcoin vs Ethereum Which Crypto Performed Better in Q2, 2024.docx
Bitcoin vs Ethereum Which Crypto Performed Better in Q2, 2024.docxBitcoin vs Ethereum Which Crypto Performed Better in Q2, 2024.docx
Bitcoin vs Ethereum Which Crypto Performed Better in Q2, 2024.docx
SFC Today
 
@Girls @Call Chennai 🛬 XXXXXXXXXX 🛬 available 24*7 cash payment book now pay ...
@Girls @Call Chennai 🛬 XXXXXXXXXX 🛬 available 24*7 cash payment book now pay ...@Girls @Call Chennai 🛬 XXXXXXXXXX 🛬 available 24*7 cash payment book now pay ...
@Girls @Call Chennai 🛬 XXXXXXXXXX 🛬 available 24*7 cash payment book now pay ...
shamrisumri
 
How Salesforce Development in the UK is Driving Digital Transformation
How Salesforce Development in the UK is Driving Digital TransformationHow Salesforce Development in the UK is Driving Digital Transformation
How Salesforce Development in the UK is Driving Digital Transformation
Sweet Potato Tec
 
202254.com全网最高清影视香蕉影视,热门电影推荐,热门电视剧在线观看,免费电影,电影在线,在线观看。球华人在线電視劇,免费点播,免费提供最新高清的...
202254.com全网最高清影视香蕉影视,热门电影推荐,热门电视剧在线观看,免费电影,电影在线,在线观看。球华人在线電視劇,免费点播,免费提供最新高清的...202254.com全网最高清影视香蕉影视,热门电影推荐,热门电视剧在线观看,免费电影,电影在线,在线观看。球华人在线電視劇,免费点播,免费提供最新高清的...
202254.com全网最高清影视香蕉影视,热门电影推荐,热门电视剧在线观看,免费电影,电影在线,在线观看。球华人在线電視劇,免费点播,免费提供最新高清的...
ffg01100
 
Girls Call Mahipalpur 000XX00000 Provide Best And Top Girl Service And No1 in...
Girls Call Mahipalpur 000XX00000 Provide Best And Top Girl Service And No1 in...Girls Call Mahipalpur 000XX00000 Provide Best And Top Girl Service And No1 in...
Girls Call Mahipalpur 000XX00000 Provide Best And Top Girl Service And No1 in...
mahigarg2024#G05
 
Kolkata @Girls @Call WhatsApp Numbers 🫦0000XX0000🫦 List For Friendship Girls ...
Kolkata @Girls @Call WhatsApp Numbers 🫦0000XX0000🫦 List For Friendship Girls ...Kolkata @Girls @Call WhatsApp Numbers 🫦0000XX0000🫦 List For Friendship Girls ...
Kolkata @Girls @Call WhatsApp Numbers 🫦0000XX0000🫦 List For Friendship Girls ...
paridubey2024#G05
 
workbook and project U5 1ºsecundaria.pdf
workbook and project U5 1ºsecundaria.pdfworkbook and project U5 1ºsecundaria.pdf
workbook and project U5 1ºsecundaria.pdf
anya2024forgya
 
optimized green synthesis characterization and evaluation
optimized green synthesis characterization and evaluationoptimized green synthesis characterization and evaluation
optimized green synthesis characterization and evaluation
ManojKumarr75
 
Trading Strategy for London silver bullet
Trading Strategy for London silver bulletTrading Strategy for London silver bullet
Trading Strategy for London silver bullet
OkgatoSemadi1
 
Megalive99 Situs Betting Online Gacor Terpercaya
Megalive99 Situs Betting Online Gacor TerpercayaMegalive99 Situs Betting Online Gacor Terpercaya
Megalive99 Situs Betting Online Gacor Terpercaya
Megalive99
 
Book dating , international dating phgra
Book dating , international dating phgraBook dating , international dating phgra
Book dating , international dating phgra
thomaskurtha9
 
AWS Networking Basic , tanapat limsaiprom
AWS Networking Basic , tanapat limsaipromAWS Networking Basic , tanapat limsaiprom
AWS Networking Basic , tanapat limsaiprom
ธนาพัฒน์ ลิ้มสายพรหม
 

Recently uploaded (20)

Draya Michele’s Son – Kniko Howard’s Rise to Fame.pptx
Draya Michele’s Son – Kniko Howard’s Rise to Fame.pptxDraya Michele’s Son – Kniko Howard’s Rise to Fame.pptx
Draya Michele’s Son – Kniko Howard’s Rise to Fame.pptx
 
Effective Tips for Creating the Best Rich Media Ads .pptx
Effective Tips for Creating the Best Rich Media Ads .pptxEffective Tips for Creating the Best Rich Media Ads .pptx
Effective Tips for Creating the Best Rich Media Ads .pptx
 
High Profile Girls Call ServiCe Chennai XX00XXX00X Tanisha Best High Class Ch...
High Profile Girls Call ServiCe Chennai XX00XXX00X Tanisha Best High Class Ch...High Profile Girls Call ServiCe Chennai XX00XXX00X Tanisha Best High Class Ch...
High Profile Girls Call ServiCe Chennai XX00XXX00X Tanisha Best High Class Ch...
 
Network Layer and its protocols mod .pptx
Network Layer and its protocols mod .pptxNetwork Layer and its protocols mod .pptx
Network Layer and its protocols mod .pptx
 
Iot-Internet-of-Things_Industrial revolution 4.0-ppt.pptx
Iot-Internet-of-Things_Industrial revolution 4.0-ppt.pptxIot-Internet-of-Things_Industrial revolution 4.0-ppt.pptx
Iot-Internet-of-Things_Industrial revolution 4.0-ppt.pptx
 
Chennai Girls Call ServiCe X00XXX00XX Tanisha Best High Class Chennai Available
Chennai Girls Call ServiCe X00XXX00XX Tanisha Best High Class Chennai AvailableChennai Girls Call ServiCe X00XXX00XX Tanisha Best High Class Chennai Available
Chennai Girls Call ServiCe X00XXX00XX Tanisha Best High Class Chennai Available
 
Jarren Duran Fuck EM T shirts Jarren Duran Fuck EM T shirts
Jarren Duran Fuck EM T shirts Jarren Duran Fuck EM T shirtsJarren Duran Fuck EM T shirts Jarren Duran Fuck EM T shirts
Jarren Duran Fuck EM T shirts Jarren Duran Fuck EM T shirts
 
Female Service Girls Call Delhi 9873940964 Provide Best And Top Girl Service ...
Female Service Girls Call Delhi 9873940964 Provide Best And Top Girl Service ...Female Service Girls Call Delhi 9873940964 Provide Best And Top Girl Service ...
Female Service Girls Call Delhi 9873940964 Provide Best And Top Girl Service ...
 
Bitcoin vs Ethereum Which Crypto Performed Better in Q2, 2024.docx
Bitcoin vs Ethereum Which Crypto Performed Better in Q2, 2024.docxBitcoin vs Ethereum Which Crypto Performed Better in Q2, 2024.docx
Bitcoin vs Ethereum Which Crypto Performed Better in Q2, 2024.docx
 
@Girls @Call Chennai 🛬 XXXXXXXXXX 🛬 available 24*7 cash payment book now pay ...
@Girls @Call Chennai 🛬 XXXXXXXXXX 🛬 available 24*7 cash payment book now pay ...@Girls @Call Chennai 🛬 XXXXXXXXXX 🛬 available 24*7 cash payment book now pay ...
@Girls @Call Chennai 🛬 XXXXXXXXXX 🛬 available 24*7 cash payment book now pay ...
 
How Salesforce Development in the UK is Driving Digital Transformation
How Salesforce Development in the UK is Driving Digital TransformationHow Salesforce Development in the UK is Driving Digital Transformation
How Salesforce Development in the UK is Driving Digital Transformation
 
202254.com全网最高清影视香蕉影视,热门电影推荐,热门电视剧在线观看,免费电影,电影在线,在线观看。球华人在线電視劇,免费点播,免费提供最新高清的...
202254.com全网最高清影视香蕉影视,热门电影推荐,热门电视剧在线观看,免费电影,电影在线,在线观看。球华人在线電視劇,免费点播,免费提供最新高清的...202254.com全网最高清影视香蕉影视,热门电影推荐,热门电视剧在线观看,免费电影,电影在线,在线观看。球华人在线電視劇,免费点播,免费提供最新高清的...
202254.com全网最高清影视香蕉影视,热门电影推荐,热门电视剧在线观看,免费电影,电影在线,在线观看。球华人在线電視劇,免费点播,免费提供最新高清的...
 
Girls Call Mahipalpur 000XX00000 Provide Best And Top Girl Service And No1 in...
Girls Call Mahipalpur 000XX00000 Provide Best And Top Girl Service And No1 in...Girls Call Mahipalpur 000XX00000 Provide Best And Top Girl Service And No1 in...
Girls Call Mahipalpur 000XX00000 Provide Best And Top Girl Service And No1 in...
 
Kolkata @Girls @Call WhatsApp Numbers 🫦0000XX0000🫦 List For Friendship Girls ...
Kolkata @Girls @Call WhatsApp Numbers 🫦0000XX0000🫦 List For Friendship Girls ...Kolkata @Girls @Call WhatsApp Numbers 🫦0000XX0000🫦 List For Friendship Girls ...
Kolkata @Girls @Call WhatsApp Numbers 🫦0000XX0000🫦 List For Friendship Girls ...
 
workbook and project U5 1ºsecundaria.pdf
workbook and project U5 1ºsecundaria.pdfworkbook and project U5 1ºsecundaria.pdf
workbook and project U5 1ºsecundaria.pdf
 
optimized green synthesis characterization and evaluation
optimized green synthesis characterization and evaluationoptimized green synthesis characterization and evaluation
optimized green synthesis characterization and evaluation
 
Trading Strategy for London silver bullet
Trading Strategy for London silver bulletTrading Strategy for London silver bullet
Trading Strategy for London silver bullet
 
Megalive99 Situs Betting Online Gacor Terpercaya
Megalive99 Situs Betting Online Gacor TerpercayaMegalive99 Situs Betting Online Gacor Terpercaya
Megalive99 Situs Betting Online Gacor Terpercaya
 
Book dating , international dating phgra
Book dating , international dating phgraBook dating , international dating phgra
Book dating , international dating phgra
 
AWS Networking Basic , tanapat limsaiprom
AWS Networking Basic , tanapat limsaipromAWS Networking Basic , tanapat limsaiprom
AWS Networking Basic , tanapat limsaiprom
 

Shift Money 2019 - Why your data quality problem is not your user's fault - Murat Abur (Suade)

  • 1. DISCOVERING FIRE OR WHY YOUR DATA QUALITY PROBLEM IS NOT THE USER’S FAULT www.suade.org/fire
  • 2. HOW DO WE PREVENT THE NEXT FINANCIAL CRISIS? REGTECH...
  • 3. R(x1, x2, ..., xn) = y where, y = metrics, reports, analytics, insights R = rules, regulations x = data THE REGULATORY EQUATION
  • 4.  We think garbage is acceptable  Garbage input is not our problem  Garbage output also not our problem TECHNOLOGY SAYS GARBAGE IN, GARBAGE OUT Leverage your data model to prevent garbage in!
  • 5. Objectives:  Quality, control and governance for Financial Institutions  Comparability and supervision of underlying data for Regulators  Seamless integration for FinTechs and third parties FINANCIAL REGULATION (FIRE) DATA FORMAT Github:  https://github.com/suadelabs/fire
  • 6. WHAT IS BAD DATA? Leverage your data model to prevent garbage in! Bad data is data with:  No definitions  No validations  Missing values  Wrong values  Flags  Derived values https://xkcd.com/1838/
  • 7. BAD DATA: NO DEFINITIONS Amount in USD Total Deposits ? ACT_DEP_BAL R10_USD_AMT GTEE_AMT LIQ14_TOT_BAL R12_AMT 90 60 60 30 120 Rule 1: No attribute without definition Data: Output:
  • 8. BAD DATA: NO VALIDATIONS Rule 2: No attribute without limitation Data: CCY START_DT CATEGORY BALANCE USDRATE USD_BAL GBP 2019-03-08 Retail 30 1.20 36 ALL 18/11/20 International -20 0.01 0 USD 18/11/19 Govt 15 100 15
  • 9. BAD DATA: MISSING/WRONG VALUES OK. Missing values tell you what you don’t know Data: CCY START_DT CATEGORY BALANCE USDRATE USD_BAL GBP 2019-03-08 Retail 30 1.20 36 ALL International -20 0.01 USD Govt 15 15
  • 10. Total USD Deposits from SMEs ? Deposits from Govts ? Deposits from other customers 15 O_REP R10_TYPE CATEGORY USD_AMT SME? GOVT? GB Central Bank Retail 30 True True UK Small Business International 20 False True DE Financial Govt 15 False False BAD DATA: FLAGS Rule 3: Flags are not your friend Output:Data:
  • 11. BAD DATA: DERIVED VALUES Amount in EUR Total Deposits ? EURUSD EURGBP GBP_AMT TYPE USD_AMT 1.00 1.16 100 DEP 120 Rule 4: Avoid attributes that can easily be deduced from others
  • 12. ADHERE TO PRINCIPLES, NOT YOUR BOSS Good data has:  Definitions  Validations  No flags  No derivable values
  • 13. FIRE GUIDING PRINCIPLES 1. Data attributes should always be true:  “...every pull request requires a corresponding, documented, legal reference, preferably to a currently in-force financial regulation.” 2. Data attributes should be atomic  “One property should not be derivable from other properties.“ 3. Data attributes should be consistent  “Schema properties should try to avoid logical inconsistencies.”
  • 14. TO CONCLUDE Death List OOOO----RENRENRENREN ISHIIISHIIISHIIISHII(added columns without defintions)  VERNITAVERNITAVERNITAVERNITA GREENGREENGREENGREEN (added columns without validations) BUDDBUDDBUDDBUDD(added a whole bunch of flags)  ELLEELLEELLEELLE DRIVERDRIVERDRIVERDRIVER (added redundant information) BILLBILLBILLBILL (allowed this to happen)