Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Measuring and Managing Credit Risk With Machine Learning and Artificial Intelligence

473 views

Published on

In recent years, technological developments have undergone in-depth analysis among banks, but we are still far from attaining mature levels both at the methodological and at the credit granting, monitoring and control process levels. Banks should equip themselves with new and more structured Model Risk frameworks to manage new Machine Learning model validation paradigms. Learn more from Accenture Finance & Risk: https://accntu.re/2qGUUMx

Published in: Technology

Comments are closed

  • Be the first to comment

  • Be the first to like this

Measuring and Managing Credit Risk With Machine Learning and Artificial Intelligence

  1. 1. MEASURING AND MANAGING CREDIT RISK WITH MACHINE LEARNING AND ARTIFICIAL INTELLIGENCE: A NEW ERA? STEFANO BONINI, ACCENTURE FINANCE & RISK GIULIANA CAIVANO, ACCENTURE FINANCE & RISK
  2. 2. TOPICS ARTIFICIAL INTELLIGENCE AND MACHINE LEARNING: A JOURNEY THROUGH TIME1 MACHINE LEARNING IN RISK MANAGEMENT2 EVOLUTION OF CREDIT RISK3 4 CONCLUSIONS AND NEXT STEPS Copyright © 2019 Accenture. All rights reserved. 2
  3. 3. ARTIFICIAL INTELLIGENCE AND MACHINE LEARNING A JOURNEY THROUGH TIME “Are there imaginable digital computers which would do well in the imitation game?“1 1950-1960 1990-2000 1970-1980 TODAY… Copyright © 2019 Accenture. All rights reserved. 3 Alan Turing proposes Turing Test as a measure of machine intelligence7 1950 1966 The MIT Computer Science & Artificial Intelligence Lab creates Eliza - the Chatbot8 20142001 UBS AG uses Sqreem Technologies Pte. Ltd. Artificial Intelligence to provide financial advice11 2019 Amazon Alexa™ is a cloud-based voice service developed by Amazon.com Inc., and used in Amazon Echo™ devices12 Robots beat humans in a simulated financial trading competition; the Robots made 7% more cash than the humans10 1955 1987 The term "Artificial Intelligence“ is first coined by computer scientist John McCarthy for the Dartmouth College AI conference2 Security Pacific National Bank introduces fraud prevention task based on artificial neural network3 2013 “KENSHO,” the financial answer machine combines latest big data and machine learning techniques to analyze how real-world events affect markets4 Google Duplex™ assistive technology, a service to allow an AI assistant to book an appointment by phone6 20182017 A top tier investment bank adopts COiN (Contract Intelligence), an Artificial Intelligence tool that analyzes legal documents and contracts using image recognition5 1997 The Deep Blue chess machine defeats world chess champion, Garry Kasparov9
  4. 4. MACHINE LEARNING IN RISK MANAGEMENT GLOBAL VIEW Degree of banks’ maturity with respect to the application of Machine Learning in credit risk Copyright © 2019 Accenture. All rights reserved. 4 MATURITY OF MACHINE LEARNING BASED ON COMPANY SIZE (TOTAL ASSETS, USD) APPLICATIONS AREAS OF MACHINE LEARNING IN CREDIT RISK $1t plus $500b-$1t $150b-$500b Under $150b 0 20 40 60 80 100 0 10 20 30 40 50 60 70 80 90 Collections Credit Monitoring Credit Scoring and decisioning Provisioning Economic Capital Stress Testing Regulatory Capital Mature Intermediate BeginnerMature Intermediate Beginner None KEY BENEFITS OF APPLYING MACHINE LEARNING MORE PRECISE MODELS BETTER DATA USAGE MORE EFFICIENCY IN MODEL DEVELOPMENT DATA DEFICIENCIES ADDRESSED Source: Institute of International Finance – Machine Learning in Credit Risk Summary Report – Nov 2018 Regulatory capital Stress Testing Economic capital Provisioning Credit classification and decisioning Credit Monitoring Collections Mature Intermediate
  5. 5. LEGENDA  Si  No, ma: No, ma lo reputo potenzialmente utile/applicabile  No, non: No, e non lo reputo potenzialmente utile/applicabile  N/A: Non sa/Non Risponde  There is a willingness to use machine learning in model estimation, given the high amount of data available to banks; despite the advantages, only 25% of banking players adopt it for internal model estimation and 19% for rating scales enhancement  A Few (8%) banks use Machine Learning techniques to do stress test and to manage non-performing loans  Over 50% of banks, while not applying them, consider Machine Learning techniques as potentially useful and applicable in each area that was previously analyzed Accenture Analysis based on the evidence from the survey pool MACHINE LEARNING IN RISK MANAGEMENT ITALIAN VIEW Copyright © 2019 Accenture. All rights reserved. 5 CREDIT RISK MODEL ESTIMATION / DEVELOPMENT RATING SCALE STRESS TEST DATA QUALITY EARLY WARNING CUSTOMER SEGMENTATION NON-PERFORMING LOANS Yes– 25.0% No, but– 52.4% Yes – 19.3% No, but – 59.1% Yes– 8.6% No, but – 75.0% Yes – 10.0% No, but – 76.2% Yes– 7.0% No, but – 71.4% Yes– 8.6% No, but – 68.7% No, but – 61.9%Yes– 7.5% No, not /NA – 22.7% No, not /NA – 21.6% No, not /NA – 13.8% No, not /NA – 30.6% No, not /NA – 21.6% No, not/NA – 23.6% No, not /NA – 16.4% No, but: No, but I consider it potentially useful/applicable *No, not/NA: No, and I don't think it's useful/ Don’t know or don’t answerLEGEND EVIDENCES Source: INTELLIGENZA ARTIFICIALE: L’APPLICAZIONE DI MACHINE LEARNING E PREDICTIVE ANALYTICS NEL RISK MANAGEMENT, AIFRIM position paper, March 13, 2019. Access at: http://www.aifirm.it/presentazione-position-paper-aifirm-intelligenza-artificiale-lapplicazione-di-machine-learning-e-predictive-analytics-nel- risk-management/
  6. 6. MAIN CHALLENGES SHOULD PROBE ALGORITHMS TO PRODUCE INTERMEDIATE RESULTS THAT EXPLAIN WHAT, HOW AND WHY COMPLEX ALGORITHMS FOLLOW A LOGIC IN WHICH THE ROUTES DEVELOP DYNAMICALLY AND ARE MORE DIFFICULT TO EXPLAIN SHOULD HAVE AN INTELLIGENT AND PERSPECTIVE VIEW OF RESULTS - CONSIDERING THE COMPLEXITY AND AMOUNT OF POSSIBLE OUTPUTS MACHINE LEARNING IN RISK MANAGEMENT CHALLENGES AND TESTING “…it is inevitable that the more AI enters our lives, the more we are not going to be willing to interact with black boxes that just tell us what to do without ever telling us why.”13 Copyright © 2019 Accenture. All rights reserved. 6  The new European Banking Authority Guidelines on loan origination and monitoring require banks to perform sensitivity analysis to test the sustainability of counterparties, simulating adverse conditions, considering both market and idiosyncratic variables  Banks that use advanced technologies for credit supply processes should take into consideration the risk deriving from these technologies (e.g. bias deriving from Machine Learning models) in their risk management frameworks and be able to adequately govern the outcomes of the models for strength THE NEW REGULATORY GUIDANCE14 INTERPRETABILITY ACCURACYINESTIMATION NEURAL NETWORK: DEEP LEARNING RANDOM FOREST SVM DECISION TREES LOGISTIC REGRESSION
  7. 7. EVOLUTION OF CREDIT RISK USE CASES EXAMPLES Copyright © 2019 Accenture. All rights reserved. 7 ESTIMATE& VALIDATIONOF INTERNAL MODELS Inclusion of new types of data for model estimation Standardization and efficiency of repetitive tasks by dedicating internal resources to challenge activities STRESS TESTING  Development of multiple scenarios in an automated and objective way  Increased objectivity in defining scenarios 2nd LEVEL CONTROLS ON CREDIT  Reduction of control execution times  Error reduction and performance increase EARLY WARNING  Automation of a large number of indicators, enhancing the predictive performance of the model  Use of social data for the preventive interception of anomalies STRESS TESTING EARLY WARNING VALIDAZI ONE CONTROL LI DI II LIVELLO
  8. 8. SFIDEOpportunity to use different data (e.g. data on real estate values of external companies) Adding information through deep learning techniques (e.g. reading the financial statements’ notes) Role of Open Banking (real-time knowledge of all system customer information, not just liabilities as a risk center) BACKTESTING JUST IN TIME EVOLUTION OF CREDIT RISK ESTIMATE AND VALIDATION OF INTERNAL MODELS OUTPUT Δ ≥ X% Y% ≥ Δ ≥ X% Δ ≤ Y% TEST OK TEST KO TEST OK APPLICATION MODEL APPLICATION BACKTESTING SAMPLE ANALYSIS MACHINE LEARNING & ROBOTICS Automatic quantitative analysis of model performance based on adaptive Machine Learning- based tools BENCHMARK PROBABILITY OF DEFAULT MODEL SUPERVISED MACHINE LEARNING BENCHMARK MODEL 1 2 3 4 5 6 7 % DR PD PROBABILITY OF DEFAULT MODEL BENCHMARK SCALE UNSUPERVISED MACHINE LEARNING BENCHMARK RATING SCALE DEVELOPMENT DATA BENCHMARKING TRAINED MACHINE Analysis of model documentation based on what has been learned from historical / regulatory documentation ANALYSIS OF MODEL REPORTS FINAL REPORT ANALYSIS OUTCOME  COMPLETENESS OF THE TOPICS COVERED  Description of the Section Template  Input Date  Defining Default Section  …  COMPLIANCE OF THE DOCUMENT STRUCTURE WITH THE LEGISLATION  … Copyright © 2019 Accenture. All rights reserved. DOCUMENT ANALYSIS USE OF NEW INFORMATION 8
  9. 9. The introduction of automation and Machine Learning methodologies can lead to important efficiency in the credit monitoring process, significantly improving the predictive performance of Early Warning models EVOLUTION OF CREDIT RISK EARLY WARNING MODELS 9  High number of experientially defined indicators  Different levels of severity assigned by experts  High % of false positives in the face of few correct ignitions INITIAL SITUATION ADVANTAGES  Significant decrease in the number of early warning indicators used  Affiliation of indicators to the appropriate level of severity  Significant decrease in false positives  Significant increase in correct indicator switch on SERIOUSVERY SERIOUS LIGHT STANDARD METHODOLOGY MACHINE EARNING Application of supervised techniques (e.g. decision trees) to identify the correct severity level of each indicator HEURISTIC APPROACH Automate the indicator selection process through a heuristic approach SOCIAL DATA USAGE Copyright © 2019 Accenture. All rights reserved. Inclusion of new data sources via social media - indicative of reviews and customer trends • Deep learning techniques that through analysis and reading are able to extrapolate information from large amounts of text • Predictive analytics techniques to assess the correlation between web info and customer creditworthiness HOW? NEW TREND
  10. 10. Machine Learning and Robotic Process Automation techniques find numerous applications in the perimeter of 2nd level controls, from Key Risk Indicators identification to control development and automation AUTOMATED COLLECTION OF INFORMATION FROM APPLICATIONS AUTOMATED FILL IN OF THE CONTROL REPORT AUTOMATED PRODUCTION OF THE SUMMARY CONTROL REPORT MACHINE LEARNING ROBOTIC PROCESS AUTOMATION  ENHANCING THE PREDICTIVE EFFECTIVENESS OF ADOPTED STATISTICAL TECHNIQUES  ENHANCING THE COMPUTATIONAL / ANALYSIS CAPACITY  ENHANCING THE INFORMATION CONTRIBUTION OF EACH VARIABLE  REDUCTION OF TIME / COSTS OF THE CONTROL PROCESS  DISPLACEMENT OF RESOURCES FROM «REPETITIVE» ACTIVITIES TO ACTIVITIES THAT ENHANCE “IT” AND INCREASE SKILLS  REDUCTION OF OPERATIONAL ERRORS ADVANTAGES ADVANTAGES Recourse to supervised / unsupervised Machine Learning techniques for: IDENTIFICATION AND ANALYSIS OF VARIABLES AUTOMATIC SAMPLE EXTRACTION EVOLUTION OF CREDIT RISK 2ND LEVEL CONTROLS ON CREDIT Copyright © 2019 Accenture. All rights reserved. 10
  11. 11. EVOLUTION OF CREDIT RISK STRESS TESTING  Current stress testing approaches are grounded in models based on a priori assumptions about the relationships between market variables and plausible idiosyncratic variables  A Machine Learning solution could allow greater effectiveness and robustness of stress testing approaches SCENARIOS DEFINED FROM THE LIMITS OF CURRENT APPROACHES TO THE NEW APPROACHES OF STRESS TESTING MACHINE LEARNING Copyright © 2019 Accenture. All rights reserved. 11  MODEL HYPOTHESES The robustness of current approaches is based on formalized hypotheses for the purposes of the models  NON-LINEAR RELATIONSHIPS They are usually not properly captured by models  HISTORICAL EVENTS The past is not always applicable to the present  IDENTIFYING BIASES The distortions of current models are not easily identifiable MODEL TRANSLATION Supervised machine learning + expert judgment MODEL VALIDATION Historical data backtesting DATA GRANULARITY& GOVERNANCE METHODOLOGICAL FLEXIBILITY Data management Data quality Data ownership Data flows  METHODOLOGIC FLEXIBILITY Machine learning approaches allow firms to independently identify the causal structure of the relationships between input variables (even non-linear) without formal a priori hypotheses  DATA CENTRALITY Machine learning approaches require a large amount of granular (idiosyncratic and market) data input, so data management is crucial  INTERPRETATION OF RESULTS The results of the machine learning models should be interpreted by experts to identify plausible scenarios  VALIDATION OF MODELS Validation plays a crucial role in maintaining the robustness of the model framework and outcomes THE COMPONENTS OF THE MACHINE LEARNING APPROACH Credit portfolio Client ID Stipulation Payment
  12. 12. CONCLUSIONS AND NEXT STEPS A NEW ERA FOR CREDIT RISK ? Copyright © 2019 Accenture. All rights reserved. 12 Regulators are starting to incorporate these innovations into new legislation by outlining new features aimed at improving and enhancing credit analysis through the use of more advanced statistical methods and enhancement techniques that can evaluate both new information sources and data in «real time» In recent years, technological developments have undergone in-depth analysis among banks, but we are still far from attaining mature levels both at the methodological and at the credit granting, monitoring and control process levels All banking functions that participate in the credit cycle (Lending, Risk, Workout) and Control functions should be equipped with tools and capabilities to:  exploit the methodological / modeling potential even in more managerial areas, not just the regulatory space  rethink operational practices through the application of various levels of Machine Learning and Artificial Intelligence sophistication to improve operational efficiency With the increase in models devoted to bank management, the impact of model risk should increase and banks should equip themselves with new and more structured Model Risk frameworks to manage new Machine Learning model validation paradigms
  13. 13. Stefano Bonini, PhD, CStat, PStat® stefano.bonini@accenture.com Giuliana Caivano, PhD giuliana.caivano@accenture.com CONTACTS Copyright © 2019 Accenture. All rights reserved. 13
  14. 14. REFERENCES 1. “Computing Machinery and Intelligence,” A.M. Turing, UMBC, 1950. Access at: https://www.csee.umbc.edu/courses/471/papers/turing.pdf 2. “A proposal for the Dartmouth summer research project on Artificial Intelligence,” J. McCarthy, August 31, 1955. Access at: http://www- formal.stanford.edu/jmc/history/dartmouth/dartmouth.html 3. “The Fraud Examiner – AI in the fight against fraud,” M. Wilder, Association of Certified Fraud Examiners, (1990). Access at: https://www.acfe.com/fraud-examiner.aspx?id=4294999437 4. “Kensho: The Financial Answer Machine,” L. Shin, Forbes, December 9, 2015. Access at: https://www.forbes.com/sites/laurashin/2015/12/09/kensho- the-financial-answer-machine/#7fe18e4ef310 5. “An AI Completed 360,000 Hours of Finance Work in Just Seconds,” D. Galeon, Futurism.com, March 8, 2017. Access at: https://futurism.com/an-ai- completed-360000-hours-of-finance-work-in-just-seconds 6. “What is Google Duplex? The smartest chatbot ever explained,” E. Rawes, Digital Trends, April 3, 2019. Access at: https://www.digitaltrends.com/home/what-is-google-duplex/ 7. “Computing Machinery and Intelligence,” A.M. Turing, UMBC, 1950. Access at: https://www.csee.umbc.edu/courses/471/papers/turing.pdf 8. “Story of ELIZA, the first chatbot developed in 1966,” M. Salecha, Analytics India Magazine, October 5, 2016. Access at: https://www.analyticsindiamag.com/story-eliza-first-chatbot-developed-1966/ 9. “Garry Kasparov and the game of artificial intelligence,” M. Sollinger, PRI, January 5, 2018. Access at: https://www.pri.org/stories/2018-01-05/garry- kasparov-and-game-artificial-intelligence 10. “Robots beat humans in trading battle,” BBC News, August 8, 2001. Access at: http://news.bbc.co.uk/2/hi/business/1481339.stm 11. “UBS Turns to Artificial Intelligence to Advise Clients,” J. Vögeli, Bloomberg, December 7, 2014. Access at: https://www.bloomberg.com/news/articles/2014-12-07/ubs-turns-to-artificial-intelligence-to-advise-wealthy-clients 12. “Why Amazon Alexa Is Always Listening To Your Conversations: Analysis” J. Su, Forbes, May 16, 2019. Access at: https://www.forbes.com/sites/jeanbaptiste/2019/05/16/why-amazon-alexa-is-always-listening-to-your-conversations-analysis/#36291ac72378 13. “Machine learning hits explainability barrier,” D. DeFrancesco, Risk.net, November 6, 2018. Access at: https://www.risk.net/risk- management/6008221/machine-learning-hits-explainability-barrier 14. “Draft Guidelines on loan origination and monitoring,” European Banking Authority, Consultation Paper, June 19, 2019. Access at: https://eba.europa.eu/documents/10180/2831176/CP+on+GLs+on+loan+origination+and+monitoring.pdf Copyright © 2019 Accenture. All rights reserved. 14
  15. 15. MEASURING AND MANAGING CREDIT RISK WITH MACHINE LEARNING & ARTIFICIAL INTELLIGENCE: A NEW ERA? About Accenture Accenture is a leading global professional services company, providing a broad range of services and solutions in strategy, consulting, digital, technology and operations. Combining unmatched experience and specialized skills across more than 40 industries and all business functions—underpinned by the world’s largest delivery network —Accenture works at the intersection of business and technology to help clients improve their performance and create sustainable value for their stakeholders. With more than 482,000 people serving clients in more than 120 countries, Accenture drives innovation to improve the way the world works and lives. Visit us at www.accenture.com Disclaimer This presentation is intended for general informational purposes only and does not take into account the reader’s specific circumstances, and may not reflect the most current developments. Accenture disclaims, to the fullest extent permitted by applicable law, any and all liability for the accuracy and completeness of the information in this presentation and for any acts or omissions made based on such information. Accenture does not provide legal, regulatory, audit, or tax advice. Readers are responsible for obtaining such advice from their own legal counsel or other licensed professionals.

×