Your SlideShare is downloading. ×
Vision 2014: Regulatory Requirements for Model Risk Governance
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Vision 2014: Regulatory Requirements for Model Risk Governance

109
views

Published on

Expert Insights: Regulatory update and best practices for complying with the latest model risk governance requirements. …

Expert Insights: Regulatory update and best practices for complying with the latest model risk governance requirements.

Published in: Economy & Finance, Business

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
109
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. ©2014 Experian Information Solutions, Inc. All rights reserved. Experian and the marks used herein are service marks or registered trademarks of Experian Information Solutions, Inc. Other product and company names mentioned herein are the trademarks of their respective owners. No part of this copyrighted work may be reproduced, modified, or distributed in any form or manner without the prior written permission of Experian. Experian Public. Regulatory requirements for model risk governance continue to evolve Is your organization prepared to meet these requirements? Linda Haran Experian Robert Stone Experian #vision2014
  • 2. 2©2014 Experian Information Solutions, Inc. All rights reserved. Experian Public.  Regulatory environment and Model Risk Governance ServicesSM  Model stress testing  Model risk governance best practices Agenda
  • 3. 3©2014 Experian Information Solutions, Inc. All rights reserved. Experian Public. The regulatory environment and Model Risk Governance ServicesSM Robert Stone
  • 4. 4©2014 Experian Information Solutions, Inc. All rights reserved. Experian Public. Bank compliance: Model Risk Governance U.S. standards – OCC Bulletin 2011-12 Bulletin 2000-16 2000 Bulletin 2011-12 2011 The Supervisory Guidance on Model Risk Management OCC Bulletin 2011–12 extends the scope beyond standard model validation to policies, practices, standards for:  Benchmarking  Model development  Model use and implementation  Model governance and controls All national banks and federal savings associations are directly impacted Other financial services institutions are indirectly impacted and should consider as industry best practice
  • 5. 5©2014 Experian Information Solutions, Inc. All rights reserved. Experian Public. Model Risk Governance Regulatory requirements Policies All financial institutions that use models in their decisioning processes must ensure that their internal policies and procedures are consistent with the guidance What is expected of institutions for compliance? Assessment of model risk Banks need to mitigate the potential risks arising from the reliance on models that may be improperly validated or tested Managing model risk Models risk can be considerably reduced through rigorous model validation procedures Validation requirements The OCC requires that financial institutions perform ongoing model validations at least once a year, include an independent review, and produce proper model documentation
  • 6. 6©2014 Experian Information Solutions, Inc. All rights reserved. Experian Public. The way forward  Financial regulators expect greater use of models by financial entities: ► “The experience and judgment of developers, as much as their technical knowledge, greatly influence the appropriate selection of inputs and processing components. The training and experience of developers exercising such judgment affects the extent of model risk.”  Therefore, regulatory examination and supervision regimes will increasing their oversight and review of the institution’s management of its models in three stages: 1. Through examination of model development, implementation, and use consistent with the 2011 guidance 2. Through examination of a sound model validation process 3. Through examination of a governance framework with defined roles and responsibilities for clear communication of the model’s limitations and assumptions as well as the authority to restrict the model’s usage by the institution Model Risk Governance The regulatory horizon
  • 7. 7©2014 Experian Information Solutions, Inc. All rights reserved. Experian Public. Model stress testing analysis Linda Haran
  • 8. 8©2014 Experian Information Solutions, Inc. All rights reserved. Experian Public. Model risk governance Model validation  Back-testing  Benchmarking  Sensitivity analysis  Stress-testing Analytical approach applies to both generic and custom built credit risk models  Stress-testing
  • 9. 9©2014 Experian Information Solutions, Inc. All rights reserved. Experian Public. Unconventional times for model governance  OCC Bulletin 2011–12 defines stress testing of a model to: ► “Evaluate model performance over a wide range of parameter input values, including extreme values, that are correlated with macro economic factors”  The model stress testing analysis will determine how the model performs in extreme economic environments Stress testing Requirements and model risk assessment
  • 10. 10©2014 Experian Information Solutions, Inc. All rights reserved. Experian Public. Capturing the impact of a recession to a risk model through its natural cycle Economic cycling Stressing the model through a downturn Stressing model performance on the overall population and by risk segments of prime and non-prime Increased utilization, delinquency behaviors Late stage recession Inquires and new trades increasing Early stage recession
  • 11. 11©2014 Experian Information Solutions, Inc. All rights reserved. Experian Public.  Does a model tend to lose its ability to rank order when stressed under economically difficult conditions?  What could cause the most risk and instability of your models in a highly stressed scenario? Stress testing Analysis hypothesis
  • 12. 12©2014 Experian Information Solutions, Inc. All rights reserved. Experian Public.  Create a series of economic attributes in the context of time ► Unemployment ► Consumer Price Index ► Gross Domestic Product  Determine what types of consumer credit characteristics correlate to changing economic conditions over time  Identify those highly correlated attributes that are included in the algorithm of an Experian bureau risk model  Determine stress values for each attributes identified  Asses model performance under stressed conditions Stress testing Defining the input parameters
  • 13. 13©2014 Experian Information Solutions, Inc. All rights reserved. Experian Public. 0 1,000 2,000 3,000 4,000 5,000 900-900 860-879 820-839 780-799 740-759 700-719 660-679 620-639 580-599 540-559 500-519 460-479 420-439 381-399 300-370 Normalized population distribution Stressed inquiry attribute distribution 0 500 1,000 1,500 2,000 900-900 860-879 820-839 780-799 740-759 700-719 660-679 620-639 580-599 540-559 500-519 460-479 420-439 381-399 300-370 Normalized population distribution Stressed inquiry attribute distribution 0 1,000 2,000 3,000 4,000 5,000 900-900 880-899 860-879 840-859 820-839 800-819 780-799 760-779 740-759 720-739 700-719 680-699 660-679 640-659 620-639 600-619 580-599 560-579 540-559 520-539 500-519 480-499 460-479 440-459 420-439 400-419 381-399 369-376 300-300 Normalized population distribution Stressed inquiry attribute distribution Stressing the model Increased inquiry behaviors Prime Subprime
  • 14. 14©2014 Experian Information Solutions, Inc. All rights reserved. Experian Public. 0 1,000 2,000 3,000 4,000 5,000 900-900 860-879 820-839 780-799 740-759 700-719 660-679 620-639 580-599 540-559 500-519 460-479 420-439 381-399 300-370 Normalized population distribution Stressed utilization attribute distribution 0 500 1,000 1,500 2,000 900-900 860-879 820-839 780-799 740-759 700-719 660-679 620-639 580-599 540-559 500-519 460-479 420-439 381-399 300-370 Normalized population distribution Stressed utilization attribute distribution 0 1,000 2,000 3,000 4,000 5,000 900-900 880-899 860-879 840-859 820-839 800-819 780-799 760-779 740-759 720-739 700-719 680-699 660-679 640-659 620-639 600-619 580-599 560-579 540-559 520-539 500-519 480-499 460-479 440-459 420-439 400-419 381-399 369-376 300-300 Normalized population distribution Stressed utilization attribute distribution Stressing the model Increased utilization behaviors Prime Subprime
  • 15. 15©2014 Experian Information Solutions, Inc. All rights reserved. Experian Public. 0 1,000 2,000 3,000 4,000 5,000 6,000 900-900 860-879 820-839 780-799 740-759 700-719 660-679 620-639 580-599 540-559 500-519 460-479 420-439 381-399 300-370 Normalized population distribution Stressed delinquency attribute distribution 0 500 1,000 1,500 2,000 900-900 860-879 820-839 780-799 740-759 700-719 660-679 620-639 580-599 540-559 500-519 460-479 420-439 381-399 300-370 Normalized population distribution Stressed delinquency attribute distribution 0 1,000 2,000 3,000 4,000 5,000 6,000 900-900 880-899 860-879 840-859 820-839 800-819 780-799 760-779 740-759 720-739 700-719 680-699 660-679 640-659 620-639 600-619 580-599 560-579 540-559 520-539 500-519 480-499 460-479 440-459 420-439 400-419 381-399 369-376 300-300 Normalized population distribution Stressed delinquency attribute distribution Stressing the model Increased delinquency behaviors Prime Subprime
  • 16. 16©2014 Experian Information Solutions, Inc. All rights reserved. Experian Public.  When an attribute in the model with a high coefficient is taken to its maximum valid value, the score becomes highly unstable  KS drops by 31 points Model performance: Under what scenario would a model dramatically under perform? 0 1,000 2,000 3,000 4,000 5,000 6,000 900-900 880-899 860-879 840-859 820-839 800-819 780-799 760-779 740-759 720-739 700-719 680-699 660-679 640-659 620-639 600-619 580-599 560-579 540-559 520-539 500-519 480-499 460-479 440-459 420-439 400-419 381-399 369-376 300-307 Normalized population distribution Stressed inquiry attribute Max inquiry attribute
  • 17. 17©2014 Experian Information Solutions, Inc. All rights reserved. Experian Public. Model risk governance best practices Robert Stone
  • 18. 18©2014 Experian Information Solutions, Inc. All rights reserved. Experian Public.  Subjective judgment is employed in all model development  Design, theory and logic should support a clear statement of purpose ► Comparison should be made among alternatives  Measure, where possible, and understand model uncertainty Model cycle Best practices Development, use and implementation
  • 19. 19©2014 Experian Information Solutions, Inc. All rights reserved. Experian Public. Model cycle Best practices Validation  In addition to model power or accuracy, validation should identify potential limitations and assumptions and assess their impact  Vendor models should be incorporated into a bank’s broader risk management framework  Model validators should be given authority to challenge and appropriate incentives Development, use and implementation
  • 20. 20©2014 Experian Information Solutions, Inc. All rights reserved. Experian Public.  Ensures effective challenge, and take prompt remedial action where necessary  Broadly divided into ownership, controls and compliance Model cycle Best practices Development, use and implementation Validation Governance
  • 21. 21©2014 Experian Information Solutions, Inc. All rights reserved. Experian Public. Model Risk Governance business review Case study – a Top-5 retail bank TimeShort Long Validationvolume/standardization High Low Data flow Model type Validation requirement Validation elements Assessment results 1 2 3 Score files exchange Risk models Asset models Balance models Line models Pricing models Point estimate business use Rank ordering business use  Actual vs. expected characteristics  Actual vs. expected score dist.  Actual vs. expected outcome dist.  Override analysis  Scorecard performance  Concordance/shifting  Future model performance  Single factor analysis  Benchmarking analysis  Calibration/redesign issues  Control limit determination  Business use guidance  Population stability analysis  K-S analysis
  • 22. 22©2014 Experian Information Solutions, Inc. All rights reserved. Experian Public. Model Risk Governance business review Case study – a community bank in the Midwest Business challenge Client was seeking a model validation of its consumer credit scoring model in accordance with the OCC Guidance 2011-12 model risk management. Solution Client leveraged Experian’s Global Consulting Practice and analytics team’s expert knowledge of best practice, data and modeling to ensure that the best model validation approach. Quantitative model validation Standard tests of the models were conducted to validate performance and accuracy, along with benchmarking to other essential models. Qualitative model validation Experian reviewed and gained a thorough understanding of the client’s modeling efforts, policies and procedures. Information on the client’s existing practices were compared to industry best practice in order to create the final deliverables, which included a gap analysis, strategic roadmap and actionable recommendation. Benefit to client The client was able to leverage the strategic roadmap and recommendations to proactively move toward best practices and fulfill regulatory demands prior to further scrutiny and review. The client leveraged Experian’s model validation team to augment existing staffing in a cost effective approach.
  • 23. 23©2014 Experian Information Solutions, Inc. All rights reserved. Experian Public. Model Risk Governance is not a new concept, but a way of thinking that needs to be an integrated part of an organization  Many organizations have invested in methods, resources, processes and technology to assess monitor, manage and model their credit risk  Changes in economic environment or misapplication of models exposes an organization  Regulators expect greater use of risk models and as a result oversight has increased to review the management of these models Summary
  • 24. 24©2014 Experian Information Solutions, Inc. All rights reserved. Experian Public. Model Risk Governance is not a new concept, but a way of thinking that needs to be an integrated part of an organization  Increased oversight leads to uncertainty of expectations and resource challenges  It’s not a one-time exercise, but a valuable ongoing re-calibration to better manage risk and return on investment  The investment leads to better decisions and enhanced business performance Summary Discover Experian’s® best-in-class Model Risk Governance services
  • 25. 25©2014 Experian Information Solutions, Inc. All rights reserved. Experian Public. For additional information, please contact: Linda.Haran@experian.com Robert.Stone@experian.com Hear the latest from Vision 2014 in the Daily Roundup: www.experian.com/vision/blog @ExperianVision | #vision2014 Follow us on Twitter
  • 26. 26©2014 Experian Information Solutions, Inc. All rights reserved. Experian Public. Visit the Experian Expert Bar to learn more about the topics and products covered in this presentation.