Successfully reported this slideshow.
Upcoming SlideShare
×

# Model Risk for Pricing and Risk Models in Finance

2,618 views

Published on

We present quantitative methods to assess the model risk for pricing and risk models.

Published in: Economy & Finance, Business
• Full Name
Comment goes here.

Are you sure you want to Yes No

Are you sure you want to  Yes  No

### Model Risk for Pricing and Risk Models in Finance

1. 1. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Model Risk and Model Control Patrick H¨aner H¨aner Consulting Berlin c 2015 H¨aner Consulting Model Risk 1 / 166
2. 2. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control The latest version of this document additonal resources examples may be found on https://github.com/haenerconsulting/modelrisk c 2015 H¨aner Consulting Model Risk 2 / 166
3. 3. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Outline 1 Model Classes 2 Credit Risk Measures 3 Model Implementation 4 Back Testing 5 Model Control c 2015 H¨aner Consulting Model Risk 3 / 166
4. 4. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Trade Models Pricing Models Overview 1 Model Classes 2 Credit Risk Measures 3 Model Implementation 4 Back Testing 5 Model Control c 2015 H¨aner Consulting Model Risk 4 / 166
5. 5. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Trade Models Pricing Models What is a Model? Deﬁnition (Model - Narrow) A Model is a mathematical framework providing answers to a speciﬁc set of Questions. Refers only to mathematics No reference to implementation No relation to markets and trading activity of institution Not a useful deﬁnition! c 2015 H¨aner Consulting Model Risk 5 / 166
6. 6. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Trade Models Pricing Models What is a Model? Deﬁnition (Model - Wider) A Model provides answers to a speciﬁc set of Questions. It consists of Information input component Processing component, applying mathematical transformations Reporting component, creating business information Input Market Data Static Data Processing Mathematical Model Reporting Business Information c 2015 H¨aner Consulting Model Risk 6 / 166
7. 7. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Trade Models Pricing Models Model - Wider Deﬁnition Emphasis on usage Covers data, software and mathematics Context of institution, trading activity and market relevant → organizational impact for validation: roles and repsonsibilites c 2015 H¨aner Consulting Model Risk 7 / 166
8. 8. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Trade Models Pricing Models What is the Question? Types of Questions How is the instrument deﬁned What is the value → pricing model What will the prices in the future be → risk model c 2015 H¨aner Consulting Model Risk 8 / 166
9. 9. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Trade Models Pricing Models What is the Question? Dependencies FutureNow Pricing Model Sensitivities State Variables Transactions VaR PFE c 2015 H¨aner Consulting Model Risk 9 / 166
10. 10. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Trade Models Pricing Models Categorization Types of Statements Prescriptive Model independent, robust statements Descriptive Explaining Predictive Falsiﬁable c 2015 H¨aner Consulting Model Risk 10 / 166
11. 11. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Trade Models Pricing Models Model Independent Replication Static Cashﬂow Replication Floating leg of swap: replicate by long/short FRA. Model Dependent Static Replication Barrier options: static replication dependent of model. c 2015 H¨aner Consulting Model Risk 11 / 166
12. 12. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Trade Models Pricing Models Model Independent Relation Across trade parameters Across trade types Trade Parameters Price of knock-out option increases with barrier height. Trade Types Barrier option is cheaper than a Plain-Vanilla Replicate Plain-Vanilla by in/out Barriers c 2015 H¨aner Consulting Model Risk 12 / 166
13. 13. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Trade Models Pricing Models Trade Model Requirements Contractual details as in term sheet need computer-readable representation: trade representation data exchange auditing Trade Repository In US: Dodd-Frank regulation require DTCC data repository (DDR) as a multi-asset class repository. c 2015 H¨aner Consulting Model Risk 13 / 166
14. 14. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Trade Models Pricing Models Pricing Model Usage No liquid prices (Mark to Model) Sensitivities Valuation under future/hypothetical scenarios c 2015 H¨aner Consulting Model Risk 14 / 166
15. 15. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Trade Models Pricing Models Pricing Model Questions Implied price w/o credit risk Implied price w credit risk: CVA/DVA Implied price range: incomplete markets Bid/ask price: liquidity c 2015 H¨aner Consulting Model Risk 15 / 166
16. 16. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Trade Models Pricing Models Pricing Model Sensitivities Which sensitivities should be reported? Aggregation How to aggregate vega sensitivities from two systems with diﬀerent models? Lognormal model: σBS Normal model: σnorm Model Independence Report sensitivities wrt. to market observables, i.e. instead of sigmaBS, σnorm use option prices. c 2015 H¨aner Consulting Model Risk 16 / 166
17. 17. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Overview 1 Model Classes 2 Credit Risk Measures 3 Model Implementation 4 Back Testing 5 Model Control c 2015 H¨aner Consulting Model Risk 17 / 166
18. 18. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Likeliness vs Severity of Credit Events Categories Which dimensions to consider? Severity How much will we lose? Likeliness What’s the chance that we lose? Granularity What does the measure refer to? c 2015 H¨aner Consulting Model Risk 18 / 166
19. 19. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Granularity of Measure Based on Defaults All Counterparties Single Counterparty Other Aggregations Global/macro economic Sector, country Trade c 2015 H¨aner Consulting Model Risk 19 / 166
20. 20. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Exposure and Recovery How to measure severity? Need to value trade: Deﬁnition (Exposure at Default) EAD(t) = max 0, p(t)|τ = t τ : time at which CP defaults Deﬁnition (Loss Given Default) Loss at time t = LGD(t)EAD(t) Deﬁnition (Recovery) R(t) = 1 − LGD(t) c 2015 H¨aner Consulting Model Risk 20 / 166
21. 21. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Severity Valuation Approaches Accrual Banking book; rarely adjust; illiquid assets Mark to market Trading book; frequently adjusted; traded assets Mark to model Trading book; frequently adjusted; complex structures Example CreditRiskMeasures.xlsx c 2015 H¨aner Consulting Model Risk 21 / 166
22. 22. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Severity Accrual Loan to Acme Ltd value is face value maximal loss is notional of loan Mark to market Buy bond of Acme Ltd; assume liquid market value is mark to market of bond value lower than in risk-free valuation c 2015 H¨aner Consulting Model Risk 22 / 166
23. 23. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Severity Mark to model Exotic interest rate swap with Acme Ltd. What is the value risk free: assuming Acme may never default risky: Acme may default risky with own risk: Acme and we may default c 2015 H¨aner Consulting Model Risk 23 / 166
24. 24. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Forward Looking Measures Assess exposure in future → model how state of the world evolves Deterministic Evolution Scenario Analysis, Stress testing Stochastic Evolution Model for risk factors c 2015 H¨aner Consulting Model Risk 24 / 166
25. 25. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Scenario Analysis/Stress Test Meanings of Stress change model parameters → pick a single path → degenerate measure (Dirac measure) Uniﬁed handling by Measure Transforms Stochstic Process (Langevin Equation) Dual Model Representations Probabiliy Measure (Path Integrals) : t ! x(t) Path µ( ) ⇠ e S( ) S( ) ⌘ Z ⌧ 0 L(x, ˙x) Action L(x, ˙x) : Lagrangian ˙x = f(x) + ⇠ c 2015 H¨aner Consulting Model Risk 25 / 166
26. 26. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Types of Stress Tests Approaches give economic scenario given loss (inverse stress) Inverse stresses c 2015 H¨aner Consulting Model Risk 26 / 166
27. 27. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Statistical Measures Single Netting Set Deﬁnition (Potential Future Exposure) PFE(t) = max 0, p(t)|τ = t τ : time at which CP defaults Deﬁnition (Expected Exposure (EE)) EE(t) = E[PFE] Deﬁnition (Expected Positive Exposure) EPE(T) = 1 T T 0 EE(t) dt c 2015 H¨aner Consulting Model Risk 27 / 166
28. 28. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Regulatory Measures Deﬁnition (Eﬀective Expected Exposure (EEE)) Maximum of EPE and past EEE: never decreasing. c 2015 H¨aner Consulting Model Risk 28 / 166
29. 29. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Statistical Measures Multiple Netting Set Deﬁnition (Losses across Netting Sets) L(t) = a χτa≤tLGDa max 0, pa(τa) a : Identiﬁer of netting set c 2015 H¨aner Consulting Model Risk 29 / 166
30. 30. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Portfolio Measures Meaningful risk measures for portfolios Deﬁnition (Coherent Risk Measure) Risk measure ρ: for portolio X: Normalization ρ(∅) = 0 empty portfolio has no risk Monotonicity X1 ≤ X2 → ρ(X1) ≥ ρ(X2) Sub-additivity ρ(X1 + X2) ≤ ρ(X1) + ρ(X2) diversiﬁcation/netting Homogeneity ρ(αX) = αρ(X) α > 0 Translation invariance ρ(X + a) = ρ(X) − a adding cash a reduces risk c 2015 H¨aner Consulting Model Risk 30 / 166
31. 31. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Portfolio Measures Quantile q% quantile: value, for which q% of outcomes are smaller/larger. Quantiles are not coherent measures. Expected Shortfall Expected loss conditioned on the loss being larget than X. The Expected Shortfall (Mean Excess Loss) is a coherent measure. Example PortfolioMeasure.xlsx c 2015 H¨aner Consulting Model Risk 31 / 166
32. 32. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Likeliness of Default Example LikelihoodExperiment.xlsx c 2015 H¨aner Consulting Model Risk 32 / 166
33. 33. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk What does probability mean? Average observers Implied ensemble Genuine ensemble Probability and Measurement Need to deﬁne Ensemble Measurement process c 2015 H¨aner Consulting Model Risk 33 / 166
34. 34. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Examples Genuine Ensemble Mathematics Physics: Identically prepared experiment Average observers Consensus of observers: Market prices Betting quota Implied Ensemble Equivalence classes: Names with same rating Price returns in diﬀerent time windows c 2015 H¨aner Consulting Model Risk 34 / 166
35. 35. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Measures for Probability of Default Deﬁnition (Survival/Default Probability, Default Intensity) Let τ be time of default S(t) = p(τ > t) S : survival probability S(t) = e−λ(t)t λ : term default intensity D(t) = 1 − S(t) D : default probability Note: D is a CDF! c 2015 H¨aner Consulting Model Risk 35 / 166
36. 36. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Forward Intensity Forward default intensity Probability d(t) of defaulting between t and dt: d(t) = dD(t) dt (1) c 2015 H¨aner Consulting Model Risk 36 / 166
37. 37. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Estimating Probability of Default Estimating λ Credit Rating Typically using historical data Market Prices Current credit spreads from bonds or CDS Implied Default Intensity Let s(t) be a credit spread s(t) = (1 − R)λ(t) R : recovery rate c 2015 H¨aner Consulting Model Risk 37 / 166
38. 38. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Uniﬁying Severity and Frequency Measures High Severity/Low Frequency vs. Low Severity High Frequency How to compare Single large deal with good counterparty Set of small deals with bad counterparties Answer Pricing including credit risk allows comparing! c 2015 H¨aner Consulting Model Risk 38 / 166
39. 39. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Approaches Top-down vs Bottom-up Top-down Pricing from ﬁrst principles Bottom-up Calculate price correction from building blocks: Exposure (EE) and PE, LGD c 2015 H¨aner Consulting Model Risk 39 / 166
40. 40. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Bottom-Up approach Assumptions Risk-free prices known Calculate EE Estimate PE, LGD Calculate correction to risk-free price c 2015 H¨aner Consulting Model Risk 40 / 166
41. 41. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Measuring the Corrections Riskiness of counterparty reduces the price: Deﬁnition (CVA) Risky price p∗ A as seen from counterparty A with counterparty B: p∗ = p − CVAB p : risk-free price CVAB : Credit Valuation Adjustment for counterparty B c 2015 H¨aner Consulting Model Risk 41 / 166
42. 42. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Measuring the Corrections Does credit risk of counterparty A also aﬀect price? Deﬁnition (DVA) Price pA as seen from counterparty A with counterparty B: p∗ = p − CVAB + DVAA p : risk-free price DVAA : Debit Valuation Adjustment for counterparty A DVA increases the price. Accounting vs. Regulatory DVA must be used for P&L but not for regulatory capital. c 2015 H¨aner Consulting Model Risk 42 / 166
43. 43. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Regulatory CVA BCBS 189, paragraph 89: Regulatory CVA Similar to regulatory capital charge for default: Assumes independence of exposure and default process. CVA = T 0 (1 − R)Df (t)EE(t)d(t) dt where d is the default probability from equation (1),Df discount factor c 2015 H¨aner Consulting Model Risk 43 / 166
44. 44. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk CVA Example CVA.xlsx c 2015 H¨aner Consulting Model Risk 44 / 166
45. 45. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Regulatory CVA Regulatory vs Trading CVA Regulatory Historic measure for EE, implied for PD Trading Both EE and PD in implied measure c 2015 H¨aner Consulting Model Risk 45 / 166
46. 46. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Pricing for Portfolio of Netting Sets As for single netting sets: pricing combines severity and likeliness. Requires knowing prices of individual netting sets at default probability of default P(χτ1≤t1 , χτ2≤t2 , . . . , χτN ≤tN ) Additional useful quantity: in terms of total losses: Deﬁnition (Loss distribution) L(l, t) = P(L(t) ≥ l) c 2015 H¨aner Consulting Model Risk 46 / 166
47. 47. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Types of Measures Severity Frequency Pricing Credit Risk Granularity of Measure in Regulatory Context Metrics used for Regulatory Purposes Focus on measures for individual counterparties. No proper modelling of collective losses required. c 2015 H¨aner Consulting Model Risk 47 / 166
48. 48. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Overview 1 Model Classes 2 Credit Risk Measures 3 Model Implementation 4 Back Testing 5 Model Control c 2015 H¨aner Consulting Model Risk 48 / 166
49. 49. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Building Blocks Model Building Process Business Analysis Materiality, speciﬁcation Model choice Find adequate model Software implementation Develop and roll out c 2015 H¨aner Consulting Model Risk 49 / 166
50. 50. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Materiality What to Model? Which risk factors material for current portfolio? How can we assess materiality without exposure model in place? Approach Simple estimation of exposure assuming future portfolio prices normally distributed estimation of ﬁrst two moments c 2015 H¨aner Consulting Model Risk 50 / 166
51. 51. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Gaussian Approximation Need to estimate E[p(T)], E[p2(Ti )] at some future times T: Performing Taylor expansion for price p around expected risk factor: p(x(T), T) ≈ p(x0(T), T) + i ∂p(x0(T), T) ∂xi ∆xi (T) + 1 2 ij ∂2p(x0(T), T) ∂xi ∂xj ∆xi (T)∆xj (T) x0(T) ≡ E[x(T)] ∆xi (T) ≡ xi (T) − x0,i (T) c 2015 H¨aner Consulting Model Risk 51 / 166
52. 52. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Gaussian Approximation The expectation value M of the price is hence M(T) ≈ p(x0(T), T) + 1 2 ij γij (T)Ωij (T) M(T) ≡ E[p(x(T), T)] γij (T) ≡ ∂2p(x0(T), T) ∂xi ∂xj Ωij (T) ≡ E[∆xi (T)∆xj (T)] For the variance V we obtain up to second order in ∆x: V (T) ≈ ij δi (T)δj (T)Ωij (T) V (T) ≡ E[(p(x(T), T) − E[p(x(T), T)])2 ] δi (T) ≡ ∂p(x0(T), T) ∂xic 2015 H¨aner Consulting Model Risk 52 / 166
53. 53. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Gaussian Approximation What can we learn? Risk factor contributions Matrix elements Ψij = δi (T)δj (T)Ωij (T) indicate contribution of risk factors ij to total variance. EE, PE Knowning mean and variance of the Gaussian distribution, any statistical quantity may be evalued. Caveat Depending on speciﬁcs of portfolio this approximation may be more or less accurate: that is why we use Monte Carlo simulations after all. c 2015 H¨aner Consulting Model Risk 53 / 166
54. 54. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Practical Implementation For t = 0: δ and γ from Market risk system. But: need netting set level aggregation → deal level granularity For t > 0 estimate future δ, γ by bumping c 2015 H¨aner Consulting Model Risk 54 / 166
55. 55. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Trade Models Requirements Represent trades/products Standardize for interoperability c 2015 H¨aner Consulting Model Risk 55 / 166
56. 56. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Trade Models Trade Parameters Product represented by parameters FpML c 2015 H¨aner Consulting Model Risk 56 / 166
57. 57. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Trade Models Trade Parameters Pro/Con ⊕ standardized logic in client c 2015 H¨aner Consulting Model Risk 57 / 166
58. 58. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Trade Models Cashﬂows Product represented by casﬂows Payoﬀ macros c 2015 H¨aner Consulting Model Risk 58 / 166
59. 59. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Trade Models Cashﬂows Pro/Con ⊕ simple not expressive enough (just cash is exchanged) single product (no interations) c 2015 H¨aner Consulting Model Risk 59 / 166
60. 60. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Trade Models Transaction Model Approach Multi agent simulation: Time Map wall clock to simulation time Market Events simulation time to events Transactions events to transactions (e.g. cashﬂows) Execution execute events Pro/Con ⊕ general ⊕ all business logic in model → easy tooling expensive c 2015 H¨aner Consulting Model Risk 60 / 166
61. 61. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Pricing & Risk Models Criteria for Model Choice Categories Independent of product Relate to Mathemathics or Physics Dependent of product Speciﬁc to product type Dependent of portfolio and market Context c 2015 H¨aner Consulting Model Risk 61 / 166
62. 62. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Pricing & Risk Models Independent of Product Coordinate Sytems From Physics we know: dynamics must not depend on choice of coordinates → dimension analysis. Interpolation How to interpolate r, σ. Interpolate dimension-less quantities: rt and σ2t. c 2015 H¨aner Consulting Model Risk 62 / 166
63. 63. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Pricing & Risk Models Product Dependent State Variables vs. Parameters Liquidity Hedge frequency, transaction costs, close-out period Completeness Unhedgeable risk, uniqueness of price c 2015 H¨aner Consulting Model Risk 63 / 166
64. 64. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Pricing & Risk Models State Variables and Parameters Indicators of Model Quality Parameter Dimensionality Avoid overparamerization Stability of Parameters Frequent recalibration: indicator of poor model performance GBM w termstructure vs Garch TS GBM Garch dimension ∞ 3 recalibration frequently for short end less frequent time-homogeneous N Y c 2015 H¨aner Consulting Model Risk 64 / 166
65. 65. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Pricing & Risk Models Arbitrage Risk Model for Volatility surface Directly modelling surface w/o arbitrage not trivial. Alternatively model option prices with HJM-like framework. c 2015 H¨aner Consulting Model Risk 65 / 166
66. 66. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Pricing & Risk Models Market Liquidity & Completeness Liquidity Hedge frequency, transaction costs, close-out period Completeness Unhedgeable risk, uniqueness of price c 2015 H¨aner Consulting Model Risk 66 / 166
67. 67. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Comparing Models Assume state of the world evolves randomly: Model as Process: Stochastic Diﬀerential Equation (Langevin Equation) dx dt = f (x) + g(x)ξ(t) Physics Notation dx = f (x)dt + g(x)dW(t) Finance Notation Wiener Process (SDE) dx = dW (t) W : Wiener Process c 2015 H¨aner Consulting Model Risk 67 / 166
68. 68. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Comparing Models Model as Measure P P : Γ → µ(Γ) probability Γ : t → x(t) some path Wiener Process (SDE) Γ ≡ {x1, . . . xN} µ(Γ) ∼ i G(xi , xi+1) G : Gaussian c 2015 H¨aner Consulting Model Risk 68 / 166
69. 69. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Pricing & Risk Models Parametric Models Error Analysis Infer from parameter uncertainty price/risk uncertainty. Parameter Uncertainty E.g. such that hedging instrument prices still in bid-ask Parameter Error Uncertainty of price/risk due to error in parameters GBM with vol uncertainty (∆p)2 = ∂p ∂σ ∆σ 2 c 2015 H¨aner Consulting Model Risk 69 / 166
70. 70. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Benchmarking Pricing/risk factor models Q, Q , empirical measure P Comparing Pricing Models Q vs Q Risk Models P vs Q c 2015 H¨aner Consulting Model Risk 70 / 166
71. 71. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Benchmarking Distances How far apart two models? Need to deﬁne metric: Expectation values E.g. diﬀerences of prices and EEs under diﬀerent measures Distributions E.g. Kullback-Leibler entropy dP P log dP P dP. Independent of quantity to average. c 2015 H¨aner Consulting Model Risk 71 / 166
72. 72. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Model Uncertainty Benchmarking giving limited answer: Calibration-Consistent Measures Deﬁne metric d to quantify goodness of calibration: pP i : model price calibration instrument i pi : market price calibration instrument i dP = i (pP i − pi )2 C = {P|dP ≤ } c 2015 H¨aner Consulting Model Risk 72 / 166
73. 73. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Model Uncertainty Non-uniqueness For d = 0: Multiple measures For single parametric measure, multiple solutions for calibration → ill behaved Incomplete market For d > 0: For single parametric measure: parameter risk c 2015 H¨aner Consulting Model Risk 73 / 166
74. 74. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Beyond Benchmarking Pricing model descriptive: Replicates prices of hedging instruments Determines no-arbitrage price of illiquit product How to asses quality of model? There are implied predictions: State variables vs parameters Prediction: parameters are constant Martingale Total price of deal and self-ﬁnancing hedges should be 0 at any point in time c 2015 H¨aner Consulting Model Risk 74 / 166
75. 75. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development State variables and parameters State variables Temporal evolution or measure Parameters Family of evolutions/measures Analysis Choice of state variables: qualitative assessment Robustness of parameters: predicted are no changes c 2015 H¨aner Consulting Model Risk 75 / 166
76. 76. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Hedge Performance If perfectly hedged: pathwise replication → P&L distribution Unbiased Sharply peaked (Dirac) Hedge Simulations Self Consistency Use state variables simulated with pricing model Performance Historical state variables c 2015 H¨aner Consulting Model Risk 76 / 166
77. 77. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Exposure Goal Estimate Credit Risk measures → need to estimate exposure/price distributions in future. The exposure e(t) at time t of a netting set is given by e(t) = max 0, i pi (x, t) − C(t) (2) where pi price of trade i x risk factors C(t) price of collateral c 2015 H¨aner Consulting Model Risk 77 / 166
78. 78. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Calculating Exposure, CVA/DVA and Losses Risk Factors&Counterparty Default Times t x t x RT1 RT2 Risk Factors Trades t x t x R1 R2 Collaterals Portfolio Prices t p t p P1 P2 Collateral Prices t c t c C1 C2 PDF of Exposures Default Times of Counterparties PDF of Exposures at Default Expected Exposure Potential Future Exposure Bootom-up CVA/DVA Top-down CVA/DVA c 2015 H¨aner Consulting Model Risk 78 / 166
79. 79. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Building Blocks Components Required for estimating risk measures for single and portfolios of netting sets: Pricing Risk-factor Collateral Netting Dependency c 2015 H¨aner Consulting Model Risk 79 / 166
80. 80. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Pricing Models Requirements Need to be fast! Ideally same as front oﬃce Perform well under stressed state variables c 2015 H¨aner Consulting Model Risk 80 / 166
81. 81. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Pricing Models Acceleration Techniques Dumb lookup Approximate price as function of few variables deﬁne variables (e.g stock price) deﬁne grid recaluclate for each gridpoint price interpolate c 2015 H¨aner Consulting Model Risk 81 / 166
82. 82. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Pricing Models Acceleration Techniques Smart lookup Approximate price as function of few variables deﬁne variables (e.g stock price) prices on grid are side eﬀect of pricing at spot; e.g. pricing on tree or AMC interpolate c 2015 H¨aner Consulting Model Risk 82 / 166
83. 83. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Risk Factor Models Pricing vs Risk Models Purpose Pricing Model Fit liquid market instruments; arbitrage-free Risk Model Predict Challenges for Risk Model Dependency Simultaneously simulate all asset classes Calibrationl Global calibration c 2015 H¨aner Consulting Model Risk 83 / 166
84. 84. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Risk Factor Models Short vs Long term prediction Long term prediction a challenge: Reducing dimensionality Economic macro factors Co-integration c 2015 H¨aner Consulting Model Risk 84 / 166
85. 85. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Risk Factor Models Pricing model Dynamics Arbitrage-free models used with risk calibration GBM HJM type of models ⊕ Well understood, tractable Not intended for risk c 2015 H¨aner Consulting Model Risk 85 / 166
86. 86. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Gaussian Dependency Modelling Goal Express random vector ξ with correlated ξi as linear combination of uncorrelated random factors ηi : ξ = Mη E[ξi ξj ] − E[ξi ]E[ξj ] ≡ Ωij E[ηi ηj ] − E[ηi ]E[ηj ] = λ2 i δij diagonal, pos. sem. def. What to consider? Ω? correlation matrix? c 2015 H¨aner Consulting Model Risk 86 / 166
87. 87. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Principal Component Analysis Dimensional Analysis Risk factors ξi not dimension-less! interest rate :[T−1] stock price :[Cash] volatility: [T−1 2 ] → Ωij may have diﬀerent dimensions,i.e. Ω in general not a physically meaningful quantity! c 2015 H¨aner Consulting Model Risk 87 / 166
88. 88. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Principal Component Analysis Solution Consider instead of Ω following matrix Φ: Φij ≡ ∂f (ξ) ∂ξi ∂f (ξ) ∂ξj Ωij f : some function For dimensionality [Φ]: [Φij ] = [f ] [ξi ] [f ] [ξj ] [ξi ][ξj ] = [f 2 ] ∀i, j (3) c 2015 H¨aner Consulting Model Risk 88 / 166
89. 89. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development GBM Risk Factor Model Multivariate GBM Xi (t + ∆t) = Xi e(µi −1 2 σi )∆t+σi √ ∆tξi (t) µ : drift σ volatility ξi : Normal random Cov(ln Xi (t + ∆t), ln Xj (t + ∆t)) = Ωij c 2015 H¨aner Consulting Model Risk 89 / 166
90. 90. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Dependent Gaussian Random Variables Given uncorrelated Gaussian random number vector ζ. Need build η: Cov(ηi , ηj ) = Ωij c 2015 H¨aner Consulting Model Risk 90 / 166
91. 91. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Calibration Deﬁnition Calibration is the process to determine model parameters. Approaches Statistical Using historical data Implied Market implied parameters Economic Macro economical relation between rates, infaltion Asumptions Statistical Past is good predictor for future Implied Information in spot market predicts future Economic Some fundamental economic laws rule future c 2015 H¨aner Consulting Model Risk 91 / 166
92. 92. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Statistical Calibration For simple models: ad hoc parameter estimation averaging ﬁtting Example SimpleEstimation.xls c 2015 H¨aner Consulting Model Risk 92 / 166
93. 93. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Maximum Likelihood Estimation Systematic way to calibrate Approach Parametric model with parameters α ↔ parametric measure µα: µα(Γ) = e−Sα(Γ) D[Γ] Assume: historical path ΓH is the most likely one. Find α∗ such that: µα∗ (ΓH) = max α µα(ΓH) c 2015 H¨aner Consulting Model Risk 93 / 166
94. 94. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Maximum Likelihood Estimation Implementation Assuming iid: µα(Γ) = m(xi ) m(x) = e−s(x) Γ = {x1, . . . , xn} Maximizing m ↔ minimizing i s(xi ) : log-likelihood c 2015 H¨aner Consulting Model Risk 94 / 166
95. 95. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Maximum Likelihood Estimation Example MLE.xls c 2015 H¨aner Consulting Model Risk 95 / 166
96. 96. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Implied Parameters Apply parameters used for pricing: Drift and Volatility Drift µ from T forward price (Covered Parity) Volatility σ T years ATM implied volatility Assumption Risk neutral measure yield good predictor for real-world measure Caveat Carry trades Supply/demand, risk premium Perform analysis before using implied parameters! c 2015 H¨aner Consulting Model Risk 96 / 166
97. 97. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Economic Calibration Parities connect for instance FX rates Inﬂation rates Real interest rates Nominal interest rates Purchansing power Example Parities.xlsx c 2015 H¨aner Consulting Model Risk 97 / 166
98. 98. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Parities Example (Relative Purchasing Power Parity) pf (t1)(1 + if )X(t2) = pd (t2)(1 + id ) pd/f : domestic/foreign price id/f : domestic/foreign 1 yr inﬂation rate X : Exchange rate Yields after averaging E[X(t2)] X(t1) = 1 + Id 1 + If where I is the expected inﬂation rate. c 2015 H¨aner Consulting Model Risk 98 / 166
99. 99. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Parities Example (International Fisher Eﬀect (Uncovered Parity)) (1 + rd/f ) = (1 + ρd/f )(1 + id/f ) rd/f : domestic/foreign nominal 1 yr interest rate ρd/f : real rdomestic/foreign 1 yr interest rate Assumingρd = ρf gives E[X(t2)] X(t1) = 1 + rd 1 + rf c 2015 H¨aner Consulting Model Risk 99 / 166
100. 100. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Issues with standard GBM model Issues rigidity: calibration short vs long horizons → term structure of parameters dimensionality → factor models underestimation of rare events and bursts (clustering) → GARCH not suitable where spread stationary process → cointegration unable to capture some behabiour like regime-switches → parametric models (Nelson-Siegel) c 2015 H¨aner Consulting Model Risk 100 / 166
101. 101. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development GBM with Term Structure Interpolation Principles Interpolate dimension-less quantities Forward Drift/Covariance Dimensionality analysis → interpolate TΩ c 2015 H¨aner Consulting Model Risk 101 / 166
102. 102. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Factor Models Issues with general covariance matrix N risk factors →∝ N2 parameters over-parametrization for empirical parameters: problems with positive deﬁniteness Idea Split return r of riskfactors into contributions from Indices fn shared by multiple risk factors Idiosycratic factors unique to each risk factor r = α + n βnfn + and assume indices uncorrelated to indosyncraticsc 2015 H¨aner Consulting Model Risk 102 / 166
103. 103. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Types of Factor Models Classiﬁcation Macroeconomic Observables like changes in inﬂation, interest rate, unemployment rate Fundamental Portfolios associated to security attributes like industry membership, book to market ratio, dividends Statistical Factor analysis of covariance matrix c 2015 H¨aner Consulting Model Risk 103 / 166
104. 104. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Macroeconomic Factor Model Fast/Slow Slow variables Macro-economic state of the economy: inﬂation, unemployment rate, GDP Fast Asset prices Pros and Cons ⊕ Designed to predict long-term evolution ⊕ Able to reﬂect systemic macro risks Empirical evidence not convincing Theories controversial c 2015 H¨aner Consulting Model Risk 104 / 166
105. 105. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Fundamental Factor Model Sector/Region 1 Deﬁne for each sector/region pair an index 2 Associate stock to sector/region 3 Regress stock return vs index return → α, β Example FactorModel.xls Pros and Cons ⊕ Designed to predict long-term evolution ⊕ Able to reﬂect systemic macro risks Empirical evidence not convincing Theories controversialc 2015 H¨aner Consulting Model Risk 105 / 166
106. 106. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Choice of Factors How to know whether factors appropriate? Analyze variance explained by factors c 2015 H¨aner Consulting Model Risk 106 / 166
107. 107. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Volatility Clustering (a) Spot (b) Log-returns Figure : GBPUSD spot c 2015 H¨aner Consulting Model Risk 107 / 166
108. 108. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Autocorrelation (a) Autocorrelation: log-returns (b) Autocorrelation: squared log-returnsc 2015 H¨aner Consulting Model Risk 108 / 166
109. 109. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Garch Model Let Xn be the log-return of some foreign exchange rate f at time tn: Xn = ln fn fn−1 (4) we may then express the foreign exchange rate fN at some future sampling point time tN by the initial value f0 at t0 and a series of returns: fN = f0e N i=1 Xi (5) The observation points ti are typically deﬁned in terms of number of business days ∆T between them. For short time horizon predictions we choose ∆T = 1 for larger horizon, we may choose a less granular time grid. c 2015 H¨aner Consulting Model Risk 109 / 166
110. 110. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Garch Model The dynamics of the returns is then assumed to follow a Garch(1,1) process Xn = µ + n t ∼ iid(0, σ2 n) (6) σ2 n+1 = α + βσ2 n + γ 2 n (7) The asymptotic value σ2 ∞ = limn→∞ E[σ2 n] is then obtained by equation (7) noting, that E[ 2] = σ2 and E[σ2 n+1] → E[σ2 n]: σ∞ = α 1 − β − γ (8) c 2015 H¨aner Consulting Model Risk 110 / 166
111. 111. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Garch Model: Limit Weak limit: Stochastic variance Mean reverting variance dXt = µXtdt + √ vtXtdWt dvt = α(vt)dt + β(vt)dZt c 2015 H¨aner Consulting Model Risk 111 / 166
112. 112. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Copula Dependence under Stress In stressed markets correlations increase between downward price movements → systematic risk implied default probabilities → contagion Deﬁnition (Copula) Separate Marginal distributions from Dependency c 2015 H¨aner Consulting Model Risk 112 / 166
113. 113. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Cointegration Long-run Relationship Variables moving together: Macro-economic Consumption-Income Prices-Wages Domestic prices - fpreign prices Exogeneous For instance managed currencies How to model processes. which stay close to each other? GBM with ρij 1 not? No! Need dynamic, where diﬀerence is stationary Deﬁnition Stochastic processes x, y are cointegrated: y(t) = a + bx(t) + ξ(t)c 2015 H¨aner Consulting Model Risk 113 / 166
114. 114. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Implementation 1 ﬁnd parameters a, b by regression 2 show residuals are stationary (e.g. Dickey-Fuller Test) Example Cointegration.xlsx c 2015 H¨aner Consulting Model Risk 114 / 166
115. 115. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Risk Factor Models Empirical Models Nelson-Siegel model r(T) = r∞ + a(T)r0 + b(T)rm r∞ : rate for long maturities r0 : rate for short maturities rm : rate for intermediate maturities a, b : decay functions c 2015 H¨aner Consulting Model Risk 115 / 166
116. 116. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Risk Factor Models Empirical Models Nelson-Siegel model Normal/inverted curves But not arbitrage-free How to introduce dynamics? E.g. PCA of (r∞, r0, rm) Example NelsonSiegel.xlsm c 2015 H¨aner Consulting Model Risk 116 / 166
117. 117. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Wrong Way Risk Types Speciﬁc Legal connection between underlying and counterparty General Dependence between prob. of default of counterparty and exposure SFT Transactions Lend cash to counterparty A accepting their stock as collateral. Emerging Market CCY swap We are long strong currency. Weakening of emerging market currency, increased prob default → increase exposure c 2015 H¨aner Consulting Model Risk 117 / 166
118. 118. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Modelling Wrong Way Risk What is wrong with standard modelling? p+ is not conditioned on default. Need to add in price function default state χ of counterparty: extending state of the world Approaches Given a model for default times either Simulating counterparty’s default Calculating price given default Example WrongWayRisk.xls c 2015 H¨aner Consulting Model Risk 118 / 166
119. 119. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Collateral Modeling Components Margin Call Process Model margin calls with correct frequency and close-out period Collateral Price E.g. model bond price if collateral is bond Simpliﬁcation Margin call process: just at spot → short-cut method All collateral as cash → haircuts c 2015 H¨aner Consulting Model Risk 119 / 166
120. 120. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Collateral Modeling Short-Cut Method Deﬁnition (Basel II Short-Cut Method) EE and PE of collateralized trades given by EE and PE for close-out period (5 days for SFT, 10d for OTC) Beneﬁts/Issues ⊕ Computationally cheap ⊕ No collateral exposure spikes at expity Assumes exposures declining over time Risk not accurately represented c 2015 H¨aner Consulting Model Risk 120 / 166
121. 121. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Dependency Modelling Among Risk Factors Standard way to model dependence: Gaussian Copula. Gaussian Copulas are Levy copulas. Replace Gaussian with other Levy coupula and obtain Levy model. Between Defaults Simulate either Default times τ E.g. by Marshall-Olkin Copulas Default state at t:χτ≤t E.g. structural models c 2015 H¨aner Consulting Model Risk 121 / 166
122. 122. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Dependency Modelling Between a Default and Risk Factors To caputure Wrong Way risk need to model dependence between risk factor and default state Example WrongWayRisk.xls Between a cross name Defaults and Risk Factors Need modelling full state of the world (x(t), {χτ1≤t, . . . χτ1≤t}). → scenario consistency is system c 2015 H¨aner Consulting Model Risk 122 / 166
123. 123. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Model Lifecycle Organisation Execution Problem Deﬁnition Analysis Implementation Test Deployment MaintananceChanges Figure : Model Development Lifcecyle c 2015 H¨aner Consulting Model Risk 123 / 166
124. 124. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Speciﬁcation Approaches Human readable Business and functional specs Machine readable Speciﬁcation ∼ test c 2015 H¨aner Consulting Model Risk 124 / 166
125. 125. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Speciﬁcation Tools ScalaTest Code c 2015 H¨aner Consulting Model Risk 125 / 166
126. 126. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Speciﬁcation Tools ScalaTest Output Part of CI: c 2015 H¨aner Consulting Model Risk 126 / 166
127. 127. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Implementation Software in-house third-party Require diﬀerent validation strategies c 2015 H¨aner Consulting Model Risk 127 / 166
128. 128. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Third Party Strategies Black-box, no code review Reverse-engineering c 2015 H¨aner Consulting Model Risk 128 / 166
129. 129. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Revision Control Requirements Audit Who changed what/when Resurrect Roll-back to previous state Collaborate Merge contributions from diﬀerent authors Approaches Plain ﬁles Tag ﬁles/directories with version information Local Local database contains version information (e.g RCS) Server Database on server (e.g. SVN) Distributed Each developer has own databse with potentially central db (e.g. Git) c 2015 H¨aner Consulting Model Risk 129 / 166
130. 130. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Revison Control Tools Approaches MyDirectoryV1.0 MyDirectoryV1.1 MyDirectoryV1.2-bugfix1 (a) File based (b) Local VCS c 2015 H¨aner Consulting Model Risk 130 / 166
131. 131. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Revison Control Tools Approaches (a) Centralized VCS (b) Distributed VCS c 2015 H¨aner Consulting Model Risk 131 / 166
132. 132. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Revision Control Tools Git Figure : Git Gui (SourceTree) c 2015 H¨aner Consulting Model Risk 132 / 166
133. 133. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Documentation Requirement Contain enough information to reverse-engineer. Tools Automated API doc (Doxygen, ScalaDoc, . . .) Internal wiki (e.g. Conﬂuence) c 2015 H¨aner Consulting Model Risk 133 / 166
134. 134. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Testing Test Types Unit Library level Integration System level c 2015 H¨aner Consulting Model Risk 134 / 166
135. 135. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Testing Unit Test c 2015 H¨aner Consulting Model Risk 135 / 166
136. 136. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Release Requirements Regression Impact analysis Sign-oﬀ Auditing Lock-down c 2015 H¨aner Consulting Model Risk 136 / 166
137. 137. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Maintance Bugs/Enhanements Tracking system Failing test cases Metrics: severity, resolution time c 2015 H¨aner Consulting Model Risk 137 / 166
138. 138. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Analysis Models Software Development Integrated Development Process Robust system should have Components Revsion Control system Build System Bug tracking system Wikin Components integrated to workﬂow with high degree of automation c 2015 H¨aner Consulting Model Risk 138 / 166
139. 139. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Regulatory Requirements Measuring Model Performance Overview 1 Model Classes 2 Credit Risk Measures 3 Model Implementation 4 Back Testing 5 Model Control c 2015 H¨aner Consulting Model Risk 139 / 166
140. 140. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Regulatory Requirements Measuring Model Performance Motivation Impact of Credit risk model Trading activity limits set by PE Capital charges regularity capital dependent of EEPE P&L EE enters CVA/DVA Model Risk Back-testing should quantify model risk aﬀecting these quantities. c 2015 H¨aner Consulting Model Risk 140 / 166
141. 141. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Regulatory Requirements Measuring Model Performance Requirements Back-testing Process Should provide Deﬁnition of measure for model risk Monitoring of metrics Mitigating actions for model deﬁciencies c 2015 H¨aner Consulting Model Risk 141 / 166
142. 142. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Regulatory Requirements Measuring Model Performance BCBS Guidance G1 G2 G3 c 2015 H¨aner Consulting Model Risk 142 / 166
143. 143. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Regulatory Requirements Measuring Model Performance BCBS Guidance G4 G5 G6 c 2015 H¨aner Consulting Model Risk 143 / 166
144. 144. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Regulatory Requirements Measuring Model Performance BCBS Guidance G7 G8 G9 c 2015 H¨aner Consulting Model Risk 144 / 166
145. 145. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Regulatory Requirements Measuring Model Performance BCBS Guidance G10 G11 G12 c 2015 H¨aner Consulting Model Risk 145 / 166
146. 146. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Regulatory Requirements Measuring Model Performance BCBS Guidance G13 G14 G15 c 2015 H¨aner Consulting Model Risk 146 / 166
147. 147. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Regulatory Requirements Measuring Model Performance BCBS Guidance G16 c 2015 H¨aner Consulting Model Risk 147 / 166
148. 148. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Regulatory Requirements Measuring Model Performance What is the Question? Types of Investigation Hypothesis testing (Answer in percentage or yes/no) Estimation of model uncertainty (Answer in cash terms) Analysis at diﬀerent levels: ﬁgure 7 c 2015 H¨aner Consulting Model Risk 148 / 166
149. 149. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Regulatory Requirements Measuring Model Performance Domains Economic Quantities Regulatory Capital Limits CVA/DVA Risk Measures EEPE PE EE Process Characterictics Marginal Distributions Auto-Correlations N-Point Functions Model-Dependent Quantities Model Parameters Driver dynamic Figure : Domains c 2015 H¨aner Consulting Model Risk 149 / 166
150. 150. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Regulatory Requirements Measuring Model Performance Deﬁnition A model is represented by a measure Q. May be generated by a stochastic process. Quantifying Diﬀerence of Models Comparing expectation values Comparing probability distributions Note: PDFs and CDFs may be expressed as expectation values c 2015 H¨aner Consulting Model Risk 150 / 166
151. 151. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Regulatory Requirements Measuring Model Performance Radon-Nikodym Derivative Distance of model Q and end empirical measure P in terms of dP dQ: EP[f ] = EQ[ dP dQ f ] (9) Compare P and Q Direct dP dQ ≈ id? Expectation values Empirical expectation measures in terms of model expectations Relative Entropy Kullback-Leibler entropy → information geometry (see [?]) c 2015 H¨aner Consulting Model Risk 151 / 166
152. 152. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Regulatory Requirements Measuring Model Performance Radon-Nikodym Derivative Let ξ be a scalar stochastic variable (e.g. portfolio price π(t)) Deﬁnition P empirical, Q model CDF Ψ : [0, 1] → [0, 1] (10) Ψ(α) = P(Q−1 (α)) (11) Radon-Nikodym derivative ψ EP[f ] = EQ[ψ(α)f ] (12) ψ(α) = dΨ(α) dα (13) c 2015 H¨aner Consulting Model Risk 152 / 166
153. 153. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Regulatory Requirements Measuring Model Performance Example (a) 0.0 0.2 0.4 0.6 0.8 1.0 α 0.0 0.2 0.4 0.6 0.8 1.0 Ψ(α) [x]=100.00;σ=0.40 [x]=110.00;σ=0.40 [x]=90.00;σ=0.40 [x]=100.00;σ=0.44 [x]=100.00;σ=0.36 0.8 1.0 1.2 1.4 1.6 1.8 ψ(α) [x]=100.00;σ=0.40 [x]=110.00;σ=0.40 [x]=90.00;σ=0.40 [x]=100.00;σ=0.44 [x]=100.00;σ=0.36 c 2015 H¨aner Consulting Model Risk 153 / 166
154. 154. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Regulatory Requirements Measuring Model Performance Cumulative Distribution Functions Cumulative distribution function (CDF) for some state variable ξ expressed as expectation: Deﬁnition P(ξ0) = EP[Θ(ξ − ξ0)] (14) where Θ is the Heaviside function. c 2015 H¨aner Consulting Model Risk 154 / 166
155. 155. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Regulatory Requirements Measuring Model Performance Estimating Ensemble averages E estimated well by time averages if ergodic stationary CDF P(ξ0) ≈ 1 N N i=1 Θ(ξ(ti ) − ξ0) (15) Ψ Ψ(α) ≈ 1 N N i=1 Θ(ξ(ti ) − Q−1 (α)) (16) c 2015 H¨aner Consulting Model Risk 155 / 166
156. 156. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Regulatory Requirements Measuring Model Performance Requirements for Estimation Process neeeds to be ergodic stationary iid price process If empirical price process is iid, the ergodic. iid process of underlying Even if underlying process the price return process of the deal may not be so, if deal not time homogeneus c 2015 H¨aner Consulting Model Risk 156 / 166
157. 157. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Regulatory Requirements Measuring Model Performance Distances Point Distance di = |Ψ(qi ) − qi | (17) Curve Distance (Weighted) quadratic distance d between functions q → Ψ(q) and q → q: d(q, Ψ(q)) = i wi (Ψ(qi ) − qi )2 (18) qi e.g (0.01, 0.05, 0.3, 0.5, 0.7, 0.95, 0.99) c 2015 H¨aner Consulting Model Risk 157 / 166
158. 158. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Regulatory Requirements Measuring Model Performance Hypothesis Testing Null-Hypothesis Null-Hypothesis, is that distances are 0. Reject Null-Hypothesis p-values smaller than some threshold Challenges estimating p-values Temporal dependence: overlap of time-windows Ensemble dependence: returns of netting sets not independent Good p values get bigger Bad Estimation tricky Need some simpliﬁcations, like eﬀective sample sizes c 2015 H¨aner Consulting Model Risk 158 / 166
159. 159. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Regulatory Requirements Measuring Model Performance Problems using metrics for Ψ Issues using metrics for Ψ Opaque no cash denominated measure Economics Product Dependent with same distance diﬀerent moments drive deviations in EE (see ﬁgure (9)) Limited usefulness Passes test if not enough data available c 2015 H¨aner Consulting Model Risk 159 / 166
160. 160. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Regulatory Requirements Measuring Model Performance Problems using metrics for Ψ 20 40 60 80 100 120 140 160 strike K 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5 EE EE [x]=100.00;σ=0.40 [x]=110.00;σ=0.40 [x]=90.00;σ=0.40 [x]=100.00;σ=0.44 [x]=100.00;σ=0.36 Figure : Comparing EEs for a forward using log-normal distributions with diﬀerent parameters c 2015 H¨aner Consulting Model Risk 160 / 166
161. 161. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Regulatory Requirements Measuring Model Performance Comparison using Cash denominated Quantities Economically Relevant Model Dependent Quantities Regulatory Capital depends on EE(t) (through EEPE) Limits impacted by CDF P&L impacted by EE(t) Measure These three quantities are functions of EQ. Their value under empirical measure P estimated through equation (12) → diﬀerence in cash terms c 2015 H¨aner Consulting Model Risk 161 / 166
162. 162. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Overview 1 Model Classes 2 Credit Risk Measures 3 Model Implementation 4 Back Testing 5 Model Control c 2015 H¨aner Consulting Model Risk 162 / 166
163. 163. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Requirements Control Processes should be Complete Accurate Consistent Timely Appropriate and Relevant Auditable c 2015 H¨aner Consulting Model Risk 163 / 166
164. 164. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Governance Board of directors and senior management to establish a strong model risk framework Roles and responsibilities: clear reporting lines and incentives, address conﬂicts of interest, suﬃcient authority to control staﬀ Firmwide model inventory should model use, products, responsible parties and planned activities Detailed documentation: understand how the model operates, limitations and key assumptions Developers, users, control and compliance units should document their work including ongoing monitoring, c 2015 H¨aner Consulting Model Risk 164 / 166
165. 165. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Audit Review policies and compliance Conﬁrm and if appropriate challange validation work Review records of model use, and conﬁrm models are subject to controls, and also account for limitations Verify accuracy and completeness of the model inventory Assess the process for establishing and monitoring limits and usage Asessments of operational systems and evaluate the reliability of data used by models Report ﬁndings to the board c 2015 H¨aner Consulting Model Risk 165 / 166
166. 166. Model Classes Credit Risk Measures Model Implementation Back Testing Model Control Policies Model Validation Model Development IT Data Management Backtesting Stresstesting c 2015 H¨aner Consulting Model Risk 166 / 166