modeFinance MORE Rating Validation Worldwide

HOW SUCCESFUL WERE the “MORE” RATINGS in PREDICTING the IMPACTSof THE GLOBAL CRISIS in 2008-2009?SummaryIntroduction ......................................................................................................................................... 2Validation of the MORE ratings on the ENTIRE bankruptcy database (Between the years: 2000-2009) .................................................................................................................................................... 3Validation of the MORE ratings on the ENTIRE bankruptcy database; only during the 2008&2009crisis...................................................................................................................................................... 5Validation of the MORE rating in Continental Macro Areas ................................................................ 8 Validation of the MORE rating in West Europe ............................................................................... 9 Validation of the MORE rating in East Europe ............................................................................... 10 Validation of the MORE rating in North America .......................................................................... 11 Validation of the MORE rating in Far East ..................................................................................... 12 Comments on Continental Macro Areas Rating Evaluation .......................................................... 13Conclusions ........................................................................................................................................ 13Appendix ............................................................................................................................................ 14 modeFinance For more information visit: AREA Science Park www.modefinance.com or send Padriciano 99 an e-mail to: 34012 Trieste ITALY info@modefinance.com Pag | 1

IntroductionTo validate a rating model, we have to demonstrate that: 1. For bankrupt companies, the assigned rating deteriorates, approaching the default date (Bankruptcy dynamics). 2. The model discriminates between profitable companies and bankrupt companies (Discriminating power of the model).For the above mentioned two steps, we need to have the information on bankrupt companies.Using the companies labeled as “bankrupt” in ORBIS (a global database which has information on60 million companies, provided by Bureau van Dijk Electronic Publishing) we have compiled thefollowing distribution (years: 2000-2009): Country Number of bankrupt companies FR 9110 BE 8172 NL 5232 IT 4529 UA 1991 RO 1207 PL 1100 FI 1098 CZ 780 LT 655 LV 598 EE 525 RU 320 SK 158 Others 318 Table 1 Number of defaulted companies in ORBIS Number of bankroupt companies 10000 9000 8000 7000 6000 5000 4000 3000 2000 1000 0 Fig. 1 Distribution of defaulted companies in ORBIS modeFinance For more information visit: AREA Science Park www.modefinance.com or send Padriciano 99 an e-mail to: 34012 Trieste ITALY info@modefinance.com Pag | 2

We proposed that the following methodology will be applied: 1. Validation of the MORE ratings on the ENTIRE bankruptcy database (years 2000-2009); 2. Validation of the MORE ratings on the ENTIRE bankruptcy database but only in 2008- 2009 to enhance the knowledge of MORE ratings in the financial crisis. 3. Validation of the MORE rating in Continental Macro Areas.In each validation, we followed two steps to demonstrate the following features of the model:Bankruptcy Dynamics & Discriminating power of the model.Validation of the MORE ratings on the ENTIRE bankruptcy database(Between the years: 2000-2009)To validate MORE ratings, first we studied the evaluation of the bankrupt companies’ ratingschecking their evolution over the years. In this case, all bankrupt companies in ORBIS were used(around 40,000 companies).We observed the ratings one, two, three and four years before the FINAL available annual report. Fig. 2 Distribution of the ratings of defaulted companies (entire ORBIS database) modeFinance For more information visit: AREA Science Park www.modefinance.com or send Padriciano 99 an e-mail to: 34012 Trieste ITALY info@modefinance.com Pag | 3

BB B CCC CC C Fig. 3 Dynamic of defaulted companies: rating distribution and mean rating (entire ORBIS database)In order to demonstrate the first feature, it is important to see ratings’ evolution for the bankruptcompanies. From Fig.3, it is possible to observe that the MORE ratings catch the performancedegradation of the companies with a high accuracy, approaching the default date.It was observed that one year before the last available annual report, 48% of the companies wereclassified as risky and 27% as vulnerable, so that 75% of companies had bad economic andfinancial ratings. It is very interesting to note that four years before the last accounting year,MORE ratings classified 64% of the defaulted companies as risky and vulnerable (or worse); and soMORE has a great ability to predict the default.As a second step, we had to demonstrate if the model discriminates between the non-defaultedcompanies and defaulted companies. To execute this, we used one of the well-known statisticalmethods: ROC (Relative or receiver operating characteristic, please see Appendix for details). It ispossible to observe within the following graphs that the model achieved very accurate results indistinguishing defaulted companies, reaching a AUC (area under the curve ) value of 86 in the lastyear, with a very promising AUC value of 73 four years before the final available annual report.This behavior can be seen when the distribution of the world rating, a typical Gaussiandistribution, where BB is the most probable rating class is compared to the distribution of thedefaulted companies in which the most probable rating classes are the poor ones. (See Fig.4) modeFinance For more information visit: AREA Science Park www.modefinance.com or send Padriciano 99 an e-mail to: 34012 Trieste ITALY info@modefinance.com Pag | 4

Fig. 4 World rating distribution: non-defaulted companies (left), defaulted companies (right). Entire ORBIS database Fig. 5 ROC graphs; one year before: up-left; four year before: down right. Entire ORBIS database.In conclusion, it is possible to assert that the model predicted the world defaulted companieswith very good accuracy even four years before bankruptcy occurred.Validation of the MORE ratings on the ENTIRE bankruptcy database; onlyduring the 2008&2009 crisisAfter evaluating the MORE ratings on the entire database, we wanted to extend the study toinvestigate if the MORE ratings recognized the financial crisis during the years 2008-2009. This isbecause understanding MORE ratings’ success in monitoring the crisis was quite important. modeFinance For more information visit: AREA Science Park www.modefinance.com or send Padriciano 99 an e-mail to: 34012 Trieste ITALY info@modefinance.com Pag | 5

To do this, the companies which went bankrupt in 2008-2009 were selected and studied in thesame way as before. In this case, the database consisted of around 4,000 companies from allaround the world. Fig .6 Distribution of the ratings of defaulted companies (2008-2009 ORBIS database) modeFinance For more information visit: AREA Science Park www.modefinance.com or send Padriciano 99 an e-mail to: 34012 Trieste ITALY info@modefinance.com Pag | 6

BB B CCC CC C Fig 7 Dynamic of defaulted companies: rating distribution and mean rating (2008-2009 ORBIS database)In the first step; the MORE ratings once again recognized the evolution of the bankrupt companiesapproaching the default date. From Fig 7 it is possible to observe that one year before the finalaccounting year, the MORE ratings classified 75% of the defaulted companies as vulnerable (orworse).It is interesting to note that the MORE ratings had a stable behavior when companies movedtowards bankruptcy, both in 2000-2009 and in the 2008-2009 crisis. This demonstrates that theMORE methodology was successful in predicting the impacts of the financial crisis.Again as the second step, we wanted to demonstrate if the model can discriminate between nondefaulted companies and defaulted ones. As we did for the world companies between the years2000-2009; we also used ROC for the world companies in the 2008- 2009 crisis. According to theAUC values, the model again achieved very accurate results in distinguishing defaulted companiesreaching an AUC value of 85 in the last year with an AUC value of 73 four years before the finalavailable annual report. This behavior is seen in the comparison between the distribution of theworld rating (the distribution where BB is the most probable rating class) and the distribution ofthe default companies in which the most probable rating classes are the poor ones. (See Fig .8) Fig .8 World rating distribution: non-defaulted companies (left), defaulted companies (right). 2008-2009 ORBIS database modeFinance For more information visit: AREA Science Park www.modefinance.com or send Padriciano 99 an e-mail to: 34012 Trieste ITALY info@modefinance.com Pag | 7

Fig. 9 ROC graphs; one year before: up-left; four year before: down right. 2008-2009 ORBIS databaseValidation of the MORE rating in Continental Macro AreasIn order to understand better the behavior of MORE rating, we evaluated the rating performancesin the different Continental Macro Area (following the definitions of ORBIS), defined in thefollowing tables.Continental Macro Area Number of bankrupt companiesWest Europe 30000East Europe 9000North America 120Far East 50Rest of the World 10For statistical reasons to apply the ROC evaluation and the rating evolution, we performed theanalysis on: West Europe, East Europe, North America and Far East. In all those cases we dontused the LAST available information of defaulted companies, but one year before the last year (asset by Basel II policy). modeFinance For more information visit: AREA Science Park www.modefinance.com or send Padriciano 99 an e-mail to: 34012 Trieste ITALY info@modefinance.com Pag | 8

Validation of the MORE rating in West Europe 100% 90% One year before the last availble 80% account 70% 30,00% 60% 50% Risky companies (CC-C-D) 25,00% 40% Vulnerable companies (B-CCC) 20,00% 30% Balanced companies (BBB-BB) 15,00% 20% Healthy companies (AAA-AA-A) 10% 10,00% 0% 5,00% One year before Two years before Three years before the last available the last available the last available 0,00% account account account AAA AA A BBB BB B CCC CC C D Two years before the last availble Three years before the last availble account account 30,00% 30,00% 25,00% 25,00% 20,00% 20,00% 15,00% 15,00% 10,00% 10,00% 5,00% 5,00% 0,00% 0,00% AAA AA A BBB BB B CCC CC C D AAA AA A BBB BB B CCC CC C D Fig 10 Dynamic of defaulted companies: rating distribution and mean rating (West Europe ORBIS database) Mean rating evolution BB 6 B 5 CCC 4 CC 3 Three years before the last available Two years before the last available One year before the last available account account account Fig 11 Mean rating evolution and ROC graph: West Europe ORBIS database modeFinance For more information visit: AREA Science Park www.modefinance.com or send Padriciano 99 an e-mail to: 34012 Trieste ITALY info@modefinance.com Pag | 9

Validation of the MORE rating in East Europe 100% 90% One year before the last availble 80% account 70% 25,0% 60% 50% Risky companies (CC-C-D) 20,0% 40% Vulnerable companies (B-CCC) 30% Balanced companies (BBB-BB) 15,0% 20% Healthy companies (AAA-AA-A) 10,0% 10% 0% 5,0% One year before Two years before Three years before the last availble the last availble the last availble 0,0% account account account AAA AA A BBB BB B CCC CC C D Two years before the last availble Three years before the last availble account account 25,0% 25,0% 20,0% 20,0% 15,0% 15,0% 10,0% 10,0% 5,0% 5,0% 0,0% 0,0% AAA AA A BBB BB B CCC CC C D AAA AA A BBB BB B CCC CC C DFig 12 Dynamic of defaulted companies: rating distribution and mean rating (East Europe ORBIS database) Mean rating evolution 6 BB 5 B CCC 4 CC 3 Three years before the last availble Two years before the last available One year before the last available account account account Fig 13 Mean rating evolution and ROC graph: East Europe ORBIS database modeFinance For more information visit: AREA Science Park www.modefinance.com or send Padriciano 99 an e-mail to: 34012 Trieste ITALY info@modefinance.com Pag | 10

Validation of the MORE rating in North America 100% 90% One year before the last availble 80% account 70% 30,0% 60% 50% Risky companies (CC-C-D) 25,0% 40% Vulnerable companies (B-CCC) 20,0% 30% Balanced companies (BBB-BB) 15,0% 20% Healthy companies (AAA-AA-A) 10% 10,0% 0% 5,0% One year before Two years before Three years before the last availble the last availble the last availble 0,0% account account account AAA AA A BBB BB B CCC CC C D Two years before the last availble Three years before the last availble account account 30,0% 35,0% 25,0% 30,0% 25,0% 20,0% 20,0% 15,0% 15,0% 10,0% 10,0% 5,0% 5,0% 0,0% 0,0% AAA AA A BBB BB B CCC CC C D AAA AA A BBB BB B CCC CC C D Fig 14 Dynamic of defaulted companies: rating distribution and mean rating (North America ORBIS database) Mean rating evolution BB 6 B 5 CCC 4 CC 3 Three years before the last availble Two years before the last available One year before the last available account account account Fig 15 Mean rating evolution and ROC graph: North America ORBIS database modeFinance For more information visit: AREA Science Park www.modefinance.com or send Padriciano 99 an e-mail to: 34012 Trieste ITALY info@modefinance.com Pag | 11

Validation of the MORE rating in Far East 100% 90% One year before the last availble 80% 70% account 60% Risky companies (CC-C-D) 35,0% 50% 40% Vulnerable companies (B-CCC) 30,0% 30% Balanced companies (BBB-BB) 25,0% 20% Healthy companies (AAA-AA-A) 20,0% 10% 0% 15,0% One year before Two years before T ree years before 10,0% the last availble the last availble the last availble account account account 5,0% 0,0% AAA AA A BBB BB B CCC CC C D Two years before the last availble Three years before the last availble account account 45,0% 45,0% 40,0% 40,0% 35,0% 35,0% 30,0% 30,0% 25,0% 25,0% 20,0% 20,0% 15,0% 15,0% 10,0% 10,0% 5,0% 5,0% 0,0% 0,0% AAA AA A BBB BB B CCC CC C D AAA AA A BBB BB B CCC CC C DFig 16 Dynamic of defaulted companies: rating distribution and mean rating (Far East ORBIS database) Mean rating evolution BB 6 B 5 CCC 4 CC 3 Three years before the last availble Two years before the last available One year before the last available account account account Fig 17 Mean rating evolution and ROC graph: Far East ORBIS database modeFinance For more information visit: AREA Science Park www.modefinance.com or send Padriciano 99 an e-mail to: 34012 Trieste ITALY info@modefinance.com Pag | 12

Comments on Continental Macro Areas Rating EvaluationFrom the graphs of the MORE rating evaluation in the different Continental Areas (Fig 10, Fig 11,Fig 12, Fig 13, Fig 14, Fig 15, Fig 16, Fig 17) it is possible to observe that the behavior of the MORErating is quite stable and accurate. In every Continental Areas there is a clear downgrade of theratings of the defaulted companies approaching to the bankrupt data.As it is possible to observe from the following table, the accuracy of the MORE rating is quite highall around the world. MORE rating obtains one slight difference only for the Far East evaluationwhere the number of bankrupt companies is very low to have an accurate statistical analysis.Continental Macro Area AUC value one year AUC value two year AUC value three year before bankrupt before bankrupt (Gini before bankrupt (Gini (Gini value) value) value)West Europe 0,83 (0,66) 0,76 (0,52) 0,72 (0,44)East Europe 0,83 (0,66) 0,77 (0,54) 0,74 (0,48)North America 0,9 (0,8) 0,83 (0,66) 0,79 (0,58)Far East 0,77 (0,54) 0,64 (0,28) 0,55 (0,1)ConclusionsIn order to validate the MORE model, we used the bankruptcy information on companies found inORBIS. The entire database and data from the 2008-2009 crisis were taken into account.Using two well-known statistical methods, ROC & Bankruptcy Dynamic the MORE ratings achievedvery ACCURATE and STABLE results for the entire database as well as data from the crisis period.The results are stable and this is essential for the validation of the rating method. From thisanalysis we were able to confirm the worth of the “world” (entire database) validation.In order to enhance the quality of the validation, the rating MORE has been evaluated on fourdifferent World macro regions. Also in those cases, the results are accurate and stable betweendifferent economical regions. modeFinance For more information visit: AREA Science Park www.modefinance.com or send Padriciano 99 an e-mail to: 34012 Trieste ITALY info@modefinance.com Pag | 13

AppendixmodeFinance visionThe modeFinance vision is to look at the fundamental economic and financial aspects of thecompany. The main idea is to evaluate the rating by observing each aspect of the economic andfinancial behavior of the company: the better the equilibrium between the different aspects, thebetter the final rating.This idea has been implemented into the MORE (Multi Objective Rating Evaluation) rating model;the MORE model permits the user to assess the creditworthiness of a company by aggregating andevaluating the most important sections of the financial and economic behaviors of a companysuch as profitability, liquidity, solvency, interest coverage and efficiency.Unlike the most used rating models, MORE does not provide an output as a statistical measure likethe probability of bankruptcy; but qualitative information about the general creditworthiness ofthe analyzed company. This information is expressed through a credit rating scale (shown below). modeFinance For more information visit: AREA Science Park www.modefinance.com or send Padriciano 99 an e-mail to: 34012 Trieste ITALY info@modefinance.com Pag | 14

Rating class Rating Macro class Assessment The companys capacity to meet its financial commitments is extremely strong. The company shows an excellent economic and financial flow and AAA fund equilibrium The company has very strong creditworthiness. It also has a good capital Healthy structure and economic and financial equilibrium. Difference from AAA is AA companies slight The company has a high solvency. The company is however more susceptible to the adverse effects of changes in circumstances and A economic conditions than companies in higher rated categories Capital structure and economic equilibrium are considered adequate. The companys capacity to meet its financial commitments could be affected BBB by serious unfavourable events Balanced A company rated BB is more vulnerable than companies rated BBB. companies Furthermore the company faces major ongoing uncertainties or exposure BB to adverse business, financial, or economic conditions The company presents vulnerable signals with regard to its fundamentals. Adverse business, financial, or economic conditions will be likely to impair B the companys capacity or willingness to meet its financial commitments Vulnerable A company rated CCC has a dangerous disequilibrium on the capital companies structure and on its economic and financial fundamentals. Adverse CCC market events and an inadequate management could affect with high probability the companys solvency The company shows signals of high vulnerability. In the event of adverse market and economic conditions, the companys strong disequilibrium CC could increase Risky The company shows considerable pathological situations. The companys C companies capacity to meet its financial commitments is very low The company has not any longer the capacity to meet its financial D commitments modeFinance For more information visit: AREA Science Park www.modefinance.com or send Padriciano 99 an e-mail to: 34012 Trieste ITALY info@modefinance.com Pag | 15

The probability of defaultThe probability of default (PD) provided by modeFinance is coupled with the rating class of eachevaluated company; it is a quantitative information that expresses the possibility that a companydeteriorates its financial and economical strengths; so the definition of PD is “the probability thata company goes in D class (the worst) within one year.” This means that the PD is the probabilitythat a company will face distress.In order to evaluate the probability of default, modeFinance observes the historical variations ofthe ratings distribution during the years by using the transition matrix theory (as shown in thefigure below). This means computing the frequency of the rating downgrades in D class byobserving a long historical series of ratings.This definition of PD comprehends even the probability of bankruptcy; because according tomodeFinance definition, a company rated D should be restructured financially for not goingbankrupt.In order to validate its rating model and the PD definition, modeFinance tested it on the capabilityto discriminate the profitable companies from the non-profitable companies (following the Basel IIrules of Bank for International Settlements and the most known validation techniques).In the following section is presented an example regarding the validation of the ratings evaluatedfor French companies.ROCThe MORE ratings have been tested by using ROC (relative or receiver operating characteristic)method and computing the AUC (area under the curve). ROC curves generalize contingency tableanalysis by providing information on the performance of a model at any cut-off that might bechosen.ROCs are constructed by scoring all credits and ordering the non-defaulters from worst to best onthe x axis and then plotting the percentage of defaults excluded at each level on the y axis. So, they axis is formed by associating every score on the x axis with the cumulative percentage of defaults modeFinance For more information visit: AREA Science Park www.modefinance.com or send Padriciano 99 an e-mail to: 34012 Trieste ITALY info@modefinance.com Pag | 16

with a score equal to or worse than that score in the test data. In other words, the y axis gives thepercentage of defaults excluded as a function of the number of non-defaults excluded.A convenient measure for summarizing the graph of the ROC is the area under the ROC (AUC),which is calculated as the proportion of the area below the ROC relative to the total area of theunit square. A value of 0.5 indicates a random model, and a value of 1.0 indicates perfectdiscrimination.A rough guide for classifying the accuracy of a diagnostic test is the traditional academic pointsystem:• 1,00-0,90: excellent• 0,90-0,80: good• 0,80-0,70: adequate• 0,70-0,60: poor• 0,60-0,50: fail Perfect model 1 A % Defaulted excluded Real model B Sorted no default Fig 18 Example of ROC analysis modeFinance For more information visit: AREA Science Park www.modefinance.com or send Padriciano 99 an e-mail to: 34012 Trieste ITALY info@modefinance.com Pag | 17

modeFinance MORE Rating Validation Worldwide

by Mattia Ciprian

Accessibility

Upload Details

Usage Rights

1 Embed 2

Statistics

modeFinance MORE Rating Validation Worldwide Document Transcript