HCS Summary, Validation & FAQ's

405 views

Published on

People Capital's Human Capital Score (HCS)
Summary, Validation and FAQ's of the exisiting Human Capital Score

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
405
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

HCS Summary, Validation & FAQ's

  1. 1. HumanCapitalScore™Summary and ModelValidationConfidential Information
  2. 2. For additional information, please visit www.people2capital.com or e-mail alan.samuels@people2capital.comIndexWhy the Human Capital Score™ is Relevant? ......................................3How the Human Capital Score™ Works...............................................3The Human Capital Score™ and Economic Cycles ................................4Validating the Human Capital Score™ Model ......................................4Frequently Asked Questions about the Human Capital Score™ ...........8 What is the Human Capital Score? ........................................................................................................... 8 Why hasnt this been done before? .......................................................................................................... 8 What insights does the Human Capital Score™ provide? ......................................................................... 8 How does the Human Capital Score™ address economic condition fluctuations? .................................. 9 On what scale is the Human Capital Score™ provided? ........................................................................... 9 So how does the Human Capital Score™ work? ....................................................................................... 9 What cant the Human Capital Score™ do (aka "the fine print")? .......................................................... 10PRIVATE & CONFIDENTIAL Page 2
  3. 3. For additional information, please visit www.people2capital.com or e-mail alan.samuels@people2capital.comWhy the Human Capital Score™ is Relevant?Traditional credit measures (e.g. the FICO© score) are oftentimes inadequate in evaluating the risk ofeducational loans since students have had little chance to develop a representative credit history andmay be incorrectly perceived as "high risk" borrowers. The Human Capital Score™ (HCS) is an innovativeproprietary credit risk measure for students developed by People Capital to overcome this informationbarrier between student borrowers and educational lenders. The HCS™ is an ideal credit risk measurefor student loans because it uses (just as the traditional credit scores do) standardized and verifiableattributes; however, these attributes are common to all students (like test scores and majors) instead ofcommon to all borrowers (like debt repayment history). Instead of predicting future default directly, themodel predicts students’ future incomes. By predicting future income, the HCS™ can better evaluate thestudents ability to repay their student loans in the future and is a more appropriate measure of risk instudent lending.How the Human Capital Score™ WorksThe HCS™ is modeled developed using historical postsecondary education data from several sources.These data allow us to evaluate the relative effect of certain attributes (such as high school, GPA,SAT/ACT scores, college, and major) on students future earnings. We then calibrate these incomeestimates to current income levels to predict students incomes for the 10 years after collegegraduation. These income predictions, while very accurate in predicting the ability for students to repayeducation debt, will not, however, inform lenders on the propensity of students to repay educationdebt out of what they earn.The HCS™ model gathers information from the student including standardized test scores, GPA, highschool, college, and major. The HCS calculator in turn provides two pieces of information about thestudent: first, a Human Capital Score™, denoting the relative "risk" of students, (this is currentlypresented on a scale of 1-9 with "+" or "-" used to denote scores at the high/low end of a range, thoughthe calculator can provide gradations on a 99 point scale); and second, the projected income path of astudent over the ten years after graduation, which includes a mean predicted income for each year aswell as a predicted income range. This "income band" illustrates the range of possible annual incomes astudent can reasonably expect to earn in a given year. Statistically speaking, a student with a given set ofattributes can expect to earn an income that falls within this income range 80% of the time; we similarlyexpect the student to earn an income that falls above this range 10% of the time and below this range10% of the time.PRIVATE & CONFIDENTIAL Page 3
  4. 4. For additional information, please visit www.people2capital.com or e-mail alan.samuels@people2capital.comThe Human Capital Score™ and Economic CyclesThe current economic downturn has had a negative effect on the employment prospects of collegegraduates, and, therefore, income expectations should be adjusted to account for the subsequent effecton graduates incomes. To account for these cyclical effects on expected incomes of college graduates,the HCS™ model incorporates a "recession adjustment" that is derived from current and historical stateunemployment rates. The unemployment rate is used as an instrument for the effect on potentialearnings because it is affected by a recessionary cycle much the same way college students earningpotential is affected; in order to earn an income one must be employed. Therefore, we expect that ahigher unemployment rate will result in lower expected earnings. For simplicity, we assume thatstudents generally find work in the same state as the undergraduate institution from which theyrecently graduated. We calculate a "recession weight" derived from the change in each statesunemployment rate between the current year and the years of the historical data used in the model.This change in state unemployment rate allows us to control for the current economic downturn whilestill using the historical data. We can adjust our predictions since we observe how the unemploymentrate in the current year differs from the unemployment rate experienced by the college students in ourdataset. We essentially adjust the predicted income of students in proportion with the stateunemployment rates of the current recession less the unemployment rate experienced by the studentsin the dataset.Validating the Human Capital Score™ ModelIn developing the HCS™ model, the People Capital team has aggregated data from several institutionaland government sources into the dataset used in the model. These data consist of US undergraduatestudents and information including their schools, majors, and earnings after college. We then use thesedata to predict each students future earnings for the decade after graduation. Since these data containinformation on realized earnings in the ten years following graduation, we are able to validate the HCS™models predictions in two ways using our original sample. First, we evaluate the accuracy of predictedincome as compared to realized income within predicted income deciles for our entire sample. Then, wesplit the sample at random into two parts—a "regression" sample and a "test" sample—in order tosimulate an "out of sample" test while still using our original data. In this simulated out of sample test,we first regress salary on attributes for the "regression" half of our initial overall sample. The results ofthat regression are then used to predict the income for the observations within the "test" half of ourinitial overall sample. Ideally, the "test" half of the sample would result in predicted incomes as accurate(i.e. as close to the corresponding realized income) as the "regression" half of the sample.Figure 1 and Figure 2 both show deciles of HCS™ predicted income and corresponding realized incomefor Years 4 and 5 after graduation. In these figures, our measure of "accuracy" is determined by theproximity of the results to the ideal: that realized income matches the predicted income value so thatPRIVATE & CONFIDENTIAL Page 4
  5. 5. For additional information, please visit www.people2capital.com or e-mail alan.samuels@people2capital.compoints lay on the 45 degree line.1 To illustrate the accuracy of the HCS™ predictions, we first sort theincome predictions into deciles. For each predicted income decile group, we calculate the decile groupsmean predicted income as a fraction of the overall samples mean realized income (i.e. the proximity ofthe mean predicted income within a decile group to the mean realized income over the whole sample).Similarly, we calculate the decile groups mean realized income as a fraction of the overall samplesmean realized income. We use these fractions instead of actual income values in order to "normalize"the results so as to both preserve the integrity of People Capitals intellectual property and also to makethe results comparable from year to year.There are two relevant values for the deciles: the mean predicted income of the decile group as afraction of the mean realized income of the whole sample, and the mean realized income of the decilegroup as a fraction of the mean realized income of the whole sample. In Figure 1 and Figure 2, eachpoint represents a decile group, the x-axis measures the decile groups predicted income fraction andthe y-axis measures the decile groups realized income fraction. Figure 1: Validating the Human Capital Score™ Income Predictions, Year 4 YEAR 4, HCS™ Predicted Income vs Realized Income (by Deciles) 1.40 Decile Actual Income Mean/Overall 1.30 Decile Actual Income Mean 1.20 Income 1.10 Fraction 1.00 45° Line (Perfect Fit) 0.90 0.80 0.70 0.70 0.80 0.90 1.00 1.10 1.20 1.30 1.40 Decile HCS™ Predicted Income Mean/Overall Actual Income Mean1 Note: When we refer to realized income, we mean actual income for 4-Year realized income and smoothedincome for 5-year realized income. The data only contain income information on select years—approximately sixout of the ten years after graduation (for example, there are no data on 5-year incomes, but there are data on 4-year incomes). Since we require income values for all ten years after graduation, we generate a smoothed incometrend for each individual. The smoothed income is calculated as follows: we first assume that income growth islinear, and then, we regress salary on years out of college for each individual to calculate the slope and intercept ofeach individuals smoothed income trend. Due to our use of smoothed income in the HCS™ model, we providetwo versions of the first part of model validation; Figure 1 (Year 4) shows the 4-year predicted income and actualincome, whereas Figure 2 (Year 5) shows the individuals 5-year predicted income and smoothed income. Wetested alternative patterns of income growth (e.g. log-linear), but the data showed that income growth is bestsmoothed linearly.PRIVATE & CONFIDENTIAL Page 5
  6. 6. For additional information, please visit www.people2capital.com or e-mail alan.samuels@people2capital.comFigure 1 illustrates HCS income results for the 4th year after college, and Figure 2 illustrates the HCSincome results for the 5th year after college. Ideally, all ten points would be on the 45° line in Figure 1and Figure 2, thus indicating that the realized income and predicted income fractions for the decilegroup are equal. As Figure 1 shows, the 4-year decile income fractions are fairly close to the 45° line.Figure 2 similarly shows that the 5-year decile income fractions are also quite close to the 45° line.Proximity of the decile data points to the 45° line indicates that the average HCS™ income predictionsare almost equal to the average realized income for a decile group in a given year (Year 4 for Figure 1and Year 5 for Figure 2). Figure 2: Validating the Human Capital Score™ Income Predictions, Year 5 YEAR 5, HCS™ Predicted Income vs Realized Income (by Deciles) 1.40 Decile Smoothed Income Mean/Overall 1.30 Smoothed Income Mean Decile 1.20 Income Fraction 1.10 1.00 45° Line (Perfect Fit) 0.90 0.80 0.70 0.70 0.80 0.90 1.00 1.10 1.20 1.30 1.40 Decile HCS™ Predicted Income Mean/Overall Smoothed Income MeanThe second portion of the HCS™ model validation involves an "out of sample" test, which is oftentimesemployed to evaluate whether or not a model can accurately forecast for an "out of sample" dataset(data not used in the creation of the model). Ideally, we would use a completely different set of data toperform this test. However, since we do not have data with all of the necessary information, we useinstead the current sample split into two random parts: the first half of our initial sample (Part A) is usedin the regression and to generate the model, while the second half of our initial sample (Part B) is usedto "test" the accuracy of the model. This test evaluates the models forecasting accuracy because aresulting income prediction may appear to be a "good fit" for "in sample" (Part A) data, but may not beaccurate in forecasting income for "out of sample" (Part B) data.To test that the HCS™ income predictions are accurate for individuals outside of the regression sample,we regress salary on attributes for the "regression" half of our original sample (Part A), and then, usingthe results from that regression, we predict the income for observations in the "test" half of our originalsample (Part B). Ideally, the income prediction coefficients based on the regression half of our originalPRIVATE & CONFIDENTIAL Page 6
  7. 7. For additional information, please visit www.people2capital.com or e-mail alan.samuels@people2capital.comsample (Part A) should produce an accurate prediction for "out of sample" observations from the testhalf of our original sample (Part B). This result would indicate that the model not only fits the “insample” data, but that it also accurately predicts income for new observations not used in the creationof the model. Figure 3 illustrates the accuracy of mean 5-year predicted income and mean 5-year realized income forthe predicted income deciles for two sets of data points—one set represents the predictions for the "insample" data (Part A) and another set that represents the predictions for the "out of sample" data (PartB). Figure 3: Out of Sample Test to Validate the Human Capital Score™ Income Predictions, Year 5 YEAR 5, HCS™ Predicted Income vs Realized Income, Out of Sample Test 1.5 Decile Smoothed Income Mean/Overall Smoothed 1.4 In Sample 1.3 [Decile Income 1.2 Fraction] Income Mean 1.1 Out of Sample 1 [Decile Income 0.9 Fraction] 45° Line 0.8 (Perfect Fit) 0.7 0.6 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 Decile HCS™ Predicted Income Mean/Overall Smoothed Income MeanIdeally, both sets of data points would reside on the 45° line, indicating that the resultant "accuracy" ofthe HCS income predictions (closeness to realized income values) does not differ between the in-sampleresults and the out-of-sample results. The proximity of both sets data points to the 45° line indicatesthat both the "in sample" and "out of sample" mean predicted incomes are close to mean realizedincome for each of the predicted income deciles. The "in sample" data points are similar in relevance tothe data points in Figure 1 and Figure 2—they show that the mean HCS™ 5-year predicted income foreach of the deciles is close to the mean 5-year smoothed income value for each of the deciles.PRIVATE & CONFIDENTIAL Page 7
  8. 8. For additional information, please visit www.people2capital.com or e-mail alan.samuels@people2capital.comHowever, a more powerful validation of the HCS™ model emerges from the "out of sample" results.These points represent the predicted income deciles aggregate accuracy of the HCS™ 5-year incomepredictions for the "test" half of the original sample—which is the closest we have to an "out of sample"dataset. The "out of sample" income predictions are almost as similar to the "out of sample" realizedincomes as the "in sample" predictions are to the "in sample" realized incomes. In terms of validation,these results indicate that the model maintains its accuracy when used with observations outside of theregression sample. This "out of sample" test validates the HCS™ models ability to forecast incomeaccurately for an out-of-sample student—perhaps a new prospective student borrower.Frequently Asked Questions about the Human Capital Score™What is the Human Capital Score?The Human Capital Score™ (HCS) is a proprietary credit risk measure developed by People Capital. Itcalculates future income potential by including variables such as GPA, standardized test scores, collegeand major. For students — who have short or no credit history — the FICO® Score is unlikely to be anappropriate measure of credit risk. While the current version of the Human Capital Score™ is bestcalibrated for Bachelors degrees, we are working on future enhancements that will make the HCSapplicable to other degree types.Why hasnt this been done before?Traditional credit scores (such as FICO® Score or VantageScore) are based on attributes that are easy toquantify and rank. More delinquencies lower the score; while a longer credit history raises the score.Evaluating student academic attributes is not so simple. How do we know which schools or majors aremore likely to correlate with the ability to earn income and the capacity to repay a loan? We mustcollect, clean, and integrate additional data about schools, majors and such. This requires expertise,time and dedicated resources.What insights does the Human Capital Score™ provide?The Human Capital Score™ (HCS) projects the possible income paths of college students in the 10 yearsafter graduation. This allows the classification of students into various risk categories which lenders canuse to consider the capacity of a given group of college students to repay loans of long- and shortmaturities. The projected income shortly after graduation is a good indicator of short term capacity topay. Longer-term loans can be assessed by looking at predicted income over a longer period. Forbenchmarking purposes, we provide a Human Capital Score™ for students for the period 2 years and 8years post-graduation. Note that the current version of the Human Capital Score™ is best calibrated forBachelors degrees, we are working on future enhancements that will make the HCS applicable to otherdegree types.Projections of the average future income should be considered with full understanding that it is acomputer based algorithm that uses historical data and a broad view of future economic conditions togenerate a result. As with any score that tries to categorize individual credit capacity, there is a range ofpossible future results. The Human Capital Score™ offers broad ranking categories as well as measuresPRIVATE & CONFIDENTIAL Page 8
  9. 9. For additional information, please visit www.people2capital.com or e-mail alan.samuels@people2capital.comof ranges of possible income paths. This makes it possible to evaluate the likelihood that income will fallbelow a certain threshold in a given year, or that average income will fall below a certain threshold inthe 10 years following graduation.How does the Human Capital Score™ address economic condition fluctuations?Traditional credit scoring models are based on historical data on defaults and credit attributes (e.g., debtoutstanding, number of credit cards, etc.). Attributes linked to low default rates in this historical dataare given high scores; attributes linked to high default rates are given low scores. Such traditionalmodels do not seek to understand why, or how, there is a link between an attribute (e.g., number ofcredit cards) and default; rather, they only reflect patterns and links from their historical data.Instead of focusing on factors that predict credit default, the Human Capital Score™ uses both academicand credit data to project future earnings potential. Our model identifies the set of earnings paths(around a projected average) possible for a borrower with a given set of attributes (e.g., major, school,SAT score, GPA, etc.). Given these earnings paths, we can determine how often an individual is likely tobe able to generate sufficient income to pay off a loan. Because we model the reason why people mightdefault (insufficient income), the model is easy to adjust to projected changes in economic conditions,even to circumstances never seen before in historical data. When income projections fall in response tochanging economic conditions, the Human Capital Score™ will reduce income projections.The Human Capital Score™ model differentiates between people whose earnings paths (under normalconditions) were projected to be often just barely sufficient to make their debt payments from thosewhose income paths were projected to be more than sufficient. If changing economic conditions reduceour income projections, this will reduce Human Capital Score™ more for the first group than the second.This flexibility makes the Human Capital Score™ a superior tool for rank ordering students who, as ageneral matter, have no significant credit history – students ability to pay is directly dependent on theirfuture earning capacity within a future economic context.On what scale is the Human Capital Score™ provided?The Human Capital Score™ is able to generate 99 gradations of income potential, however, PeopleCapital chooses to use a public ranking scale with 9 categories (1-9) with "+" and "-" to denote scoresthat are at the higher or lower ends of the category.So how does the Human Capital Score™ work?The Human Capital Score™ combines credit-risk tools and metrics with academic achievementinformation to generate possible earnings and insight into each borrowers future creditworthiness.The model driven calculation is based on statistical data on a large number of students. We know theirmajors, schools, grades, scores, and a host of other attributes. We know how much these studentsearned in the years after they graduated. We can use this information to create projections of incomefor students based on each students specific academic attributes. In overly simplistic terms, if studentsin our data who study engineering and have good grades had high and growing incomes aftergraduation, the Human Capital Score™ will assign high and growing incomes to engineering studentswith good grades who ask for a Human Capital Score™.PRIVATE & CONFIDENTIAL Page 9
  10. 10. For additional information, please visit www.people2capital.com or e-mail alan.samuels@people2capital.comThe Human Capital Score™ also incorporates additional information on how much students with variousmajors earn, the attributes of the various schools, etc. This allows the model to make quality projectionsof the future potential incomes of students, even when we dont have data on many (or even any)students who went to that school or had that major. The score is not limited by the data provided bystudents who request a score.Of course, we cant get data on the incomes of students 10 years after graduation - except fromstudents who graduated at least 10 years ago. To ensure that the Human Capital Score™ reflects themost recent patterns in graduate incomes, we consider the most recent trends in the overall incomedistribution of college graduates and adjust based on current economic trends. Also, the current versionof the Human Capital Score™ is best calibrated for Bachelors degrees (but we are working on futureenhancements that will make the HCS applicable to other degree types).Because we have individual-level data on many students, we can project both average likely income andthe range of possible income paths. Students from a given major and school may all have relativelysimilar incomes; another major or school may have wide variation in graduates incomes. The HumanCapital Score™ will be able to provide a variety of statistics relevant for repayment, not just expectedincome. The model can also estimate the likelihood that income will fall below a certain value, or fall inthe worst 10 percent group. We can compute the probability that lifetime income will fall below a giventhreshold.Note that the current version of the Human Capital Score™ is best calibrated for Bachelors degrees; weare working on future enhancements that will make the HCS applicable to other degree types.What cant the Human Capital Score™ do (aka "the fine print")?While the Human Capital Score™ calculates potential future income, and thus broadly estimates theability to pay, it does not measure the willingness to pay. If someone with a high income is unwilling tomake loan payments, or someone with no income still makes loan payments, this isnt captured by themodel. Human Capital Score™ measures ability to pay, not propensity to pay.While the Human Capital Score™ ranks future income projections for college students in the 10 yearsafter graduation, we do not have a crystal ball. These projections are based on data about the incomesof people who have already graduated from college and are working now. If economic conditions shift inunexpected ways, we wont capture that. Most obviously, if the current recession reduces the incomesof college graduates in the coming years, our estimates of income will be systematically too high. Thatsaid, it will continue to show the relative ranking of college students. So it will continue to show whichstudents are relatively better options than others. Note that the current version of the Human CapitalScore™ is best calibrated for Bachelors degrees; we are working on future enhancements that will makethe HCS applicable to other degree types.The Human Capital Score™ can only project income using standardized attributes and ignores specificstudent interests or plans. An engineering major from MIT with high scores and grades will have a highHuman Capital Score™ because past engineers from MIT with high scores and grades have on averageenjoyed high incomes after graduation. If this particular student plans to join the circus (no disrespect tothis particular career path intended, just that it traditionally affords a lower income level) aftergraduation, the Human Capital Score™ cannot, and does not attempt to, reflect this. We are not able toPRIVATE & CONFIDENTIAL Page 10
  11. 11. For additional information, please visit www.people2capital.com or e-mail alan.samuels@people2capital.comreliably verify or validate information about a specific students work plans or expectations. We can onlyrely on information that we can verify.The Human Capital Score™ and any income projections and ranges are solely as statements of opinionand should not be construed as statements of fact. Human Capital Scores are not recommendations tobuy, sell or hold any security or to lend to any borrower. People Capital relies on information providedby borrowers and performs only limited verification of this information. The use of the Human CapitalScore™ should not be construed as an endorsement of the accuracy of any of the data or conclusions, oras an attempt to independently assess or vouch for the financial condition of any borrower.PRIVATE & CONFIDENTIAL Page 11

×