Classic FICO¹ has been the credit score used by the mortgage industry to underwrite mortgages and model mortgage cash flows for decades. While enhanced credit scores have been developed by the industry, classic FICO is still relied upon to assess credit risk in mortgage lending. To accelerate the adoption of improved credit scoring, the Federal Housing Finance Agency (FHFA) disclosed plans to transition away from classic FICO and approve the use of FICO 10T and VantageScore 4.0 (Vantage) models.

On July 11, 2024, FHFA announced the publication of Vantage historical credit scores. This data disclosure allows market participants to analyze and compare this credit score model against the traditionally used classic FICO score.

This article analyzes the recently published Vantage data and provides comparisons against classic FICO. The aim of the analysis is to evaluate the credit score models, and to evaluate the potential impact that the transition could have on mortgage modeling.

A similar analysis will be performed on FICO 10T once the data is publicly available.

Data

The analysis presented in this article uses loan-level credit score data made publicly available by Fannie Mae and Freddie Mac (the government-sponsored enterprises, or GSEs). The data includes:

Interact with the data

Click here to explore the Vantage 4.0 and Classic FICO data on a more granular level.

• Sample size: 4.07 million loans.
• Origination years covered: 2011-2023.
• Sample contains both classic FICO and Vantage score.
• Loans that were missing one of the two credit scores were omitted from the analysis.

Given that the data made available by FHFA is a sample of the population of loans and may not be representative of the full dataset across all periods, some of the results in the following sections might diverge from trends we would see if the full population of loans were available and analyzed.

Figure 1 provides the number of loans and average credit score by origination year for the loans with a Vantage score.

Vantage: Range and summary statistics

Similar to classic FICO, the Vantage credit score ranges from 300 to 850. Summary metrics indicate that Vantage credit scores produced, on average, a higher average score when compared to classic FICO for the same population. Further, the Vantage score data has a higher standard deviation relative to classic FICO. This means that distribution of credit scores is wider for Vantage relative to classic FICO for the scored population. Based on the sample selected, loans had Vantage scores between 361 and 850, while the same loans had classic FICO scores between 430 and 848.

Without any kind of stratification applied to the loans, the average difference between Vantage and classic FICO is 1.44% (Vantage being higher). Figure 2 lists summary statistics calculated by Milliman for both credit score models by credit score range. From Figure 2, one thing to note is that most of the difference in dispersion between scores is caused by loans in the lowest (less than 620) and highest (greater than 780) categories. Classic FICO rarely produces credit scores more than 800; the Vantage score results have more observations above 800 relative to classic FICO.

This gap between the average classic FICO score and the Vantage score is generally consistent across time, with loans originated in 2023 having a small difference between average scores. Figure 3 shows the difference between average credit scores for each score by vintage.

Credit score distribution and correlation

The above data looks at average credit score data. With credit scores, it is important to evaluate both the average and distribution of scores across the range of credit scores. The classic FICO distribution can be described as a triangular distribution skewed to the left, with a peak point around the 790-800 range. This unimodal distribution differs from the bimodal distribution under the same loan population with the Vantage model, with two “peaks” or “local maxima” areas (the 700-730 and 790-800 ranges). There are also a greater number of loans with a score higher than 800 under the Vantage model when compared to the classic FICO model. Figure 4 plots both distributions.

The classic FICO distribution has very few observations with a credit score less than 620. This is because the data is censored to originations that were purchased by the GSEs. The Vantage data includes loans with a score below 620. These loans likely have credit attributes in the borrower’s credit report indicating they would be a greater risk. Therefore, it is possible these loans would not have been originated if Vantage were used for the underwriting decision. Classic FICO tends to have limited observations with a credit score above 800 while Vantage produces a greater number of scores above 800. This observation is consistent with Milliman’s analysis of FICO 10T using data provided to Milliman on behalf of FICO. ²

The correlation coefficient between scores is 0.73. While positive, there are several mortgages that have large differences in credit scores. The differences are both positive (meaning the borrower may have a higher credit score under Vantage) or negative (meaning the borrower may have a lower credit score under Vantage). Milliman analyzed the differences by additional risk attributes. When looking at the data by loan-to-value (LTV) ratio, the average difference between scores declines for loans with higher loan-to-value ratios, moving from an average difference in scores of 2.60% (less than 60 LTV) to -0.82% (greater than 95 LTV). Figure 5 shows the score relationship in scatterplot form, while Figure 6 breaks down the average score differences by debt-to-income (DTI) and LTV ratios.

Default rates by credit score model

This section compares the observed default rates between scores by credit score range. The apparent “score inflation” under Vantage might be expected to translate into higher default rates in high-score ranges, but the analysis in this section suggests that this phenomenon is not prevalent in most credit score ranges.

To calculate the default rates by delinquency status and credit score range, the same loan sample was used. For each delinquency status category (D30, D60, D90, D120, D150), the default rates represent an “ever delinquent” rate. For example, the D90 delinquency rate is calculated as the number of loans that were ever 90 days delinquent divided by the total number of loans. Most buckets in this section contained 15,000+ loans; however, there are some cases where the data contained a smaller number of loans. Note that Milliman analyzed the full history of data through March 2020. The first section provides default rates for the full history of data, and the second section censors the data through March 2020. In 2020, many borrowers went delinquent and received special COVID-19 forbearance; therefore, default rates are elevated in the data after calendar year 2020.

Across all levels of delinquency, the observed delinquency rates in the data have a similar shape between classic FICO and Vantage. Default rates under the Vantage model are, on average, slightly higher across delinquency statuses for high credit score ranges. Loans assigned a credit score of less than 620 under the Vantage score model experienced a slightly lower default rate for every delinquency status analyzed. The data for classic FICO is limited for loans with a credit score of less than 620.

Figure 7 illustrates the 90+-day delinquency rate results for both credit score models. Overall, the graph curve shapes remain constant. Additional delinquency levels were evaluated by Milliman, and the trends were generally consistent with the 90+-day delinquency rates. Only the 90+-day delinquency rates are provided in this publication.

From Figure 7, both models show strong predictive capabilities to rank-order credit risk. That is, there is a consistent relationship of lower default rates for higher credit scores for both scoring models. The exception is loans with a classic FICO less than 620, where the number of observations is limited.

For high credit score ranges, classic FICO has a lower default rate when compared to Vantage. Figure 8 provides the ratio of the default rate from the Vantage models divided by the default rate under classic FICO for the same credit score range. For credit score ranges between 640 and 740, the default rate between models is generally consistent. For credit score ranges above 740, the default rate under Vantage is approximately 41% greater than the default rate under classic FICO.

With the data available, it is difficult to understand the cause for this observed relationship as the loan population and evaluation time is the same for each score. For modeling, this means it is likely that an adjustment is needed in the models to accurately ingest Vantage into existing models to produce unbiased model estimates. Milliman is evaluating the data further to determine whether this relationship is consistent across economic cycles and to develop a methodology to calibrate Milliman’s mortgage models to accept either classic FICO or Vantage.

Delinquency rates before COVID-19

To remove the effect of COVID-19 on the default rates calculated in the previous section and generate a “pre-COVID-19” loan sample, the data was filtered to exclude loans that originated before 2020. Additionally, any performance data after March 2020 was removed from the data. This way, the default rates analyzed excluded the impact of COVID-19-related delinquencies resulting from COVID-19-related forbearance. Figure 9 shows the 90+-day delinquency rates by credit score range for both classic FICO and Vantage using the pre-COVID-19 data sample. Note that the default rates in Figure 9 are materially lower than the default rates in Figure 7 above, demonstrating the significant impact of COVID-19-related delinquencies in the data.

The observed delinquency rates pre-pandemic retain a shape that is similar to the analysis performed in the previous section. For credit scores above 700, the default rates using the Vantage score are higher for the same range relative to the classic FICO default rate. For credit scores less than 700, the default rates using the Vantage score are lower for the same range relative to the classic FICO range. With the data provided, it is difficult to understand what is the source for this observation and why the relationship varies with and without COVID-19-related delinquencies.

Figure 10 provides a relativity of default rates between each scoring model for the same credit score ranges. For credit scores above 740, the default rates are about 17% higher using Vantage relative to classic FICO. These results further support the necessity to calibrate mortgage performance models to include the Vantage score in the models.

Default rate by score combination

Given the relatively low correlation between classic FICO and Vantage for the same loans and scoring periods, Milliman wanted to evaluate whether there is additional information in classic FICO or Vantage that can be used to estimate default rates. In other words, if a borrower is scored a classic FICO of 650 and a Vantage score of 750, does that borrower perform differently when compared to a borrower with a classic FICO of 650 and a Vantage score of 650? From the above default rate analysis, each score successfully produced declining default rates as credit scores increased. Therefore, we want to evaluate how borrowers perform given different indications of risk from each score. Figure 11 breaks down observed D90+ default rates using the pre-COVID-19 dataset.

From Figure 11, we see there is information gained from both classic FICO and Vantage. That is, for the same classic FICO range, observed default rates decrease with higher Vantage scores and, for the same Vantage score range, observed default rates decrease with higher classic FICO scores. This indicates that both classic FICO and Vantage have different assessments of borrower creditworthiness at origination. Vantage uses more data and trended data relative to classic FICO. This is because classic FICO was developed prior to 2000, when this data was not available. Nevertheless, there are variables and data elements considered by classic FICO that are not included in the Vantage models or at least that provide additional predictive information that is not provided by the Vantage score.

For a modeling perspective, this indicates that models that ingest both classic FICO and Vantage would outperform models that only include one score. However, it is likely that mortgage underwriting will transition to FICO 10T and Vantage and not use classic FICO; therefore, it is not clear at this time whether mortgage data will include both scores or if there is value in building such a model.

Key takeaways

• Vantage is an updated credit score that produces a wider distribution of credit scores relative to classic FICO.
• Vantage leverages additional data that was not available when classic FICO was created and therefore scores borrowers differently because of this data.
• Vantage and classic FICO are correlated but there is sufficient divergence between scores to demonstrate they use different signals.
• Default rates for Vantage and classic FICO are both strong models to ordinally rank borrowers from higher credit risk to lower credit risk.
• The default rates are generally consistent between scores, but caution should be used when using Vantage directly in existing mortgage models. The credit scoring models are independent from each other, and a calibration process is needed.
• There is information in using both credit scores in evaluating mortgage default risk.

¹ Classic FICO refers to FICO score versions 2, 4 and 5.

² FICO Blog (August 29, 2023). Executive Summary: Milliman FICO Score 10 and FICO Score 10T Model Assessment, June 2023. Retrieved September 2, 2024, from https://www.fico.com/blogs/executive-summary-milliman-ficor-score-10-and-ficor-score-10-t-model-assessment-june-2023.