Sensitivity analysis for model risk management
Sensitivity testing with dependence has the potential for a wide range of applications in reporting, such as for Solvency II, IFRS 17, and balance sheet valuation.
There has been much discussion among employers trying multiple approaches to entice their employees to get vaccinated against COVID-19. One approach that has been in the news recently is an employee contribution differential (a surcharge or an incentive) between vaccinated and unvaccinated employees such that unvaccinated employees pay more toward their health benefits than their vaccinated counterparts. For example, in late August, Delta Airlines announced that unvaccinated employees would have to pay an additional $200 each month toward their medical benefits compared to their vaccinated colleagues.1 An article in Forbes suggests the typical “surcharge” under consideration is lower, more like $20 to $50 per month.2
The monthly employee contribution incentive or surcharge amount is not necessarily intended to tie directly to actual expected differences in medical costs between the two groups; it may be intended partly to cover additional medical costs, partly to cover other employer costs such as lost productivity and other types of claims (e.g., life insurance, disability insurance, legal claims, and workers’ compensation3), and partly a simple incentive. In addition, employers may be limited in the amount of the incentive or surcharge; they have to meet legal requirements for any contribution differential to provide “affordable” coverage as defined by the Patient Protection and Affordable Care Act (ACA), and the wellness program differential cannot exceed 30% of total cost as defined under HIPAA (and amended by the ACA).4
On September 9, 2021, the Biden administration disseminated an executive order prescribing mandates that widely impact COVID-19 vaccination requirements for employees. The order further clarified the mandates that would cover employees of the federal government and entities doing business with the federal government. This is in addition to federal mandates that cover the Department of Defense, Department of Veterans Affairs, Indian Health Service, National Institutes of Health, and the healthcare facilities that receive federal Medicare or Medicaid funds.5 In addition, the executive order asks the Occupational Safety and Health Administration (OSHA) to issue an Emergency Temporary Standard (ETS) to require all employers with more than 100 workers to enact programs to ensure all employees are vaccinated or be subject to weekly COVID-19 testing.
However, as of this publication, no rule has been issued by OSHA, so many of the details of the employer mandate are unclear. Additionally, these executive orders are being met with considerable resistance, and are expected to meet a flurry of litigation and legislation.6 For example, on September 23, Senator Mike Lee introduced Senate Bill S.2843, which would prohibit penalties for violation of a COVID-19 vaccine mandate.7 As such, the future of these mandates is a question mark at this point. However, employers need not wait for a federal resolution and may still be considering measures to encourage employee vaccination, like a medical employee contribution credit/surcharge depending on vaccination status.
This concept of incentives and surcharges raises the question, “do unvaccinated employees incur higher healthcare costs, and if so, how much more?” Although actual healthcare claims data from patients with COVID-19 related hospitalizations is still emerging, it is possible to estimate the magnitude of these differences using prior published Milliman research, published COVID-19 surveillance data, and a few key assumptions. Using two separate sets of COVID-19 case and hospitalization rates, we developed two scenarios where the per member per month (PMPM) claim cost difference between vaccinated and unvaccinated employees were estimated to be $8.85 PMPM and $46.29 PMPM. Although these data points are not intended to represent the full range of potential values, these results illustrate the magnitude of potential health benefit cost impacts from non-vaccinated employees.
To perform our simple analysis, we used published probabilities of contracting COVID-19 in a month; then applied published probabilities of a having hospital stay for COVID-19 patients. These case rate and hospitalization probabilities were estimated separately for fully vaccinated and not fully vaccinated employees.
We then divided the hospital stays by type using published percentages of hospital care severity for COVID-19 patients: severe without critical care, critical care without a ventilator, and critical care with a ventilator. Finally, we assigned the average cost of each type of hospital case using published research.
To get the final costs per month, we multiplied the probabilities and costs to arrive at separate composite costs PMPM and calculated the difference between the costs of vaccinated and unvaccinated employees.
We developed two scenarios, A and B, using different assumptions for the infection and hospitalization rates.
Milliman previously published estimated mean hospital payments per admission by inpatient severity level for commercially insured patients using claims experience for COVID-19 inpatient treatment in 2020 for hospital admission dates from April 1 through July 31. The distribution of the claims data used for this analysis was unknown and may not represent a nationwide average.8 We applied a 5.0% annual trend to project to July 2021 allowed cost per case. These cost estimates are presented in Figure 1.
|Severity Level – Anticipated Treatment||Allowed Per Case July 2021|
|Inpatient Critical – No Ventilator||$60,445|
|Inpatient Critical – Ventilator||$126,410|
Although COVID-19 patients are likely to use a full spectrum of medical services, hospitalizations represent the majority of the medical costs related to COVID-19 treatment. Given the available data, we have limited this analysis to hospital costs, although this approach will understate the actual costs.
We used these same hospitalization costs for both scenarios.
Research disseminated by the Centers for Disease Control and Prevention (CDC) on September 10, 2021, quantified the difference in case (infection) rates, hospitalization rates, and death rates between vaccinated and unvaccinated populations for 13 U.S. jurisdictions for the period from April 4 through July 17, 2021.9
Meanwhile, the highly contagious Delta variant has caused an increase in infection rate, ramping up through August. More recent data published by New York City’s Department of Health (NYC DOH), including data accessed on September 21, 2021, suggest much higher infection and hospitalization rates, particularly among the unvaccinated, and an increased spread in infection and hospitalization rates between unvaccinated and vaccinated.10
We relied on the June 20 through July 17, 2021, national average weekly case and hospitalization rates from the CDC research to inform Scenario A, and average weekly case and hospitalization rates from the NYC DOH from August 1 through September 5, 2021, to inform Scenario B.
COVID-19 surveillance data from the CDC were used to allocate hospitalizations into the type of care received: ventilator (assumed to be in the intensive care unit [ICU]11), non-ventilator ICU, and other hospital stays.12 We blended the monthly percentages from February through June 2021 for ages 18 to 64 based on a nationwide average commercially insured working age population. We used these hospitalization type of care probabilities in both scenarios.
Note that we present two different point estimates based on publicly available data. These scenarios are not intended to capture the full range of possible outcomes.
Using the hospitalization type of care probabilities from the CDC combined with the hospitalization claims costs in Figure 1, we developed the tree diagrams in Figures 2 and 3 that show the various COVID-19 pathways for employees culminating in estimated per member per month (PMPM) costs for COVID-19-related hospitalizations for Scenarios A and B. Monthly infection rates are shown at the first level, and then conditional probabilities are shown at each subsequent level.*
Figure 2 presents the pathways using nationwide infection and hospitalization rates from the CDC (Scenario A), and Figure 3 presents the pathways using more recent (but more localized) infection and hospitalization rates from NYC DOH (Scenario B). For both scenarios, we used the same splits of hospitalizations into severity of care, and the same hospitalization average costs.
Figure 2 key assumptions and caveats: Monthly infection and hospitalization rates are based on a September 10, 2021, CDC study using data from June 20 to July 17, 2021. These rates were blended assuming a nationwide average commercial working age 18-64 population. ICU and ventilator rates were based on a CDC COVID-19 hospitalization study using data from February to June 2021. Hospital costs are from Milliman published estimated mean hospital payments per admission by inpatient severity level for commercially insured patients. The above scenario represents one possible estimate of cost differential between vaccinated and not fully vaccinated employees, and not a lowest possible estimate.
For example, the right branch of Figure 2 illustrates how an employer with 100,000 unvaccinated employees could expect 390 employees (0.39%), on average, to be infected with COVID-19 in a month (given the assumed monthly rates of infection in this scenario). Among those infected, 25 (6.48%) are expected to have a hospital stay. Of those with a hospital stay, 4.5 (17.96%) are expected to require ICU care. Of those in the ICU, 2.3 (51.56%) are expected to require a ventilator and 2.2 (48.44%) are expected not to require a ventilator.
Compositing the costs using the infection and hospitalization rates from the CDC shown in Figure 2, we have a total estimated difference in hospitalization costs (these severe cases comprise most of the medical cost differential) between vaccinated and unvaccinated employees of roughly $8.85 per member per month.
Figure 3 key assumptions and caveats: Monthly infection and hospitalization rates are based on NYC DOH published data for August 1 through September 5. ICU and ventilator rates were based on a CDC COVID-19 hospitalization study using data from February to June 2021. Hospital costs are from Milliman published estimated mean hospital payments per admission by inpatient severity level for commercially insured patients. The above scenario represents a one possible estimate of cost differential between vaccinated and not fully vaccinated employees, and not a highest possible estimate.
Compositing the costs using the monthly infection and hospitalization rates from NYC DOH shown in Figure 3, we have a total estimated difference in hospitalization costs (these severe cases comprise most of the medical cost differential) between vaccinated and unvaccinated employees of roughly $46.29 per member per month.
For context, the 2020 average premium for a single employee was $622.50 per month (of which covered workers, on average, contribute 17%), according to the Kaiser Family Foundation (KFF) 2020 Employer Health Benefits Survey.13 Given that, we estimated the differential in estimated COVID-19 hospitalization costs to be between 1.4% and 7.4% of nationwide average monthly premium.
As discussed above, this estimated hospitalization cost differential represents just a portion of an incentive or surcharge and employer may choose to implement.
There are many potential nuances in the implementation of an incentive or surcharge. Some compliance-related considerations have been discussed elsewhere, for example by the Society for Human Resource Management (SHRM)14 and JacksonLewis.15,16 In addition, the Equal Employment Opportunity Commission (EEOC) has provided some specific guidance on this topic.17
Other nuances stem from the newness of the disease itself, and the vaccines available to inoculate employees against it. It is not currently known whether having the disease once or getting vaccinated against it will impart lasting immunity. That leads to the question as to how long a potential incentive or surcharge should last, and how often does it need to be revisited and revised? If an employee is vaccinated in March 2021, should the financial advantage last for a year? Two years? Five years? Does it only continue if booster shots are taken? How does an employer verify vaccination status?
As the pandemic wears on, new treatments will be developed to treat COVID-19. For example, a new pill that is reported to reduce COVID-19 hospitalizations and death by half was announced recently by Merck.18 This is an example of one of many treatment options that may be used in the future to reduce risk of costly hospitalizations and progression of the disease, further eroding the medical cost differential between vaccinated and unvaccinated employees over time.
Meanwhile, there is some debate as to the value of natural immunity versus vaccinated immunity. A study by the CDC using data from Kentucky found 2.3 times better immunity against reinfection following vaccination compared to natural (post-disease) immunity.19 However, an Israeli study with a much larger sample size found disease-naïve vaccinated patients had a risk of subsequent infection 13 times greater compared to previously infected patients.20 Regardless, the immunity level of an unvaccinated group will increase over time as its members experience and (mostly) recover from the disease. That implies that the difference in case rates and hospitalizations between the vaccinated and unvaccinated groups will diminish over time, which would reduce the $8.85 to $46.29 estimated medical cost differential.
Employers, particularly medical care providers, have required vaccination against various illnesses as a condition for employment previously; this is not a new concept with COVID-19.21 However, these newer mandates have attracted lawsuits, many of which have been resolved in favor of the employer. For whatever reason, the vaccination mandate has become a hot-button issue. This may be, in part, because the COVID-19 vaccines were initially disseminated prior to full approval by the U.S. Food and Drug Administration (FDA). It also may be, in part, due to the touchy political climate in the United States at present.
Our analysis showed a modest estimated cost differences of $8.85 and $46.29 PMPM between vaccinated and unvaccinated employees in terms of the most impactful medical expense for COVID-19 (hospitalization). Employers have implemented (or considered) much larger cost differentials between the two groups as a medical benefit contribution incentive or surcharge, for example the $200 surcharge for unvaccinated Delta Airlines employees mentioned above. This could be due to not having an estimated medical cost differential, and/or wanting to put some “teeth” into the incentive or surcharge to influence employee behavior.
It is important to reiterate that our estimated difference in cost between the vaccinated and unvaccinated employees is focused solely on the cost from COVID-19-related hospitalizations. There could potentially be other cost differences between the two populations, including (but not limited to) other medical costs related to COVID-19 treatment, productivity differences due to missed work from COVID-19 illness, disability claims, life insurance claims, workers’ compensation claims, and litigation costs. This analysis focused on COVID-19-related hospitalizations as one of the more expensive health benefit events that would differentiate the two groups.
Other possible employer requirements, such as mandated social distancing or masking, may impact the cost estimates estimate above. It would make sense that the more effective an employer is at other methods for slowing spread, the lower the value of the vaccine incentive or surcharge and vice versa.
If the Biden administration’s executive order is ultimately implemented, employers may still choose to impose an incentive or surcharge on unvaccinated employees. Employers will still be on the hook for excess costs for unvaccinated employees, possibly including the required weekly COVID-19 testing going forward. If employers are required to verify and submit periodic data to OSHA as proof of vaccination or testing, it could create an additional administrative cost burden for employers as well.
And, while COVID-19 is typically described in the context of a current pandemic, the World Health Organization predicts it to become endemic; that is, it is here to stay.22
As mentioned above, the key sources for our analysis include:
The situation is very fluid regarding infection and hospitalization rates and treatments; these values are constantly changing. These sources will need to be updated for specific time periods and populations as the COVID-19 pandemic evolves.
Data available for this simple analysis are still emerging, and therefore the values used are highly likely to be revised over time. The distribution of the claims data analyzed by market and geographic region is unknown and may not represent a nationwide average. Hospital payments vary significantly by region, meaning payments to hospitals in some locations may differ significantly from the nationwide estimates presented in this white paper. Differences between the claims data summarized in this white paper and current or future nationwide payment amounts may result from evolution in disease treatment patterns, payer-specific payment structures, geographic variation in the incidence of COVID-19 admissions, new legislation, federal rulemaking, and other unforeseen events that affect the per-admission hospital payment for COVID-19 admissions.
We note the following limitations with respect to the claims costs used in this analysis:
We focused solely on the costs of COVID-19-related hospitalizations. Other COVID-19-related costs, such as costs for COVID-19 testing, office visits, emergency room visits, and other medical and nonmedical costs, are not reflected in these estimates.
Hospitalization percentages by type of hospitalization (ICU with ventilator, ICU total, and other) are the same for the vaccinated and unvaccinated populations in our analysis because we did not have these rates specifically for each of the two populations. It may be that vaccinated and unvaccinated hospital patients have different likelihoods of needing various levels of hospital care.
Our analysis is premised on assumptions regarding the spread of the disease, including assumptions as to how many people are hospitalized in a population, how hospitalization rates vary by age and gender, the severity of healthcare services utilized by those infected, and other variables. Scientific knowledge of these items is incomplete and new data on the spread of COVID-19 in the United States is still emerging. In addition, actions taken by governmental authorities and the healthcare system related to the COVID-19 pandemic are rapidly changing. Consequently, our results will evolve as new information becomes available and new actions are taken by the authorities and other stakeholders. Due to the limited information available on the pandemic, any analysis is subject to a substantially greater than usual level of uncertainty.
Guidelines issued by the American Academy of Actuaries require actuaries to include their professional qualifications in all actuarial communications. Jill Van Den Bos and Shelley Moss are members of the American Academy of Actuaries and meet the qualification standards for performing the analyses documented in this article.
A special thank you to our peer reviewers, Brent Jensen and Sue Taranto, and many other colleagues who provided valuable feedback.
*Note that the Mayo Clinic estimates that as of September 19, 2021, 54.9% of the U.S. population is fully vaccinated. For adults ages 18 to 64, the percentage of fully vaccinated people ranges by state from 39.1% in West Virginia to 75.0% in Massachusetts. See https://www.mayoclinic.org/coronavirus-covid-19/vaccine-tracker.
**CDC, Monitoring Incidence of COVID-19 Cases, Hospitalizations, and Deaths, by Vaccination Status – 13 U.S. Jurisdictions, April 4-July 17, 2021. See https://www.cdc.gov/mmwr/volumes/70/wr/mm7037e1.htm.
***NYC Health, Latest COVID-19 Data. See https://www1.nyc.gov/site/doh/covid/covid-19-data.page#daily.
1Singh, R.K. (August 25, 2021). Delta Air Lines to add $200 monthly health insurance charge for unvaccinated staff. Reuters. Retrieved October 3, 2021, from https://www.reuters.com/business/aerospace-defense/delta-add-200-monthly-health-insurance-charge-unvaccinated-staff-2021-08-25/.
2 Japsen, B. (August 8, 2021). Coming soon for the unvaccinated: A $50 monthly paycheck deduction from your employer. Forbes. Retrieved October 3, 2021, from https://www.forbes.com/sites/brucejapsen/2021/08/08/coming-soon-for-the-unvaccinated-a-50-monthly-paycheck-deduction-from-your-employer/?sh=4501554747ae.
3 Provines, A. & DiCenso, S.R. (September 16, 2021). Vaccine mandates and vaccine injuries: An unexpected cost to the workers’ compensation system. Milliman Insight. Retrieved October 3, 2021, from https://www.milliman.com/en/insight/vaccine-mandates-vaccine-injuries-an-unexpected-cost-workers-compensation.
4 Nathsonson, N.M. (September 20, 2021). Money Money Money: How Much Can the Health Plan Surcharge on Unvaccinated Employees Be? Benefits Law Advisor. Retrieved October 3, 2021, from https://www.benefitslawadvisor.com/2021/09/articles/vaccination-surcharge/money-money-money-how-much-can-the-health-plan-surcharge-on-unvaccinated-employees-be/.
5 National Law Review (September 10, 2021). President Biden Issues COVID-19 Vaccine Mandate for Certain Workers and Federal Contractors and Subcontractors. Retrieved October 3, 2021, from https://www.natlawreview.com/article/president-biden-issues-covid-19-vaccine-mandate-certain-workers-and-federal.
6Tucker, E. & Richer, A.D. (September 10, 2021). Biden’s vaccine rules to set off barrage of legal challenges. AP News. Retrieved October 3, 2021, from https://apnews.com/article/joe-biden-health-lawsuits-business-coronavirus-pandemic-0f668373b0a2f0f6af6e5b39e2ceaae9.
7 Congress.gov. All Information (except text) for S.2843 – No Taxation Without Congressional Consent Act. Retrieved October 3, 2021, from https://www.congress.gov/bill/117th-congress/senate-bill/2843/all-info.
8Kramer, M., Bazell, C., & Mraz, M. (May 2021). Hospital Payments for COVID-19 Inpatient Treatment. Milliman White Paper. Retrieved October 3, 2021, from https://us.milliman.com/-/media/milliman/pdfs/2021-articles/5-13-21-hospital_payments_for_covid-19.ashx.
9CDC (September 10, 2021). Monitoring Incidence of COVID-19 Cases, Hospitalizations, and Deaths, by Vaccination Status – 13 U.S. Jurisdictions, April 4-July 17, 2021. Morbidity and Mortality Weekly Report (MMWR). Retrieved October 3, 2021, from https://www.cdc.gov/mmwr/volumes/70/wr/mm7037e1.htm.
10NYC Health. Latest Data. COVID-19: Data. Retrieved October 3, 2021, from https://www1.nyc.gov/site/doh/covid/covid-19-data.page#daily.
11American Thoracic Society. Mechanical Ventilation. Retrieved October 3, 2021, from https://www.thoracic.org/patients/patient-resources/resources/mechanical-ventilation.pdf.
12COVID-NET. Laboratory-Confirmed COVID-19-Associated Hospitalizations. Retrieved October 3, 2021, from https://gis.cdc.gov/grasp/covidnet/COVID19_5.html.
13Kaiser Family Foundation (October 8, 2020). Figure 1.1: Average Annual Premiums for Covered Workers, Single and Family Coverage, by Plan Type, 2020. 2020 Employer Health Benefits Survey. Retrieved October 3, 2021, from https://www.kff.org/report-section/ehbs-2020-section-1-cost-of-health-insurance/#figure11.
14Lazzarotti, J.J. (August 16, 2021). Health Plan Premium Surcharges for the Unvaccinated? Employer Considerations.SHRM. Retrieved October 3, 2021, from https://www.shrm.org/resourcesandtools/hr-topics/benefits/pages/employers-ponder-health-plan-premium-surcharges-for-the-unvaccinated.aspx.
15Lazzarotti, J.J. (August 10, 2021). Health Plan Premium Surcharges for Those Not Vaccinated for COVID-19? Benefits Law Advisor. Retrieved October 3, 2021, from https://www.benefitslawadvisor.com/2021/08/articles/affordable-care-act/health-plan-premium-surcharges-for-those-not-vaccinated-for-covid-19/.
17EEOC (May 28, 2021). EEOC Issues Updated COVID-19 Technical Assistance. Press release. Retrieved October 3, 2021, from https://www.eeoc.gov/newsroom/eeoc-issues-updated-covid-19-technical-assistance.
18Merck (October 1, 2021). Merck and Ridgeback’s Investigational Oral Antiviral Molnupiravir Reduced the Risk of Hospitalization or Death by Approximately 50 Percent Compared to Placebo for Patients with Mild or Moderate COVID-19 in Positive Interim Analysis of Phase 3 Study. Retrieved October 5, 2021, from https://www.businesswire.com/news/home/20211001005189/en/
19CDC (August 13, 2021). Reduced Risk of Reinfection With SARS-CoV-2 After COVID-19 Vaccination – Kentucky, May-June 2021. Morbidity and Mortality Weekly Report (MMWR). Retrieved October 3, 2021, from https://www.cdc.gov/mmwr/volumes/70/wr/mm7032e1.htm?s_cid=mm7032e1_w.
20Gazit, S. et al. (August 25, 2021). Comparing SARS-CoV-2 natural immunity to vaccine-induced immunity: Reinfections versus breakthrough infections. MedRxiv. Retrieved October 3, 2021, from https://www.medrxiv.org/content/10.1101/2021.08.24.21262415v1.
21Rosenfeld, R. (August 11, 2021). Employer Vaccination Mandates: Are They Legal?. JD Supra. Retrieved October 3, 2021, from https://www.jdsupra.com/legalnews/employer-vaccination-mandates-are-they-2946823/.
22BBC News (May 14, 2020). Coronavirus may never go away, World Health Organization warns. Retrieved October 3, 2021, from https://www.bbc.com/news/world-52643682.