SOA Study Cuts Underwriting Bias Gap to 0.071 for 0.001 of AUC

At a 2% referral rate, a GLM in the Society of Actuaries Research Institute's auto simulation flags one audit group at 1.654 times the rate of the other on identical simulated claim behavior. An adversarial neural network trained on the same records brings the ratio to 0.879. It also ranks claims slightly better, at an AUC of 0.6833 against the GLM's 0.6766, according to the report the Institute published on October 27.

Hong Li and Jiangang He wrote the 66-page study, Evaluating Bias Mitigation Strategies in AI Models for Insurance Pricing and Risk Assessment. They built five synthetic books (life accelerated underwriting, auto, credit, mortgage and health), each with one observed input measured with a group-correlated shift. A GLM, a standard neural network and an adversarial neural network were fit to each and audited at the top 1%, 2% and 5% of predicted risk.

Key Takeaways

The life GLM and the adversarial network sit 0.001 of AUC apart, while their 2% true-positive-rate gaps are 0.114 and 0.071, a 38% difference.
In the auto case the adversarial model wins on both axes, lifting AUC to 0.6833 from 0.6766 while cutting the demographic-parity ratio to 0.879 from 1.654.
No model sees the group label; distortion enters through a credit score shifted down for one group and recorded violations averaging 2.70 in high-monitoring territories against an unobserved 0.72.
Disparities peak at the strictest cutoff: the life parity ratios are charted between about 2.5 and 5 across the 1%, 2% and 5% action rates, highest at 1%.
The adversary targets observed territory, so mitigation needs no protected-class data, yet the report says mitigation "did not eliminate all disparities."

How the distortion enters the data

Each simulation keeps the group label out of prediction, the adversarial target, penalty selection and refitting, the report says. It appears only after the models are fixed, to compute two audit numbers on a held-out sample. The demographic-parity ratio divides one group's high-risk selection rate by the other's. The absolute true-positive-rate gap compares how often each group's truly adverse cases are caught.

In the life book of 100,000 policies, 34,805 sit in the disadvantaged group and 14,929 die within the horizon, an event rate of about 15% (SOA Research Institute, October 2026). Its insurer observes a credit-related underwriting score equal to the true, group-neutral score minus a fixed shift for that group, plus noise. A territory flag that is in the model is set for about 60% of the disadvantaged group and 20% of the other.

Auto separates driving behavior from its recording. Unobserved violations average about 0.72 in both territories, the report says. Recorded violations add a Poisson uplift with mean 2.00 in the high-monitoring territory, so the recorded mean there is about 2.70. That territory holds most of the disadvantaged group, and the claim rate is about 15%.

A GLM reads each recorded violation as the same claim signal wherever it was written, so it over-rates drivers in the uplifted territory and under-ranks claims overall. The adversarial model, penalised when territory can be recovered from its hidden layer, therefore gains AUC in this one case.

Health shifts an observed morbidity score up by 2.20 for the disadvantaged group. That group is 25% of the portfolio and carries a high area-deprivation flag 55% of the time, against 25% for the rest (SOA Research Institute, October 2026).

The frontier at the referral cutoff

Penalty strength is chosen without the audit label. Candidate penalties whose cross-validated AUC is within 0.005 of the best are retained, and among them the one with the largest adversary loss is selected, the report says. Selected penalties were 2 in life, 0.5 in auto and 0.3 in health. This table reads the frontier charts at the 2% operating point.

Line	GLM AUC	ADV-NN AUC	GLM TPR gap	ADV-NN TPR gap	GLM parity ratio	ADV-NN parity ratio
Life (accelerated UW)	0.8038	0.8028	0.114	0.071	charted only	charted only
Auto	0.6766	0.6833	0.0159	0.0065	1.654	0.879
Credit	about 0.782	about 0.780	about 0.064	about 0.062	charted only	charted only
Mortgage	0.8262	0.8264	0.0716	0.0446	2.628	1.790
Health	0.8325	0.8322	0.1144	0.0836	2.740	2.403

Source: SOA Research Institute, October 2026, Figures 8, 18, 19, 28, 38, 39, 47 and 48, held-out sample at the 2% operating point. Life and credit parity ratios are plotted, not tabulated.

A validation memo reporting AUC would miss the life row: 0.8038 against 0.8028. At the 2% cutoff, the share of applicants who go on to die and are routed to full underwriting differs between the groups by 11.4 points under the GLM and 7.1 under the adversarial model.

In the life case the parity ratios are charted on a scale from 2.5 to 5, with the GLM highest at 1% and all three models falling as the action rate loosens to 5%. The true-positive-rate gaps move the other way, rising as it loosens. An audit run at a 5% action rate when the program refers 1% reports a different number from the one the decision produces.

Standard neural networks landed on no consistent side. In auto the plain network's parity ratio is 0.773, across one from the GLM's 1.654, so it under-flags the group the GLM over-flags. Credit shows the cost side: the adversarial model trims the gap to about 0.062 from 0.064 while AUC falls to about 0.780 from 0.782; "stronger adversarial penalties can reduce predictive performance," the executive summary says.

How the regulators' tests read the same model

No US regulator has yet written the report's metric, a ratio of selection rates at the action rate, into a rule. Colorado's September 2023 draft for life insurers would have required Bayesian Improved First Name Surname Geocoding to infer race, then a logistic regression of approval rates on policy type, face amount, age, gender and tobacco. Further testing followed only where the difference was significant at a p-value below .05 and reached five percentage points.

That draft was never adopted. Bulletin B-10.004, issued October 18, 2024 and revised October 22, 2025, waives the testing description from life insurers' annual reports because "the Division has not yet adopted rules establishing the required quantitative testing," the Division of Insurance said. Colorado extended the governance rule itself to private passenger auto and health benefit plan insurers from October 15, 2025, with first compliance reports due July 1, 2026, covered in the site's July piece.

On the report's group shares, a parity ratio of 1.654 at a 2% action rate works out to selection rates of about 2.7% and 1.6%, a gap of roughly one percentage point. Colorado's draft threshold was five points, which at a 2% action rate is only reachable if nearly every flagged applicant comes from the smaller group. At the referral rates the SOA report models, only a ratio test registers the gap.

New York's Insurance Circular Letter No. 7, issued July 11, 2024, asks insurers to assess whether external data or an AI system "produces disproportionate adverse effects" and to search for a less discriminatory alternative. That analysis runs "prior to putting AIS into production and on a regular cadence." It adds that insurers "are not expected to collect additional data from or about individuals" for the purpose. Adopted December 4, 2023, the NAIC model bulletin goes no further than encouraging "verification and testing methods to identify errors and bias."

The SOA design meets that constraint on one side only. Its adversary trains against observed territory, a field every carrier already holds, so mitigation runs without a protected-class label. The two audit numbers need the label, which the simulation has by construction and a carrier would have to infer, the step Colorado's draft mandated and New York leaves to the insurer.

Health reverses the reading. There, the authors say, a high-risk flag "may represent a triage or intervention decision rather than an unfavorable underwriting action." A GLM ratio of 2.740 means the disadvantaged group is enrolled in care management at 2.74 times the other group's rate on the same underlying morbidity. Mitigation lowers that to 2.403, which here means less outreach. A true-positive-rate gap of 0.1144, falling to 0.0836, reads the same way in both settings, because it measures how many truly high-morbidity members each group's flag catches.

SOA Study Cuts Underwriting Bias Gap to 0.071 for 0.001 of AUC

Key Takeaways

How the distortion enters the data

The frontier at the referral cutoff

How the regulators' tests read the same model

Further Reading

Sources