Explainable AI in Underwriting: Why Black-Box Models Won’t Survive Regulatory Scrutiny
There is a saying that has emerged from AI practitioners working with insurance regulators: ‘AI that can’t be explained is AI that can’t be adopted.’ It sounds simple. The implications are not.
In group health underwriting specifically, the pressure toward explainability is coming from three directions simultaneously — regulators, reinsurers, and underwriters themselves. Each group has different reasons for demanding it, but the operational requirement is the same: every pricing decision needs a traceable, auditable rationale that exists somewhere other than inside the model’s weights.
The Regulatory Pressure
The NAIC Model Bulletin on AI, issued in late 2023, established a framework principle that has since shaped state-level regulatory guidance across most major insurance markets: carriers are responsible for the decisions their AI systems make. The model is not a separate actor. If an AI-assisted underwriting decision produces outcomes that are unfair, discriminatory, or inconsistent with filed rates, the carrier bears the liability.
This has practical implications. In a traditional underwriting environment, an examiner reviewing a book of business can ask an underwriter to walk through the logic of a pricing decision. With AI-assisted underwriting, that walkthrough needs to be producible from the system itself — not reconstructed from memory or from a spreadsheet that was used to double-check the model’s output.
The EU AI Act, which classifies insurance AI as high-risk, takes this further by requiring documented governance, model validation records, and bias monitoring as conditions of deployment. While the EU Act doesn’t directly govern US carriers, its framework is influencing how domestic regulators are thinking about AI audit requirements. Carriers building explainability into their systems now are building for the regulatory environment of 2027, not just 2026.
The Reinsurer Pressure
Treaty reinsurers underwriting stop-loss books have always had the right to audit underwriting files. In practice, that audit focused on whether specific claims were correctly adjudicated and whether aggregate attachment points were appropriate for the groups rated.
As AI-assisted underwriting produces decisions at higher volume and faster speed, the audit question is shifting. Reinsurers are increasingly asking not just ‘was this group priced correctly?’ but ‘was the pricing logic consistent across this book?’ Inconsistency in rule application — groups with similar risk profiles priced materially differently because different underwriters applied different manual adjustments — is a governance flag that affects treaty terms.
An explainable, rules-based system produces a consistent audit trail. Every group rated at a given deductible level, every adjustment factor applied, every laser decision made — all attributable to the same underlying logic. That’s the evidence base a reinsurer needs to trust the book.
The Underwriter Pressure
The most underappreciated source of explainability pressure is the underwriters themselves. AI systems that produce outputs without explanation get ignored. An underwriter who receives a risk score of 0.73 with no context for what drove that score has no way to assess whether to trust it. In practice, they default to their own judgment and use the AI output as a secondary reference at best.
Explainable AI changes the interaction. When the system shows not just the score but the contributing factors — ‘GLP-1 loading: +8%, high-risk SIC adjustment: +5%, age band shift: +3%’ — the underwriter can engage with the model’s reasoning. They can confirm it where it aligns with their assessment, override it where their judgment differs, and document both. That’s a genuinely productive human-AI collaboration. A black box is not.
What Explainability Actually Requires
Explainability in underwriting AI is not about using simpler models. A sophisticated machine learning model can be made explainable through the right architecture and logging discipline. What it requires operationally:
-
- Every rating factor and its contribution to the final premium must be logged at the decision level, not just the model level
- Override decisions by underwriters must be logged alongside the AI recommendation and the reason for override
- Rule changes must be version-controlled — the same group re-rated six months later should produce either the same result or a traceable explanation of why the result changed
- Bias monitoring must be ongoing — the system should flag if pricing outcomes are producing statistically significant variation across demographic characteristics that should not affect rating
- Documentation should be producible on demand for any individual submission, not just summarised at the portfolio level
The Competitive Advantage of Building This Now
Carriers that build explainability into their AI systems now gain a regulatory and reinsurance advantage as the scrutiny intensifies. The carriers that will struggle are those that deploy capable but opaque models and then face the governance retrofit when regulators start asking questions.
The infrastructure investment is not large relative to the risk. It is primarily a data and workflow discipline investment — ensuring that logging, version control, and audit trail are built into the system from day one rather than added as an afterthought.
Every pricing decision in DataHub’s SmartRules Engine is logged, attributable, and auditable. Rule changes are version-controlled, and underwriter overrides are documented automatically. Contact the DataHub team to see how the audit trail works in practice.