In March 2026, a major European insurance company’s AI underwriting system denied health coverage to a 47-year-old diabetic patient. The decision was made in 0.3 seconds. The reasons were opaque. When the patient appealed, no human at the company could explain exactly why. When journalists investigated, it emerged the model had been trained on data with socioeconomic proxies that correlated with race. No individual had made a discriminatory decision. The algorithm had. Who was responsible?
The Accountability Gap
AI systems are making decisions that affect human lives at scale and speed that existing frameworks were never designed to handle:
- Content moderation algorithms determining what billions of people see online
- Hiring algorithms screening resumes before any human reads them
- Predictive policing systems influencing where officers are deployed
- Credit scoring models determining who gets loans and at what rates
- Medical diagnostic AI flagging which patients warrant specialist attention
In each case, accountability when something goes wrong is genuinely contested. The developer? The company that deployed it? The human who chose to rely on it? Current legal frameworks were designed for human decision-makers, not algorithmic ones.
What Regulators Are Actually Doing
The EU’s AI Act, which began phased enforcement in 2024-2025, is the world’s most comprehensive AI governance framework. It classifies AI systems by risk level:
- Unacceptable risk (prohibited): Social scoring by governments, real-time remote biometric identification in public spaces, systems that manipulate behavior by exploiting vulnerabilities.
- High risk (strictly regulated): AI in critical infrastructure, education, employment, essential services, law enforcement. These require conformity assessments, human oversight, transparency obligations.
- Limited risk: Chatbots and AI-generated content must disclose their AI nature.
The US has taken a fragmented approach: executive orders, agency guidance, and voluntary commitments rather than comprehensive legislation. The contrast creates compliance challenges for global companies.
Explainability: The Technical Challenge
Regulatory requirements for AI explainability bump against a fundamental reality: the most capable models are the least interpretable. Transformer-based models with billions of parameters make decisions through processes that cannot be reduced to simple rule-based explanations.
Techniques like LIME, SHAP, and attention visualization provide partial explanations. Anthropic’s interpretability research aims to understand what concepts models internally represent. But rigorous, legally-defensible explanations for black-box model decisions remain elusive.
What Organizations Should Do Today
- Establish AI ethics review boards with real authority to delay or stop deployments
- Conduct bias audits on training data and model outputs before deployment
- Maintain human oversight for consequential decisions even when models could handle them autonomously
- Build audit trails that log AI decisions and the data they were based on
- Engage proactively with industry standards bodies and regulatory processes
The companies that treat AI governance as a core design principle will be better positioned — ethically and competitively — when the reckoning from high-profile AI failures comes.