Most banks and health systems already have governance frameworks for new technology — vendor risk, change management, model risk management. The mistake is treating customer-facing AI as a new category that needs an entirely new framework, instead of extending the ones already in place.

Governance failures with AI rarely look like a single dramatic incident. They look like a chatbot that quietly started giving inconsistent answers after a policy update nobody flagged to the team maintaining the model, or a model that was retrained without anyone re-validating it against fair lending requirements. The institutions that get burned aren't usually the ones deploying reckless technology — they're the ones who deployed reasonable technology without a reasonable process wrapped around it.

Who owns the AI, exactly?

A surprising number of AI deployments in regulated institutions don't have a clear, single owner. IT owns the infrastructure. A business unit owns the use case. Compliance reviews it once at launch. No one owns it on an ongoing basis — which means no one is accountable when the underlying policy changes, when the model's behavior drifts, or when a customer complaint surfaces a pattern worth investigating.

A workable governance model assigns three distinct roles, even if one person wears more than one hat at a smaller institution:

  • Business owner. Accountable for the use case working correctly — the actual policy or decision logic being represented accurately.
  • Technical owner. Accountable for the system running correctly — uptime, accuracy of the underlying rules engine, escalation paths functioning as designed.
  • Compliance owner. Accountable for the deployment staying within regulatory bounds on an ongoing basis, not just at initial launch.

Change control: the part most teams skip

Model risk management frameworks that banks already use for credit models apply just as well to AI concierge deployments, and the discipline transfers directly: no change to the underlying rules or model goes into production without documented review, testing, and sign-off from the compliance owner. This sounds bureaucratic until you consider the alternative — a support-team update to a fee-waiver policy that gets pushed straight to production without anyone checking whether it still satisfies fair lending documentation requirements.

If your policy changes and your AI's behavior doesn't change with it — on record, on a specific date — you have a governance gap, not a technology gap.

Practically, this means version-controlling the business rules that drive customer-facing decisions the same way engineering teams version-control code: every change has an author, a date, a reason, and a rollback path. When an examiner asks what the policy was on a specific date six months ago, the answer should be a lookup, not a reconstruction project.

Oversight that doesn't require a compliance officer to read model weights

One reason governance frameworks stall out around AI is that they're written as if the compliance team needs to understand the technology at a deep technical level to oversee it. That's the wrong bar. A compliance officer doesn't need to understand how a language model's attention mechanism works. They need to be able to answer three questions on demand:

  • What decision was made, and what rule or policy produced it?
  • Was this decision within the bounds the system is authorized to make on its own, or did it require human sign-off?
  • If the underlying policy has changed since, what was it at the time of the decision?

A system built around a deterministic core with a sealed audit trail answers all three without requiring a data science background. A system built as an undifferentiated black box does not, no matter how good its outputs look in a demo.

What belongs in an AI governance framework, specifically

"AI governance framework" gets used to describe everything from a two-page policy memo to a full enterprise risk program. For a bank or health system deploying customer-facing AI, a working framework needs five components, not a general statement of principles:

  • An inventory. Every AI system in production, what it's authorized to decide, and who owns it — updated when a new use case ships, not reconstructed annually for an audit.
  • A risk tiering method. Not every AI use case carries the same stakes; a framework that treats a routine FAQ bot the same as a claims-denial system wastes oversight capacity on the wrong things. (See our companion piece on AI risk management in practice for a working tiering approach.)
  • Change-control procedure. How a rule or model update gets proposed, tested, approved, and logged — the mechanism, not just the principle that changes should be reviewed.
  • An escalation and incident process. What happens when something goes wrong — an inconsistent answer, a customer complaint pattern, an examiner inquiry — and who's accountable for the response.
  • A review cadence. A fixed schedule for re-assessing each in-production use case against current regulation and current performance, not a one-time approval that's assumed to hold indefinitely.

Most institutions have pieces of this scattered across existing model risk management and vendor risk programs. The work is less about inventing something new and more about making sure AI-specific deployments are actually routed through the existing machinery, rather than treated as a special IT initiative that skips the usual controls because it "isn't really a model."

Responsible AI as a governance input, not a marketing term

"Responsible AI" is often used as a values statement — a paragraph about fairness and safety that sits in a company's public communications. For governance purposes, it needs to be more concrete: a responsible AI policy should specify, in writing, what fairness testing is performed before a use case ships, what data the system is and isn't permitted to use, and what the institution's position is on customer disclosure — does a customer get told they're interacting with AI, and does that disclosure change based on what the AI is authorized to decide. These are governance decisions with real operational consequences, not branding choices, and they belong in the same document as the ownership and change-control structure, not a separate aspirational statement.

Building the governance model, not just the policy document

A governance policy that lives in a document nobody re-reads after the launch review isn't governance — it's paperwork. The distinguishing feature of a working model is that it's operational: ownership is assigned to actual people who get notified when something needs review, changes are blocked from shipping without sign-off, and the audit trail updates itself rather than depending on someone remembering to log it.