A wrong output is a mistake; a wrong action is an incident
A conversational assistant's output is text someone reads and decides what to do with. An assistant that can send a message, change a record, or call an external system removes that checkpoint — the output isn't advice anymore, it's an action already taken by the time anyone sees it.
That shift is the entire reason governance for assistants looks different from content moderation or output review. The question isn't just "was the answer good" — it's "was the assistant allowed to do that, and can we reconstruct why it did."
What actually needs to be in place
These aren't alternatives — a production assistant that takes real actions needs all four, because each one covers a failure the others don't.
Permission scoping
The assistant acts with the specific user's actual access, not a broad shared service account. Without this, the assistant becomes a way to reach data or actions its user was never entitled to on their own — the failure mode usually called the confused-deputy problem.
Human-in-the-loop approval
High-impact or irreversible actions pause for explicit sign-off before they execute. This is the layer that turns "the assistant did something wrong" into "the assistant proposed something wrong, and it was caught" — a materially different outcome.
Policy enforcement
An explicit allow-list of what the assistant may do, with everything else denied by default. Enforced at the point where the action would actually execute, not only suggested in the prompt — a prompt is guidance the model can misread; a policy check is a gate it can't talk its way past.
Audit logging
Every decision and action recorded with enough context — what was requested, what the assistant decided, what it actually did — to reconstruct the full chain after the fact. Without this, the first three layers are unverifiable claims rather than something you can actually check.
Classify before you write a single policy
The actual design decision is sorting every action the assistant could take into a tier. The policy language that follows is just enforcement of a decision that's already been made — so make the decision first, explicitly, action by action.
Frequently asked
Why do AI assistants need different governance than a chatbot?
A chatbot's output is text someone reads before deciding what to do with it. An assistant that takes actions — sending messages, changing records, calling external systems — turns a wrong output into an incident rather than a mistake someone catches. The controls have to match that shift from advice to action.
What are the main layers of AI assistant governance?
Four layers cover most of it: permission scoping, so the assistant acts with the calling user's actual access rather than a broad service account; human-in-the-loop approval for high-impact or irreversible actions; policy enforcement that allow-lists what the assistant may do and denies the rest by default; and audit logging detailed enough to reconstruct what happened and why after the fact.
What is permission scoping for an AI assistant?
Ensuring the assistant can only take actions the specific user it's acting for is actually authorized to take, rather than inheriting broad access from a shared service account. Without it, an assistant becomes a way to reach data or actions its user was never entitled to — sometimes called the confused-deputy problem.
How should a team start building governance for an AI assistant?
By classifying every action the assistant can take into a tier — allow it automatically, require a review before it runs, or block it outright — before writing any policy language. The classification is the actual design decision; the policy is just enforcing what the classification already decided.