What It Takes to Make AI Governable: Six Small Projects

Rolling AI out inside a company is easy to start. You buy seats, hand out an API key, and people get on with it. The hard part arrives a few months later, as a list of questions nobody can answer.

Who can use this, and did the person who left last month lose access? Did confidential data go to a provider that keeps it for thirty days? Who spent four thousand dollars, and who agreed to it? What can the assistant reach in our other systems, and with whose credentials? We changed the model on Tuesday, does everything that depended on it still work? Can you show me, for March?

None of those can be answered from inside one application. They need a platform that every request passes through, owned by a team whose job it is. I wanted to understand that platform properly, so I built it, as six small projects that fit together. Five of them are public so far.

This article is a tour of what each piece does and the one thing each taught me. I will also be clear about what is real and what is still mocked, because that is the first thing I would ask if I were reading this.

How the pieces fit

flowchart LR
    T["Entitlements as code<br/>(Terraform, identity provider)"] -->|"groups, tokens"| G
    U["People and services"] --> G["AI gateway<br/>identity → policy → spend<br/>→ routing → audit"]
    G --> M["Models"]
    G --> X["Tools"]
    S["MCP server<br/>(acts as the user)"] -.->|"may this user<br/>use this tool?"| G
    E["Eval gate<br/>(GitHub Action)"] -.->|"before a model change"| G
    G -->|"decision log"| I["System inventory<br/>and evidence"]
    V["Vendor bills,<br/>seat roster"] --> F["Spend and seat FinOps"]

The gateway is the centre. The others either feed it, ask it, check it, or report on it.

1. The gateway: one path for every request

ai-gateway started life as a routing proxy: one endpoint in front of several model providers, with fallback, rate limits and cost tracking. The second version put a governance layer in front of that. Every request, from a person or a service, to a model or a tool, now passes five stages in order: identity, policy, spend, routing, audit.

People sign in through the company's identity provider. A reviewed YAML file maps their groups to roles and tiers and says which model may hold which class of data. Budgets have a soft cap that warns and a hard cap that refuses. Every decision, allowed or denied, is recorded with who, what, why and the version of the policy in force.

What it taught me: a token is a snapshot. The obvious design is to put a user's groups in their login token and read them from there. Then someone removed from a group keeps their access until the token expires. The fix is to use the token only to prove who is calling, and read groups from a directory the identity provider keeps current over SCIM, on every request. A leaver loses access on their next call, with a token that is still valid.

The second lesson: governance cannot be bolted on from outside. "Confidential data never reaches a model that retains it" sounds like a check at the front door. But a fallback chain exists to send a request somewhere else when the first choice fails. Check only the first model and the first outage breaks the promise. So the chain itself is filtered to the eligible models before routing starts, and if the only eligible model is down, the request fails rather than leaks.

2. An MCP server that acts as the user

The gateway's own tools act as an app. That is fine for reading pull requests and wrong for Jira, where what you may see depends on who you are. mcp-atlassian-governed is an MCP server for Jira and Confluence where every call runs with the signed-in person's own permissions.

It has three levels of control. Read tools return only what the user could open themselves. Write tools do not exist unless an operator has named them in an allow-list, and need an extra scope. The one destructive tool, deleting a comment, asks the person to confirm through their client, and shows them the real comment fetched from Jira rather than the model's description of it.

What it taught me: you may not pass a token along. My first instinct for "act as the user" was to forward the user's token to Atlassian. The MCP authorisation spec forbids that, and it is right to: a token issued for one service should be useless at another. So there are two grants per person. One says who is calling the server. The other is that person's own consent at Atlassian, stored encrypted and refreshed as needed.

And: someone has to prove who is at the browser. Linking an account means opening a URL. If the server trusts whoever opens it, an attacker can send their own link to a victim and end up with the victim's Jira attached to the attacker's identity. So the browser signs in first, and the flow stops unless it comes back as the same user the link was made for.

3. Spend and seats: what the bill says

A gateway only sees traffic that goes through it. Seat subscriptions and anything calling a vendor directly never touch it. The vendor's bill is the only complete picture. ai-spend-finops pulls costs from vendor admin APIs and the seat roster from the identity provider into a local store, and produces four reports: reconciliation against what was committed to each vendor, spend against each team's cap, anomalies, and a seat true-up that lists paid seats nobody uses and what reclaiming them would save.

What it taught me: make the reports pure functions. Sync copies the data into a local file. The reports read only that file and the config, with no network and no clock inside them. That makes them trivial to test, and it means last month can be re-run and give the same answer. It also made the anomaly check easy to get right: it compares a team's day with the median of its own last two weeks, so one earlier spike does not raise the bar for the next one.

4. Entitlements as code

Who gets which AI tool is decided in the identity provider's admin console, by clicking. Nobody reviews a click. ai-entitlements-terraform turns one small data file into groups, group rules, app assignments and token policies, so an access change is a pull request with the plan attached.

Team groups are filled by rules on profile attributes, so joiners, movers and leavers follow HR data with no ticket. The exception is the group that approves spending. That one is a list of at least two named people and can never be filled by a rule, because a change to somebody's department must not be able to make them an approver.

What it taught me: plan and apply are different levels of trust. The pipeline holds no static cloud key. GitHub Actions gets a short-lived identity token and trades it for a role. There are two roles: any pull request can assume the read-only one to plan, and only the protected environment can assume the one that applies. Each trusts one exact subject. A wildcard there would trust every repository in the organisation.

5. An eval gate for model changes

Changing a model is one line in a config file, and every feature moves at once. model-eval-action is a GitHub Action that runs a handful of golden tasks on the current model and the candidate, checks the real answers and tool calls, and comments on the pull request with quality, tool use, cost per task and latency.

What it taught me: compare, do not score. The gate has no pass mark. A candidate fails when it loses a task the current model passes. A task both fail is information, not a blocker. That removes the endless argument about thresholds. When I ran the original harness between two local models it caught something I would never have spotted by reading: the smaller model called the case lookup tool to add 17 and 25.

6. An inventory that is checked against reality

The last piece is the one auditors ask for first: a list of AI systems with owners, purposes, data, vendors and evidence. It is usually a spreadsheet that was true once. Mine is a data file that is checked against the gateway's policy, so a system that claims confidential data but whose model may not hold any is a failing check with a name. Evidence is pulled from the gateway's decision log and written as a pack with a hash for every file, so an edited pack no longer matches its manifest.

What it taught me: an audit log is only useful if it never holds content. Every record in these projects says who, what, when, allowed or refused and why. None holds a prompt, an answer or a tool argument. That one rule is what lets the logs be kept, queried and handed over without a redaction step.

I have not published this last project yet.

What is real, and what is not

I would rather say this plainly than have someone find out.

Every repository has a "what's real vs. mocked" table in its README that says the same in more detail. Mocks check logic and shape. They do not check that a vendor's API behaves the way its documentation says, and the first live run of each of these will find something.

What I would tell someone starting

Start with the path, not the features. Decide that every request goes through one place, and what that place records. Most of the rest follows.

Put decisions in reviewed files. The access policy, the caps, the entitlements, the inventory and the golden tasks are all data files changed by pull request and validated in CI. That is the cheapest review process there is.

Fail closed, and say so. No rule means no data. No approver group means nobody approves. The gateway not answering means the tool does not run. Each of those is a line in a README, with the trade-off next to it.

Write down what is mocked. It is the most useful section to write early, because it forces a claim about every part, and the list of gaps falls out of it.

The code, the decision records and the runbooks are all in the repositories linked above.