Trust at Scale

The Argument

AI agents are doing real work. Not as demos. In production. Contracts are being reviewed, medical records triaged, incident reports drafted, invoices processed, code written, research synthesized. The shift from “AI as tool” to “AI as worker” is structural, and it is happening faster than the infrastructure can keep up.

A consulting engagement has agreements about scope, subcontractors, data access, payment, and acceptance. When work is challenged, those records help the parties reconstruct what happened. They are imperfect, but they give responsibility a structure. Agent workflows need an equivalent structure that survives delegation across services and organizations.

Now do the same thing with AI agents.

An orchestrator may call a document agent, which calls an extraction model, which queries another service. Each component may have its own logs and controls. The difficult questions span the whole chain: what was authorized, what did it cost, which evidence supports the result, and who can respond when it is wrong?

The agent economy is running without paperwork.

The gap is in how existing capabilities compose. Billing, access control, audit logs, and evaluation already exist. The proposal is to connect them through shared records and enforceable contracts, so a handoff does not lose the constraints or evidence that made the work trustworthy.

Historical standards such as accounting conventions and secure communication protocols show how common rules can support activity across institutional boundaries. They offer useful precedents, not a guarantee that one architecture will inevitably win or serve the public interest.

The agent economy has its revolution. It is waiting for its ledger.

The Four Voids

The accountability gap has four components. Solving one without the others leaves the system ungoverned.

Cost Attribution

A monthly invoice may aggregate costs across many clients and workflows. Per-call logs help, but organizations also need a way to combine them into an attributable execution history across providers. That history should distinguish measured costs, estimates, and commitments that have not yet settled.

The consequence is already visible: six-figure surprise bills, pricing done by guesswork, and no ability to distinguish between a $0.50 analysis and a $15.00 one until the monthly invoice arrives.

Data Provenance

Provenance records where data came from, what operations used it, and which sources are offered in support of a conclusion. It helps a reviewer investigate a claim. It does not establish that the source is true, that it supports the inference, or that every influence on the model has been captured.

The Mata v. Avianca case, in which an attorney’s brief contained six fictitious case citations generated by ChatGPT, demonstrated that AI-generated citations have no inherent relationship to reality. The text looks authoritative. The formatting is perfect. The content is invented.

Governance Boundaries

A delegation needs a purpose, bounded permissions, and evidence of the authority to grant them. Those restrictions must remain effective when another agent or organization joins the work. Creating a child agent must not expand the parent's authority, and required protections must not disappear when a peer lacks support for them.

Calibrated Confidence

This is the fourth void, and perhaps the most insidious.

When a human professional produces work, they communicate uncertainty naturally. “I’m fairly confident in this analysis, but the Southeast Asian data is thin; I’d want to verify those numbers before the board.” The consumer knows where to focus review.

When an AI system produces work, it communicates with uniform certainty. Every statement carries the same typographic weight, the same authoritative tone. A fact drawn from a source document looks identical to a hallucinated statistic. The output is a wall of equal-weight assertions, and the human reviewer has no signal about where to focus attention.

A model's stated confidence may differ substantially from its measured accuracy. The gap depends on the task and deployment. Confidence scores need independent evaluation before they can support decisions about which work to review. A system that is certain and wrong can be more dangerous than one that admits uncertainty: it steers attention away from the mistake.

Calibration asks whether comparable predictions with a score of 80% are correct about 80% of the time under the evaluation conditions. Ensembles and other scorers can help estimate this, but agreement alone is insufficient. Monitor changing inputs, shared blind spots, and the quality of human labels.

Confidence is one routing signal. The stakes, reversibility, and accumulated consequences of a workflow also matter. Review needs time, evidence, and authority to stop the action, including ways to challenge decisions after they have been made. When the reviewer has neither time to investigate nor authority to intervene, human-in-the-loop governance is theater.

What This Looks Like in Healthcare

Imagine a physician reviewing a patient's blood-pressure readings. One medication list is in the clinic's records; a note about a previous reaction is in another system. The AI assembles a summary and proposes a next step with “73% confidence.” The physician needs more than a confident sentence.

Seventy-three percent of what? Was the proposed step judged appropriate in comparable cases, or is the model merely likely to generate those words? Which records did it read? Did it find the note about the earlier reaction? A useful system makes those questions answerable. Even a well-calibrated score leaves the clinical decision with the physician and patient.

The institution also needs to control access, document decisions, and respond to corrections. Permission to process a record does not by itself make every use of it appropriate. Patients and clinicians need ways to question the result, and the organization needs someone responsible for resolving the problem. The technical controls support those responsibilities.

The Stack

The proposed architecture organizes seven responsibilities. The numbering makes the framework easier to discuss; it is not a strict build order. Identity, authorization, and evidence are needed throughout the system.

Layer 1 (Accountability) provides a shared record of actions, costs, sources, and applied rules. Context flows down and results flow up. The record needs access controls, retention limits, and a correction history. Traceability should enable investigation without becoming unlimited collection.

Layer 2 (Compute) sits above accountability because every compute operation generates costs and governance events. Dynamic allocation (request, allocate, release, report) replaces static provisioning and feeds directly into the cost tree. Worker self-registration means adding capacity is zero-configuration: start a node, point it at the broker, and it appears in the pool.

Layer 3 (Exchange) is the least mature and one of the most important. When multiple organizations collaborate on a workflow, someone needs to settle the bill. Today this happens through monthly invoices, manual estimates, and disputes that take weeks. The exchange layer automates reconciliation from the exact cost trees that Layer 1 already produces.

Layer 4 (Trust) records performance in context: the task, versions, evaluation method, uncertainty, and disputed outcomes. A million transactions do not establish authority if the outcomes were never checked or the deployment has changed. Reputation needs independent verification and a route to correction.

Layer 5 (Enforcement) checks whether an operation may proceed. Explicit permissions and required approvals can be enforced outside the acting agent. Judgments about meaning, harm, and evidence remain fallible. The system must state those limits, monitor its reviewers, and route consequential uncertainty to people able to act.

Layer 6 (Agency) solves identity. Today most agents authenticate with shared API keys. The agency layer gives every agent a unique cryptographic identity, explicit authorization chains (who authorized this agent to do this thing), capability-based permissions, and consent management at every organizational boundary.

Layer 7 (Marketplace) is the capstone. Discovery, trust evaluation, terms negotiation, execution, safety monitoring, settlement: every step depends on the layers below. Without the stack, a marketplace is just a directory. With the stack, it is the infrastructure that makes large-scale agent commerce possible.

The Human Shift

A police officer in Ontario finishes a call: a domestic disturbance, a break-and-enter, a collision with injuries. The event is over. The paperwork begins. Forty percent of the shift is administrative. An AI system transcribes the narrative, extracts parties, looks up offense codes, formats the report. The officer reviews, corrects, approves. Time: 10 minutes instead of 45.

The officer did not disappear. The work changed. The human moved from execution to governance.

This pattern can change parts of many professions. It can also reduce staffing, create new work, or shift authority away from employees. The division between execution and judgment will keep moving as AI capabilities change. The outcome depends on how institutions redesign work and share its gains.

But here is the part that “human in the loop” advocates rarely address: the human governor cannot review everything. The accountant cannot examine every one of the 10,000 invoices the system processed overnight. If the human must review everything, the AI saved no time.

Selective review needs more than a confidence threshold. It needs task-specific evaluation, sampling of apparently easy cases, attention to consequences, and a reviewer who can investigate. A queue that forces rapid approval can preserve the appearance of oversight while exhausting the person expected to provide it.

Interpreting calibrated uncertainty becomes a professional skill, as fundamental to the AI-augmented professional as reading financial statements is to the traditional accountant. The physician needs to understand what it means when the AI is 70% confident in a medication recommendation: what the number says statistically, and what it means clinically, for this patient, with this history.

Education must teach both practice and evaluation. Learners need to attempt problems, explain their reasoning, compare with automated answers, and receive feedback. Automating the apprenticeship work creates a responsibility to replace the learning opportunities it provided.

Governance can be meaningful work, but it is not the whole purpose of human life. Better service, affordable access, time for care and creativity, and the freedom to learn should count alongside productivity. Those gains require choices about training, ownership, and participation.

The Convergence

Seven independent forces are converging on the same requirement, and none of them are coordinating.

Regulatory and institutional. Legislation, procurement requirements, and voluntary frameworks create different obligations and expectations. They need evidence appropriate to their context; a protocol alone does not establish compliance.

Economic. CFOs discovering six-figure AI bills with no cost attribution. Organizations unable to price their AI-assisted services because they cannot calculate per-workflow costs. The 72% of AI projects that stall at pilot because the integration infrastructure is missing.

Safety. Models making consequential decisions (clinical, legal, financial) without calibrated confidence. Hallucinated citations in court filings. PII leaking across organizational boundaries.

Responsibility. When a workflow causes harm, the relevant people need evidence about authorization, actions, and outcomes. A record can support investigation, but institutions must still decide responsibility and provide remedies.

Environmental. Carbon reporting requirements that cannot be met when compute consumption is unmetered. Organizations required to report environmental impact with no way to attribute energy usage to specific AI operations.

Geopolitical. The CLOUD Act, data sovereignty requirements, sovereign compute strategies. Nations recognizing that AI infrastructure controlled by foreign corporations is a strategic vulnerability.

Enterprise. CISOs responsible for AI governance in organizations deploying agents at scale. AI maturity declining 20% year-over-year because organizations bought the tools without building the operational infrastructure.

These forces do not need to coordinate. They are all demanding the same thing: a way to account for what AI agents do. Cost attribution, data provenance, governance enforcement, and calibrated confidence are not features of a product. They are infrastructure, the same way TCP/IP, HTTPS, and double-entry bookkeeping are infrastructure.

These needs create a strong reason to build compatible infrastructure. The design should be tested through independent implementations, useful deployments, and the ability to change vendors without losing essential records.

That is the design problem of the decade.

The Architecture Question

I am betting on organizations owning more of the capability they depend on. Sovereignty is the ability to say no and keep working. That requires control over where essential work runs, who holds the keys, which updates are accepted, and how to leave a provider with usable records. Local compute makes that independence practical for suitable workloads. External services should be dependencies the organization deliberately chooses and can replace.

In January 2025, OpenAI and its partners announced Stargate, with a plan to invest $500 billion over four years in US AI infrastructure. That is an announced ambition, not a tally of completed spending. It makes the stakes visible: who will own the capability that everyone else depends on?

Distributed hardware does not automatically distribute power. A shared control plane, identity provider, marketplace, or reputation system can still become a gatekeeper. Sovereignty therefore includes usable exports, independent implementations, clear policy authority, and a tested ability to revoke access and recover from failures.

A compact summary, a small model, or a shared protocol can reduce the burden of doing work. Each must preserve the evidence and uncertainty needed to question the result. Efficiency is valuable when it increases people's options, rather than quietly removing them.

Affected people need a voice too. Someone denied a service should be able to understand the relevant reasons, correct information, and reach a reviewer with authority to change the outcome. The organization paying for the workflow is not always the only party whose interests matter.

This is the direction I want AceTeam to support: bounded delegation, useful records, fallible judgments made visible, and a practical ability to challenge, leave, and repair the system. We earn confidence by testing what happens when work crosses boundaries, permission changes, and something goes wrong. The final section of the book, The Future We Choose, connects it to a walk with my son and the human purposes I want it to serve. For a simple story about the same ideas, start with Who painted the room black?

The Complete Argument

This essay is the short version. The book traces the full arc: the structural shift, the accountability void, the historical pattern, the seven-layer stack, the transitions, and what comes next. Available with inline audio.