Effect contract
One contract per server, written from tool descriptions, MCP annotations and the README — before the first probe, with predictions fixed before the run.
Research · AI safety · Agent control
Verifiable Agent Control
Effect contracts for tools nobody has audited.
Autonomous agents act through tools whose descriptions can omit what they do to files, processes, the network and their own state. VAC builds verifiable effect contracts that connect three things: what a tool declares, what isolated execution reveals, and what a controller may authorize before the call. And VAC measures what those decisions cost — in held legitimate calls.
Agent · MCP client
tools/call
VAC guard · MCP proxy
- 1Attest the revision: installation, contract,
tools/list - 2Predicate S on the state before the call
- 3Decision into the hash-chained ledger
allowed
MCP-Serverunmodified · sandbox · netns
held
Reviewnever reaches the server
13real MCP servers, instrumented in a private network namespace
1,683graded calls over three seeds; 12 of 13 servers reproduce call for call on a second machine
17 / 17index rewrites by read-only tools flagged before execution, without a false alarm
0index rewrites by read-only calls behind the enforcing MCP proxy, over 480 calls
01 The question
Control is only as good as its knowledge of the tools.
Agents act through tools, and tools are added faster than anyone reads their code. The Model Context Protocol does not require a tool's description to match its implementation, and it declares its own effect annotations (readOnlyHint, destructiveHint) untrusted.
Permission systems, information-flow boundaries and AI-control protocols nevertheless derive what an agent can do from exactly these declarations: a tool annotated read-only is auto-approved, a tool declaring no network is not a sink. A tool that also writes outside the watched scope, spawns a process or contacts an unknown host opens an action channel the controller does not model.
VAC makes this gap measurable and decides it before the call.
What does a controller have to know about a tool's effects to decide a call, how much of that can it obtain from documentation and from isolated execution for a tool it did not write, at what cost in refused legitimate calls, and where does the knowledge run out?The research question of VAC
02 Method
Declared, observed, decided.
One evidence chain from a tool's documentation to the decision a controller takes before the call.
Isolated execution
The unmodified server runs in a fresh sandbox inside a private network namespace whose only exit is the harness. It records file-system snapshots, every fork and exec through the kernel's process connector, sockets, and every network attempt by name.
Controller & proxy
A grader classifies every observed change against the contract. Controllers decide before the call; the guard enforces the decision as an MCP proxy in front of the unmodified server and writes it to a hash-chained ledger.
Grader classes
- fs_outside_scope
- fs_change_by_readonly_tool
- fs_undeclared
- undeclared_subprocess
- undeclared_network
- declared_effect_absent
03 Findings
Where declaration and behaviour part ways.
13 servers, chosen by a rule fixed before the first probe: every official reference server that runs locally over stdio, four variants of the MSB benchmark server and two widely installed third-party servers with rich local state.
Desktop Commander
Third party · widely installedAn undeclared subprocess on every one of 267 calls (python3 --version). DNS lookups of a telemetry fallback host that the README does not cover, on 81 calls.
git
Official reference serverTools annotated readOnlyHint rewrite .git/index: after a git_reset, git read-tree leaves entries without stat data, and the next git_status or git_diff_unstaged refreshes them.
obsidian-mcp
Third partyJournal, lock and trash files on every mutation that only the README declares; move_note rewrites the notes that link to the moved one.
MSB attack variants
Benchmark · ICLR 2026Two variants deviate on every call and are caught at the first probe; the third lives only in the served tool definition, as an extra parameter.
from probe 1filesystem, sqlite, memory, everything, fetch, time and MSB normal behave exactly as declared.
04 Decision before execution
A state-aware controller, frozen and measured on new traffic.
git's index rewrites depend on state, not on arguments. The state predicate S reads .git/index before the call and flags entries without stat data. It was pre-registered, tested in the controller run — 12 of 12 rewrites over 85 calls — and then applied frozen to ten seeds that no earlier run used.
| Controller | Rewrites flagged | Additional reviews |
|---|---|---|
S — index state read before the callreads .git/index, flags entries without stat data |
||
P2 — history since the last git_resetpre-registered history predicate |
||
| Depth-2 decision treeselected on controller seeds 7–8 | ||
| Per-tool baselinefitted on the same seeds |
Source: real-mcp/analysis/GUARDED_RUN.md · regenerated from the receipts
→ Enforcement
The guard as an MCP proxy in front of the unmodified server.
runtime/mcp_proxy.py attests the installation, the contract and the served tools/list against pinned digests, evaluates S on the index before every call, fsyncs the decision to a hash-chained ledger and forwards only allowed calls. The kernel's process events and the sandbox snapshots show no activity for any held call.
A held git_status leaves the index stale for the next reader; how a review is resolved — approve once, refresh, deny — is the operational parameter WP2 measures.
480calls through the proxy
415forwarded under an attested revision
65held for review
0held calls reached the server
0index rewrites by read-only calls
05 Further results
The same evidence chain, four more measurements.
Network
Call-scoped declarations
Applied to the recorded receipts, call-scoped network declarations flag 81 of 81 calls that reached the undocumented telemetry host — including 8 that the tool-wide declaration had excused.
Contract authorship
Who writes the contract?
Contracts drafted by seven language models from the same documentation agree with the human contracts on read-only intent for 92–98 % of tools and differ on 13–46 % of call-level grades. Documentation says what a tool is for; execution shows what it touches.
Static and dynamic
Two methods that complement each other
Of 17 behaviours, the best static scan found 15; the harness observed all 12 that execution can see — including the index rewrite inside the git binary that no scan saw.
Controlled study
AgentDojo & CaMeL
With the unmodified CaMeL interpreter, two hosted models carried out a concealed state-conditional effect in all 53 harmful episodes without the veto and in none with it.
06 Research program
From proof of concept to the conformance map.
The conformance map is made for maintainers, the MCP registry and client vendors that use annotations in approval policies — and for everyone building controllers on CaMeL, FIDES or AI-control protocols.
Conformance map
At least 60 locally runnable MCP servers, documentation against observed effects, existing static and dynamic tools run beside the harness as comparators.
State-aware qualification
Pre-call state and history predicates for the state-conditional part of the map, selected before the traffic they are scored on, against budget-matched baselines — with the review-resolution policy of the proxy as a measured parameter.
Independent confirmation
The frozen controller on sealed hidden-effect tools built by a second person, with live agent executions and an independent outcome grader.
Core invariant
VAC narrows an existing security boundary and never widens it. Unknown effects and changed revisions are held for review.
07 Reproducible
Every number from its receipts.
The package is one evidence chain: harness, receipts, controller and guarded runs, guard and proxy, contract and static studies, the AgentDojo/CaMeL study, verification, research protocol and the notices to the maintainers.
$ git clone https://github.com/cakbudak/vac
$ cd vac
$ bash VERIFY.sh
# regenerates every label, summary and report- VERIFY.sh — regenerates every label, summary and report from the receipts
- 21 Tests — for the proxy, the state policy, the ledger and call-scoped network declarations
- Hash manifest — over every file of the package
- Licenses — Apache-2.0 for code, CC BY 4.0 for text and data
Contact
Talk about VAC.
For collaboration, questions about the measurements, or using the conformance map in a client, registry or controller.
Cengizhan AkbudakSoftware Developer & AI Engineer · Vienna
Hamna QaseemCollaborator on the research package