Claim · Mixed

This compute runs inference, not training

The claim is that a declared cluster is used only to run existing models to produce outputs, not to train new or more capable models.

Several agreement proposals would let existing AI models keep serving users while restricting further training. The claim combines a positive part (the declared inference is happening) with a negative part (nothing else, especially training, is). It matters because it could let most AI compute stay in productive use under a training restriction.

It is hard because the same chips can do both, workloads can be disguised, and the negative part requires accounting for all of a facility's activity. Proposed approaches include capturing and recomputing the traffic that enters and leaves a facility, limiting bandwidth between groups of chips so that large training cannot be coordinated, and classifying workloads from telemetry. Each rests on open assumptions about numerical nondeterminism, hidden capacity and side channels.

State of verification

Declared inference can be checked with components that are demonstrated (R2) or in production (R3), but the absence of training cannot yet be verified. That half rests on full-stack designs (AI 2040 inference-only verification stack, Low-trust AI compute verification system overview, SASH confidential network logger) that are proposed (R1), and a team building the components reports nothing past a proof-of-principle prototype ¹³.

For example, a verifier may want to know that a cluster declared for inference is not training a new model.

Network taps (R1) record the cluster's front-end traffic with its users, and sampled recomputation (R3) re-runs sampled requests on the declared model. DiFR tolerates numerical noise in these re-runs on open-weight models of 8 to 30 billion parameters ¹¹.
SASH's network logger (R1) is a public prototype of both. It passes every request through a logger and re-runs it on a separate cluster, with a 270-million-parameter model and no stated adversary ¹⁷.
This covers the positive half at most. It shows that sampled outputs match the declared model. A tap on the cluster's external links does not stop covert workloads. It aims only to stop their results leaving over those links ¹⁸.
The negative half needs the rest of the cluster accounted for. Training traffic runs on back-end fabric that is harder to tap ⁵. Bandwidth limits (R2 for software monitoring) between groups of chips target that fabric, since distributed training must exchange gradients ⁶. Attestable proposes proofs of useful work (R1) to keep declared hardware busy with approved or protocol-defined work, leaving little spare capacity ¹⁵.
Known routes remain. Reinforcement-learning rollouts are inference, so declared servers could generate them while hidden compute updates the model ¹⁶. In one scenario, JoshC estimates that more than 95% of computation must be accounted for to constrain that strategy ¹⁶. Workload classification (R2) is a lighter alternative. It detects training with 98.2% accuracy on its own corpus, but 43–87% on the most challenging disguised workloads held out from its training, and current GPUs lack the protections its telemetry needs to be trusted ⁸ ¹⁴.

On this page

Mechanisms

R3Pearl proof-of-useful-work blockchainprimary
Proves matrix-multiplication work for consensus; not applied to bounding the spare capacity of declared hardware that could run training.
R2On-chip telemetry from timing, memory and performance counters ⚠primary
Classifying training versus other workloads from counters ³ or timing ¹.
R2Workload classification from telemetry and side channelsprimary
Classifies observed activity as training, inference or non-ML work.
R1AI 2040 inference-only verification stackprimary
The stack's stated purpose: retrofitted data centres run only inference.
R1Low-trust AI compute verification system overviewprimary
Challenged records are screened for inference versus training.
R1Network taps and certifiersprimary
Supplies the committed record of cluster I/O that recomputation checks against declared inference ¹ ⁴.
R1Proofs of useful work for capacity accountingprimary
Bounds the spare capacity of declared hardware that could run unauthorised training; Attestable pairs work accounting with ZK inference proofs.
R1SASH confidential network loggerprimary
SASH describes the aim as distinguishing inference from training in data centres ¹.
R1Whole-workload recomputation (reproducible packets)primary
Proposed as the correctness check for an inference-only retrofit ¹.
R3Deterministic and bit-exact inferencesupporting
Bit-exact recomputation of declared inference removes the tolerance an operator could hide other work in ¹.
R3Sampled inference recomputationsupporting
Proposed as the correctness check in inference-only retrofit plans. Completeness needs other mechanisms ³ ⁶.
R3TEE remote attestation for AI workloads ⚠supporting
Attests a declared inference deployment, but not that the same chips ran no other workloads ¹¹.
R2Bandwidth limits and compartmentalizationsupporting
Intended to leave inference workable while making large training impractical ¹ ².
R2Bounding unexplained information in outputssupporting
In the compute-agreement framing, outputs must be predictable by policy-compliant computation, which limits what undeclared workloads can export ¹.
R2Tamper evidence for verifier devicessupporting
Protects the integrity of taps, gateways and recomputation hardware used for inference-only verification ¹² ¹³.
R2Timed challenge-response and memory-occupation challengessupporting
Listed as an alternative inference-verification direction that may not need a hardware retrofit ²; bounds spare memory ¹.
R2Zero-knowledge proofs of inferencesupporting
Attestable proposes using proofs to show accounted workloads used an approved, unchanged model.
R1Attestable zero-knowledge inference proversupporting
Proposed use: showing an accounted workload used an approved, unchanged model.
R1Memory wiping and proofs of secure erasuresupporting
Periodic wipes are proposed so that only verified inference outputs persist ¹; Amodo frames wipes as verifying completeness of declared workloads ⁵.
R1RAND secure inference data center (SIDC) designsupporting
Scoped to serving already-trained models.
R1Side-channel suppression for isolated facilitiessupporting
Inference-only designs count on suppressing unmonitored physical channels so that all significant traffic passes the taps ².

Why it matters

Several proposals to restrict frontier AI development target training while preserving the use of existing models ¹ ². Sastry and colleagues note that most AI compute is now used for inference ³.

A draft international agreement restricts the scale of AI training. Its chip-use verification is meant to distinguish inference on existing systems from the training of new models ¹.
The AI 2040 verification plan would convert data centres to inference-only operation, preventing training runs while models continue to serve users ².
RAND's framework treats accurate declaration of AI inference as its own verification subgoal, separate from declared training ⁴. It lists deterministic replication of neural-network inference as a research problem ⁴.
A low-trust system overview names distinguishing inference from training, and deploying only approved models, among its core objectives ⁵.

Why it is hard

The same hardware can do both. Scher and Thiergart expect that some current inference-specialised chips could be repurposed for training without significant effort ⁶. Shavit notes that there is no straightforward way to determine whether an ML chip is running a training job or an unrelated one ⁷.
Classifiers invite evasion. The Open Problems survey notes that adversarial customers may obfuscate their activities, for example by adding noise to how they use computational resources ⁹. One adversarial study of GPU-telemetry classifiers reports 98.2% accuracy at identifying training across its corpus, but 43–87% on the most challenging disguised workloads held out from its training ⁸.
Training traffic is harder to observe. Inference produces token-level input and output data on the front-end links between a data centre and its users ⁵. Training traffic runs over back-end fabric that has much higher bandwidth, is latency-sensitive, and is harder to tap ⁵. Cankaya describes front-end tapping as the most viable option, and notes that back-end tapping may require sampling rather than full capture ¹⁰. The system overview leaves open how far a facility can be required to make all egress traffic explainable by its ingress ⁵.
Recomputation must cope with numerical noise. Checks on sampled outputs handle it either statistically ¹¹ or by exact reproduction ¹².
Bandwidth limits may erode. Limits between pods of chips could prevent the gradient exchange that distributed training needs ⁶. Sastry and colleagues note that more viable decentralised training could undermine the detectability of training ³.
Hidden capacity and channels. The negative part of the claim requires that no capacity or channel is hidden. The system overview addresses this with memory wiping and side-channel suppression, and lists open problems for both ⁵.
Inference can be part of training. JoshC analyzes a covert reinforcement-learning strategy that generates rollouts on declared inference servers while updating the model on hidden compute ¹⁶. Observing inference outputs alone would not rule it out.

Sources

BA. Scher et al. (2025). An International Agreement to Prevent the Premature Creation of Artificial Superintelligence. Machine Intelligence Research Institute. Source recordSupports: restricting the scale of training; chip use verification distinguishing inference on existing systems from training · abstract; Article VII (as summarised)
CR. Dean (2026). Verification Plan. AI 2040. Source recordSupports: data centres converted to inference-only operation; network taps, recomputation and reproducible packets · phases; verification mechanisms
BG. Sastry et al. (2024). Computing Power and the Governance of Artificial Intelligence. arXiv. Source recordSupports: majority of AI compute used for inference; decentralised training could undermine detectability · training vs inference; limitations
BM. Baker et al. (2025). Verifying International Agreements on AI: Six Layers of Verification for Rules on Large-Scale AI Development and Deployment. RAND Corporation. Source recordSupports: declared inference (1.A.2) as a distinct subgoal; deterministic replication of inference as an R&D problem · §3.2; Appendix A.9
BN. Cankaya (2026). A System Overview for Near-Term, Low-Trust AI Compute Verification. Machine Intelligence Research Institute. Source recordSupports: distinguishing inference from training; token-level front-end evidence; back-end harder to tap; egress explainable by ingress as open question; memory wiping; side channels · verification goals; inference vs training; open problems
BA. Scher & L. Thiergart (2025). Mechanisms to Verify International Agreements About AI Development. arXiv. Source recordSupports: inference-specialised chips repurposable for training; pods with limited external bandwidth · Verifying that known compute is not being used for a large training run
BY. Shavit (2023). What does it take to catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring. arXiv. Source recordSupports: no straightforward way to tell whether a chip is running training or another job · open problems
BR. Rahman & S. Tajdari (2026). Detecting Hidden ML Training With Zero-Overhead Telemetry. ICML 2026 Workshop on Technical AI Governance Research. Source recordSupports: telemetry classifier accuracy overall and on adversarially disguised workloads; required telemetry protections · abstract; §5.2; deployment requirements
AA. Reuel et al. (2025). Open Problems in Technical AI Governance. Transactions on Machine Learning Research. Source recordSupports: workload classification; adversarial customers may obfuscate by adding noise · §3.2.2 / §5.2.2 open problems
CN. Cankaya (2026). The Fundamentals and Feasibility of Secure Network Taps for Verifying AI Datacenter Use. The Datacenter Lie Detector. Source recordSupports: front-end tapping most viable; back-end requires sampling · frontend vs backend
BA. Karvonen et al. (2025). DiFR: Inference Verification Despite Nondeterminism. ICML 2026 Workshop on Technical AI Governance Research. Source recordSupports: recomputation despite benign numerical noise on 8–30B open-weight models · abstract; §5
BN. Cankaya (2026). Bit-Exact AI Inference Verification Without Performance Tradeoffs. ICML 2026 Workshop on Technical AI Governance Research. Source recordSupports: bit-exact inference verification across GPU variants · abstract
CT. Milton et al. (2026). Verifying international AI deals: Plan A, the state-of-play, and what you can do to help. Amodo (Substack). Source recordSupports: no verification component past a proof-of-principle prototype · introduction
BS. K. Monfared et al. (2026). Timing and Memory Telemetry on GPUs for AI Governance. arXiv. Source recordSupports: current GPUs expose limited trusted telemetry · abstract
CAttestable (2026). Pacing AI Requires Proof. Attestable blog. Source recordSupports: spare capacity could run an unauthorised training job; approved inference plus protocol-defined work fills a required work budget (provider proposal) · blog post
Cjoshc (2026). Can governments quickly and cheaply slow AI training?. AI Alignment Forum. Source recordSupports: public analysis of covert reinforcement-learning rollouts on declared inference servers and updates on hidden compute; share of computation that must be accounted for (scenario estimate) · §2.5; §3.4; §4
BSingapore AI Safety Hub (SASH) (2026). inference-verification: Inference Verification Prototype. GitHub. Source recordSupports: SASH prototype re-runs every request through a logger on a separate cluster, with Gemma 3 270M and a demo switch as the only adversary · README.md; implementation
BN. Cankaya et al. (2026). Fingerprinting All AI Cluster I/O Without Mutually Trusted Processors. arXiv. Source recordSupports: a tap on external links does not prevent covert workloads, only the exfiltration of their results