{ "name": "AI Verification Tech Map", "schema_version": "1.2.0", "rubric_version": "1.1", "generated": "2026-09-29T17:15:49.747Z", "includes_drafts": true, "license": "CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)", "notes": "url is the record's page on this site; a source record gives the original work's link in source_url. Fields marked assessment:true are editorial judgments under the published rubric, not statements of fact. flags_all includes computed flags (review-overdue, critical-flaw-open). provenance.reviewed_by and readiness.assessed_by list reviewers by handle, including AI agents such as codex-review; an agent review is not independent expert review.", "records": [ { "id": "M-0014", "slug": "bandwidth-limits-and-compartmentalization", "title": "Bandwidth limits and compartmentalization", "aliases": [ "Traffic shaping", "Isolated inference units", "Interconnect limits" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Capping or removing network links between groups of accelerators, so that serving within each group still works but large training across groups becomes far slower.", "summary": "Bandwidth limits cap or remove the links between groups of AI accelerators (\"pods\"), so that serving models within each pod keeps working but large training runs across pods become slow and costly. Lucid Computing's design caps each 72-GPU pod at 1 Gbps per direction. Lucid's modelled estimates put covert frontier training at about 350 times less efficient, or about 140 times if two unproven techniques both succeed. Both figures assume an auditor randomizes how pods connect to routers; with operator-chosen routing and compression, the inefficiency can fall to about 25 times. A small software prototype shows the traffic gap between training and inference, but as of September 2026 no cap that a verifier can check has been built or red-teamed. The main obstacles are showing that all traffic crosses the capped links and that the enforcing devices can be trusted. The largest known weakness is progress in low-communication training.", "technical": "- **Pod and cap.** Lucid Computing's Traffic Shaping design models a pod as one NVIDIA GB200 NVL72 rack (72 GPUs). A shaper at each pod's north-south uplink enforces a recommended cap of β = 1 Gbps (125 MB/s) per direction per pod, averaged over 60 seconds, about 14 Mbps per GPU [[S-1301]].\n- **Adversary.** The reference served model is a \"Fable-sized model (estimated 6T parameter MoE)\"; the adversary's target is about 10x larger (60T parameters). Strategies analysed: centralized training adapted to the cap, DiLoCo-family low-communication data parallelism (including streaming and quantized variants), and SWARM-family decentralized pipeline parallelism. The analysis grants the adversary perfect overlap of communication and computation, zero pipeline bubbles, zero straggler and fault overhead, free coordination compute outside the pods, aggressive low-precision recipes and the best published communication-reduction techniques [[S-1301]].\n- **Results.** The headline is a lower bound on the inefficiency multiplier of about 350x, with central estimates of 1,000x–10,000x depending on training architecture. For SWARM-style pipelines the floor falls to about 140x only if two further unproven techniques both succeed (8x lossless activation compression at frontier scale and an unusually narrow 60T-parameter architecture). The appendix summary claims at least 100x, with roughly 1.4–3.5x of margin, after stacking every charitable and speculative assumption [[S-1301]].\n- **Conditions.** The auditor randomizes the assignment of pods to routers and re-randomizes it periodically, and per-pod local storage is declared, capped and inspected. If the operator could instead assign whole cells of 100 or more pods to one pipeline stage, the bound would collapse to about 90–220x uncompressed and as low as about 25x with compression [[S-1301]].\n- **Amodo's node-level limiter.** NVIDIA BlueField-3 DPUs replace the NICs on a 400G Ethernet link, reaching 390 Gbps unencrypted and 193 Gbps encrypted between two DPUs [[S-1313]].", "category": "isolation-architecture", "secondary_categories": [ "off-chip-devices" ], "verifies": [ { "claim": "C-0008", "role": "primary", "note": "Caps or removes communication between declared groups of accelerators (S-0067, S-1301)." }, { "claim": "C-0004", "role": "supporting", "note": "Intended to leave inference workable while making large training impractical (S-0067, S-1301)." }, { "claim": "C-0007", "role": "supporting", "note": "Bounds the size of model that can be trained efficiently across pods (S-1301)." }, { "claim": "C-0009", "role": "supporting", "note": "A cap on outgoing bandwidth bounds how much weight data can leave a facility in a given time (S-1508)." } ], "threat_model": "adversarial", "adversarial_evaluation": "analysis", "hardware_requirement": "retrofit-device", "prover_cooperation": "required", "confidentiality": "preserving", "depends_on": [ { "target": "M-0013", "note": "Monitored links are needed to show that all traffic leaving a group crosses the capped boundary." }, { "target": "M-0017", "note": "Shaping and monitoring devices must resist tampering and bypass." } ], "readiness": { "assessment": true, "level": "R2", "scope": "detecting traffic above a limit between accelerator groups, not enforcing a cap", "rubric_version": "1.1", "rationale": "R2, narrowly: a public software prototype on data-centre GPUs monitors inter-node traffic against a limit and detects low-communication training, but no cap that a verifier can check has been built.\n\n- **R1** met: the AI 2040 plan proposes isolated inference units created by removing back-end networking, on the grounds that high-bandwidth links are mostly needed only for training [[S-0067]]. Lucid Computing gives a detailed design with a cap, a pod definition, adversary strategies, favourable assumptions and residual risks [[S-1301]]. The MIRI overview describes perimeters of monitored links sized to pods for inference [[S-0018]].\n- **R2** met: MIRI's Technical Governance Team published an end-to-end prototype, with code, that monitors inter-node traffic against a threshold on two nodes with four A100 GPUs [[S-3220]]. It tested violating training, compliant inference and a stated adversary, DiLoCo training [[S-3220]]. The prototype alerts rather than throttles, and the authors call its software-only design \"trivially spoofable\" [[S-3220]]. Amodo reports building a DPU-based per-node rate limiter on 400G links, for weight security under a trusted controller [[S-1313]]. Lucid states that its pod-level cap is \"still at the design stage and not yet implemented or red-teamed\" [[S-1301]]. The mechanism's implementations, [[I-0010]] and [[I-0011]], are proposed architectures at R1.\n- **R3** not met: no production-grade cap or monitor for this use is publicly available, and no reliance on one by a party other than its developer is documented.\n\nConfidence is low: the R2 evidence is one small, software-only prototype described in a blog post, and it monitors a limit without enforcing one.", "evidence": [ "S-0067", "S-1301", "S-0018", "S-3220", "S-1313" ], "next_level_gaps": [ "A production-grade cap or bandwidth monitor for this use, or reliance on one by a party other than its developer for a verification decision.", "Enforcement outside the prover's control, such as a shaper at each pod uplink, built and tested on data-centre hardware (Lucid reports a proof of concept in development).", "Measured, not only modelled, training inefficiency under a cap at realistic scale, including low-communication methods.", "Red-teaming of cap bypass (for example through parallel scale-up switches) and of trust in the shaping devices." ], "confidence": "low", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Low-communication training reduces the bandwidth training needs", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "DiLoCo matched fully synchronous training on 8 workers while communicating 500 times less. Rahman writes that this family of methods theoretically allows large-scale training with less than 100 Mbps. Lucid includes these methods in its bounds, but notes that extreme activation compression, architectures with unusually small inter-layer widths, or modular paradigms could erode the margin.", "sources": [ "S-1314", "S-0060", "S-1301" ], "response": null }, { "assessment": true, "title": "Operator control of pod routing collapses the bound", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Lucid's analysis finds that if the operator can freely assign pods to routers, it could dedicate a whole cell of 100 or more pods to one pipeline stage. The bound then falls to about 90–220x uncompressed and as low as about 25x with compression. The proposed mitigation, auditor-controlled random assignment that is periodically re-randomized, has not been implemented.", "sources": [ "S-1301" ], "response": null }, { "assessment": true, "title": "Undeclared local storage raises per-pod capacity", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "More memory or storage per pod helps an adversary. Lucid requires per-pod storage to be declared, capped and physically inspected.", "sources": [ "S-1301" ], "response": null }, { "assessment": true, "title": "Training within one pod is not covered", "kind": "open-question", "severity": "significant", "status": "open", "description": "Lucid's bounds concern pre-training models larger than the pods are sized for. Training models that fit in one pod, fine-tuning and reinforcement-learning post-training within one pod are outside the modelled threat.", "sources": [ "S-1301" ], "response": null }, { "assessment": true, "title": "Parallel scale-up switches are hard enforcement points", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "In GB200 topologies, GPUs reach GPUs in other nodes through NVSwitches without a NIC on the path. Amodo notes that limits are hard to enforce there because many switches work in parallel, so compromising one or two would bypass the limit.", "sources": [ "S-1313" ], "response": null } ], "blockers": [ { "text": "No cap that a verifier can check has been implemented or red-teamed.", "theme": "adversarial-validation", "blocked_by": null, "sources": [ "S-1301" ] }, { "text": "The verifier must know that all traffic leaving a pod crosses the capped, monitored links.", "theme": "coverage-hidden-compute", "blocked_by": "M-0013", "sources": [ "S-0018" ] }, { "text": "Shaping devices and routing assignments must be trusted by both parties; Amodo has not yet fully analysed resilience to a compromised DPU.", "theme": "hardware-trust", "blocked_by": "M-0017", "sources": [ "S-1301", "S-1313" ] }, { "text": "Advances in low-communication training could shrink the margin that the cap enforces.", "theme": "capacity-bounds", "blocked_by": null, "sources": [ "S-1314", "S-0060", "S-1301" ] } ], "challenge_themes": [ "capacity-bounds", "coverage-hidden-compute", "hardware-trust", "adversarial-validation", "performance-compatibility" ], "organizations": [ "O-0101", "O-0180", "O-0201", "O-0202" ], "people": [], "sources": [ { "source": "S-0067", "supports": "isolated inference units by removing back-end networking; rationale; retrofit illustration", "locator": "inference-only retrofitting proposal" }, { "source": "S-1301", "supports": "Traffic Shaping design, cap, pod model, adversary strategies and assumptions, results and their conditions, residual risks, status", "locator": "summary; main text; appendices A.7–A.9" }, { "source": "S-1508", "supports": "egress limits cap how much can be stolen", "locator": "§5.1" }, { "source": "S-0018", "supports": "perimeter of monitored links; perimeter size for inference and training", "locator": "§5.1.1" }, { "source": "S-3220", "supports": "interconnect-limit monitoring prototype, workloads, measured traffic, DiLoCo detection, alerting without throttling, spoofability, longer training, LoRA and RL", "locator": "whole post" }, { "source": "S-3221", "supports": "Anthropic's egress bandwidth controls for model weights", "locator": "security section" }, { "source": "S-1313", "supports": "DPU-based node bandwidth boundary, throughput, NVSwitch limitation, compromised-DPU caveat, security purpose", "locator": "whole note" }, { "source": "S-1314", "supports": "500x less communication on 8 workers", "locator": "abstract" }, { "source": "S-0060", "supports": "DiLoCo-family bandwidth needs; bandwidth caps on evaders judged infeasible", "locator": "§1; appendix G.1" }, { "source": "S-0026", "supports": "workshop abstract proposes workload compartmentalization proofs to distinguish inference from frontier training; no protocol details assessed", "locator": "ICML 2026 TAIGR workshop abstract" }, { "source": "S-3565", "supports": "expert-parallel MoE inference involves all-to-all cross-device communication", "locator": "abstract" } ], "concepts": [ "K-0017", "K-0021", "K-0025", "K-0016", "K-0018" ], "complements": [ "M-0005", "M-0001", "M-0003", "M-0015", "M-0016" ], "alternatives": [], "type": "mechanism", "implementations": [ { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "I-0010", "title": "RAND secure inference data center (SIDC) design", "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/" } ], "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/", "source_file": "content/mechanisms/bandwidth-limits-and-compartmentalization.md", "flags_all": [], "body_markdown": "## How it works\n\nTraining and serving use the network differently [[S-0067]] [[S-1301]]. The AI 2040 plan notes that AI servers \"have a lot of high bandwidth communication between them, but this is (mostly) only necessary for training\" [[S-0067]]. Lucid Computing gives the same rationale: inference sends little external traffic, while training moves gradients, model state and activations among many machines [[S-1301]].\n\nTwo families of design use this asymmetry [[S-0067]] [[S-1301]]:\n\n- The AI 2040 plan would remove a significant amount of back-end networking to \"convert datacenters into groups of isolated inference-units\" [[S-0067]]. Its illustration retrofits 40% of US compute into inference units of four server racks [[S-0067]].\n- Lucid's Traffic Shaping design caps links with a network shaper at the north-south uplink of each pod, modelled as one GB200 NVL72 rack [[S-1301]]. It recommends a cap of 1 Gbps per direction per pod, averaged over 60 seconds [[S-1301]]. The parties would configure and agree on the shaping devices, built on commodity hardware or open standards through a mediated co-design process [[S-1301]].\n\nThe MIRI system overview frames the same boundary as a perimeter: untrusted devices are grouped so that only monitored links communicate in and out [[S-0018]]. For inference, that perimeter can be as small as pods of a few hundred accelerators [[S-0018]]. Monitored links are covered in [[M-0013]].\n\nAmodo reports building a related enforcement point for weight security [[S-1313]]. It replaced a node's NICs with NVIDIA BlueField-3 DPUs, reasoning that \"all the traffic in and out of a node goes through a NIC\", which creates \"a clean bandwidth boundary\" [[S-1313]].\n\nReuter and colleagues propose a separate \"proofs of compartmentalization\" approach. Their ICML 2026 workshop abstract argues that inference workloads can be compartmentalized while frontier training cannot, and proposes checking that property without revealing confidential workloads [[S-0026]]. The abstract gives no protocol or performance results; it does not establish that their approach has been implemented.\n\n## What it establishes\nLucid aims to let a third party verify that frontier training either is not happening in a declared facility or would cost so much more that training a model ten times the current frontier becomes economically infeasible [[S-1301]]. It estimates a lower bound of about 350x on the inefficiency of covert frontier training, with central estimates of 1,000x to 10,000x, if the auditor controls pod routing and per-pod storage is capped [[S-1301]]. For decentralized pipeline training, the floor falls to about 140x if two unproven techniques both succeed. Lucid calls its figures \"engineering estimates, not proofs\" [[S-1301]]. This addresses [[C-0008]], and indirectly [[C-0004]] and [[C-0007]].\n\nIt does not establish:\n\n- A cap alone does not show what the pods compute, so the AI 2040 plan pairs inference units with taps and recomputation ([[M-0001]], [[M-0003]]) [[S-0067]].\n- Lucid places training that fits within one pod outside its modelled threat, along with fine-tuning and reinforcement-learning post-training within one pod [[S-1301]].\n- The caps apply only inside declared facilities. Outside them, Rahman judges it \"infeasible to prevent evaders from getting fast internet connections\" [[S-0060]].\n\n## Threat model\n- **Favourable adversary.** Lucid's analysis grants the adversary perfect overlap of communication and computation, zero pipeline bubbles and \"the best published communication-reduction techniques\", among other advantages [[S-1301]].\n- **Auditor-controlled routing.** The bound holds only if the auditor randomizes the assignment of pods to routers and periodically re-randomizes it [[S-1301]].\n- **Declared storage.** Per-pod storage must be declared, capped and inspected [[S-1301]].\n- **Complete perimeter.** The shaper sits at each pod's uplink [[S-1301]]. The verifier must also know that no other path leaves the pod, which the MIRI overview handles with a perimeter where only monitored links communicate in and out [[S-0018]]; see [[M-0013]] and [[M-0017]].\n\n## Evidence\n- **Designs.** The AI 2040 plan, Lucid's design brief and the MIRI overview are designs and analysis [[S-0067]] [[S-1301]] [[S-0018]]. Lucid describes its design as \"still at the design stage and not yet implemented or red-teamed\" [[S-1301]]. It reports that its engineers are building a proof of concept for red-teaming at a partner cluster [[S-1301]].\n- **Monitoring prototype.** MIRI's Technical Governance Team monitored inter-node traffic on a two-node Azure cluster with four A100 GPUs joined by one 100 Gb Ethernet link, and published its code [[S-3220]]. Fully sharded fine-tuning of Llama 3.1 8B and a mixture-of-experts model averaged 2–3 GB/s between the nodes, while LLM inference averaged about 10 kB/s [[S-3220]]. DiLoCo cut total cross-node traffic from about 10 TB to about 100 GB, but its regular spikes of hundreds of MB/s remained distinguishable from inference [[S-3220]]. The prototype raises alerts and does not throttle, and the authors call their software-only version \"trivially spoofable\", because the prover controls the whole measurement pipeline [[S-3220]].\n- **Node-level limiter.** Amodo reports 390 Gbps unencrypted and 193 Gbps encrypted throughput between two DPUs on a 400G link [[S-1313]]. That demonstration targets weight security under a trusted controller, and Amodo has \"not yet fully analysed resilience to a single compromised DPU\" [[S-1313]].\n- **Operator egress limits.** Anthropic reports that its ASL-3 security measures include egress bandwidth controls, which limit the rate of outbound traffic from environments that hold model weights [[S-3221]].\n\n## Limitations\n- **Low-communication training.** DiLoCo matched fully synchronous training on 8 workers while communicating 500 times less [[S-1314]]. Rahman writes that this family theoretically allows large-scale training at under 100 Mbps [[S-0060]]. Lucid's bounds include DiLoCo- and SWARM-family methods, but it notes that extreme activation compression or modular paradigms could erode the margin [[S-1301]].\n- **Inference that crosses pod boundaries.** Expert-parallel mixture-of-experts inference uses all-to-all communication between devices holding different experts, which can be a bottleneck [[S-3565]]. A cap calibrated for token-only ingress and egress may disrupt that serving architecture.\n- **Low-traffic training.** LoRA fine-tuning and reinforcement learning involve less inter-node traffic than ordinary training, and the MIRI authors leave whether a limit would stop them to future work [[S-3220]]. They judge that training for longer at lower throughput is not a major concern, because a reasonable limit would slow training by a prohibitive factor, such as 100x [[S-3220]].\n- **Routing control.** If the operator can group pods freely behind routers, the bound falls to about 90–220x uncompressed and as low as about 25x with compression [[S-1301]].\n- **Hidden storage.** Undeclared per-pod storage weakens the bound [[S-1301]].\n- **Scale-up bypass.** In GB200 topologies, GPUs reach other nodes through NVSwitches without a NIC on the path, and compromising one or two parallel switches would bypass a limit [[S-1313]].\n- **Durability.** Lucid positions the design as \"a short-to-medium-term deterrent and Phase-1 milestone\" [[S-1301]].", "body_text": "How it works Training and serving use the network differently [S-0067] [S-1301]. The AI 2040 plan notes that AI servers \"have a lot of high bandwidth communication between them, but this is (mostly) only necessary for training\" [S-0067]. Lucid Computing gives the same rationale: inference sends little external traffic, while training moves gradients, model state and activations among many machines [S-1301]. Two families of design use this asymmetry [S-0067] [S-1301]: - The AI 2040 plan would remove a significant amount of back-end networking to \"convert datacenters into groups of isolated inference-units\" [S-0067]. Its illustration retrofits 40% of US compute into inference units of four server racks [S-0067]. - Lucid's Traffic Shaping design caps links with a network shaper at the north-south uplink of each pod, modelled as one GB200 NVL72 rack [S-1301]. It recommends a cap of 1 Gbps per direction per pod, averaged over 60 seconds [S-1301]. The parties would configure and agree on the shaping devices, built on commodity hardware or open standards through a mediated co-design process [S-1301]. The MIRI system overview frames the same boundary as a perimeter: untrusted devices are grouped so that only monitored links communicate in and out [S-0018]. For inference, that perimeter can be as small as pods of a few hundred accelerators [S-0018]. Monitored links are covered in Network taps and certifiers. Amodo reports building a related enforcement point for weight security [S-1313]. It replaced a node's NICs with NVIDIA BlueField-3 DPUs, reasoning that \"all the traffic in and out of a node goes through a NIC\", which creates \"a clean bandwidth boundary\" [S-1313]. Reuter and colleagues propose a separate \"proofs of compartmentalization\" approach. Their ICML 2026 workshop abstract argues that inference workloads can be compartmentalized while frontier training cannot, and proposes checking that property without revealing confidential workloads [S-0026]. The abstract gives no protocol or performance results; it does not establish that their approach has been implemented. What it establishes Lucid aims to let a third party verify that frontier training either is not happening in a declared facility or would cost so much more that training a model ten times the current frontier becomes economically infeasible [S-1301]. It estimates a lower bound of about 350x on the inefficiency of covert frontier training, with central estimates of 1,000x to 10,000x, if the auditor controls pod routing and per-pod storage is capped [S-1301]. For decentralized pipeline training, the floor falls to about 140x if two unproven techniques both succeed. Lucid calls its figures \"engineering estimates, not proofs\" [S-1301]. This addresses Communication between compute groups is bounded, and indirectly This compute runs inference, not training and A training run stayed within declared limits. It does not establish: - A cap alone does not show what the pods compute, so the AI 2040 plan pairs inference units with taps and recomputation (Sampled inference recomputation, Whole-workload recomputation (reproducible packets)) [S-0067]. - Lucid places training that fits within one pod outside its modelled threat, along with fine-tuning and reinforcement-learning post-training within one pod [S-1301]. - The caps apply only inside declared facilities. Outside them, Rahman judges it \"infeasible to prevent evaders from getting fast internet connections\" [S-0060]. Threat model - Favourable adversary. Lucid's analysis grants the adversary perfect overlap of communication and computation, zero pipeline bubbles and \"the best published communication-reduction techniques\", among other advantages [S-1301]. - Auditor-controlled routing. The bound holds only if the auditor randomizes the assignment of pods to routers and periodically re-randomizes it [S-1301]. - Declared storage. Per-pod storage must be declared, capped and inspected [S-1301]. - Complete perimeter. The shaper sits at each pod's uplink [S-1301]. The verifier must also know that no other path leaves the pod, which the MIRI overview handles with a perimeter where only monitored links communicate in and out [S-0018]; see Network taps and certifiers and Tamper evidence for verifier devices. Evidence - Designs. The AI 2040 plan, Lucid's design brief and the MIRI overview are designs and analysis [S-0067] [S-1301] [S-0018]. Lucid describes its design as \"still at the design stage and not yet implemented or red-teamed\" [S-1301]. It reports that its engineers are building a proof of concept for red-teaming at a partner cluster [S-1301]. - Monitoring prototype. MIRI's Technical Governance Team monitored inter-node traffic on a two-node Azure cluster with four A100 GPUs joined by one 100 Gb Ethernet link, and published its code [S-3220]. Fully sharded fine-tuning of Llama 3.1 8B and a mixture-of-experts model averaged 2–3 GB/s between the nodes, while LLM inference averaged about 10 kB/s [S-3220]. DiLoCo cut total cross-node traffic from about 10 TB to about 100 GB, but its regular spikes of hundreds of MB/s remained distinguishable from inference [S-3220]. The prototype raises alerts and does not throttle, and the authors call their software-only version \"trivially spoofable\", because the prover controls the whole measurement pipeline [S-3220]. - Node-level limiter. Amodo reports 390 Gbps unencrypted and 193 Gbps encrypted throughput between two DPUs on a 400G link [S-1313]. That demonstration targets weight security under a trusted controller, and Amodo has \"not yet fully analysed resilience to a single compromised DPU\" [S-1313]. - Operator egress limits. Anthropic reports that its ASL-3 security measures include egress bandwidth controls, which limit the rate of outbound traffic from environments that hold model weights [S-3221]. Limitations - Low-communication training. DiLoCo matched fully synchronous training on 8 workers while communicating 500 times less [S-1314]. Rahman writes that this family theoretically allows large-scale training at under 100 Mbps [S-0060]. Lucid's bounds include DiLoCo- and SWARM-family methods, but it notes that extreme activation compression or modular paradigms could erode the margin [S-1301]. - Inference that crosses pod boundaries. Expert-parallel mixture-of-experts inference uses all-to-all communication between devices holding different experts, which can be a bottleneck [S-3565]. A cap calibrated for token-only ingress and egress may disrupt that serving architecture. - Low-traffic training. LoRA fine-tuning and reinforcement learning involve less inter-node traffic than ordinary training, and the MIRI authors leave whether a limit would stop them to future work [S-3220]. They judge that training for longer at lower throughput is not a major concern, because a reasonable limit would slow training by a prohibitive factor, such as 100x [S-3220]. - Routing control. If the operator can group pods freely behind routers, the bound falls to about 90–220x uncompressed and as low as about 25x with compression [S-1301]. - Hidden storage. Undeclared per-pod storage weakens the bound [S-1301]. - Scale-up bypass. In GB200 topologies, GPUs reach other nodes through NVSwitches without a NIC on the path, and compromising one or two parallel switches would bypass a limit [S-1313]. - Durability. Lucid positions the design as \"a short-to-medium-term deterrent and Phase-1 milestone\" [S-1301].", "referenced_by": [ { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "I-0010", "title": "RAND secure inference data center (SIDC) design", "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "K-0017", "title": "Compartmentalization", "url": "https://trustbutveri.fyi/concepts/compartmentalization/" }, { "id": "K-0025", "title": "Inference and training workloads", "url": "https://trustbutveri.fyi/concepts/inference-and-training-workloads/" }, { "id": "K-0021", "title": "Interconnect bandwidth", "url": "https://trustbutveri.fyi/concepts/interconnect-bandwidth/" }, { "id": "K-0003", "title": "Positive and negative claims", "url": "https://trustbutveri.fyi/concepts/positive-and-negative-claims/" }, { "id": "O-0201", "title": "AI Futures Project", "url": "https://trustbutveri.fyi/organizations/ai-futures-project/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" }, { "id": "O-0180", "title": "Lucid Computing", "url": "https://trustbutveri.fyi/organizations/lucid-computing/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" } ] }, { "id": "M-0024", "slug": "bounding-unexplained-information", "title": "Bounding unexplained information in outputs", "aliases": [ "Unexplained-information bound", "Output compressibility bounds", "Egress limiting by compression" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Limits the hidden information a facility's outputs can carry by measuring how much of those outputs the declared computation fails to predict.", "summary": "Bounding unexplained information measures how much of the information leaving a facility the declared work cannot explain. If a verifier can predict outputs from the declared model and recorded inputs, little room remains to smuggle out model weights or the results of hidden work. One proposed architecture routes all traffic through a verifier-controlled interlock and challenges the operator to show that random outputs follow from compliant computation. As of September 2026 no prototype results have been published. An instance for language-model inference, with public code, cut the information an attacker could hide to under 0.5% on a 30-billion-parameter model, under benign prompts and at a false-positive rate below 0.01%. An independent study showed that an attacker who chooses the prompts roughly doubles the leakage per token. The main obstacles are closing every other channel, including side channels, and tolerating numerical noise without leaving room for a covert channel.", "technical": "- **Fixed-seed sampling likelihood.** The verifier knows the seed and recomputes the next-token distribution for each logged position, then scores whether each emitted token is plausible under honest sampling. Gumbel-Max and inverse-probability-transform estimators are given, and three theorems bound the rate at which an adversary can embed information while passing the check [[S-0015]].\n- **Reported result.** On MoE-Qwen-30B under benign prompt traffic, exfiltratable information fell to under 0.5% at a false-positive rate below 0.01%, a slowdown of more than 200 times [[S-0015]].\n- **Attack result.** Across six instruction-tuned models of 1 to 32 billion parameters and three seeds, the strongest prompt attack raised leakage from 0.067–0.119 to 0.145–0.286 bits per token and cut the slowdown from 146–254 times to 60–118 times, because the verifier's admissible-token set grows with the model's output entropy [[S-1507]].\n- **Compression bounds.** LLM-based arithmetic coding with domain-adapted LoRA adapters is 2 times better than the base model; succinct rewrites reach compression ratios of about 0.03; and a question-asking protocol reaches 0.0006–0.004 [[S-1508]]. An earlier proposal reports compression below 1 bit per token by arithmetic coding against a weaker reference model, in an experiment with fine-tuned Pythia models [[S-1509]].", "category": "isolation-architecture", "secondary_categories": [ "cryptographic-computational" ], "verifies": [ { "claim": "C-0009", "role": "primary", "note": "Bounds how much weight or other undeclared information can leave in checked outputs; does not close other channels." }, { "claim": "C-0004", "role": "supporting", "note": "In the compute-agreement framing, outputs must be predictable by policy-compliant computation, which limits what undeclared workloads can export (S-0019)." }, { "claim": "C-0008", "role": "supporting", "note": "Bounds effective unexplained output bandwidth rather than raw link bandwidth." } ], "threat_model": "adversarial", "adversarial_evaluation": "independent-red-team", "hardware_requirement": "retrofit-device", "prover_cooperation": "required", "confidentiality": "partial", "depends_on": [ { "target": "M-0001", "note": "Predictions of honest outputs come from recomputing declared workloads." }, { "target": "M-0014", "note": "The bound is meaningful only if outputs through the interlock are the prover's only channel." }, { "target": "M-0022", "note": "Physical side channels bypass output checks and must be suppressed separately." }, { "target": "M-0013", "note": "An interlock or tap must record commitments to all traffic." } ], "readiness": { "assessment": true, "level": "R2", "scope": "bounding how much hidden information can leave in checked inference outputs", "rubric_version": "1.1", "rationale": "R2 through the inference-output instance, which has public code and an independent attack study; the facility-level architecture is still a proposal.\n\n- **R1** met: Petrie and Mühlhäuser publish an architecture, the claim it verifies and its setting, in which neither party trusts the other's hardware [[S-0019]].\n- **R2** met through the instance for inference outputs. Rinberg et al. bound the covert information in LLM responses beyond what honest sampling from the declared model explains. They publish code and report results on a 30-billion-parameter mixture-of-experts model against a stated adversary who controls the inference server [[S-0015]]. The facility-level architecture remains at R1: its paper outlines protocol details, attacks and prototyping plans, not results [[S-0019]].\n- **R3** not met: no party other than the developers relies on the bound for a verification decision, and no production-grade system is available.\n- **R4** not met. Its evaluation criterion holds for the inference instance only: an independent researcher attacked it in practice, and the flaw shown widens the bound rather than defeating it, so the evaluation left no critical flaw open [[S-1507]]. The facility-level design has not been independently evaluated.\n\nConfidence is low. The R2 evidence covers token outputs of a single inference service, not the general bound over all facility outputs [[S-0019]], and the demonstrated bound degrades when the attacker controls prompts [[S-1507]].", "evidence": [ "S-0019", "S-0015", "S-1507" ], "next_level_gaps": [ "A prototype of the facility-level architecture (interlock, commitments, challenge-based prediction) with published results.", "A bound that holds against an adversary who controls the prompt distribution, for example with entropy-calibrated tolerances, evaluated independently.", "Reliance by a party other than the developer on an unexplained-information bound for a verification decision, or a production-grade deployment." ], "confidence": "low", "assessed_by": [ "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Prompt-controlled entropy inflation widens the covert channel", "kind": "demonstrated-attack", "severity": "significant", "status": "open", "description": "Gumbel-based inference verification tolerates token choices that honest GPU nondeterminism could produce, and the size of that tolerated set grows with the model's output entropy. Kezins, an independent researcher, showed that an adversary who controls the prompt distribution can raise output entropy and roughly double the bits leaked per token. Across six models of 1 to 32 billion parameters, this cut the slowdown from 146–254 times under benign prompts to 60–118 times. Kezins argues that architectures built on the same unexplained-information bound inherit this attack surface, and recommends calibrating tolerances against local token entropy rather than benign traffic.", "sources": [ "S-1507", "S-0015" ], "response": null }, { "assessment": true, "title": "Information the declared computation explains is not bounded", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "The bound limits unexplained bits only. Outputs that the declared computation fully explains can still carry valuable information: a compression study notes that an adversary with inference access can extract more proprietary information per bit than naive transmission allows.", "sources": [ "S-1508", "S-0019" ], "response": null }, { "assessment": true, "title": "Channels other than checked outputs are outside the bound", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "The inference-verification scheme treats side channels as out of scope. A low-trust system design argues that suppressing physical covert bandwidth below kilobits per second is much more achievable than aiming for zero, and that a malicious device can leak one bit of information by deliberately outputting a wrong result.", "sources": [ "S-0015", "S-0018" ], "response": null }, { "assessment": true, "title": "The facility-level design is untested", "kind": "open-question", "severity": "significant", "status": "open", "description": "The compute-verification architecture is described with protocol details, potential attacks and prototyping plans, but no prototype results have been published.", "sources": [ "S-0019" ], "response": null } ], "blockers": [ { "text": "The prover's compute must be isolated so that all traffic passes through the verifier's interlock; any unmonitored path voids the bound.", "theme": "coverage-hidden-compute", "blocked_by": "M-0014", "sources": [ "S-0019" ] }, { "text": "Physical side channels need separate suppression, and one design treats a low residual bandwidth, rather than zero, as the realistic target.", "theme": "coverage-hidden-compute", "blocked_by": "M-0022", "sources": [ "S-0018", "S-0015" ] }, { "text": "Tolerance for numerical nondeterminism sets the size of the residual channel; bit-exact replay would remove it but needs full hardware and software metadata.", "theme": "protocol-soundness", "blocked_by": "M-0002", "sources": [ "S-1507", "S-0018" ] }, { "text": "Recomputation over confidential weights and inputs needs a protected setting: prover recomputation in a verifier-controlled enclosure, verifier recomputation in a prover-controlled enclosure, or zero-knowledge proofs.", "theme": "privacy-leakage", "blocked_by": null, "sources": [ "S-0019" ] }, { "text": "No prototype of the facility-level architecture exists to red-team.", "theme": "adversarial-validation", "blocked_by": null, "sources": [ "S-0019" ] } ], "challenge_themes": [ "coverage-hidden-compute", "protocol-soundness", "privacy-leakage", "adversarial-validation" ], "organizations": [], "people": [], "sources": [ { "source": "S-0019", "supports": "architecture: isolation, interlock, commitments, challenge-based prediction; principle; three confidentiality options; stage of work", "locator": "abstract (read via the ICML 2026 virtual poster page; the OpenReview PDF was not reachable)" }, { "source": "S-0015", "supports": "fixed-seed sampling likelihood; theorems; threat model and assumptions; results; code release; side channels out of scope", "locator": "abstract; §4; §5 (Theorems 5.1-5.3); §6; Appendix E" }, { "source": "S-1507", "supports": "entropy-inflation attack, results and recommended mitigation; applicability to unexplained-information architectures", "locator": "abstract; introduction; method; conclusion" }, { "source": "S-1508", "supports": "compressibility of LLM text; egress limiting rationale; dual-use note", "locator": "abstract; §5.1; §5.3" }, { "source": "S-1509", "supports": "upload limits with compression against a weaker model; below 1 bit per token; assumptions; hidden-distillation route; author's uncertainty and probability estimate", "locator": "whole post" }, { "source": "S-0018", "supports": "sanitized egress and one-bit fault leakage; side-channel suppression target; exact-replay metadata", "locator": "§4.3.3; §5.2.2; §5.3.1" }, { "source": "S-1300", "supports": "residual covert egress of about 40 Mbit/s for a 200k-GPU inference cluster at about 0.1 bits per token after replay checks", "locator": "§5.2" } ], "concepts": [ "K-0008", "K-0009", "K-0013", "K-0016", "K-0017", "K-0020", "K-0022", "K-0024" ], "complements": [ "M-0002", "M-0015" ], "alternatives": [], "type": "mechanism", "implementations": [], "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/", "source_file": "content/mechanisms/bounding-unexplained-information.md", "flags_all": [], "body_markdown": "## How it works\n\nThe idea is to measure the information that leaves a facility against what its declared work would produce [[S-0019]]. Petrie and Mühlhäuser propose verifying AI compute agreements by bounding the amount of unexplained information in network outputs, without either party trusting the other's hardware [[S-0019]].\n\n- **Isolation and commitment.** The prover's compute nodes are isolated, and all traffic passes through a verifier interlock that records cryptographic commitments of the encrypted traffic [[S-0019]].\n- **Prediction challenges.** From time to time the verifier challenges the prover to show that randomly selected outputs can be predicted by a policy-compliant computation on the measured inputs [[S-0019]].\n- **Principle.** The better compliant computation predicts the outputs, the less bandwidth is left for exporting the results of undeclared workloads [[S-0019]].\n- **Confidentiality.** The prover can recompute inside a verifier-controlled enclosure, the verifier can recompute inside a prover-controlled enclosure, or zero-knowledge proofs can be used [[S-0019]].\n\nFor language-model inference, Rinberg et al. make this concrete. Honest inference follows a well-defined generative process, so hiding data in responses must make them deviate from expected model behaviour [[S-0015]]. A verification server re-runs sampled requests with the trusted model and the shared random seed, scores how plausible each emitted token is, and bounds the rate at which hidden bits can pass unnoticed [[S-0015]]. See [[M-0001]].\n\nA related line of work uses compression [[S-1509]] [[S-1508]]. One proposal compresses each response with a weaker reference model, using arithmetic coding, so that only bits the reference model did not predict count against an upload limit [[S-1509]]. Rinberg and co-authors note that limiting egress rests on a simple fact: if only 10 GB leaves a data centre, no attacker can steal more than 10 GB [[S-1508]]. Language-model text compresses far better than model weights, so a facility can apply stricter egress limits [[S-1508]].\n\n## What it establishes\nIt can put an upper bound on the covert information in checked outputs, which slows exfiltration rather than stopping it [[S-0015]]. Under benign prompt traffic, Rinberg et al. report that the exfiltratable fraction fell to under 0.5%, not zero [[S-0015]]. One estimate puts the remaining channel at about 40 Mbit/s for a 200,000-GPU inference cluster at full load, assuming about 0.1 bits per token survive replay checks [[S-1300]].\n\nIt does not show that explained outputs are harmless, that no smaller model is being prepared, or anything about other channels:\n\n- The bound covers only unexplained information [[S-0019]]. Compression research notes that someone with inference access can extract more proprietary information per bit than naive transmission allows [[S-1508]].\n- One upload-limit proposal lists running a distillation job inside the data centre, to produce a smaller model that is easier to move, as a residual route [[S-1509]]. Its author judges this hard to do without detection [[S-1509]].\n- The inference scheme treats side channels as out of scope [[S-0015]].\n\n## Threat model\n- **Facility-level design.** Neither party trusts the other's hardware, and the prover's compute can be isolated behind the interlock [[S-0019]].\n- **Inference scheme.** An insider controls the server's outputs, and outsiders may fully control the prompts [[S-0015]]. The scheme also assumes trusted verification code, secure logging, a private choice of which outputs to check that the adversary cannot manipulate, and randomness that is trusted and non-malleable [[S-0015]]. The reported bound was measured on benign prompts from a public chat dataset, and the authors call its size under adversarial prompts an open empirical question [[S-0015]]. An independent study has since measured it [[S-1507]].\n- **Upload limits.** The proposal assumes strong physical security [[S-1509]]. It also expects that a lab's most powerful model will not generate more than one or two orders of magnitude more tokens than it saw in training [[S-1509]].\n\n## Evidence\n- **Inference outputs.** Rinberg et al. tested models from 3 to 30 billion parameters, including two mixture-of-experts models, and published code [[S-0015]]. On the 30-billion-parameter mixture-of-experts model, under benign prompt traffic, they report under 0.5% exfiltratable information at a false-positive rate below 0.01%, a slowdown of more than 200 times [[S-0015]].\n- **Independent attack.** Kezins tested six instruction-tuned models of 1 to 32 billion parameters over three seeds [[S-1507]]. Under benign prompts the slowdown was 146–254 times [[S-1507]]. Engineered prompts roughly doubled leakage per token and reduced the slowdown to 60–118 times [[S-1507]].\n- **Compressibility.** In a question-asking protocol, ten yes-or-no answers from a larger model recovered 23% to 72% of the capability gap between a small and a large model on standard benchmarks, at compression ratios of 0.0006 to 0.004 [[S-1508]].\n- **Facility-level architecture.** Petrie and Mühlhäuser publish it as protocol details, attack vectors and prototyping plans [[S-0019]].\n\n## Limitations\n- **Entropy inflation.** An attacker who controls prompts widens the set of tokens the verifier tolerates [[S-1507]]. Kezins argues that designs built on the same unexplained-information bound inherit this weakness, and recommends calibrating tolerances against local token entropy [[S-1507]].\n- **Noise tolerance.** The covert channel lives in the tolerance allowed for GPU nondeterminism [[S-1507]]. Bit-exact replay needs the hardware model, weights, parallelism layout, software versions and batch size [[S-0018]]; see [[M-0002]].\n- **Residual physical bandwidth.** One design argues that suppressing covert physical bandwidth below kilobits per second is much more achievable than aiming for zero [[S-0018]]. See [[M-0022]] and [[M-0014]].\n- **Uncertain volumes.** The upload-limit author is uncertain about the ratio of generated tokens to model parameters [[S-1509]]. In 2024 he estimated about a 25% chance that the approach can cheaply ensure that weights cannot be taken without bypassing physical security [[S-1509]].", "body_text": "How it works The idea is to measure the information that leaves a facility against what its declared work would produce [S-0019]. Petrie and Mühlhäuser propose verifying AI compute agreements by bounding the amount of unexplained information in network outputs, without either party trusting the other's hardware [S-0019]. - Isolation and commitment. The prover's compute nodes are isolated, and all traffic passes through a verifier interlock that records cryptographic commitments of the encrypted traffic [S-0019]. - Prediction challenges. From time to time the verifier challenges the prover to show that randomly selected outputs can be predicted by a policy-compliant computation on the measured inputs [S-0019]. - Principle. The better compliant computation predicts the outputs, the less bandwidth is left for exporting the results of undeclared workloads [S-0019]. - Confidentiality. The prover can recompute inside a verifier-controlled enclosure, the verifier can recompute inside a prover-controlled enclosure, or zero-knowledge proofs can be used [S-0019]. For language-model inference, Rinberg et al. make this concrete. Honest inference follows a well-defined generative process, so hiding data in responses must make them deviate from expected model behaviour [S-0015]. A verification server re-runs sampled requests with the trusted model and the shared random seed, scores how plausible each emitted token is, and bounds the rate at which hidden bits can pass unnoticed [S-0015]. See Sampled inference recomputation. A related line of work uses compression [S-1509] [S-1508]. One proposal compresses each response with a weaker reference model, using arithmetic coding, so that only bits the reference model did not predict count against an upload limit [S-1509]. Rinberg and co-authors note that limiting egress rests on a simple fact: if only 10 GB leaves a data centre, no attacker can steal more than 10 GB [S-1508]. Language-model text compresses far better than model weights, so a facility can apply stricter egress limits [S-1508]. What it establishes It can put an upper bound on the covert information in checked outputs, which slows exfiltration rather than stopping it [S-0015]. Under benign prompt traffic, Rinberg et al. report that the exfiltratable fraction fell to under 0.5%, not zero [S-0015]. One estimate puts the remaining channel at about 40 Mbit/s for a 200,000-GPU inference cluster at full load, assuming about 0.1 bits per token survive replay checks [S-1300]. It does not show that explained outputs are harmless, that no smaller model is being prepared, or anything about other channels: - The bound covers only unexplained information [S-0019]. Compression research notes that someone with inference access can extract more proprietary information per bit than naive transmission allows [S-1508]. - One upload-limit proposal lists running a distillation job inside the data centre, to produce a smaller model that is easier to move, as a residual route [S-1509]. Its author judges this hard to do without detection [S-1509]. - The inference scheme treats side channels as out of scope [S-0015]. Threat model - Facility-level design. Neither party trusts the other's hardware, and the prover's compute can be isolated behind the interlock [S-0019]. - Inference scheme. An insider controls the server's outputs, and outsiders may fully control the prompts [S-0015]. The scheme also assumes trusted verification code, secure logging, a private choice of which outputs to check that the adversary cannot manipulate, and randomness that is trusted and non-malleable [S-0015]. The reported bound was measured on benign prompts from a public chat dataset, and the authors call its size under adversarial prompts an open empirical question [S-0015]. An independent study has since measured it [S-1507]. - Upload limits. The proposal assumes strong physical security [S-1509]. It also expects that a lab's most powerful model will not generate more than one or two orders of magnitude more tokens than it saw in training [S-1509]. Evidence - Inference outputs. Rinberg et al. tested models from 3 to 30 billion parameters, including two mixture-of-experts models, and published code [S-0015]. On the 30-billion-parameter mixture-of-experts model, under benign prompt traffic, they report under 0.5% exfiltratable information at a false-positive rate below 0.01%, a slowdown of more than 200 times [S-0015]. - Independent attack. Kezins tested six instruction-tuned models of 1 to 32 billion parameters over three seeds [S-1507]. Under benign prompts the slowdown was 146–254 times [S-1507]. Engineered prompts roughly doubled leakage per token and reduced the slowdown to 60–118 times [S-1507]. - Compressibility. In a question-asking protocol, ten yes-or-no answers from a larger model recovered 23% to 72% of the capability gap between a small and a large model on standard benchmarks, at compression ratios of 0.0006 to 0.004 [S-1508]. - Facility-level architecture. Petrie and Mühlhäuser publish it as protocol details, attack vectors and prototyping plans [S-0019]. Limitations - Entropy inflation. An attacker who controls prompts widens the set of tokens the verifier tolerates [S-1507]. Kezins argues that designs built on the same unexplained-information bound inherit this weakness, and recommends calibrating tolerances against local token entropy [S-1507]. - Noise tolerance. The covert channel lives in the tolerance allowed for GPU nondeterminism [S-1507]. Bit-exact replay needs the hardware model, weights, parallelism layout, software versions and batch size [S-0018]; see Deterministic and bit-exact inference. - Residual physical bandwidth. One design argues that suppressing covert physical bandwidth below kilobits per second is much more achievable than aiming for zero [S-0018]. See Side-channel suppression for isolated facilities and Bandwidth limits and compartmentalization. - Uncertain volumes. The upload-limit author is uncertain about the ratio of generated tokens to model parameters [S-1509]. In 2024 he estimated about a 25% chance that the approach can cheaply ensure that weights cannot be taken without bypassing physical security [S-1509].", "referenced_by": [ { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0009", "title": "Recomputation", "url": "https://trustbutveri.fyi/concepts/recomputation/" }, { "id": "K-0022", "title": "Weight exfiltration", "url": "https://trustbutveri.fyi/concepts/weight-exfiltration/" } ] }, { "id": "M-0018", "slug": "chip-location-verification", "title": "Chip location verification", "aliases": [ "Delay-based location verification", "Ping-based location attestation", "Latency-based geolocation of chips" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Timing a chip's signed replies to trusted servers at known places, so that the speed of light bounds how far away the chip can be.", "summary": "Location verification aims to show that an AI chip is where its owner says, or at least not in a restricted region. Trusted \"landmark\" servers at known places exchange cryptographic challenges with the chip, which signs its replies with a key unique to it. Signals cannot travel faster than light, so a fast reply caps the chip's distance from each landmark. A slow reply does not show that a chip is far away, because internet routes add delay. The designers estimate under $1 million for firmware and software, plus $2.5–12.5 million a year for 100–500 landmarks. Ulyssean describes an H100 prototype on its demonstration website. NVIDIA is reportedly developing a version that uses its own servers. The main weaknesses are extraction of a chip's key, manipulated network delay and compromised landmarks. The check says nothing about chips in transit or how they are used.", "technical": "Brass and Aarne give the distance bound as D = T × 200, where D is the maximum distance in km and T the one-way delay in ms, taking about 200,000 km/s as the speed of light in optical fibre [[S-1400]]. In their case study, a 4.665 ms one-way delay bounds the chip to within 933 km of the landmark [[S-1400]]. They note that a limit based on the vacuum speed of light, about 300,000 km/s, covers worst cases such as out-of-band radio or satellite links and cannot be beaten physically [[S-1400]]. The cost is more false negatives: honest chips near a landmark may not reply fast enough, which the authors' moderate experimentation suggested could happen \"perhaps even in more than 50% of cases\" [[S-1400]]. Calibrating landmark delay factors might reduce this rate but needs further work [[S-1400]]. Recent geolocation algorithms reach a median error under 10 km given excellent landmark distribution and coverage; delay-based methods in general give about 10 km to 1,000 km, depending on the algorithm [[S-1400]].\n\nThe Sovereignty Certificates draft ([[I-0009]]) casts the protocol as a RATS (RFC 9334) attestation flow with Entity Attestation Token (RFC 9711) claims [[S-1404]]. The verifier issues a nonce and a signed directory of anchors. The attester probes anchors from inside a TEE, and each anchor returns a signed receipt with a high-precision timestamp and a nonce. The verifier then runs a deterministic multilateration to compute a feasible region [[S-1404]]. Per-cycle ephemeral keys are bound into the hardware root of trust's attestation quote to prevent replay [[S-1404]].\n\nTee and Happel propose identifying a GPU by a hardware fingerprint instead of an extractable on-chip key. Their proof of concept races atomic operations across streaming multiprocessors [[S-1403]]. Over 480 runs on 24 NVIDIA H200 GPUs rented from a cloud provider, re-identification accuracy was 98.8% from a single run and 100% from paired runs, with each run taking about 2.9 s [[S-1403]]. The authors state that the fingerprint still has to be validated on more GPUs, shown to be stable over time and conditions, and shown to resist an adversary who runs the function faster or simulates the GPU [[S-1403]].", "category": "accounting-provenance", "secondary_categories": [ "on-chip" ], "verifies": [ { "claim": "C-0002", "role": "primary", "note": "Bounds how far a responding chip can be from trusted landmark servers at the time of the check." }, { "claim": "C-0010", "role": "supporting", "note": "Can flag enrolled chips that stop responding or answer from outside declared regions; says nothing about chips outside the scheme." } ], "threat_model": "adversarial", "adversarial_evaluation": "analysis", "hardware_requirement": "existing-features", "prover_cooperation": "required", "confidentiality": "preserving", "depends_on": [ { "target": "M-0008", "note": "Binding a timed reply to one physical chip relies on a per-chip key held in secure hardware, as in remote attestation." } ], "readiness": { "assessment": true, "level": "R1", "scope": "bounding how far a chip is from trusted landmark servers when checked", "rubric_version": "1.1", "rationale": "The design is public and detailed, but the one reported prototype has a single published result, which cannot be checked.\n\n- **R1** met: Brass and Aarne give a full design with adversary classes, attacks and cost estimates [[S-1400]]. Aarne, Fist and Withers describe landmark-based verification [[S-0056]], and Avellar and Grunewald set out how a regulator could run it [[S-1402]].\n- **R2** not met: the IAPS brief reports a rudimentary H100 prototype and one result, a Singapore landmark bounding a chip in Singapore to within 300 miles [[S-1401]]. Ulyssean describes an H100 and AMD SEV-SNP setup with 38 cloud landmarks and a timing method, but publishes no code or systematic end-to-end location results or error rates [[S-3570]]. The single result cannot be reproduced from the available information. NVIDIA's delay-based scheme, reported by Avellar and Grunewald [[S-1402]], has no published design or results. Tee and Happel publish reproducible results for GPU fingerprinting, which is a component, not end-to-end location verification [[S-1403]]. The most mature implementation for this use, the draft Sovereignty Certificates specification ([[I-0009]]), is itself R1 [[S-1404]].\n\nConfidence is medium: the demo describes its setup and timing method, but the code is not public and its single location result lacks the data needed for independent reproduction [[S-3570]] [[S-1401]].", "evidence": [ "S-1400", "S-1401", "S-3570", "S-1402", "S-1404", "S-1403", "S-0056" ], "next_level_gaps": [ "A public implementation, or reproducible end-to-end results, on data-centre accelerators with a real landmark network.", "Published measurements of false-positive and false-negative rates under realistic internet routing.", "An evaluation against a stated adversary covering delay manipulation, faster network paths, landmark compromise and key extraction." ], "confidence": "medium", "assessed_by": [ "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Extracting a chip's key lets another device answer for it", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Ping-based protocols rely on cryptographic keys stored on the chip. Tee and Happel argue that an adversary with physical access could extract these keys and so compromise location verification [[S-1403]]. They propose GPU fingerprints as a mitigation, so far tested on 24 GPUs [[S-1403]]. Brass and Aarne assume the keys are stored securely, for example in a TPM [[S-1400]].", "sources": [ "S-1403", "S-1400" ], "response": null }, { "assessment": true, "title": "Added delay can shift an estimated position", "kind": "demonstrated-attack", "severity": "significant", "status": "open", "description": "Brass and Aarne cite internet-geolocation research in which artificially increased round-trip times moved the estimated location by up to 1,000 km, with a 74% chance of avoiding detection [[S-1400]]. Avellar and Grunewald list inflated ping times from circuitous routing as an evasion route [[S-1402]]. Added delay only loosens a distance bound, and Brass and Aarne propose a hard time limit as the counter: a chip that replies too slowly cannot be ruled out of a restricted location [[S-1400]].", "sources": [ "S-1400", "S-1402" ], "response": null }, { "assessment": true, "title": "Faster-than-assumed network paths", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Brass and Aarne list dark fibre and other private high-speed interconnects as ways to lower measured delays artificially. They judge that leasing dark fibre would probably not be a considerable challenge for covertly or openly adversarial actors [[S-1400]]. Avellar and Grunewald note that this can make a chip appear to be somewhere else entirely [[S-1402]]. A limit set at the vacuum speed of light cannot be beaten, but it makes honest chips fail more often [[S-1400]].", "sources": [ "S-1400", "S-1402" ], "response": null }, { "assessment": true, "title": "Compromised landmarks can falsify measurements", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "A party that controls landmark servers can report false timing. Brass and Aarne cite research in which manipulating a third of the landmarks shifted the estimated location by about 700 km [[S-1400]]. Avellar and Grunewald note that compromised landmarks let adversaries spoof travel-time measurements directly [[S-1402]]. The draft specification asks verifiers to require anchors in diverse places, run by several independent operators [[S-1404]].", "sources": [ "S-1400", "S-1402", "S-1404" ], "response": null } ], "blockers": [ { "text": "No public code or reproducible end-to-end location results are available for the reported H100 prototype.", "theme": "adversarial-validation", "blocked_by": null, "sources": [ "S-1401", "S-3570" ] }, { "text": "Per-chip keys must be provisioned and protected against extraction; hardware-integrated, tamper-resistant versions still need R&D.", "theme": "hardware-trust", "blocked_by": null, "sources": [ "S-1400", "S-1403", "S-0007" ] }, { "text": "The time limit forces a trade-off: a limit at the speed of light in fibre can be beaten by faster links, while one at the vacuum speed of light makes honest chips fail often.", "theme": "protocol-soundness", "blocked_by": null, "sources": [ "S-1400" ] }, { "text": "A trusted landmark network must be built and secured, and who should operate it, under what oversight, is unsettled.", "theme": "access-governance", "blocked_by": null, "sources": [ "S-1400", "S-1402" ] } ], "challenge_themes": [ "hardware-trust", "protocol-soundness", "adversarial-validation", "coverage-hidden-compute", "access-governance" ], "organizations": [ "O-0180", "O-0204", "O-0207", "O-0140" ], "people": [], "sources": [ { "source": "S-1400", "supports": "design, problem framing, adversary classes, distance bound, fibre versus vacuum speed limits and false negatives, precision, attacks and hard time limits, costs, key storage", "locator": "Detailed Summary; Solution requirements and threat models; Delay-based methods sections; Three adversarial strategies; Proposed Solution Requirements" }, { "source": "S-1401", "supports": "reported H100 prototype (builder not named) and its single Singapore result; summary of Brass and Aarne's 2024 report; development and landmark network cost estimates", "locator": "issue brief, pp. 1-2" }, { "source": "S-3570", "supports": "Ulyssean's account of the H100 and AMD SEV-SNP demonstration, 38 landmarks, timing method, code status", "locator": "Technical details, public JavaScript asset" }, { "source": "S-1402", "supports": "regulator workflow; maturity and effectiveness ratings; costs; transit gap; evasion and landmark compromise; NVIDIA confirmed developing delay-based verification with NVIDIA-run servers (citing Reuters, December 2025)", "locator": "§1.6; Executive Summary" }, { "source": "S-1413", "supports": "NVIDIA's opt-in, customer-installed fleet-management service with read-only telemetry and an agent to be open-sourced; NVIDIA's statement that its GPUs lack hardware tracking, kill switches and backdoors (provider self-description)", "locator": "blog post" }, { "source": "S-3180", "supports": "Fleet Intelligence (May 2026): read-only agent released as open source; telemetry and GPU integrity attestation, no location check described (provider self-description)", "locator": "blog post" }, { "source": "S-0056", "supports": "speed-of-light upper bound; landmark-server illustration; hundreds of landmarks", "locator": "'Location Verification', p. 11" }, { "source": "S-0007", "supports": "feasibility rating; physical attacks on embedded mechanisms", "locator": "§3.1 (M6); §4.3" }, { "source": "S-1403", "supports": "key-extraction weakness; fingerprinting proof of concept and its stated limitations", "locator": "Abstract; threat model; results; limitations" }, { "source": "S-1404", "supports": "draft protocol structure, threat model, anchor diversity and tunnelling check", "locator": "§0.3, §4.2, §6.3, §8.1, §8.3.3, §8.3.4" } ], "concepts": [ "K-0004", "K-0005", "K-0006", "K-0007", "K-0018" ], "complements": [ "M-0019", "M-0017" ], "alternatives": [], "type": "mechanism", "implementations": [ { "id": "I-0009", "title": "Lucid sovereignty (location) certificates", "url": "https://trustbutveri.fyi/implementations/lucid-location-certificates/" } ], "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/", "source_file": "content/mechanisms/chip-location-verification.md", "flags_all": [], "body_markdown": "## How it works\n\nChip location verification tries to establish where an AI chip physically is [[S-1400]]. Brass and Aarne judge delay-based methods, also called ping-based or latency-based verification, the most promising approach [[S-1400]]. A trusted \"landmark\" server at a known location sends the chip a cryptographic challenge, and the chip replies using a secret key unique to it [[S-1400]]. Signals cannot travel faster than light, so the reply time gives an upper bound on the chip's distance from the landmark [[S-1400]] [[S-0056]]. Measurements from several landmarks narrow the region where the chip can be [[S-1404]].\n\nBrass and Aarne separate a general problem, locating a chip anywhere, from a narrower anti-smuggling problem: checking only that a chip is not in a restricted location [[S-1400]]. They recommend solving the narrower problem first [[S-1400]]. Aarne, Fist and Withers illustrate it with a trusted server in Paris [[S-0056]]. A reply in under 9 ms would show that a chip lies inside a circle that excludes countries to which exports are restricted [[S-0056]].\n\nEstimates of the landmark network differ:\n\n- Brass and Aarne's 2024 report suggests several dozen landmarks at most [[S-1400]].\n- An IAPS issue brief from 2025, summarising that report, assumes 100 to 500, depending on which countries are included [[S-1401]].\n- Aarne, Fist and Withers expect hundreds worldwide [[S-0056]].\n\nFor export control, Avellar and Grunewald place landmark servers in or near major data-centre hubs [[S-1402]]. The regulator would periodically ask exporters or end users to have chips ping the nearest landmark [[S-1402]].\n\nThe chip's key must stay secret [[S-1400]]. Brass and Aarne write that per-chip keys would need secure storage, such as a TPM, and that the trusted execution environment on NVIDIA H100 GPUs \"could likely be used to implement location attestation\" [[S-1400]]. A draft industry specification, [[I-0009]], packages the approach as a remote-attestation protocol that issues short-lived location certificates [[S-1404]]. In it, an attester inside a trusted execution environment probes anchor servers and the verifier computes the feasible region [[S-1404]]. Related timing-based checks appear in [[M-0016]].\n\n## What it establishes\nWhat a timed reply can show:\n\n- A reply with one-way delay T ms shows that a device holding the chip's key was at most T × 200 km from the landmark at that moment, if signals travel no faster than light in fibre [[S-1400]].\n- For the anti-smuggling problem, a fast enough reply to a landmark outside restricted regions shows that the chip is not in those regions [[S-0056]] [[S-1400]].\n- Detours and tunnels add latency, so they generally cannot make a device appear closer to a landmark than it is [[S-1404]]. A path faster than the signal speed the verifier assumes can [[S-1400]].\n\nWhat it cannot show:\n\n- **Distance from a slow reply.** Internet traffic does not travel in straight lines, so an honest chip near a landmark may reply too slowly to be cleared [[S-1400]]. Under a hard time limit, a slow reply only means the chip cannot be ruled out of a restricted location [[S-1400]].\n- **A precise position.** Delay-based methods give positions to within about 10 km to 1,000 km, depending on the algorithm [[S-1400]].\n- **The chip itself.** The check binds the reply to whatever holds the key; an extracted key lets another device answer [[S-1403]].\n- **Location between checks.** It shows nothing about where chips were between checks, and gives little visibility during shipping and warehousing [[S-1402]].\n- **Who uses the chips, or for what.** Avellar and Grunewald treat end-location, end-user and end-use verification as separate problems [[S-1402]].\n\nBrass and Aarne suggest combining location checks with a centralized chip registry ([[M-0019]]) [[S-1400]].\n\n## Threat model\nBrass and Aarne define three adversary classes [[S-1400]]:\n\n- **Minimally adversarial** actors spend little and are very averse to discovery.\n- **Covertly adversarial** actors spend substantial resources but still want to avoid being caught.\n- **Openly adversarial** actors spend heavily and do not mind being discovered.\n\nThey argue that even an imperfect scheme has value, because it can occasionally catch cheaters and deter minimally adversarial actors [[S-1400]].\n\nThe scheme assumes the following:\n\n- **Chip keys stay secret.** Each chip's key is provisioned and stored so that the owner cannot extract it [[S-1400]]. Tee and Happel question this against adversaries with physical access [[S-1403]].\n- **Signals are no faster than assumed.** The bound holds only if no path beats the signal speed the verifier uses: the speed of light in fibre, or in a vacuum for a limit that cannot be beaten [[S-1400]].\n- **Landmarks are trustworthy.** Landmarks are honest and correctly located [[S-1400]]. The draft specification requires a signed directory of anchors and keys protected in hardware security modules [[S-1404]].\n- **The chip hardware is trusted.** The draft specification lets the attacker control the network and the host's root account, but trusts the hardware root of trust and TEE, and leaves sophisticated physical attacks as a residual risk [[S-1404]].\n\n## Evidence\n- **Prototype.** An IAPS issue brief from May 2025, which summarises Brass and Aarne's 2024 report, states that a rudimentary version has been prototyped on NVIDIA H100 chips [[S-1401]]. It shows one result, a landmark in Singapore bounding a chip in Singapore to within 300 miles [[S-1401]]. Ulyssean says it built the demonstration with an H100, an AMD SEV-SNP CPU and 38 Google Cloud Run landmarks. Its site describes a toy GPU task, five requests per landmark and shortest-response timing, but gives no end-to-end location result series; it says the code will be released after cleanup [[S-3570]].\n- **NVIDIA.** Avellar and Grunewald report, citing Reuters reporting from December 2025, that NVIDIA has confirmed it is developing location verification that estimates a chip's location from communication delays with NVIDIA-run servers [[S-1402]]. NVIDIA's own announcement from that month describes an opt-in fleet-management service that customers install and that reports read-only telemetry, with a client agent slated to be open-sourced [[S-1413]]. The announcement states that NVIDIA GPUs do not have hardware tracking technology, kill switches or backdoors [[S-1413]]. In May 2026 NVIDIA announced Fleet Intelligence, a fleet-management service whose read-only agent it released as open source. NVIDIA reports that the service collects GPU telemetry and verifies GPU integrity, and its announcement describes no location check [[S-3180]].\n- **Costs.** The 2024 report estimates that a solution would cost less than $1 million to set up and maintain for several years [[S-1400]]. The 2025 brief puts the chip firmware and software update at under $1 million, and 100 to 500 landmarks at $25,000 each a year, or $2.5 million to $12.5 million a year [[S-1401]]. Avellar and Grunewald repeat the brief's figures [[S-1402]].\n- **Ratings.** Ansari rates software-based delay methods as near-term and hardware-integrated, tamper-resistant versions as needing R&D [[S-0007]]. Avellar and Grunewald rate delay-based location verification as novel in maturity, high in effectiveness and not invasive [[S-1402]].\n- **Component result.** Tee and Happel's GPU fingerprinting re-identified 24 rented H200 GPUs with 98.8% accuracy from a single run [[S-1403]]. It tests chip identification, not location [[S-1403]].\n- **Specification.** The Sovereignty Certificates specification ([[I-0009]]) is a draft, version 0.1.0, dated 2025-10-21 [[S-1404]]. As of September 2026 its repository holds no reference implementation [[S-1404]].\n\n## Limitations\n- **Delay inflation.** Added delay moved estimated positions by up to 1,000 km in research Brass and Aarne cite; they propose a hard time limit as the counter [[S-1400]].\n- **Faster paths.** Dark fibre and other private high-speed links can lower measured delays artificially [[S-1400]] [[S-1402]].\n- **False negatives.** A limit set at the vacuum speed of light cannot be beaten, but honest chips may often fail it [[S-1400]].\n- **Landmark compromise.** Manipulating a third of the landmarks shifted estimates by about 700 km in cited research [[S-1400]], and compromised landmarks can spoof measurements directly [[S-1402]].\n- **Key extraction.** An extracted key breaks the link between reply and chip [[S-1403]]. Ansari lists glitching and focused-ion-beam editing among physical attacks on embedded mechanisms [[S-0007]].\n- **Tunnelling.** The draft specification argues that tunnels add latency and produce inconsistent geometry, and requires rejecting such results [[S-1404]].\n- **Coverage gaps.** Chips in transit or storage cannot be checked [[S-1402]].", "body_text": "How it works Chip location verification tries to establish where an AI chip physically is [S-1400]. Brass and Aarne judge delay-based methods, also called ping-based or latency-based verification, the most promising approach [S-1400]. A trusted \"landmark\" server at a known location sends the chip a cryptographic challenge, and the chip replies using a secret key unique to it [S-1400]. Signals cannot travel faster than light, so the reply time gives an upper bound on the chip's distance from the landmark [S-1400] [S-0056]. Measurements from several landmarks narrow the region where the chip can be [S-1404]. Brass and Aarne separate a general problem, locating a chip anywhere, from a narrower anti-smuggling problem: checking only that a chip is not in a restricted location [S-1400]. They recommend solving the narrower problem first [S-1400]. Aarne, Fist and Withers illustrate it with a trusted server in Paris [S-0056]. A reply in under 9 ms would show that a chip lies inside a circle that excludes countries to which exports are restricted [S-0056]. Estimates of the landmark network differ: - Brass and Aarne's 2024 report suggests several dozen landmarks at most [S-1400]. - An IAPS issue brief from 2025, summarising that report, assumes 100 to 500, depending on which countries are included [S-1401]. - Aarne, Fist and Withers expect hundreds worldwide [S-0056]. For export control, Avellar and Grunewald place landmark servers in or near major data-centre hubs [S-1402]. The regulator would periodically ask exporters or end users to have chips ping the nearest landmark [S-1402]. The chip's key must stay secret [S-1400]. Brass and Aarne write that per-chip keys would need secure storage, such as a TPM, and that the trusted execution environment on NVIDIA H100 GPUs \"could likely be used to implement location attestation\" [S-1400]. A draft industry specification, Lucid sovereignty (location) certificates, packages the approach as a remote-attestation protocol that issues short-lived location certificates [S-1404]. In it, an attester inside a trusted execution environment probes anchor servers and the verifier computes the feasible region [S-1404]. Related timing-based checks appear in Timed challenge-response and memory-occupation challenges. What it establishes What a timed reply can show: - A reply with one-way delay T ms shows that a device holding the chip's key was at most T × 200 km from the landmark at that moment, if signals travel no faster than light in fibre [S-1400]. - For the anti-smuggling problem, a fast enough reply to a landmark outside restricted regions shows that the chip is not in those regions [S-0056] [S-1400]. - Detours and tunnels add latency, so they generally cannot make a device appear closer to a landmark than it is [S-1404]. A path faster than the signal speed the verifier assumes can [S-1400]. What it cannot show: - Distance from a slow reply. Internet traffic does not travel in straight lines, so an honest chip near a landmark may reply too slowly to be cleared [S-1400]. Under a hard time limit, a slow reply only means the chip cannot be ruled out of a restricted location [S-1400]. - A precise position. Delay-based methods give positions to within about 10 km to 1,000 km, depending on the algorithm [S-1400]. - The chip itself. The check binds the reply to whatever holds the key; an extracted key lets another device answer [S-1403]. - Location between checks. It shows nothing about where chips were between checks, and gives little visibility during shipping and warehousing [S-1402]. - Who uses the chips, or for what. Avellar and Grunewald treat end-location, end-user and end-use verification as separate problems [S-1402]. Brass and Aarne suggest combining location checks with a centralized chip registry (Chip registries and manufacturing records) [S-1400]. Threat model Brass and Aarne define three adversary classes [S-1400]: - Minimally adversarial actors spend little and are very averse to discovery. - Covertly adversarial actors spend substantial resources but still want to avoid being caught. - Openly adversarial actors spend heavily and do not mind being discovered. They argue that even an imperfect scheme has value, because it can occasionally catch cheaters and deter minimally adversarial actors [S-1400]. The scheme assumes the following: - Chip keys stay secret. Each chip's key is provisioned and stored so that the owner cannot extract it [S-1400]. Tee and Happel question this against adversaries with physical access [S-1403]. - Signals are no faster than assumed. The bound holds only if no path beats the signal speed the verifier uses: the speed of light in fibre, or in a vacuum for a limit that cannot be beaten [S-1400]. - Landmarks are trustworthy. Landmarks are honest and correctly located [S-1400]. The draft specification requires a signed directory of anchors and keys protected in hardware security modules [S-1404]. - The chip hardware is trusted. The draft specification lets the attacker control the network and the host's root account, but trusts the hardware root of trust and TEE, and leaves sophisticated physical attacks as a residual risk [S-1404]. Evidence - Prototype. An IAPS issue brief from May 2025, which summarises Brass and Aarne's 2024 report, states that a rudimentary version has been prototyped on NVIDIA H100 chips [S-1401]. It shows one result, a landmark in Singapore bounding a chip in Singapore to within 300 miles [S-1401]. Ulyssean says it built the demonstration with an H100, an AMD SEV-SNP CPU and 38 Google Cloud Run landmarks. Its site describes a toy GPU task, five requests per landmark and shortest-response timing, but gives no end-to-end location result series; it says the code will be released after cleanup [S-3570]. - NVIDIA. Avellar and Grunewald report, citing Reuters reporting from December 2025, that NVIDIA has confirmed it is developing location verification that estimates a chip's location from communication delays with NVIDIA-run servers [S-1402]. NVIDIA's own announcement from that month describes an opt-in fleet-management service that customers install and that reports read-only telemetry, with a client agent slated to be open-sourced [S-1413]. The announcement states that NVIDIA GPUs do not have hardware tracking technology, kill switches or backdoors [S-1413]. In May 2026 NVIDIA announced Fleet Intelligence, a fleet-management service whose read-only agent it released as open source. NVIDIA reports that the service collects GPU telemetry and verifies GPU integrity, and its announcement describes no location check [S-3180]. - Costs. The 2024 report estimates that a solution would cost less than $1 million to set up and maintain for several years [S-1400]. The 2025 brief puts the chip firmware and software update at under $1 million, and 100 to 500 landmarks at $25,000 each a year, or $2.5 million to $12.5 million a year [S-1401]. Avellar and Grunewald repeat the brief's figures [S-1402]. - Ratings. Ansari rates software-based delay methods as near-term and hardware-integrated, tamper-resistant versions as needing R&D [S-0007]. Avellar and Grunewald rate delay-based location verification as novel in maturity, high in effectiveness and not invasive [S-1402]. - Component result. Tee and Happel's GPU fingerprinting re-identified 24 rented H200 GPUs with 98.8% accuracy from a single run [S-1403]. It tests chip identification, not location [S-1403]. - Specification. The Sovereignty Certificates specification (Lucid sovereignty (location) certificates) is a draft, version 0.1.0, dated 2025-10-21 [S-1404]. As of September 2026 its repository holds no reference implementation [S-1404]. Limitations - Delay inflation. Added delay moved estimated positions by up to 1,000 km in research Brass and Aarne cite; they propose a hard time limit as the counter [S-1400]. - Faster paths. Dark fibre and other private high-speed links can lower measured delays artificially [S-1400] [S-1402]. - False negatives. A limit set at the vacuum speed of light cannot be beaten, but honest chips may often fail it [S-1400]. - Landmark compromise. Manipulating a third of the landmarks shifted estimates by about 700 km in cited research [S-1400], and compromised landmarks can spoof measurements directly [S-1402]. - Key extraction. An extracted key breaks the link between reply and chip [S-1403]. Ansari lists glitching and focused-ion-beam editing among physical attacks on embedded mechanisms [S-0007]. - Tunnelling. The draft specification argues that tunnels add latency and produce inconsistent geometry, and requires rejecting such results [S-1404]. - Coverage gaps. Chips in transit or storage cannot be checked [S-1402].", "referenced_by": [ { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/" }, { "id": "I-0009", "title": "Lucid sovereignty (location) certificates", "url": "https://trustbutveri.fyi/implementations/lucid-location-certificates/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "K-0007", "title": "Hardware-enabled mechanism (HEM)", "url": "https://trustbutveri.fyi/concepts/hardware-enabled-mechanism/" }, { "id": "K-0004", "title": "Remote attestation", "url": "https://trustbutveri.fyi/concepts/remote-attestation/" }, { "id": "O-0207", "title": "Center for a New American Security", "url": "https://trustbutveri.fyi/organizations/center-for-a-new-american-security/" }, { "id": "O-0204", "title": "Institute for AI Policy and Strategy", "url": "https://trustbutveri.fyi/organizations/institute-for-ai-policy-and-strategy/" }, { "id": "O-0180", "title": "Lucid Computing", "url": "https://trustbutveri.fyi/organizations/lucid-computing/" }, { "id": "O-0140", "title": "NVIDIA", "url": "https://trustbutveri.fyi/organizations/nvidia/" } ] }, { "id": "M-0019", "slug": "chip-registries-and-manufacturing-records", "title": "Chip registries and manufacturing records", "aliases": [ "AI chip registry", "Chain-of-custody tracking for AI chips", "Commitments to manufacturing records" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Recording each AI chip's identity and owner from the fab onwards, and cryptographically fixing manufacturing records, so that chips can be accounted for later.", "summary": "A chip registry records unique identifiers for AI chips, with their owners and sometimes their locations, from manufacture to destruction. A verifier can then sample chips and check that declared chains of custody match what is physically there. A related proposal protects the foundry's own record of chip IDs by publishing a cryptographic fingerprint of it, so that later changes would show. As of September 2026 these are published designs, and no AI chip registry or record commitment is in operation. The main obstacle is institutional: someone must run the registry, and covering re-exports needs cooperation from re-exporters and foreign governments. The main weaknesses are that records cover only chips that were recorded, that documents and serial numbers can be forged, and that a commitment cannot show the records were accurate when made. A commitment shows what was made, not where it went.", "category": "accounting-provenance", "secondary_categories": [ "cryptographic-computational" ], "verifies": [ { "claim": "C-0001", "role": "primary", "note": "Gives a baseline of which chips were made and who declared owning them." }, { "claim": "C-0010", "role": "supporting", "note": "Supports checks that recorded chips have not been assembled into undeclared clusters." }, { "claim": "C-0002", "role": "supporting", "note": "Records declared locations, which inspections or location checks can test." } ], "threat_model": "semi-trusted", "adversarial_evaluation": "analysis", "hardware_requirement": "existing-features", "prover_cooperation": "required", "confidentiality": "partial", "depends_on": [], "readiness": { "assessment": true, "level": "R1", "scope": "a checkable record of which chips were made and who declared owning them", "rubric_version": "1.1", "rationale": "Registry and record-commitment designs are public, but none has been built or tested.\n\n- **R1** met: Baker et al. describe an AI chip registry for ownership declarations, with sampled chain-of-custody checks from manufacture to destruction, and state the goal and assumptions [[S-0002]]. Avellar and Grunewald describe how a regulator could run a registry of chip ownership and use it for random return requests [[S-1402]]. Cankaya proposes cryptographic commitments to foundry chip-ID records [[S-1408]].\n- **R2** not met: as of September 2026 no AI chip registry or manufacturing-record commitment has a public implementation, and no end-to-end results have been published [[S-0002]] [[S-1402]] [[S-1408]]. Export documentation checks are established practice, but they are not a registry, and Avellar and Grunewald rate them low in effectiveness as a verification tool [[S-1402]]. Ansari rates registry systems as near-term, not deployable now [[S-0007]].", "evidence": [ "S-0002", "S-1402", "S-1408", "S-0007" ], "next_level_gaps": [ "A public pilot registry or published commitment to manufacturing records at realistic scale.", "End-to-end results for sampled chain-of-custody checks, including how hard-to-spoof IDs are read and matched.", "An adversarial evaluation of record falsification, forged serial numbers and unrecorded chips." ], "confidence": "medium", "assessed_by": [ "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Records cover only chips that were recorded", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "A registry or commitment accounts only for chips entered into it. Cankaya asks how a verifier would know it had found all chips, or how much \"dark compute\" remains, and notes that a fraudulent original record would mean unregistered chips had been made in advance [[S-1408]]. Halstead and Larsen propose reconstructing earlier production by auditing upstream suppliers [[S-1410]].", "sources": [ "S-1408", "S-1410" ], "response": null }, { "assessment": true, "title": "Documents and serial numbers can be forged", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Avellar and Grunewald note that export documents can be forged, that companies can hide information behind obscure corporate structures, and that it may be possible to forge serial numbers on chips and racks. They recommend cryptographic attestation of a powered-on chip as an extra check [[S-1402]].", "sources": [ "S-1402" ], "response": null }, { "assessment": true, "title": "Insiders could alter records before they are fixed", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Cankaya argues that insiders who can photograph process secrets could also tamper with production records [[S-1408]]. A commitment makes changes after publication detectable, but it cannot show that the records were accurate when committed [[S-1408]].", "sources": [ "S-1408" ], "response": null } ], "blockers": [ { "text": "No AI chip registry operates, and covering re-exports would need cooperation from re-exporters and foreign governments that may not be feasible everywhere.", "theme": "access-governance", "blocked_by": null, "sources": [ "S-1402", "S-0002" ] }, { "text": "Linking records to physical chips needs hard-to-spoof unique IDs and inspections.", "theme": "hardware-trust", "blocked_by": null, "sources": [ "S-0002", "S-1402", "S-1408" ] }, { "text": "Chips produced before a registry starts must be reconstructed from supplier records.", "theme": "coverage-hidden-compute", "blocked_by": null, "sources": [ "S-1408", "S-1410" ] } ], "challenge_themes": [ "coverage-hidden-compute", "hardware-trust", "evidence-binding", "access-governance" ], "organizations": [ "O-0200", "O-0201", "O-0204" ], "people": [], "sources": [ { "source": "S-0002", "supports": "AI chip registry concept; sampled chain-of-custody checks; subgoal on undeclared clusters", "locator": "§3.2 and its footnote on the AI chip registry; §4.2.1.2" }, { "source": "S-1402", "supports": "centralized ownership registry; random return requests; serial checks; forgery limits; effectiveness ratings", "locator": "§1.1, §1.2, §1.5" }, { "source": "S-1408", "supports": "ECID golden record; Merkle-root commitment; TAP readout; eFuse tamper evidence; dark compute; insider threat", "locator": "whole post" }, { "source": "S-0053", "supports": "international AI chip registry listed among visibility mechanisms", "locator": "§4.A Visibility" }, { "source": "S-0056", "supports": "on-chip mechanisms need ownership tracking, supply-chain tracking and KYC", "locator": "'What Would Effective On-Chip Governance Look Like?', pp. 9-10" }, { "source": "S-0007", "supports": "chain-of-custody framing; PUFs; blockchain serialization; customs data; manufacturing concentration; feasibility", "locator": "§3.1 (M7)" }, { "source": "S-1410", "supports": "auditing upstream supply chain to reconstruct production", "locator": "section on preventing compute acquisition" }, { "source": "S-1400", "supports": "recommendation to combine location verification with a registry", "locator": "Detailed Summary" }, { "source": "S-3382", "supports": "recommendation to report exports, re-exports and ownership transfers of controlled AI chips", "locator": "recommendation 2" }, { "source": "S-3383", "supports": "December 2025 prosecution alleging removal and replacement of GPU labels before export", "locator": "criminal complaint summary" } ], "concepts": [ "K-0024", "K-0016", "K-0020", "K-0015" ], "complements": [ "M-0018", "M-0020", "M-0017" ], "alternatives": [], "type": "mechanism", "implementations": [], "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/", "source_file": "content/mechanisms/chip-registries-and-manufacturing-records.md", "flags_all": [], "body_markdown": "## How it works\n\nA chip registry records unique identifiers for AI chips, together with their owners and, in some designs, their locations [[S-0002]]. The aim is to account for chips over their lifetime [[S-0002]] [[S-0007]]. Sastry et al. list an international AI chip registry among possible mechanisms for regulatory visibility into compute [[S-0053]]. Baker et al. describe an AI chip registry as one possible way to report who owns AI compute [[S-0002]]. They note that declaring individual chips is not strictly necessary but makes verification easier [[S-0002]]. A verifier could check the locations and owners of random samples of chips, from manufacture to destruction, using inspections, possibly video cameras, and hard-to-spoof unique IDs [[S-0002]].\n\nAnsari describes lifecycle registries as a chain-of-custody layer similar to nuclear material accountancy [[S-0007]]. He cites proposals to identify chips with physical unclonable functions and to serialize them on a blockchain [[S-0007]]. Aarne, Fist and Withers write that on-chip governance mechanisms would need a way to track who owns data-centre AI chips, supported by supply-chain tracking and Know Your Customer policies [[S-0056]].\nGrunewald and Fist recommend notifying the regulator of exports, re-exports and ownership transfers of controlled AI chips [[S-3382]].\n\nIn an export-control setting, Avellar and Grunewald describe a centralized registry of chip ownership [[S-1402]]. The regulator could pick random chip IDs from it and ask the owners to return those chips at short notice [[S-1402]]. During on-site inspections, inspectors would check the serial numbers of a random sample of chips against asset inventories and the records reported to the regulator [[S-1402]].\n\n**Manufacturing records.** Cankaya argues that the foundry already holds a \"golden record\" of the AI chips it has made [[S-1408]]. He writes that modern processors carry Electronic Chip IDs (ECIDs), burned into one-time-programmable fuses at wafer test, that encode the wafer lot, the wafer number and the die position [[S-1408]]. He proposes that the foundry build a Merkle tree over these IDs and publish only its root hash after each batch [[S-1408]]. Later tampering would then be detectable, while the IDs themselves stay private [[S-1408]]. An inspector could later read a chip's ECID through its test access port without opening the package [[S-1408]].\n\n**Upstream audits.** Halstead and Larsen propose auditing the semiconductor supply chain upstream, across logic fabrication, memory and packaging suppliers [[S-1410]]. This would give a record of past production that is hard to under-report without creating inconsistencies [[S-1410]].\n\n## What it establishes\nWhat these tools can show:\n\n- A registry, with sampled inspections, tests whether declared chains of custody match what is physically present [[S-0002]] [[S-1402]].\n- For recorded chips, a registry supports the wider goal of checking that large quantities of chips have not been assembled into undeclared clusters [[S-0002]].\n- A published commitment makes later changes to foundry records detectable [[S-1408]].\n\nWhat they cannot show:\n\n- **Unrecorded chips.** Chips that were never recorded are not covered [[S-1408]].\n- **Where chips went.** Cankaya notes that a commitment shows what was made, and that \"the harder question is where it went\" [[S-1408]].\n- **Accuracy at the start.** A commitment cannot show that the records were correct when committed [[S-1408]].\n- **Strong assurance from documents alone.** Avellar and Grunewald rate export documentation checks low in effectiveness and useful mainly against actors of low to moderate sophistication [[S-1402]].\n\nBrass and Aarne suggest pairing a registry with location verification ([[M-0018]]) [[S-1400]].\n\n## Threat model\nThese designs trust some parties:\n\n- **Owners** declare truthfully, or are caught by sampled inspections [[S-0002]].\n- **Foundries** keep accurate ID records; Cankaya's proposal assumes the foundry holds the ECID data [[S-1408]].\n- **IDs** are hard to alter [[S-0002]]. Cankaya argues that an attacker could corrupt an ECID by blowing more of its one-time fuses, but could not set an arbitrary value without focused-ion-beam work [[S-1408]]. He adds that analysis labs can check whether a die has been re-assembled or re-packaged [[S-1408]].\n\nThe main threats named in the sources are forged documents and serial numbers [[S-1402]], opaque corporate structures [[S-1402]] and insiders altering production records [[S-1408]]. Ansari argues that the concentration of advanced manufacturing, with fewer than two dozen facilities making sub-14 nm chips as of 2023, makes a registry easier to enforce than for a commodity with distributed production [[S-0007]].\n\n## Evidence\n- **Designs only.** Baker et al., Avellar and Grunewald, and Cankaya publish designs; none reports an implementation [[S-0002]] [[S-1402]] [[S-1408]].\n- **Feasibility ratings.** Ansari rates registry systems as near-term [[S-0007]]. He rates trade-data analysis as deployable now, citing work that uses existing customs data to find circumvention patterns [[S-0007]].\n- **Export-control ratings.** Avellar and Grunewald rate export documentation checks as relatively established but low in effectiveness, and random return requests from a registry as novel and high in effectiveness [[S-1402]].\n\n## Limitations\n- **Unrecorded chips.** Chips outside the record are not covered, and a fraudulent original record would mean unregistered chips were made in advance [[S-1408]]. Reconstructing earlier production depends on supplier records [[S-1410]].\n- **Forgery.** Documents can be forged, and serial numbers on chips and racks may be forgeable [[S-1402]]. In a December 2025 prosecution, the US Justice Department alleged that workers replaced GPU labels with a fictitious brand before export or attempted export [[S-3383]].\n- **Insider tampering.** Records can be altered before they are committed [[S-1408]].\n- **Limited reach.** Covering re-exports would need cooperation from re-exporters and foreign governments, which may not be feasible everywhere [[S-1402]].\n- **Custody versus location.** A commitment fixes what was made, but tracking where chips are needs inspections or location checks [[S-1408]] [[S-0002]].", "body_text": "How it works A chip registry records unique identifiers for AI chips, together with their owners and, in some designs, their locations [S-0002]. The aim is to account for chips over their lifetime [S-0002] [S-0007]. Sastry et al. list an international AI chip registry among possible mechanisms for regulatory visibility into compute [S-0053]. Baker et al. describe an AI chip registry as one possible way to report who owns AI compute [S-0002]. They note that declaring individual chips is not strictly necessary but makes verification easier [S-0002]. A verifier could check the locations and owners of random samples of chips, from manufacture to destruction, using inspections, possibly video cameras, and hard-to-spoof unique IDs [S-0002]. Ansari describes lifecycle registries as a chain-of-custody layer similar to nuclear material accountancy [S-0007]. He cites proposals to identify chips with physical unclonable functions and to serialize them on a blockchain [S-0007]. Aarne, Fist and Withers write that on-chip governance mechanisms would need a way to track who owns data-centre AI chips, supported by supply-chain tracking and Know Your Customer policies [S-0056]. Grunewald and Fist recommend notifying the regulator of exports, re-exports and ownership transfers of controlled AI chips [S-3382]. In an export-control setting, Avellar and Grunewald describe a centralized registry of chip ownership [S-1402]. The regulator could pick random chip IDs from it and ask the owners to return those chips at short notice [S-1402]. During on-site inspections, inspectors would check the serial numbers of a random sample of chips against asset inventories and the records reported to the regulator [S-1402]. Manufacturing records. Cankaya argues that the foundry already holds a \"golden record\" of the AI chips it has made [S-1408]. He writes that modern processors carry Electronic Chip IDs (ECIDs), burned into one-time-programmable fuses at wafer test, that encode the wafer lot, the wafer number and the die position [S-1408]. He proposes that the foundry build a Merkle tree over these IDs and publish only its root hash after each batch [S-1408]. Later tampering would then be detectable, while the IDs themselves stay private [S-1408]. An inspector could later read a chip's ECID through its test access port without opening the package [S-1408]. Upstream audits. Halstead and Larsen propose auditing the semiconductor supply chain upstream, across logic fabrication, memory and packaging suppliers [S-1410]. This would give a record of past production that is hard to under-report without creating inconsistencies [S-1410]. What it establishes What these tools can show: - A registry, with sampled inspections, tests whether declared chains of custody match what is physically present [S-0002] [S-1402]. - For recorded chips, a registry supports the wider goal of checking that large quantities of chips have not been assembled into undeclared clusters [S-0002]. - A published commitment makes later changes to foundry records detectable [S-1408]. What they cannot show: - Unrecorded chips. Chips that were never recorded are not covered [S-1408]. - Where chips went. Cankaya notes that a commitment shows what was made, and that \"the harder question is where it went\" [S-1408]. - Accuracy at the start. A commitment cannot show that the records were correct when committed [S-1408]. - Strong assurance from documents alone. Avellar and Grunewald rate export documentation checks low in effectiveness and useful mainly against actors of low to moderate sophistication [S-1402]. Brass and Aarne suggest pairing a registry with location verification (Chip location verification) [S-1400]. Threat model These designs trust some parties: - Owners declare truthfully, or are caught by sampled inspections [S-0002]. - Foundries keep accurate ID records; Cankaya's proposal assumes the foundry holds the ECID data [S-1408]. - IDs are hard to alter [S-0002]. Cankaya argues that an attacker could corrupt an ECID by blowing more of its one-time fuses, but could not set an arbitrary value without focused-ion-beam work [S-1408]. He adds that analysis labs can check whether a die has been re-assembled or re-packaged [S-1408]. The main threats named in the sources are forged documents and serial numbers [S-1402], opaque corporate structures [S-1402] and insiders altering production records [S-1408]. Ansari argues that the concentration of advanced manufacturing, with fewer than two dozen facilities making sub-14 nm chips as of 2023, makes a registry easier to enforce than for a commodity with distributed production [S-0007]. Evidence - Designs only. Baker et al., Avellar and Grunewald, and Cankaya publish designs; none reports an implementation [S-0002] [S-1402] [S-1408]. - Feasibility ratings. Ansari rates registry systems as near-term [S-0007]. He rates trade-data analysis as deployable now, citing work that uses existing customs data to find circumvention patterns [S-0007]. - Export-control ratings. Avellar and Grunewald rate export documentation checks as relatively established but low in effectiveness, and random return requests from a registry as novel and high in effectiveness [S-1402]. Limitations - Unrecorded chips. Chips outside the record are not covered, and a fraudulent original record would mean unregistered chips were made in advance [S-1408]. Reconstructing earlier production depends on supplier records [S-1410]. - Forgery. Documents can be forged, and serial numbers on chips and racks may be forgeable [S-1402]. In a December 2025 prosecution, the US Justice Department alleged that workers replaced GPU labels with a fictitious brand before export or attempted export [S-3383]. - Insider tampering. Records can be altered before they are committed [S-1408]. - Limited reach. Covering re-exports would need cooperation from re-exporters and foreign governments, which may not be feasible everywhere [S-1402]. - Custody versus location. A commitment fixes what was made, but tracking where chips are needs inspections or location checks [S-1408] [S-0002].", "referenced_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "C-0001", "title": "Compute stock is at most a declared amount", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "K-0016", "title": "Undeclared compute", "url": "https://trustbutveri.fyi/concepts/undeclared-compute/" }, { "id": "O-0201", "title": "AI Futures Project", "url": "https://trustbutveri.fyi/organizations/ai-futures-project/" }, { "id": "O-0207", "title": "Center for a New American Security", "url": "https://trustbutveri.fyi/organizations/center-for-a-new-american-security/" }, { "id": "O-0204", "title": "Institute for AI Policy and Strategy", "url": "https://trustbutveri.fyi/organizations/institute-for-ai-policy-and-strategy/" }, { "id": "O-0200", "title": "RAND", "url": "https://trustbutveri.fyi/organizations/rand/" } ] }, { "id": "M-0025", "slug": "confidential-multi-party-verification", "title": "Confidential multi-party verification", "aliases": [ "Confidential audits", "Attested confidential workflows", "Trustless audits" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Lets mutually distrusting parties run an agreed check over private models or records inside attested enclaves or zero-knowledge proofs, revealing only the result.", "summary": "Confidential multi-party verification runs an agreed check over assets that their owners will not show anyone, such as a developer's model weights, an auditor's test data or users' logs, so that no participant can see the others' inputs. The check runs either inside hardware-isolated enclaves that sign what code ran or with zero-knowledge proofs. Only the result is released, with evidence of how it was produced. In a 2026 cloud GPU pilot, an outside evaluator tested a proprietary Gemini model on private prompts, neither side seeing the other's inputs. Research prototypes compose multi-step audit workflows, limit usage monitoring to a jointly signed plan, and prove audits of small models with zero-knowledge proofs. No independent red-team of these systems has been published. The main obstacles are scaling to frontier models, and trust in hardware vendors and the cloud host. Even a one-bit verdict can leak information about the private inputs.", "technical": "- **Cove object model.** An artifact is a unit of private data with exactly one owner, encrypted locally before upload. A workflow is a directed acyclic graph with a canonical serialization, and its hash names the bundle. Compilation to per-node manifests is deterministic, and the manifest hash is the node's identity. Owners record allow rules that bind each artifact to approved manifest hashes [[S-1505]].\n- **Cove run and verification.** At run time a node fetches and verifies upstream certificates, evaluates preconditions, obtains decryption keys only after attestation, runs the workload and emits a certificate whose TEE report data commits to the certificate-body hash. Report data is two SHA-256 hashes, of a fixed label and of a canonical payload. Verifiers start from a terminal certificate, the workflow bytes and the TEE vendor's roots, and check each certificate recursively [[S-1505]].\n- **ZkAudit.** The provider publishes commitments to its dataset and weights, with a zero-knowledge proof that the committed weights result from training. It then answers each audit request by computing a function F privately and releasing F's output with a second proof [[S-0022]]. A copyright or demographic audit of a MobileNet v2 model on Flowers-102 cost $108 in total (about 10 cents per image), and a counterfactual audit of a recommender on MovieLens cost $8,456 [[S-0022]].\n- **Minimal Information Disclosure.** It chooses the evidence mechanism to minimize the conditional mutual information between protected properties and the released evidence, subject to a floor on information about the authorized target. One variant proves a linear projection with a Groth16 zk-SNARK [[S-1506]].", "category": "cryptographic-computational", "secondary_categories": [ "on-chip" ], "verifies": [ { "claim": "C-0005", "role": "primary", "note": "Binds audit or capability-evaluation results to the model that is served, without revealing weights (S-0009, S-0011)." }, { "claim": "C-0006", "role": "supporting", "note": "Plan-scoped monitoring runs an agreed classifier over private usage records (S-1503)." }, { "claim": "C-0007", "role": "supporting", "note": "Zero-knowledge audits can prove properties of committed training data and weights (S-0022)." } ], "threat_model": "semi-trusted", "adversarial_evaluation": "analysis", "hardware_requirement": "existing-features", "prover_cooperation": "required", "confidentiality": "preserving", "depends_on": [ { "target": "M-0008", "note": "TEE-based designs rely on measured launch and remote attestation." } ], "readiness": { "assessment": true, "level": "R2", "scope": "audits or evaluations of a private model that reveal neither party's inputs", "rubric_version": "1.1", "rationale": "A 2026 pilot evaluated a proprietary model on a generally available cloud enclave service, but the evaluation workflows are pilots or research prototypes, and no party is documented as relying on their results for a verification decision.\n\n- **R1** met: designs that state what is verified and what is trusted are published for TEE workflows [[S-0011]] [[S-1505]] and zero-knowledge audits [[S-0022]].\n- **R2** met. In the double-blind pilot, AVERI evaluated Gemini 2.5 Flash Lite against private benchmark prompts in a Google Cloud enclave with an NVIDIA H100, with the prompts kept from Google DeepMind and the weights from the evaluators (participant-reported) [[S-3320]]. Cove has an open-source reference implementation on Intel TDX via Phala Cloud's dstack, demonstrated end to end on a benchmark workflow [[S-0011]] [[S-1505]]. ZkAudit reports peer-reviewed end-to-end audits of MobileNet image classifiers and a recommender model [[S-0022]]. [[I-0007|Attestable Audits]] ran safety benchmarks on an 8-billion-parameter model in AWS Nitro Enclaves [[S-0009]].\n- **R3** not met for this use. Google reports that Confidential Space, which releases each data owner's data only to an attested workload that meets the owner's conditions, is generally available, on H100 GPUs since April 2026 [[S-3321]] [[S-3322]]. That release step is [[M-0008|TEE attestation]], assessed on its own record. The evaluation workflow that ran on it, OpenMined's [[I-0017|PySyft]], is documented only as a pilot [[S-3320]]. Pour Demain, an outside auditor, ran interpretability evaluations of a 744-billion-parameter open-weights model on Tinfoil's production confidential-computing platform, with its governance layer shown for single sessions only [[S-3361]]. No party is documented as relying on such a result for a verification decision.\n- **R4** not met: no independent audit or red-team of Confidential Space for this use, Cove, Attestable Audits or ZkAudit has been published. An independent analysis of one monitoring tool's published evaluation found that its one-bit verdicts leak information [[S-1506]]. Attackers with physical access to the memory bus have forged the Intel TDX attestation the pilot relied on [[S-1202]] [[S-3126]].\n\nConfidence is medium. The R2 evidence includes peer-reviewed results [[S-0022]] and a pilot on a proprietary model [[S-3320]]. Whether a general-purpose enclave service can carry this use to R3 is a judgment call.", "evidence": [ "S-3320", "S-3321", "S-3322", "S-3361", "S-0011", "S-1505", "S-0022", "S-0009", "S-1506", "S-1202", "S-3126" ], "next_level_gaps": [ "A production-grade, available confidential evaluation or audit workflow, or reliance by a party other than its developer on such a result for a verification decision.", "An independent public evaluation (audit, red-team or peer-reviewed security analysis) of a confidential multi-party verification system, such as Confidential Space with PySyft or Cove, that leaves no critical flaw open.", "Confidential evaluation on multi-node enclave clusters at the scale of the largest frontier models, or zero-knowledge audits at that scale." ], "confidence": "medium", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-28", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Released verdicts can leak information about private inputs", "kind": "demonstrated-attack", "severity": "significant", "status": "open", "description": "Even a one-bit result can reveal more than intended. Abdelghafar and Kulp used the published evaluation of Auditor-in-a-Box, whose output filter is meant to disclose at most one bit. Given only the valid or invalid decision on a new request, a simple predictor identified which of two request sets it came from (borderline or proxy requests versus ordinary research requests) with 72.4% balanced accuracy, against 50% without the decision. In a second experiment, which distinguished reinforcement-learning workloads from others, several candidate one-bit reports revealed exact-workload information.", "sources": [ "S-1506" ], "response": null }, { "assessment": true, "title": "Memory-bus interposition extracts attestation keys and forges attestations", "kind": "demonstrated-attack", "severity": "significant", "status": "open", "description": "With physical access to a server's DDR5 memory bus, researchers extracted a per-CPU Intel attestation provisioning key and forged TDX attestations. Against AMD SEV-SNP the same attack recovered a signing key used inside the virtual machine, not an AMD attestation key. By pairing forged TDX attestations with genuine H100 attestations relayed from rented hardware, they made a workload without TEE protection appear to run under GPU confidential computing. Intel, AMD and NVIDIA acknowledged the findings, and the researchers report that Intel and AMD consider interposer attacks out of scope. A second team forged attestation reports on an up-to-date Intel TDX platform with an active DDR5 interposer, DDRop. Other attacks have forged SEV-SNP attestation: Battering RAM with an interposer on DDR4 servers, and RMPocalypse from a malicious hypervisor on platforms without AMD's firmware fix. Cove's reference implementation trusts Intel TDX attestation roots, and the double-blind pilot ran on Intel TDX with an H100.", "sources": [ "S-1202", "S-3126", "S-1210", "S-1212", "S-1213", "S-1505", "S-3320" ], "response": null }, { "assessment": true, "title": "Guarantees depend on the host software stack and on review", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Cove's developers state that compromise of the Docker daemon, host kernel or TEE stack breaks all guarantees. They also state that Docker policy alone cannot prove that guest code cannot generate a quote if the platform exposes quote instructions globally, and that compiled workflow bundles are hashed and reviewable but not signed by a publisher key.", "sources": [ "S-1505" ], "response": null }, { "assessment": true, "title": "Completeness is not established", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "A confidential workflow proves facts about the records and models submitted to it. A governance analysis notes that an auditor also needs assurance that all activity is accounted for, since a host could start a second confidential virtual machine that uses a different model or bypasses monitoring.", "sources": [ "S-0014" ], "response": null }, { "assessment": true, "title": "Zero-knowledge audits reveal model architecture", "kind": "theoretical-argument", "severity": "minor", "status": "open", "description": "ZkAudit keeps weights and data secret but reveals the model architecture, and it does not protect against data poisoning.", "sources": [ "S-0022" ], "response": null } ], "blockers": [ { "text": "Frontier inference typically needs the resources of several GPUs, and published confidential evaluations ran on a single server, so the double-blind pilot's authors name many-node confidential H100 or B200 clusters as the next milestone.", "theme": "performance-compatibility", "blocked_by": "M-0008", "sources": [ "S-0014", "S-3361", "S-3320" ] }, { "text": "Zero-knowledge audits have been shown on image classifiers and a recommender model, not language models at frontier scale, and a counterfactual audit of the recommender cost $8,456.", "theme": "performance-compatibility", "blocked_by": "M-0004", "sources": [ "S-0022" ] }, { "text": "Trust rests on a small number of hardware vendors, and a per-CPU Intel attestation key has been extracted by physical attack.", "theme": "hardware-trust", "blocked_by": "M-0008", "sources": [ "S-0014", "S-1202" ] }, { "text": "Parties must negotiate the plan or workflow and handle false positives and appeals, which the Auditor-in-a-Box authors list as open problems.", "theme": "access-governance", "blocked_by": null, "sources": [ "S-1504" ] }, { "text": "Released evidence must be designed to limit collateral leakage, which requires declaring protected properties in advance and calibrating on labelled executions.", "theme": "privacy-leakage", "blocked_by": null, "sources": [ "S-1506" ] } ], "challenge_themes": [ "privacy-leakage", "hardware-trust", "performance-compatibility", "access-governance", "coverage-hidden-compute" ], "organizations": [ "O-0142", "O-0202", "O-0141", "O-0215" ], "people": [], "sources": [ { "source": "S-0011", "supports": "problem statement; framework; three applications; open-source implementation on Intel TDX via dstack", "locator": "abstract (read via the ICML 2026 virtual poster page; the OpenReview PDF was not reachable)" }, { "source": "S-1505", "supports": "object model; lifecycle; certificates; trust boundary; residual risks", "locator": "README; docs/internal/architecture.md; docs/internal/security_model.md" }, { "source": "S-0022", "supports": "ZkAudit protocol; models and datasets; accuracy; costs; assumptions; architecture disclosure; data poisoning", "locator": "abstract; §5; Tables 1-4; limitations" }, { "source": "S-0009", "supports": "multi-party enclave audit protocol; transparency log; prototype and throughput; CPU versus GPU cost and slowdown; vendor trust", "locator": "§3; §4; §5; Table 2" }, { "source": "S-1503", "supports": "plan-scoped monitoring protocol", "locator": "abstract" }, { "source": "S-1504", "supports": "plan definition; reference implementation; process problems; limitations", "locator": "whole post" }, { "source": "S-1506", "supports": "minimal information disclosure framework; one-bit leakage findings; Groth16 variant; limitations", "locator": "abstract; introduction; Appendix A; Figure 5; limitations" }, { "source": "S-0014", "supports": "completeness and second-CVM gap; vendor root of trust; GPU TEE maturity; multi-GPU inference; treaty threat model", "locator": "resource accounting; hardware auditability; physical attack surface" }, { "source": "S-3320", "supports": "double-blind evaluation pilot: participants, model, benchmark, GCP Confidential Space on H100 with Intel TDX, PySyft, mutual attestation checks, overhead figure cited from NVIDIA, uninspected model code, Google in the verification path, scaling to many-node clusters", "locator": "abstract; architecture; limitations; future work" }, { "source": "S-3361", "supports": "Pour Demain's interpretability evaluations of GLM-5.1 on Tinfoil (Intel TDX, eight H200 GPUs): enclave-bound tensors, bounded signed exports, overheads, single-session governance", "locator": "whole post" }, { "source": "S-3321", "supports": "Confidential Space: multi-party roles; data released only to attested workloads; operator has no access; supported TEEs", "locator": "overview" }, { "source": "S-3322", "supports": "Confidential Space generally available, including on H100 GPUs from 2026-04-29", "locator": "release notes, 2023-03-28 and 2026-04-29" }, { "source": "S-3126", "supports": "DDRop forges attestation reports on an up-to-date Intel TDX platform with an active DDR5 interposer", "locator": "site summary" }, { "source": "S-1202", "supports": "physical extraction of Intel attestation keys and SEV-SNP signing keys; forged attestations against NVIDIA GPU confidential computing; vendor acknowledgement and positions", "locator": "project site summary; paper abstract and disclosure" }, { "source": "S-1210", "supports": "Battering RAM forges SEV-SNP attestation with a DDR4 interposer" }, { "source": "S-1212", "supports": "RMPocalypse forges SEV-SNP attestation from a malicious hypervisor" }, { "source": "S-1213", "supports": "AMD firmware fixes for RMPocalypse (CVE-2025-0033)" } ], "concepts": [ "K-0001", "K-0002", "K-0004", "K-0005", "K-0006", "K-0010", "K-0018", "K-0024" ], "complements": [ "M-0023", "M-0012" ], "alternatives": [], "type": "mechanism", "implementations": [ { "id": "I-0017", "title": "PySyft double-blind evaluations", "url": "https://trustbutveri.fyi/implementations/pysyft-double-blind-evaluations/" } ], "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/", "source_file": "content/mechanisms/confidential-multi-party-verification.md", "flags_all": [], "body_markdown": "## How it works\n\nAudits often need sensitive artifacts, such as model weights and evaluation data, held by parties who do not trust each other [[S-0011]]. Model providers have business reasons to keep models and data secret, while society needs algorithmic transparency [[S-0022]]. Confidential multi-party verification moves the agreed check to a place where no single party sees the others' inputs [[S-0011]] [[S-0022]].\n\n- **Composable enclave workflows.** Cove uses trusted execution environments (TEEs) to compose reusable multi-stage audit workflows [[S-0011]]. In its developers' design, each data owner encrypts its artifact locally and approves only specific, hashed workflow steps [[S-1505]]. A key is released only to an enclave whose attestation matches an approved step [[S-1505]]. Each step emits a certificate that the hardware binds to its code and outputs, and a verifier checks the whole chain, starting from the final certificate [[S-1505]].\n- **Commercial confidential workspaces.** Google reports that its Confidential Space runs a workload on data from several parties and releases each party's data only to a workload whose attestation meets that party's conditions. The operator has no access to the data [[S-3321]]. OpenMined's [[I-0017|PySyft evaluation workflow]] used it in a 2026 pilot [[S-3320]].\n- **Enclave-run audits.** In [[I-0007|Attestable Audits]], the model provider and the auditor each encrypt their inputs to an enclave key [[S-0009]]. The enclave runs the benchmark and publishes to a transparency log an attestation that binds the model hash, the hash of the audit code and data, and the result [[S-0009]].\n- **Plan-scoped monitoring.** A monitoring party and a monitored party co-sign a plan, which an open-source classifier runs inside an attested TEE; scope changes need fresh signatures from both parties [[S-1503]]. The plan fixes the inputs, the computation steps and the outputs to be released [[S-1504]].\n- **Zero-knowledge audits.** In ZkAudit, a provider publishes cryptographic commitments to its dataset and weights, plus a zero-knowledge proof that the weights came from training on that data [[S-0022]]. It then answers audit requests by computing a function privately and releasing the output with a proof that the computation was correct [[S-0022]].\n- **Designing what is disclosed.** Minimal Information Disclosure treats the content of the released evidence as a design choice, and aims to minimize what the evidence reveals beyond the authorized result [[S-1506]].\n\n## What it establishes\nIt can establish:\n\n- that an agreed computation ran on committed or attested inputs and produced the released result, without revealing the inputs [[S-0022]] [[S-1505]];\n- in Attestable Audits, that the model answering a user is the one that was audited [[S-0009]].\n\nIt cannot establish:\n\n- that nothing was left out of the workflow, since auditors also need assurance that all activity is accounted for [[S-0014]];\n- that the released result reveals nothing else: in one analysis, a one-bit verdict let a predictor tell which of two groups a private request came from with 72.4% balanced accuracy, against 50% by chance [[S-1506]];\n- in ZkAudit, secrecy of the model architecture, which the proofs reveal [[S-0022]].\n\n## Threat model\n- **Double-blind pilot.** Its guarantees hold for a user who trusts that the cloud provider and the hardware manufacturer have insufficient incentives to collude against them [[S-3320]]. Google's services sign and verify the attestation report, which the participants note places Google in the verification path [[S-3320]].\n- **Cove.** Its developers state that it trusts Intel TDX and its attestation roots, Docker's enforcement of the generated configuration, pinned first-party components, and public source code with human review [[S-1505]]. It does not trust its own storage server, network routing or orchestration [[S-1505]].\n- **Attestable Audits.** All parties must trust the vendor of the confidential-computing technology, which was AWS in the prototype [[S-0009]].\n- **ZkAudit.** It rests on cryptographic assumptions, including binding and hiding commitments [[S-0022]].\n- **Treaty setting.** A governance analysis contrasts the confidential-computing adversary, a dishonest operator, with the adversary in a treaty setting: a state with physical access to data centres and leverage over hardware vendors [[S-0014]].\n\n## Evidence\n- **Double-blind pilot.** Google DeepMind, OpenMined, AVERI, MLCommons and the Singapore AI Safety Institute report evaluating Gemini 2.5 Flash Lite against private AILuminate benchmark prompts in Google Cloud's Confidential Space, on one NVIDIA H100 with Intel TDX, using OpenMined's PySyft [[S-3320]]. Each party checked the enclave's attestation before uploading its assets, the prompts stayed hidden from Google DeepMind and the weights from the evaluators, and AVERI staff decrypted and scored the outputs [[S-3320]]. The participants note that not all model code could be inspected or allowlisted, and that the guest operating system builds are not independently reproducible [[S-3320]]. The report's figure of under 5% compute overhead is cited from an NVIDIA blog, not measured in the pilot [[S-3320]].\n- **Pour Demain on Tinfoil.** Pour Demain reports running five interpretability workloads on GLM-5.1, a 744-billion-parameter open-weights model, on Tinfoil's production confidential-computing platform, using Intel TDX with eight H200 GPUs [[S-3361]]. Raw tensors stayed inside the enclave, and only bounded, aggregated, signed and budget-capped exports left it [[S-3361]]. It measured 5–7% added wall time from confidential computing, rising to 33–38% with interpretability instrumentation, and it demonstrated the governance layer for single sessions only [[S-3361]].\n- **Cove.** Its authors show how its primitives express three applications: capability-attested inference, attested confidential benchmarks and bilateral capability verification [[S-0011]]. They report an open-source reference implementation on Intel TDX via Phala Cloud's dstack, demonstrated end to end on the benchmark workflow only [[S-0011]] [[S-1505]].\n- **Attestable Audits.** The authors ran MMLU, XSum and ToxicChat on a 4-bit Llama-3.1-8B model in CPU-only AWS Nitro Enclaves [[S-0009]]. They report that CPU inference cost 21.7 times as much per token as GPU inference and ran about 100 times slower [[S-0009]].\n- **ZkAudit.** Peer-reviewed at ICML 2024 [[S-0022]]. The authors audited MobileNet v2 image classifiers on three datasets, with accuracy 0.5–0.7 percentage points below full precision, and a small recommender whose error matched full precision [[S-0022]].\n- **Auditor-in-a-Box.** A reference implementation runs in Tinfoil confidential virtual machines [[S-1504]]. Its authors state that user data and plan execution in the demo are not actually secure, and that it has not been stress-tested by a counterparty [[S-1504]].\n\n## Limitations\n- **Verdict leakage.** Abdelghafar and Kulp found that one-bit reports can reveal sensitive attributes [[S-1506]]. They propose designing the evidence itself to limit this [[S-1506]].\n- **Physical attacks on TEEs.** Researchers interposing on the memory bus extracted a per-CPU Intel attestation key and forged TDX attestations [[S-1202]], and a second team forged TDX attestation reports with an active interposer [[S-3126]]. Other attacks forged AMD SEV-SNP attestations, one through a DDR4 interposer and one from software alone on platforms without AMD's fix [[S-1210]] [[S-1212]].\n- **Stack trust.** Compromise of the Docker daemon, host kernel or TEE stack breaks Cove's guarantees [[S-1505]].\n- **Scale.** Frontier model inference typically needs the resources of several GPUs [[S-0014]]. Pour Demain's evaluation used one server with eight H200 GPUs, and the double-blind pilot one H100 [[S-3361]] [[S-3320]]. ZkAudit was shown on image classifiers and a recommender model, not frontier-scale language models [[S-0022]].\n- **Process.** Plan negotiation, false positives and appeals remain open problems [[S-1504]].\n\nFor attesting that a declared safeguard ran on a single service, see [[M-0023]].", "body_text": "How it works Audits often need sensitive artifacts, such as model weights and evaluation data, held by parties who do not trust each other [S-0011]. Model providers have business reasons to keep models and data secret, while society needs algorithmic transparency [S-0022]. Confidential multi-party verification moves the agreed check to a place where no single party sees the others' inputs [S-0011] [S-0022]. - Composable enclave workflows. Cove uses trusted execution environments (TEEs) to compose reusable multi-stage audit workflows [S-0011]. In its developers' design, each data owner encrypts its artifact locally and approves only specific, hashed workflow steps [S-1505]. A key is released only to an enclave whose attestation matches an approved step [S-1505]. Each step emits a certificate that the hardware binds to its code and outputs, and a verifier checks the whole chain, starting from the final certificate [S-1505]. - Commercial confidential workspaces. Google reports that its Confidential Space runs a workload on data from several parties and releases each party's data only to a workload whose attestation meets that party's conditions. The operator has no access to the data [S-3321]. OpenMined's PySyft evaluation workflow used it in a 2026 pilot [S-3320]. - Enclave-run audits. In Attestable Audits, the model provider and the auditor each encrypt their inputs to an enclave key [S-0009]. The enclave runs the benchmark and publishes to a transparency log an attestation that binds the model hash, the hash of the audit code and data, and the result [S-0009]. - Plan-scoped monitoring. A monitoring party and a monitored party co-sign a plan, which an open-source classifier runs inside an attested TEE; scope changes need fresh signatures from both parties [S-1503]. The plan fixes the inputs, the computation steps and the outputs to be released [S-1504]. - Zero-knowledge audits. In ZkAudit, a provider publishes cryptographic commitments to its dataset and weights, plus a zero-knowledge proof that the weights came from training on that data [S-0022]. It then answers audit requests by computing a function privately and releasing the output with a proof that the computation was correct [S-0022]. - Designing what is disclosed. Minimal Information Disclosure treats the content of the released evidence as a design choice, and aims to minimize what the evidence reveals beyond the authorized result [S-1506]. What it establishes It can establish: - that an agreed computation ran on committed or attested inputs and produced the released result, without revealing the inputs [S-0022] [S-1505]; - in Attestable Audits, that the model answering a user is the one that was audited [S-0009]. It cannot establish: - that nothing was left out of the workflow, since auditors also need assurance that all activity is accounted for [S-0014]; - that the released result reveals nothing else: in one analysis, a one-bit verdict let a predictor tell which of two groups a private request came from with 72.4% balanced accuracy, against 50% by chance [S-1506]; - in ZkAudit, secrecy of the model architecture, which the proofs reveal [S-0022]. Threat model - Double-blind pilot. Its guarantees hold for a user who trusts that the cloud provider and the hardware manufacturer have insufficient incentives to collude against them [S-3320]. Google's services sign and verify the attestation report, which the participants note places Google in the verification path [S-3320]. - Cove. Its developers state that it trusts Intel TDX and its attestation roots, Docker's enforcement of the generated configuration, pinned first-party components, and public source code with human review [S-1505]. It does not trust its own storage server, network routing or orchestration [S-1505]. - Attestable Audits. All parties must trust the vendor of the confidential-computing technology, which was AWS in the prototype [S-0009]. - ZkAudit. It rests on cryptographic assumptions, including binding and hiding commitments [S-0022]. - Treaty setting. A governance analysis contrasts the confidential-computing adversary, a dishonest operator, with the adversary in a treaty setting: a state with physical access to data centres and leverage over hardware vendors [S-0014]. Evidence - Double-blind pilot. Google DeepMind, OpenMined, AVERI, MLCommons and the Singapore AI Safety Institute report evaluating Gemini 2.5 Flash Lite against private AILuminate benchmark prompts in Google Cloud's Confidential Space, on one NVIDIA H100 with Intel TDX, using OpenMined's PySyft [S-3320]. Each party checked the enclave's attestation before uploading its assets, the prompts stayed hidden from Google DeepMind and the weights from the evaluators, and AVERI staff decrypted and scored the outputs [S-3320]. The participants note that not all model code could be inspected or allowlisted, and that the guest operating system builds are not independently reproducible [S-3320]. The report's figure of under 5% compute overhead is cited from an NVIDIA blog, not measured in the pilot [S-3320]. - Pour Demain on Tinfoil. Pour Demain reports running five interpretability workloads on GLM-5.1, a 744-billion-parameter open-weights model, on Tinfoil's production confidential-computing platform, using Intel TDX with eight H200 GPUs [S-3361]. Raw tensors stayed inside the enclave, and only bounded, aggregated, signed and budget-capped exports left it [S-3361]. It measured 5–7% added wall time from confidential computing, rising to 33–38% with interpretability instrumentation, and it demonstrated the governance layer for single sessions only [S-3361]. - Cove. Its authors show how its primitives express three applications: capability-attested inference, attested confidential benchmarks and bilateral capability verification [S-0011]. They report an open-source reference implementation on Intel TDX via Phala Cloud's dstack, demonstrated end to end on the benchmark workflow only [S-0011] [S-1505]. - Attestable Audits. The authors ran MMLU, XSum and ToxicChat on a 4-bit Llama-3.1-8B model in CPU-only AWS Nitro Enclaves [S-0009]. They report that CPU inference cost 21.7 times as much per token as GPU inference and ran about 100 times slower [S-0009]. - ZkAudit. Peer-reviewed at ICML 2024 [S-0022]. The authors audited MobileNet v2 image classifiers on three datasets, with accuracy 0.5–0.7 percentage points below full precision, and a small recommender whose error matched full precision [S-0022]. - Auditor-in-a-Box. A reference implementation runs in Tinfoil confidential virtual machines [S-1504]. Its authors state that user data and plan execution in the demo are not actually secure, and that it has not been stress-tested by a counterparty [S-1504]. Limitations - Verdict leakage. Abdelghafar and Kulp found that one-bit reports can reveal sensitive attributes [S-1506]. They propose designing the evidence itself to limit this [S-1506]. - Physical attacks on TEEs. Researchers interposing on the memory bus extracted a per-CPU Intel attestation key and forged TDX attestations [S-1202], and a second team forged TDX attestation reports with an active interposer [S-3126]. Other attacks forged AMD SEV-SNP attestations, one through a DDR4 interposer and one from software alone on platforms without AMD's fix [S-1210] [S-1212]. - Stack trust. Compromise of the Docker daemon, host kernel or TEE stack breaks Cove's guarantees [S-1505]. - Scale. Frontier model inference typically needs the resources of several GPUs [S-0014]. Pour Demain's evaluation used one server with eight H200 GPUs, and the double-blind pilot one H100 [S-3361] [S-3320]. ZkAudit was shown on image classifiers and a recommender model, not frontier-scale language models [S-0022]. - Process. Plan negotiation, false positives and appeals remain open problems [S-1504]. For attesting that a declared safeguard ran on a single service, see Safeguard attestation.", "referenced_by": [ { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0017", "title": "PySyft double-blind evaluations", "url": "https://trustbutveri.fyi/implementations/pysyft-double-blind-evaluations/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" }, { "id": "K-0006", "title": "Trusted execution environment (TEE)", "url": "https://trustbutveri.fyi/concepts/trusted-execution-environment/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" }, { "id": "O-0141", "title": "Tinfoil", "url": "https://trustbutveri.fyi/organizations/tinfoil/" }, { "id": "O-0142", "title": "University of Cambridge", "url": "https://trustbutveri.fyi/organizations/university-of-cambridge/" } ] }, { "id": "M-0002", "slug": "deterministic-inference", "title": "Deterministic and bit-exact inference", "aliases": [ "Bit-exact inference", "Batch-invariant inference", "Reproducible inference" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "one_liner": "Making model inference reproducible bit for bit, so that a verifier's re-run must match the provider's output exactly rather than approximately.", "summary": "Deterministic inference makes an AI model's outputs reproducible bit for bit, so a verifier's re-run must match exactly. Otherwise, re-runs on the same input often differ slightly, because floating-point results depend on the order of operations, which shifts with batch size, hardware and software. This noise forces recomputation checks to accept approximate matches, which a cheating provider could exploit. Two methods remove it: kernels whose results do not depend on batch size, or recording enough about hardware and software for a verifier to reproduce every rounding error. vLLM and SGLang offer deterministic modes, and Gensyn and Eigen Labs report verification services built on exact replay. A published emulator predicts, bit for bit, the outputs of dense transformer blocks on four NVIDIA GPU models. The obstacles are the throughput cost of batch-invariant kernels, gaps in the emulator's coverage, no independent security evaluation, and the provider's need to disclose its full configuration.", "technical": "Two routes lead to exact results.\n\n- **Invariance.** Kernels fix the reduction order for each output element regardless of batch size. Thinking Machines made RMSNorm, matrix multiplication and attention batch-invariant, the last with a fixed split size for the key-value dimension rather than a fixed number of splits [[S-1009]]. vLLM exposes this behind VLLM_BATCH_INVARIANT=1 on NVIDIA GPUs of compute capability 8.0 or higher and on Intel XPUs, in beta [[S-1013]]. SGLang integrated batch-invariant attention for its FlashInfer, FlashAttention 3 and Triton backends [[S-1012]]. LLM-42 instead decodes on a non-deterministic fast path and replays candidate tokens under a fixed-shape reduction schedule, rolling back any that are inconsistent [[S-1014]].\n- **Record and replay.** Stock engines are deterministic but not invariant. Outputs are bitwise reproducible if the verifier knows the hardware model, the exact deployed weights, the parallelism topology (separately for prefill and decode), software versions including custom kernels, and the batch size of each forward pass [[S-0018]] [[S-0020]]. Of these, only batch size changes during serving, and it costs one extra integer per forward pass to record [[S-0020]]. A software emulator reproduces the rounding of other GPU models by modelling tensor-core accumulation and kernel-specific reduction trees [[S-0020]]. Hawkeye reproduces tensor-core matrix multiplication exactly on a CPU for Ampere, Hopper and Ada Lovelace GPUs in FP16, BF16 and FP8 [[S-1010]].\n\nWith exact replay, verification is pass/fail, and the chance of catching at least one false output in k samples is 1 − (1 − p)^k for a false-output rate p [[S-0020]].", "category": "cryptographic-computational", "secondary_categories": [], "verifies": [ { "claim": "C-0005", "role": "primary", "note": "Enables exact-match recomputation checks that the declared model, weights and software setup produced the outputs." }, { "claim": "C-0004", "role": "supporting", "note": "Bit-exact recomputation of declared inference removes the tolerance an operator could hide other work in (S-0020)." }, { "claim": "C-0009", "role": "supporting", "note": "Removes the tolerance margin that steganographic exfiltration could use (S-0020)." }, { "claim": "C-0010", "role": "supporting", "note": "Unreported batch elements alter the numerics, so covert computation inside batches becomes detectable (S-0020)." } ], "threat_model": "adversarial", "adversarial_evaluation": "analysis", "hardware_requirement": "none", "prover_cooperation": "required", "confidentiality": "partial", "depends_on": [], "readiness": { "assessment": true, "level": "R3", "scope": "exact-replay checks that the declared model and setup produced the outputs", "rubric_version": "1.1", "rationale": "Gensyn reports using exact replay in production so that anyone can check markets it settles with REE, but the production evidence comes from Gensyn, and no independent security evaluation of bit-exact verification has been published.\n\n- **R1** met: the verification claim, a covert-adversary threat model and the information a verifier needs are published [[S-0020]].\n- **R2** met: public code predicts dense LLM blocks bit for bit on A100, L40, L40S and H100 GPUs running unmodified vLLM and Hugging Face engines, against a stated adversary [[S-0020]]. Batch-invariant modes are public in vLLM (beta) and SGLang [[S-1013]] [[S-1012]], and batch invariance was shown on a 235-billion-parameter model [[S-1009]]. Exact CPU reproduction of GPU matrix multiplication is peer-reviewed [[S-1010]].\n- **R3** met through [[I-0015|Gensyn's REE]], on its developer's account. Gensyn reports that Delphi, its information-market app, is live on its mainnet, and that markets settled by open models inside REE produce receipts that anyone can re-run to verify the answer [[S-3022]]. The latest documented use is from May 2026 [[S-3022]]. REE is publicly available [[S-1812]], and its current release is v0.6.0, which Gensyn does not label alpha or beta [[S-3023]]. Eigen Labs' service does not count towards R3, because Eigen Labs launched it as a mainnet alpha [[S-3021]]. No party other than a developer is documented relying on exact replay for a verification decision.\n- **R4** not met: as of September 2026 no independent audit, red-team or peer-reviewed security analysis of bit-exact verification has been published.\n\nConfidence is low: R3 rests on Gensyn's account of its own service.", "evidence": [ "S-0020", "S-1009", "S-1010", "S-1012", "S-1013", "S-1812", "S-3021", "S-3022", "S-3023" ], "next_level_gaps": [ "An independent public evaluation (audit, red-team or peer-reviewed security analysis) of bit-exact verification that leaves no critical flaw open." ], "confidence": "low", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Some kernels remain genuinely nondeterministic", "kind": "open-question", "severity": "minor", "status": "open", "description": "The bit-exact work separates kernels that are deterministic but not batch-invariant from truly nondeterministic ones that use atomic functions. Some integer de-quantization kernels use atomic additions and remain nondeterministic, so exact replay needs backends that avoid them [[S-0020]].", "sources": [ "S-0020" ], "response": null }, { "assessment": true, "title": "Cross-hardware replay relies on reverse-engineered, closed behaviour", "kind": "open-question", "severity": "significant", "status": "open", "description": "Emulating one GPU's rounding on another requires reverse-engineering tensor-core arithmetic and modelling proprietary kernel choices. Hawkeye covers a subset of NVIDIA architectures and states that attention and other higher-level operations need further reverse engineering [[S-1010]]. For the bit-exact emulator, a proprietary Hopper kernel family is an open edge case [[S-0020]].", "sources": [ "S-1010", "S-0020" ], "response": null } ], "blockers": [ { "text": "Batch-invariant kernels cost throughput: in Thinking Machines' Qwen3-8B test, an improved deterministic build took 42 s against 26 s for vLLM's default, and SGLang reports an average 34.35% slowdown on its FlashInfer and FlashAttention 3 backends.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-1009", "S-1012" ] }, { "text": "Coverage is incomplete: the bit-exact emulator targets dense blocks on NVIDIA GPUs and excludes mixture-of-experts inference and training, and vLLM's batch-invariant mode is in beta, with open work on AMD hardware and speculative decoding.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-0020", "S-1013", "S-1814" ] }, { "text": "Amodo's status page for the AI 2040 verification plan rates a reproducible inference stack for that plan as 'not started'.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-1008" ] }, { "text": "Exact replay requires the prover to disclose weights, software versions, parallelism and batch sizes to whoever recomputes.", "theme": "privacy-leakage", "blocked_by": null, "sources": [ "S-0020", "S-0018" ] } ], "challenge_themes": [ "performance-compatibility", "protocol-soundness", "privacy-leakage", "adversarial-validation" ], "organizations": [], "people": [], "sources": [ { "source": "S-0020", "supports": "covert-adversary threat model; deterministic but non-invariant engines; required metadata; software emulator and its results; limitations; comparison with statistical schemes; best-paper award and public code (arXiv comments)", "locator": "abstract; §1; results; limitations section; arXiv comments" }, { "source": "S-0015", "supports": "logging of inferences and random sampling for verification as components separate from recomputation", "locator": "§4.2, assumptions 2 and 3 (v3)" }, { "source": "S-0018", "supports": "replay metadata in a low-trust verification system", "locator": "§5.2.2" }, { "source": "S-0016", "supports": "sources of benign nondeterminism; fixed-hardware determinism vs heterogeneous deployments; >98% token agreement", "locator": "§2; §3; §7" }, { "source": "S-0017", "supports": "floating-point non-associativity; fuzzy comparison in recomputation schemes", "locator": "determinism discussion" }, { "source": "S-0067", "supports": "reproducibility required for packet correctness checks", "locator": "Concrete inference-only retrofitting proposal" }, { "source": "S-1008", "supports": "status of reproducible inference stack", "locator": "status items" }, { "source": "S-1009", "supports": "batch invariance as main cause; kernels made invariant; Qwen3-235B experiment; timings", "locator": "whole post" }, { "source": "S-1010", "supports": "exact CPU reproduction of tensor-core matrix multiplication; scope limits", "locator": "abstract; §8; §9" }, { "source": "S-1011", "supports": "provider-reported end-to-end batch-invariant and deterministic kernels", "locator": "§3.3" }, { "source": "S-1012", "supports": "SGLang deterministic mode and its overhead", "locator": "whole post" }, { "source": "S-1013", "supports": "vLLM batch-invariance flag, supported hardware (NVIDIA compute capability 8.0+, Intel XPUs), beta status", "locator": "whole page" }, { "source": "S-1014", "supports": "scheduling-based determinism alternative that mostly reuses existing kernels", "locator": "abstract" }, { "source": "S-1813", "supports": "Thinking Machines' batch-invariant kernel library (MIT)", "locator": "README" }, { "source": "S-1814", "supports": "vLLM developers' statement that batch-invariance support is based on the Thinking Machines post; open work on AMD hardware, NVFP4 and speculative decoding" }, { "source": "S-1809", "supports": "Verde refereed delegation and RepOps reproducible operators", "locator": "abstract; §3.2" }, { "source": "S-1810", "supports": "Gensyn's report of Verde and RepOps in production (provider-reported)" }, { "source": "S-1812", "supports": "REE public as binaries with an MIT-licensed SDK (provider-reported)", "locator": "README" }, { "source": "S-3020", "supports": "EigenAI deterministic engine and optimistic re-execution protocol; same-SKU and A100-vs-H100 determinism results (provider-reported)", "locator": "abstract; Table 5" }, { "source": "S-3021", "supports": "EigenAI mainnet alpha launch; stake not yet exposed to slashing (provider-reported)" }, { "source": "S-3022", "supports": "Delphi live on Gensyn's mainnet; REE settlement receipts that anyone can re-run (provider-reported)", "locator": "settlement section" }, { "source": "S-3023", "supports": "REE current release v0.6.0, with no alpha or beta label (provider-reported)", "locator": "whole page" }, { "source": "S-3500", "supports": "TAO: operator-level tolerance bounds instead of bitwise equality on heterogeneous hardware; Merkle-anchored dispute game; Ethereum testnet deployment", "locator": "abstract; §1; evaluation" } ], "concepts": [ "K-0008", "K-0009", "K-0016" ], "complements": [ "M-0003", "M-0004" ], "alternatives": [ "M-0001" ], "type": "mechanism", "implementations": [ { "id": "I-0016", "title": "Batch-invariant inference kernels (Thinking Machines)", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/" }, { "id": "I-0015", "title": "Verde and RepOps (Gensyn)", "url": "https://trustbutveri.fyi/implementations/gensyn-verde-repops/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" } ], "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/", "source_file": "content/mechanisms/deterministic-inference.md", "flags_all": [ "provider-reported" ], "body_markdown": "## How it works\n\nFloating-point arithmetic is not associative, so the same sum computed in a different order can round differently [[S-0017]] [[S-1009]]. In LLM serving, the order changes with [[S-0016]]:\n\n- batch size and kernel strategy, which depend on server load;\n- GPU type, CUDA version and kernel implementations;\n- in mixture-of-experts models, routing that depends on other tokens in the batch.\n\nThinking Machines Lab argues that the main reason inference endpoints are nondeterministic is that load, and so batch size, varies while kernels are not batch-invariant [[S-1009]]. The bit-exact work separates true nondeterminism, caused by atomic functions, from non-invariance: deterministic computation that follows different reduction trees [[S-0020]].\n\nThere are two routes to exact results:\n\n- Batch-invariant kernels fix the reduction order for each output element whatever the batch size [[S-1009]]. vLLM offers a batch-invariant mode, currently in beta [[S-1013]], and SGLang offers a deterministic inference mode [[S-1012]]. DeepSeek reports end-to-end bitwise batch-invariant and deterministic kernels, built with the goal of bitwise alignment among its pre-training, post-training and inference pipelines [[S-1011]]. LLM-42 enforces determinism through scheduling, replaying candidate tokens under a fixed reduction schedule, and mostly reuses existing kernels unchanged [[S-1014]].\n- Record and replay uses stock engines, which already give deterministic outputs that a verifier can reproduce bit for bit, if the verifier knows the key factors and no atomic functions are called [[S-0020]]. The factors are the hardware model, exact weights, parallelism layout, software versions and the batch size of each forward pass [[S-0018]] [[S-0020]]. Software emulation removes the need for identical hardware [[S-0020]], and Hawkeye re-executes GPU matrix multiplications on a CPU without precision loss [[S-1010]].\n\nFor verification, exactness turns a recomputation check ([[M-0001]]) into a pass/fail test [[S-0020]]. [[M-0004|Zero-knowledge proofs of inference]] need determinism as a precondition [[S-0020]], and packet-based schemes ([[M-0003]]) need workloads to be reproducible [[S-0067]].\n\n## What it establishes\nUnder exact replay, the accumulated rounding errors become an auditable signature of the software and hardware used [[S-0020]]. The bit-exact work names three attacks that exploit the tolerance of approximate checks: steganography, unreported changes to inference software, and covert computation in unreported batch elements [[S-0020]]. It argues that statistical schemes can bound the covert bandwidth these leave, but cannot close it [[S-0020]].\n\nDeterminism does not capture traffic or choose samples; those come from recording and sampling mechanisms such as [[M-0013]] and [[M-0001]] [[S-0015]] [[S-0067]]. Batch-invariant kernels give identical results only while the model, inference implementation and device stay fixed [[S-0016]].\n\nFor varied inference stacks and mixed GPU types, DiFR's authors expect statistical verification to remain necessary [[S-0016]]. TAO, peer-reviewed at EuroSys 2026, is a tolerance-based alternative for heterogeneous hardware [[S-3500]]. It accepts each operator's output if it falls within bounds derived from IEEE-754 worst-case error and empirical profiles, and settles disputes with a Merkle-anchored dispute game [[S-3500]]. Its authors deployed its contract layer on an Ethereum testnet [[S-3500]].\n\n## Threat model\n- **Adversary.** The bit-exact work targets covert adversaries, who comply with monitoring only when the chance of detection is high [[S-0020]].\n- **Full disclosure.** Exact replay assumes the verifier learns every factor that affects the numerics [[S-0020]]. A reference architecture for low-trust verification lists the same replay metadata [[S-0018]]. Recording the batch size is described as negligible overhead for the prover [[S-0020]].\n- **No atomic functions.** Backends must avoid atomic functions [[S-0020]].\n- **Correct hardware model.** Cross-hardware emulation assumes the hardware's rounding, subnormal handling and accumulation order have been characterized correctly [[S-1010]] [[S-0020]].\n\n## Evidence\n- **Nondeterminism measured.** For 1,000 temperature-0 completions of one prompt on Qwen3-235B-A22B, Thinking Machines reports 80 unique outputs with default kernels [[S-1009]]. With batch-invariant kernels, all 1,000 were identical [[S-1009]].\n- **Bit-exact emulation.** On Qwen3 4B blocks, the emulator reports zero BF16 differences for feed-forward blocks on A100, L40, L40S and H100 GPUs [[S-0020]]. It reports zero differences out of 71 million elements for FlashAttention-2 at 4,000 tokens [[S-0020]]. The paper received a best-paper award at the ICML 2026 TAIGR workshop, and its code is public [[S-0020]].\n- **Matrix multiplication on CPU.** Hawkeye, peer-reviewed at MLSys 2026, reports 100% success replicating 4096 × 4096 matrix multiplications on Ampere, Hopper and Lovelace GPUs [[S-1010]].\n- **Engines.** vLLM documents its batch-invariant mode [[S-1013]]. SGLang reports an average slowdown of 34.35% for its deterministic mode on FlashInfer and FlashAttention 3 backends [[S-1012]].\n- **Without determinism.** In DiFR's tests with synchronized seeds, over 98% of tokens already match exactly between provider and verifier [[S-0016]].\n- **[[I-0016|Batch-invariant kernels]].** Thinking Machines' MIT-licensed kernels underpin SGLang's deterministic mode, and vLLM's developers state that its batch-invariant mode is based on the same work [[S-1813]] [[S-1012]] [[S-1814]].\n- **[[I-0015|Verde and RepOps]].** Gensyn fixes the order of floating-point operations so that honest compute providers get bitwise-identical results, and settles their disagreements by re-running a single operation [[S-1809]]. It reports running Verde and RepOps in production [[S-1810]], and settling markets in its Delphi app with REE, its reproducible runtime, whose receipts anyone can re-run [[S-3022]].\n- **EigenAI.** Eigen Labs reports a deterministic engine built on llama.cpp with its own matrix-multiplication and reduction kernels. Its outputs were bitwise identical across 10,000 runs on one GPU model, including across hosts, and never matched between A100 and H100 GPUs [[S-3020]]. An optimistic protocol has a committee re-execute challenged outputs inside TEEs [[S-3020]]. Eigen Labs launched the service on mainnet in September 2025, as an alpha whose stake was not yet exposed to slashing [[S-3021]].\n\n## Limitations\n- **Throughput.** In Thinking Machines' test on Qwen3-8B, vLLM's default took 26 s, the unoptimized deterministic build 55 s, and the build with an improved attention kernel 42 s [[S-1009]].\n- **Coverage.** The emulator does not yet cover mixture-of-experts inference, non-NVIDIA GPUs, the proprietary nvjet kernel family on Hopper, or training [[S-0020]]. Hawkeye covers matrix multiplication only. Attention and convolutions need further reverse engineering [[S-1010]]. vLLM's batch-invariant mode is in beta, and its tracking issue lists open work on AMD hardware, NVFP4 and speculative decoding [[S-1013]] [[S-1814]].\n- **Residual nondeterminism.** Some integer de-quantization kernels use atomic additions and remain truly nondeterministic [[S-0020]].\n- **Disclosure.** Replay requires exact weights and configuration details [[S-0020]].\n- **Maturity.** Amodo's status page for the AI 2040 verification plan rates a reproducible inference stack, and red-teaming of recomputation schemes, as not started [[S-1008]].", "body_text": "How it works Floating-point arithmetic is not associative, so the same sum computed in a different order can round differently [S-0017] [S-1009]. In LLM serving, the order changes with [S-0016]: - batch size and kernel strategy, which depend on server load; - GPU type, CUDA version and kernel implementations; - in mixture-of-experts models, routing that depends on other tokens in the batch. Thinking Machines Lab argues that the main reason inference endpoints are nondeterministic is that load, and so batch size, varies while kernels are not batch-invariant [S-1009]. The bit-exact work separates true nondeterminism, caused by atomic functions, from non-invariance: deterministic computation that follows different reduction trees [S-0020]. There are two routes to exact results: - Batch-invariant kernels fix the reduction order for each output element whatever the batch size [S-1009]. vLLM offers a batch-invariant mode, currently in beta [S-1013], and SGLang offers a deterministic inference mode [S-1012]. DeepSeek reports end-to-end bitwise batch-invariant and deterministic kernels, built with the goal of bitwise alignment among its pre-training, post-training and inference pipelines [S-1011]. LLM-42 enforces determinism through scheduling, replaying candidate tokens under a fixed reduction schedule, and mostly reuses existing kernels unchanged [S-1014]. - Record and replay uses stock engines, which already give deterministic outputs that a verifier can reproduce bit for bit, if the verifier knows the key factors and no atomic functions are called [S-0020]. The factors are the hardware model, exact weights, parallelism layout, software versions and the batch size of each forward pass [S-0018] [S-0020]. Software emulation removes the need for identical hardware [S-0020], and Hawkeye re-executes GPU matrix multiplications on a CPU without precision loss [S-1010]. For verification, exactness turns a recomputation check (Sampled inference recomputation) into a pass/fail test [S-0020]. Zero-knowledge proofs of inference need determinism as a precondition [S-0020], and packet-based schemes (Whole-workload recomputation (reproducible packets)) need workloads to be reproducible [S-0067]. What it establishes Under exact replay, the accumulated rounding errors become an auditable signature of the software and hardware used [S-0020]. The bit-exact work names three attacks that exploit the tolerance of approximate checks: steganography, unreported changes to inference software, and covert computation in unreported batch elements [S-0020]. It argues that statistical schemes can bound the covert bandwidth these leave, but cannot close it [S-0020]. Determinism does not capture traffic or choose samples; those come from recording and sampling mechanisms such as Network taps and certifiers and Sampled inference recomputation [S-0015] [S-0067]. Batch-invariant kernels give identical results only while the model, inference implementation and device stay fixed [S-0016]. For varied inference stacks and mixed GPU types, DiFR's authors expect statistical verification to remain necessary [S-0016]. TAO, peer-reviewed at EuroSys 2026, is a tolerance-based alternative for heterogeneous hardware [S-3500]. It accepts each operator's output if it falls within bounds derived from IEEE-754 worst-case error and empirical profiles, and settles disputes with a Merkle-anchored dispute game [S-3500]. Its authors deployed its contract layer on an Ethereum testnet [S-3500]. Threat model - Adversary. The bit-exact work targets covert adversaries, who comply with monitoring only when the chance of detection is high [S-0020]. - Full disclosure. Exact replay assumes the verifier learns every factor that affects the numerics [S-0020]. A reference architecture for low-trust verification lists the same replay metadata [S-0018]. Recording the batch size is described as negligible overhead for the prover [S-0020]. - No atomic functions. Backends must avoid atomic functions [S-0020]. - Correct hardware model. Cross-hardware emulation assumes the hardware's rounding, subnormal handling and accumulation order have been characterized correctly [S-1010] [S-0020]. Evidence - Nondeterminism measured. For 1,000 temperature-0 completions of one prompt on Qwen3-235B-A22B, Thinking Machines reports 80 unique outputs with default kernels [S-1009]. With batch-invariant kernels, all 1,000 were identical [S-1009]. - Bit-exact emulation. On Qwen3 4B blocks, the emulator reports zero BF16 differences for feed-forward blocks on A100, L40, L40S and H100 GPUs [S-0020]. It reports zero differences out of 71 million elements for FlashAttention-2 at 4,000 tokens [S-0020]. The paper received a best-paper award at the ICML 2026 TAIGR workshop, and its code is public [S-0020]. - Matrix multiplication on CPU. Hawkeye, peer-reviewed at MLSys 2026, reports 100% success replicating 4096 × 4096 matrix multiplications on Ampere, Hopper and Lovelace GPUs [S-1010]. - Engines. vLLM documents its batch-invariant mode [S-1013]. SGLang reports an average slowdown of 34.35% for its deterministic mode on FlashInfer and FlashAttention 3 backends [S-1012]. - Without determinism. In DiFR's tests with synchronized seeds, over 98% of tokens already match exactly between provider and verifier [S-0016]. - Batch-invariant kernels. Thinking Machines' MIT-licensed kernels underpin SGLang's deterministic mode, and vLLM's developers state that its batch-invariant mode is based on the same work [S-1813] [S-1012] [S-1814]. - Verde and RepOps. Gensyn fixes the order of floating-point operations so that honest compute providers get bitwise-identical results, and settles their disagreements by re-running a single operation [S-1809]. It reports running Verde and RepOps in production [S-1810], and settling markets in its Delphi app with REE, its reproducible runtime, whose receipts anyone can re-run [S-3022]. - EigenAI. Eigen Labs reports a deterministic engine built on llama.cpp with its own matrix-multiplication and reduction kernels. Its outputs were bitwise identical across 10,000 runs on one GPU model, including across hosts, and never matched between A100 and H100 GPUs [S-3020]. An optimistic protocol has a committee re-execute challenged outputs inside TEEs [S-3020]. Eigen Labs launched the service on mainnet in September 2025, as an alpha whose stake was not yet exposed to slashing [S-3021]. Limitations - Throughput. In Thinking Machines' test on Qwen3-8B, vLLM's default took 26 s, the unoptimized deterministic build 55 s, and the build with an improved attention kernel 42 s [S-1009]. - Coverage. The emulator does not yet cover mixture-of-experts inference, non-NVIDIA GPUs, the proprietary nvjet kernel family on Hopper, or training [S-0020]. Hawkeye covers matrix multiplication only. Attention and convolutions need further reverse engineering [S-1010]. vLLM's batch-invariant mode is in beta, and its tracking issue lists open work on AMD hardware, NVFP4 and speculative decoding [S-1013] [S-1814]. - Residual nondeterminism. Some integer de-quantization kernels use atomic additions and remain truly nondeterministic [S-0020]. - Disclosure. Replay requires exact weights and configuration details [S-0020]. - Maturity. Amodo's status page for the AI 2040 verification plan rates a reproducible inference stack, and red-teaming of recomputation schemes, as not started [S-1008].", "referenced_by": [ { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0016", "title": "Batch-invariant inference kernels (Thinking Machines)", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/" }, { "id": "I-0015", "title": "Verde and RepOps (Gensyn)", "url": "https://trustbutveri.fyi/implementations/gensyn-verde-repops/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" }, { "id": "I-0008", "title": "SASH confidential network logger", "url": "https://trustbutveri.fyi/implementations/sash-confidential-network-logger/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0008", "title": "Numerical nondeterminism", "url": "https://trustbutveri.fyi/concepts/numerical-nondeterminism/" }, { "id": "O-0212", "title": "Gensyn", "url": "https://trustbutveri.fyi/organizations/gensyn/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" }, { "id": "O-0121", "title": "Pearl Research Labs", "url": "https://trustbutveri.fyi/organizations/pearl-research/" }, { "id": "O-0214", "title": "Thinking Machines Lab", "url": "https://trustbutveri.fyi/organizations/thinking-machines-lab/" } ] }, { "id": "M-0009", "slug": "flexheg-guarantee-processors", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "aliases": [ "flexHEG", "Flexible hardware-enabled guarantees", "Guarantee processor", "Interlock", "Hardware-enabled governance mechanisms (HEMs)", "On-chip governance mechanisms" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Proposed chip add-ons, a guarantee processor inside a tamper-protected enclosure, that would check and enforce agreed rules on how AI accelerators are used.", "summary": "flexHEG (flexible hardware-enabled guarantees) is a proposed chip add-on that pairs an open, auditable guarantee processor, which sees all data and instructions going to and from an AI accelerator, with a secure enclosure that reveals or responds to tampering. Like other hardware-enabled governance mechanisms, it would let a verifier check, or the chip itself enforce, rules about how AI compute is used. Rule changes, such as to a cap on training compute, could require approval from several parties. RAND and CNAS describe related designs, such as offline licensing. All are designs; the one prototype mentioned is unpublished. The main obstacles are physical security against state attackers, chipmaker cooperation, and international trust in chip design and manufacturing. The authors estimate that integrated hardware would take 3.7–7.9 years from the start of chipmaker work to displace other accelerators at the frontier. Whether a use is malicious cannot be checked on the chip.", "technical": "- **Interlock.** A flexHEG component on the data path is called an Interlock. As the only path for data and instructions into the accelerator, it could monitor or control that flow and encrypt what leaves the enclosure. The candidate points are HBM, NVLink, PCIe and the cluster NIC [[S-1204]].\n- **Encryption cost.** Part II roughly estimates that encrypting all NVLink traffic with AES-256-GCM would take about 3% of computing power, energy or board space, and encrypting all HBM traffic about 11% of power [[S-1204]].\n- **Checking and logging.** Part II estimates that for a guarantee processor to double-check one streaming multiprocessor (SM) of a Blackwell chip would take roughly 1/192 of the chip's compute, assuming 192 SMs per chip [[S-1204]]. NVIDIA states that the Blackwell GPUs in its HGX B200 system normally have 148 SMs [[S-1611]]. Storing a FLOP log for 1M unique sources, such as a very large cluster, would take roughly 30 MB [[S-1204]].\n- **Updates.** An update, or the lack of one, could require approval by k of n parties. Periodic licenses could specify a minimum firmware version, and signatures should ideally be quantum-resistant [[S-1204]]. A guarantee processor could be required to install a firmware update regularly, for example every three months, or it would block the chip [[S-0035]].\n- **Enclosure.** Tamper evidence is sufficient for verification, while guarantees about future use need tamper response. Options include capacitor discharge or antifuses that disable the chip, PUF-derived keys, and sensors for radiation, voltage glitching, lasers and temperature [[S-1204]].\n- **Manufacturing oversight.** Part III proposes extensive testing of a random sample, for example 1/1000 of finished devices, which would give a 63% chance of catching a batch of 1000 compromised devices at that stage [[S-1205]].\n- **RAND designs.** An offline license might authorize 10^18 operations before the GPU falls back to 1% performance. A \"fixed set\" restricts high-bandwidth links to pre-authorized pods of fewer than 65 chips [[S-0057]].", "category": "on-chip", "secondary_categories": [ "isolation-architecture" ], "verifies": [ { "claim": "C-0007", "role": "primary", "note": "Verifiable claims about total training compute, and enforcement of compute thresholds (S-0035)." }, { "claim": "C-0005", "role": "supporting", "note": "Deployment only to approved flexHEG devices, and verification of evaluation scores (S-0035)." }, { "claim": "C-0002", "role": "supporting", "note": "Automated verification of approximate chip location (S-0035); see M-0018." }, { "claim": "C-0008", "role": "supporting", "note": "Interlocks on NVLink or NICs, and RAND's fixed-set pods, would bound communication (S-1204, S-0057)." }, { "claim": "C-0006", "role": "supporting", "note": "Could require deployment-time safeguards on approved devices (S-0035)." } ], "threat_model": "adversarial", "adversarial_evaluation": "analysis", "hardware_requirement": "new-chip", "prover_cooperation": "required", "confidentiality": "preserving", "depends_on": [ { "target": "M-0008", "note": "Builds on existing secure boot, device identity and remote attestation; flexHEG would extend confidential-computing attestation (S-0035, S-1204)." }, { "target": "M-0019", "note": "Governance through flexHEG assumes a registry of equipped chips; covering non-flexHEG compute is a separate problem (S-1205)." } ], "readiness": { "assessment": true, "level": "R1", "scope": "checking and enforcing training-compute limits on chips, against adversaries up to states", "rubric_version": "1.1", "rationale": "The design is detailed, but nothing has been built or tested in public.\n\n- **R1** met: the flexHEG series sets out the components, the claims that could be verified or enforced, the update governance and a threat model that includes state-level adversaries [[S-0035]] [[S-1204]] [[S-1205]]. RAND and CNAS describe related HEM designs and their threats [[S-0057]] [[S-0056]].\n- **R2** not met. As of September 2026 no public implementation or reproducible end-to-end results have been published, and no Implementation record realises this mechanism. Part II mentions that \"an existing FlexHEG prototype\" uses high-resolution power measurements, but gives no details or results [[S-1204]]. Claimed results that are not public do not count. Because that prototype could not be checked, confidence is medium rather than high.", "evidence": [ "S-0035", "S-1204", "S-1205", "S-0057", "S-0056", "S-0006" ], "next_level_gaps": [ "A public prototype of a guarantee processor or Interlock on a real accelerator data path, with published end-to-end results.", "A secure enclosure evaluated against invasive physical attacks, with published cost-to-circumvent estimates.", "A specified ruleset language and a multi-party update protocol implemented and analysed.", "Chipmaker engagement, needed for integrated designs." ], "confidence": "medium", "assessed_by": [ "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "State attackers can likely defeat current secure enclosures", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "The flexHEG authors write that \"nation-state attackers can likely compromise the best current secure enclosures\", and that the marginal cost of circumvention per device is hard to estimate [[S-1204]]. RAND similarly judges that anti-tamper measures \"would not be insurmountable for a determined and well-resourced adversary\", although they raise costs and can reveal tampering [[S-0057]].", "sources": [ "S-1204", "S-0057" ], "response": null }, { "assessment": true, "title": "Firmware-only retrofits rely on Secure Boot, which fault injection can bypass", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Part II notes that the most common attack on Secure Boot replaces the firmware and applies a voltage glitch while the signature is being checked. It also notes that sophisticated actors may use microprobing or laser voltage probing to read key registers [[S-1204]].", "sources": [ "S-1204" ], "response": null }, { "assessment": true, "title": "Many important rules cannot be checked on-chip", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Malicious intent \"is not a technical property observable on-chip\", and misuse depends on what is done with a computation's results [[S-0035]]. A guarantee processor cannot easily tell whether a network is the whole system or one expert in a mixture-of-experts system [[S-0035]]. Part III judges that a fully local ruleset \"may not be entirely feasible\" for the same reason [[S-1205]].", "sources": [ "S-0035", "S-1205" ], "response": null }, { "assessment": true, "title": "FLOP accounting can be laundered through external data", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Results of earlier or parallel workloads could be hidden in the \"external data\" fed to a device, which would falsify the total FLOP count unless the inputs are explained or time delays are imposed [[S-1204]].", "sources": [ "S-1204" ], "response": null }, { "assessment": true, "title": "Supply-chain diversion and hidden backdoors", "kind": "open-question", "severity": "significant", "status": "open", "description": "Components could be diverted before a guarantee processor is added, and backdoors could be introduced during design or manufacturing. Open-source designs and physical scans of randomly selected chips are proposed as countermeasures [[S-1204]]. Part III proposes international oversight of production and extensive testing of a random sample of finished devices [[S-1205]].", "sources": [ "S-1204", "S-1205" ], "response": null }, { "assessment": true, "title": "Coverage stops at flexHEG-equipped chips", "kind": "open-question", "severity": "significant", "status": "open", "description": "Motivated actors will always be able to use some compute that is not flexHEG-equipped [[S-0035]]. Recalling existing consumer GPUs would likely be impractical, and reaching perfect coverage, or conclusively proving that no secret government data centres exist, would be \"practically quite difficult\" [[S-1205]].", "sources": [ "S-0035", "S-1205" ], "response": null } ], "blockers": [ { "text": "Integrated flexHEG needs substantial help from the accelerator manufacturer, and the authors estimate 3.7–7.9 years, from when the manufacturer starts work, for such hardware to displace other accelerators in frontier development.", "theme": "access-governance", "blocked_by": null, "sources": [ "S-1204" ] }, { "text": "State-level attackers who hold the hardware can likely compromise the best current secure enclosures.", "theme": "hardware-trust", "blocked_by": "M-0017", "sources": [ "S-1204", "S-0057" ] }, { "text": "Rival states would need to trust the design and manufacture of guarantee processors and enclosures, for example through open design, redundant processors from each side or oversight of production.", "theme": "hardware-trust", "blocked_by": null, "sources": [ "S-1205", "S-0035" ] }, { "text": "Restricting future rule updates would need a formal language for rules, which the authors judge most likely infeasible for early flexHEG versions.", "theme": "protocol-soundness", "blocked_by": null, "sources": [ "S-0035" ] }, { "text": "Governing all relevant chips depends on knowing where they are, through chip registries and detection of undeclared facilities.", "theme": "coverage-hidden-compute", "blocked_by": "M-0019", "sources": [ "S-1205" ] } ], "challenge_themes": [ "hardware-trust", "access-governance", "protocol-soundness", "coverage-hidden-compute", "adversarial-validation" ], "organizations": [ "O-0200", "O-0207" ], "people": [], "sources": [ { "source": "S-0035", "supports": "flexHEG definition, components, verifiable claims, rulesets, update restriction, limitations, relation to confidential computing", "locator": "Executive Summary; Conceptual Overview; Appendix A-B" }, { "source": "S-1204", "supports": "Interlock, secure enclosure, integration options, update authorization, overhead estimates (including the 1/192 SM estimate and its 192-SM assumption), timelines, prototype mention, attacks", "locator": "sections on Interlock-Based Design, Secure Enclosure, Potential Accelerator Modifications" }, { "source": "S-1611", "supports": "NVIDIA states that the Blackwell GPUs in an HGX B200 system normally have 148 SMs", "locator": "MLOPart section" }, { "source": "S-1205", "supports": "verification- vs ruleset-based agreements, states as primary attackers, redundant processors, manufacturing oversight and sampling, coverage limits, algorithmic efficiency", "locator": "Creating an Internationally Trustworthy FlexHEG Ecosystem; Overseeing Production" }, { "source": "S-0057", "supports": "HEM definition, offline licensing and fixed set, threat actor tiers, attack classes, anti-tamper limits", "locator": "pp. viii-x, 4, 19-27" }, { "source": "S-0056", "supports": "on-chip governance with existing features, hardening need, 18 months to 4 years estimate", "locator": "Key findings" }, { "source": "S-0006", "supports": "workshop agenda on HEMs (four uses) and open questions on licensing and pod attestation", "locator": "§2; §2.3.4, §2.5.4" }, { "source": "S-3162", "supports": "NVIDIA's stated position that its GPUs do not and should not have kill switches, and its distinction for optional user-controlled features", "locator": "blog post" } ], "concepts": [ "K-0007", "K-0015", "K-0005", "K-0004", "K-0023", "K-0021", "K-0016", "K-0018" ], "complements": [ "M-0010", "M-0011", "M-0014", "M-0017", "M-0018" ], "alternatives": [ "M-0013" ], "type": "mechanism", "implementations": [], "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/", "source_file": "content/mechanisms/flexheg-guarantee-processors.md", "flags_all": [], "body_markdown": "## How it works\n\nRAND defines hardware-enabled governance mechanisms (HEMs) as controls built into AI hardware that enable \"enforcement and compliance verification\" of policies [[S-0057]]. It analyses two designs [[S-0057]]:\n\n- **Offline licensing** lets a GPU run a set amount of work under a renewable license, then refuse or slow further work.\n- **The fixed set** limits high-bandwidth links to a small pod of pre-authorized chips.\n\nCNAS proposes a hardened security module that enforces valid firmware and up-to-date operating licenses and supports remote attestation. It would be rolled out in stages, from firmware changes to tamper-evident and then tamper-proof hardware [[S-0056]]. An interdisciplinary workshop report by O'Gara et al. discusses HEMs for four uses: verifiable AI training and inference, verifiable cluster configuration, location verification and offline licensing [[S-0006]].\n\nflexHEG (flexible hardware-enabled guarantees) combines two components [[S-0035]]:\n\n- **Guarantee processor**: \"an open-source, standardized, general-purpose processor that sits between the AI chip and the rest of the world\". It can see all information and instructions going to and from the chip [[S-0035]].\n- **Secure enclosure**: it houses the chip and the guarantee processor. If tampering is detected, secrets are wiped [[S-0035]].\n\nWhen the guarantee processor sits directly on the data path, the design is called an **Interlock**. As the only way in, it can monitor or control the flow and encrypt data leaving the enclosure [[S-1204]]. The rules it applies form a **ruleset**, which anyone from the chip's owner alone up to \"a quorum of states\" might set [[S-0035]]. Changes could require k-of-n approval, and periodic licenses could force devices to stay current [[S-1204]].\n\nPart II of the flexHEG series describes a range of implementations [[S-1204]]:\n\n- **Integrated hardware**, built into the accelerator by its manufacturer.\n- **Retrofits**, such as modified firmware or a replacement network interface card acting as the Interlock.\n- **TEE-backed software**, which Part II judges relatively fast to deploy but probably not secure against sophisticated actors with unrestricted physical access.\n\nPart III contrasts two kinds of agreement. In verification-based agreements, states detect violations and retaliate. In ruleset-based agreements, the devices enforce the rules and no state can rewrite them alone [[S-1205]].\n\n## What it establishes\nThe authors list claims flexHEG could support [[S-0035]]:\n\n- the total compute used to train a model;\n- whether a model's score on an automated test stayed below, or rose above, an agreed level;\n- deployment only to approved devices;\n- a chip's approximate location;\n- which architectures and training techniques were used.\n\nA guarantee processor could also block any operation that would push a training run past a threshold [[S-0035]].\n\nIt cannot establish:\n\n- **intent or downstream misuse**, which is not observable on the chip [[S-0035]];\n- **whether a network is the whole system** or one expert in a larger one [[S-0035]];\n- **a true FLOP count** if earlier results are hidden in external inputs [[S-1204]];\n- **anything about compute without flexHEG hardware** [[S-0035]] [[S-1205]].\n\nFully automatic, on-device compliance checking \"appears unlikely to be feasible\" [[S-0035]]. Part III therefore discusses compromises that combine enforced rules with verification [[S-1205]].\n\n## Threat model\nIn international use, \"the primary attackers ... are the participating states themselves\" [[S-1205]]. Devices would need to be open source, auditable and \"robust to tampering even from state-level adversaries\" [[S-0035]]. CNAS stresses that existing features \"need to be hardened\" before adversarial use [[S-0056]].\n\nRAND defines three kinds of threat actor: unsubsidized commercial, state-subsidized commercial, and state. It groups attacks into four classes [[S-0057]]:\n\n- **non-invasive**, for example side channels;\n- **semi-invasive**, such as fault injection;\n- **fully invasive**, such as focused ion beams and electron microscopy;\n- **non-physical**, including design theft and cryptographic, firmware, supply-chain and resource-pooling attacks.\n\nThe design also assumes a trustworthy supply chain. Part III proposes manufacturing equipment locked to standard designs, independent testing, camera oversight, tamper-evident shipping, and extensive testing and detailed physical imaging of a random sample of finished devices [[S-1205]].\n\n## Evidence\n- **Design reports.** RAND and CNAS published design and analysis reports in 2024 [[S-0057]] [[S-0056]]. The three flexHEG parts followed in 2025, commissioned by ARIA [[S-0035]] [[S-1204]] [[S-1205]].\n- **Timelines.** CNAS estimates that the needed hardware security could take as little as 18 months, and up to 4 years, of technical effort by leading firms [[S-0056]]. Part II estimates 3.7–7.9 years for integrated flexHEG to displace other accelerators at the frontier, counted from when a manufacturer starts work. It argues that to reach its full potential, flexHEG would likely need to be deployed at scale \"by 2027 or sooner\" [[S-1204]].\n- **Prototype.** Part II says that \"an existing FlexHEG prototype uses high resolution power measurements\", with no further detail [[S-1204]].\n- **Related work.** Part II notes TEE-based efforts by Mithril Security and EQTY Lab [[S-1204]].\n- **Open questions.** O'Gara et al. ask how to verify license authenticity at scale and how to attest the integrity of fixed-set pods remotely [[S-0006]].\n\n## Limitations\n- **Enclosures.** The authors write that nation-state attackers \"can likely compromise the best current secure enclosures\" [[S-1204]].\n- **Retrofits.** Designs that rely on Secure Boot are exposed to voltage glitching and probing [[S-1204]].\n- **Supply chain.** Diversion and backdoors are open problems [[S-1204]] [[S-1205]].\n- **Evaluations.** Making capability evaluations robust to models trained to underperform is \"a very difficult open problem\" [[S-0035]].\n- **Algorithmic efficiency.** Gains lower the compute needed for a given capability over time [[S-1205]].\n- **Chipmaker position.** An August 2025 NVIDIA blog post states that NVIDIA GPUs \"do not and should not have kill switches and backdoors\". It distinguishes optional software features controlled by the user from a kill switch hardwired into a chip [[S-3162]].\n\nPhysical protection is covered in [[M-0017]], interconnect limits in [[M-0014]] and throttling in [[M-0011]].", "body_text": "How it works RAND defines hardware-enabled governance mechanisms (HEMs) as controls built into AI hardware that enable \"enforcement and compliance verification\" of policies [S-0057]. It analyses two designs [S-0057]: - Offline licensing lets a GPU run a set amount of work under a renewable license, then refuse or slow further work. - The fixed set limits high-bandwidth links to a small pod of pre-authorized chips. CNAS proposes a hardened security module that enforces valid firmware and up-to-date operating licenses and supports remote attestation. It would be rolled out in stages, from firmware changes to tamper-evident and then tamper-proof hardware [S-0056]. An interdisciplinary workshop report by O'Gara et al. discusses HEMs for four uses: verifiable AI training and inference, verifiable cluster configuration, location verification and offline licensing [S-0006]. flexHEG (flexible hardware-enabled guarantees) combines two components [S-0035]: - Guarantee processor: \"an open-source, standardized, general-purpose processor that sits between the AI chip and the rest of the world\". It can see all information and instructions going to and from the chip [S-0035]. - Secure enclosure: it houses the chip and the guarantee processor. If tampering is detected, secrets are wiped [S-0035]. When the guarantee processor sits directly on the data path, the design is called an Interlock. As the only way in, it can monitor or control the flow and encrypt data leaving the enclosure [S-1204]. The rules it applies form a ruleset, which anyone from the chip's owner alone up to \"a quorum of states\" might set [S-0035]. Changes could require k-of-n approval, and periodic licenses could force devices to stay current [S-1204]. Part II of the flexHEG series describes a range of implementations [S-1204]: - Integrated hardware, built into the accelerator by its manufacturer. - Retrofits, such as modified firmware or a replacement network interface card acting as the Interlock. - TEE-backed software, which Part II judges relatively fast to deploy but probably not secure against sophisticated actors with unrestricted physical access. Part III contrasts two kinds of agreement. In verification-based agreements, states detect violations and retaliate. In ruleset-based agreements, the devices enforce the rules and no state can rewrite them alone [S-1205]. What it establishes The authors list claims flexHEG could support [S-0035]: - the total compute used to train a model; - whether a model's score on an automated test stayed below, or rose above, an agreed level; - deployment only to approved devices; - a chip's approximate location; - which architectures and training techniques were used. A guarantee processor could also block any operation that would push a training run past a threshold [S-0035]. It cannot establish: - intent or downstream misuse, which is not observable on the chip [S-0035]; - whether a network is the whole system or one expert in a larger one [S-0035]; - a true FLOP count if earlier results are hidden in external inputs [S-1204]; - anything about compute without flexHEG hardware [S-0035] [S-1205]. Fully automatic, on-device compliance checking \"appears unlikely to be feasible\" [S-0035]. Part III therefore discusses compromises that combine enforced rules with verification [S-1205]. Threat model In international use, \"the primary attackers ... are the participating states themselves\" [S-1205]. Devices would need to be open source, auditable and \"robust to tampering even from state-level adversaries\" [S-0035]. CNAS stresses that existing features \"need to be hardened\" before adversarial use [S-0056]. RAND defines three kinds of threat actor: unsubsidized commercial, state-subsidized commercial, and state. It groups attacks into four classes [S-0057]: - non-invasive, for example side channels; - semi-invasive, such as fault injection; - fully invasive, such as focused ion beams and electron microscopy; - non-physical, including design theft and cryptographic, firmware, supply-chain and resource-pooling attacks. The design also assumes a trustworthy supply chain. Part III proposes manufacturing equipment locked to standard designs, independent testing, camera oversight, tamper-evident shipping, and extensive testing and detailed physical imaging of a random sample of finished devices [S-1205]. Evidence - Design reports. RAND and CNAS published design and analysis reports in 2024 [S-0057] [S-0056]. The three flexHEG parts followed in 2025, commissioned by ARIA [S-0035] [S-1204] [S-1205]. - Timelines. CNAS estimates that the needed hardware security could take as little as 18 months, and up to 4 years, of technical effort by leading firms [S-0056]. Part II estimates 3.7–7.9 years for integrated flexHEG to displace other accelerators at the frontier, counted from when a manufacturer starts work. It argues that to reach its full potential, flexHEG would likely need to be deployed at scale \"by 2027 or sooner\" [S-1204]. - Prototype. Part II says that \"an existing FlexHEG prototype uses high resolution power measurements\", with no further detail [S-1204]. - Related work. Part II notes TEE-based efforts by Mithril Security and EQTY Lab [S-1204]. - Open questions. O'Gara et al. ask how to verify license authenticity at scale and how to attest the integrity of fixed-set pods remotely [S-0006]. Limitations - Enclosures. The authors write that nation-state attackers \"can likely compromise the best current secure enclosures\" [S-1204]. - Retrofits. Designs that rely on Secure Boot are exposed to voltage glitching and probing [S-1204]. - Supply chain. Diversion and backdoors are open problems [S-1204] [S-1205]. - Evaluations. Making capability evaluations robust to models trained to underperform is \"a very difficult open problem\" [S-0035]. - Algorithmic efficiency. Gains lower the compute needed for a given capability over time [S-1205]. - Chipmaker position. An August 2025 NVIDIA blog post states that NVIDIA GPUs \"do not and should not have kill switches and backdoors\". It distinguishes optional software features controlled by the user from a kill switch hardwired into a chip [S-3162]. Physical protection is covered in Tamper evidence for verifier devices, interconnect limits in Bandwidth limits and compartmentalization and throttling in Hardware performance throttling and licensing.", "referenced_by": [ { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0007", "title": "Hardware-enabled mechanism (HEM)", "url": "https://trustbutveri.fyi/concepts/hardware-enabled-mechanism/" }, { "id": "K-0015", "title": "Tamper evidence and tamper resistance", "url": "https://trustbutveri.fyi/concepts/tamper-evidence/" }, { "id": "O-0207", "title": "Center for a New American Security", "url": "https://trustbutveri.fyi/organizations/center-for-a-new-american-security/" }, { "id": "O-0200", "title": "RAND", "url": "https://trustbutveri.fyi/organizations/rand/" } ] }, { "id": "M-0011", "slug": "hardware-performance-throttling", "title": "Hardware performance throttling and licensing", "aliases": [ "Offline licensing", "Performance limits", "Microarchitectural throttling", "Compute licensing", "Usage limits" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "On-chip mechanisms that cut an AI accelerator's performance when a license expires or a trusted trigger fires, bounding what the hardware can do.", "summary": "Throttling mechanisms would let the hardware itself enforce limits on AI computation. One proposal is offline licensing: a chip runs at full speed only while it holds a valid license for a set amount of work. Another is a set of microarchitectural throttles that, when triggered, shrink L2 cache capacity, add cache latency, cap cache bandwidth or limit shared-memory access. A 2026 simulation of an NVIDIA A100 found that such throttles could cut the performance of large-language-model kernels by up to 80% at one-eighth of resource availability, at a cost of under about 10,000 flip-flops each. The results come from simulation, not real chips. For verification, throttling matters only if a verifier can confirm the limit is in place and cannot be bypassed. That depends on attestation, tamper resistance and a secure trigger or licensing path. RAND judges that anti-tamper protection would not be insurmountable for a determined, well-resourced adversary.", "technical": "- **Knobs evaluated by Ma et al.** L2 capacity through way masking, L2 latency through configurable request buffering, L2 bandwidth through credit-based rate limiting, and shared-memory port access through bank arbitration. None is exposed to a software-visible interface [[S-0036]].\n- **Setup.** The simulator was AccelSim, configured as an NVIDIA A100, with utilization metrics validated on a real GPU with Nsight Compute. The workloads were CUTLASS StreamK GEMM kernels, for prefill (M=4096) and decode (M=128), and FlashAttention, with shapes from DeepSeek-V3, Llama-3-70B and Mixtral-8x7B, plus six non-LLM CUDA sample workloads [[S-0036]].\n- **Results.** At one-eighth of resource availability, performance fell by up to 80% (L2 latency for decode; L2 associativity for prefill). After a throttle was applied, performance settled within about 5K cycles for shared-memory ports, 5–7K cycles for L2 response rate and about 80K cycles for L2 associativity. Each mechanism needs fewer than about 10K flip-flops [[S-0036]].\n- **Selectivity.** Memory-side knobs affected LLM kernels selectively. Disabling compute cores degraded nearly every workload together [[S-0036]].\n- **Offline licensing (RAND).** A secure message authenticated on the GPU might authorize, for example, 10^18 arithmetic operations, after which the GPU would fall back to 1% of full performance [[S-0057]].\n- **Embedded off-switch (Petrie).** Each security block uses public-key signatures, with nonces against replay, to check for recent authorization. Petrie estimates roughly 40,000 transistors per block, and about 0.5% of the die for 10,000 blocks [[S-3160]]. The proof-of-concept README reports signature checks of about 5 million cycles for ECDSA and about 300,000 cycles for HSS/LMS [[S-3161]].", "category": "on-chip", "secondary_categories": [ "accounting-provenance" ], "verifies": [ { "claim": "C-0001", "role": "supporting", "note": "A verified performance cap bounds the effective capacity of declared hardware; needs attestation that the cap is active." }, { "claim": "C-0007", "role": "supporting", "note": "Licenses that authorize a fixed amount of work would bound compute per license period (S-0057, S-0006)." }, { "claim": "C-0003", "role": "supporting", "note": "Unlicensed hardware falls back to reduced capacity or shuts down (S-0006)." } ], "threat_model": "adversarial", "adversarial_evaluation": "analysis", "hardware_requirement": "new-chip", "prover_cooperation": "partial", "confidentiality": "preserving", "depends_on": [ { "target": "M-0010", "note": "Licenses denominated in work need secure meters for the licensed quantities (S-0006)." }, { "target": "M-0008", "note": "A verifier needs attested configuration to know a cap is in force; licensing relies on secure boot and on-chip authentication (S-0057)." } ], "readiness": { "assessment": true, "level": "R1", "scope": "performance limits a verifier can rely on, against an operator trying to bypass them", "rubric_version": "1.1", "rationale": "The designs are published, but the only quantitative results are kernel-level simulations and the only public code is a simulated proof of concept.\n\n- **R1** met: RAND and O'Gara et al. describe offline licensing, including its claims and threats [[S-0057]] [[S-0006]]. Ma et al. describe microarchitectural throttles with a stated adversary [[S-0036]]. Petrie describes a license-gated off-switch with a threat model up to states with physical access [[S-3160]]. The flexHEG reports describe license-gated operation [[S-1204]].\n- **R2** not met. The quantitative evidence is one cycle-accurate simulation of a modelled A100, at the level of kernels rather than end-to-end workloads [[S-0036]]. The paper announces no code or hardware release; code is not required, but the results are not end-to-end. Petrie's public proof of concept gates an example Int8 adder in simulation. It leaves out the redundant blocks and constant-time arithmetic that the design's threat model relies on, so it does not test resistance to the attacks those features address [[S-3161]]. The existing features that sources mention, thermal and power clock throttling and Intel On Demand feature licensing, were not built or evaluated for this use [[S-0057]] [[S-0006]].", "evidence": [ "S-0036", "S-0057", "S-0006", "S-1204", "S-3160", "S-3161" ], "next_level_gaps": [ "A public hardware or FPGA prototype, or end-to-end results described in enough detail to repeat (released simulator changes and scripts would also serve).", "End-to-end evaluation on training and inference workloads on current architectures.", "A trigger or licensing path that is cryptographically validated and shown to resist interception.", "Attestation by which a remote verifier can confirm the throttle state." ], "confidence": "medium", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Trigger and licensing path security is unresolved", "kind": "open-question", "severity": "significant", "status": "open", "description": "Ma et al. assume the trigger path stays within the same trust domain as the throttle. They note that an externally sourced trigger from a trusted authority may be intercepted or blocked by the model or by adversarial humans. They state that the trigger must be cryptographically validated and point to prior kill-switch work, but they treat detection and triggering as separate from the throttles they evaluate [[S-0036]]. O'Gara et al. list the scalable verification of license authenticity, and how licenses should be issued, as open questions [[S-0006]].", "sources": [ "S-0036", "S-0006" ], "response": null }, { "assessment": true, "title": "Physical and firmware attacks on the enforcing hardware", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "RAND's threat analysis includes invasive and semi-invasive physical attacks, fault injection, and firmware and supply-chain attacks. It judges that anti-tamper measures would not be insurmountable for a determined and well-resourced adversary [[S-0057]]. Ma et al. assume that neither the model nor human attackers can alter on-chip hardware [[S-0036]]. Petrie's design aims to raise the cost of invasive editing by spreading thousands of security blocks through the chip, so that an attack would need thousands of precise edits per chip [[S-3160]].", "sources": [ "S-0057", "S-0036", "S-3160" ], "response": null }, { "assessment": true, "title": "Sensitivity is architecture-specific and some knobs behave non-monotonically", "kind": "open-question", "severity": "minor", "status": "open", "description": "The authors caution that exact sensitivity curves and knob rankings \"may shift across configurations\". L2 associativity throttling showed non-monotonic performance because of address-mapping effects [[S-0036]].", "sources": [ "S-0036" ], "response": null }, { "assessment": true, "title": "Workloads can adapt to a throttled resource", "kind": "theoretical-argument", "severity": "minor", "status": "open", "description": "A throttled AI could switch to a simpler model, which the authors call \"precisely the intended effect\". They argue that highly optimized kernels leave little headroom for further adaptation [[S-0036]].", "sources": [ "S-0036" ], "response": null } ], "blockers": [ { "text": "The throttles need new microarchitecture in future chips, and chipmakers would have to adopt it.", "theme": "access-governance", "blocked_by": null, "sources": [ "S-0036", "S-0056", "S-3162" ] }, { "text": "Secure licensing and trigger infrastructure is missing, such as a guarantee processor that issues or checks licenses.", "theme": "protocol-soundness", "blocked_by": "M-0009", "sources": [ "S-1204", "S-0036" ] }, { "text": "Licenses denominated in work need secure meters for the licensed quantities.", "theme": "hardware-trust", "blocked_by": "M-0010", "sources": [ "S-0006" ] }, { "text": "No throttle has been evaluated on real hardware or against red-team attempts at bypass.", "theme": "adversarial-validation", "blocked_by": null, "sources": [ "S-0036" ] } ], "challenge_themes": [ "hardware-trust", "protocol-soundness", "adversarial-validation", "access-governance" ], "organizations": [ "O-0200", "O-0207" ], "people": [], "sources": [ { "source": "S-0036", "supports": "microarchitectural throttling knobs, threat model, simulation setup and results, limitations", "locator": "Abstract; §3-§6" }, { "source": "S-0057", "supports": "offline licensing design and example, existing clock throttling, threat actors, anti-tamper limits", "locator": "pp. viii-ix, 19-27" }, { "source": "S-0006", "supports": "licenses as keys for a set amount of work, throttling actions, meters, Intel On Demand, open questions", "locator": "§2.5, §2.5.2, §2.5.4" }, { "source": "S-0056", "supports": "operating licenses in a hardened security module; effort estimate for adequate hardware security", "locator": "Key findings" }, { "source": "S-0035", "supports": "guarantee processor blocking operations above a threshold", "locator": "Executive Summary" }, { "source": "S-1204", "supports": "periodic licenses with minimum version; multi-party update approval", "locator": "section on updates" }, { "source": "S-3160", "supports": "embedded off-switch design, threat actors, per-block transistor and area estimates, redundancy against invasive edits; no built hardware", "locator": "Abstract; threat model; overhead and attack sections" }, { "source": "S-3161", "supports": "SystemVerilog proof of concept gating an example Int8 adder in simulation; signature-check cycle counts and omitted production features (README)", "locator": "README" }, { "source": "S-3162", "supports": "NVIDIA's stated position that its GPUs do not and should not have kill switches, and its distinction for optional user-controlled features", "locator": "blog post" } ], "concepts": [ "K-0007", "K-0023", "K-0005", "K-0018" ], "complements": [ "M-0009" ], "alternatives": [], "type": "mechanism", "implementations": [], "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/", "source_file": "content/mechanisms/hardware-performance-throttling.md", "flags_all": [], "body_markdown": "## How it works\n\nTwo families of throttling are proposed.\n\n**Licensing.** Several designs gate performance on a license:\n\n- **RAND's offline licensing.** A renewable license temporarily authorizes certain hardware features. Once the license is used up, the GPU refuses the relevant operations or runs them \"at a much lower rate\" [[S-0057]].\n- **O'Gara et al.** They describe licenses as cryptographic keys that unlock \"a specified amount of computational work\". Throttling could limit one capability, such as memory transfers, or all of them, and meters would track the licensed quantities. They cite Intel On Demand, which unlocks chip features by license, as a commercial precedent [[S-0006]].\n- **CNAS.** Its proposed security module would enforce up-to-date operating licenses, among other checks [[S-0056]].\n- **flexHEG.** Periodic licenses could set a minimum firmware version for devices [[S-1204]]. A guarantee processor could also block operations that would take a training run past a threshold [[S-0035]].\n- **Petrie's embedded off-switch.** Thousands of redundant security blocks spread through an accelerator would each block essential operations unless they have recently received signed authorization [[S-3160]].\n\n**Microarchitectural throttles.** Ma et al. add knobs that reduce L2 cache capacity, add L2 latency, cap L2 bandwidth or limit access to shared-memory ports. None of these knobs is visible to software [[S-0036]]. A trigger switches them on. It might come from an on-chip classifier or from an external trusted authority, provided the trigger path stays in the throttle's trust domain [[S-0036]].\n\nNVIDIA GPUs already throttle their clocks when power or thermal limits are exceeded, and users can set clock speeds through NVIDIA tools [[S-0057]].\n\n## What it establishes\nThrottling enforces a limit. It does not by itself produce evidence for a verifier. A verifier could rely on a cap only if the throttle state were attested ([[M-0008]]) and, for licenses denominated in work, securely metered ([[M-0010]]) [[S-0006]]. With both in place, a cap could support claims about capacity ([[C-0001]]) or about compute used in a license period ([[C-0007]]).\n\nMa et al. find that memory-side knobs hit LLM kernels selectively, while disabling compute cores slows nearly every workload [[S-0036]]. The throttle does not detect intent. The authors call detection of misaligned behaviour \"an active and orthogonal research problem\" [[S-0036]].\n\n## Threat model\nThe sources assume adversaries of different strength:\n\n- **Ma et al.** The adversary may be an autonomous AI agent, a malicious human, or both. The AI may have broad software access to the host, and a human may also have physical access to the chip, but neither can alter the on-chip hardware [[S-0036]].\n- **RAND** considers actors up to states, with physical and non-physical attacks. It concludes that anti-tamper protection can raise costs and reveal tampering but is not insurmountable [[S-0057]].\n- **CNAS** judges that existing on-chip features must be hardened before they can be relied on in adversarial settings [[S-0056]].\n- **Petrie** designs against a well-funded non-state project trying to unlock chips covertly at scale and, as a more ambitious goal, a state with physical access trying to unlock them overtly [[S-3160]].\n\nO'Gara et al. ask which quantities should be metered and how to do so securely [[S-0006]].\n\n## Evidence\n- **Ma et al.** They simulate an NVIDIA A100 in AccelSim on GEMM and attention kernels shaped like those of recent large models. Performance fell by up to 80% at one-eighth of resource availability, and settled 5–80K cycles after a throttle was applied. Each mechanism needs fewer than about 10K flip-flops. The paper announces no code or hardware release [[S-0036]].\n- **Licensing designs.** RAND and O'Gara et al. present designs and open questions, not prototypes [[S-0057]] [[S-0006]].\n- **Embedded off-switch.** Petrie's area estimate comes from gate counts, and the paper reports no built or measured hardware [[S-3160]]. A public SystemVerilog proof of concept gates an example Int8 adder in simulation. Its README lists omitted production features, including constant-time arithmetic and redundant blocks [[S-3161]].\n- **Development effort.** CNAS estimates that adequate hardware security for on-chip mechanisms, particularly tamper evidence and tamper-proofing, could take as little as 18 months, and up to 4 years, of technical effort by leading firms [[S-0056]].\n\n## Limitations\nThe weaknesses the sources recognize are:\n\n- the security of the trigger or license path [[S-0036]] [[S-0006]];\n- physical, firmware and supply-chain attacks on the enforcing hardware [[S-0057]];\n- sensitivity that varies by architecture, including non-monotonic L2 behaviour [[S-0036]];\n- workloads adapting to the throttle [[S-0036]].\n\nMa et al. ran their experiments only on A100-like configurations, and expect the trends to hold on newer GPUs [[S-0036]]. Secure boot and voltage-glitching risks for license-checking firmware are discussed under [[M-0009]] [[S-1204]].\n\nMa et al. present their throttles as mechanisms for GPU vendors to adopt [[S-0036]]. An August 2025 NVIDIA blog post states that NVIDIA GPUs \"do not and should not have kill switches and backdoors\". It distinguishes optional software features controlled by the user from a kill switch hardwired into a chip [[S-3162]].", "body_text": "How it works Two families of throttling are proposed. Licensing. Several designs gate performance on a license: - RAND's offline licensing. A renewable license temporarily authorizes certain hardware features. Once the license is used up, the GPU refuses the relevant operations or runs them \"at a much lower rate\" [S-0057]. - O'Gara et al. They describe licenses as cryptographic keys that unlock \"a specified amount of computational work\". Throttling could limit one capability, such as memory transfers, or all of them, and meters would track the licensed quantities. They cite Intel On Demand, which unlocks chip features by license, as a commercial precedent [S-0006]. - CNAS. Its proposed security module would enforce up-to-date operating licenses, among other checks [S-0056]. - flexHEG. Periodic licenses could set a minimum firmware version for devices [S-1204]. A guarantee processor could also block operations that would take a training run past a threshold [S-0035]. - Petrie's embedded off-switch. Thousands of redundant security blocks spread through an accelerator would each block essential operations unless they have recently received signed authorization [S-3160]. Microarchitectural throttles. Ma et al. add knobs that reduce L2 cache capacity, add L2 latency, cap L2 bandwidth or limit access to shared-memory ports. None of these knobs is visible to software [S-0036]. A trigger switches them on. It might come from an on-chip classifier or from an external trusted authority, provided the trigger path stays in the throttle's trust domain [S-0036]. NVIDIA GPUs already throttle their clocks when power or thermal limits are exceeded, and users can set clock speeds through NVIDIA tools [S-0057]. What it establishes Throttling enforces a limit. It does not by itself produce evidence for a verifier. A verifier could rely on a cap only if the throttle state were attested (TEE remote attestation for AI workloads) and, for licenses denominated in work, securely metered (On-chip telemetry from timing, memory and performance counters) [S-0006]. With both in place, a cap could support claims about capacity (Compute stock is at most a declared amount) or about compute used in a license period (A training run stayed within declared limits). Ma et al. find that memory-side knobs hit LLM kernels selectively, while disabling compute cores slows nearly every workload [S-0036]. The throttle does not detect intent. The authors call detection of misaligned behaviour \"an active and orthogonal research problem\" [S-0036]. Threat model The sources assume adversaries of different strength: - Ma et al. The adversary may be an autonomous AI agent, a malicious human, or both. The AI may have broad software access to the host, and a human may also have physical access to the chip, but neither can alter the on-chip hardware [S-0036]. - RAND considers actors up to states, with physical and non-physical attacks. It concludes that anti-tamper protection can raise costs and reveal tampering but is not insurmountable [S-0057]. - CNAS judges that existing on-chip features must be hardened before they can be relied on in adversarial settings [S-0056]. - Petrie designs against a well-funded non-state project trying to unlock chips covertly at scale and, as a more ambitious goal, a state with physical access trying to unlock them overtly [S-3160]. O'Gara et al. ask which quantities should be metered and how to do so securely [S-0006]. Evidence - Ma et al. They simulate an NVIDIA A100 in AccelSim on GEMM and attention kernels shaped like those of recent large models. Performance fell by up to 80% at one-eighth of resource availability, and settled 5–80K cycles after a throttle was applied. Each mechanism needs fewer than about 10K flip-flops. The paper announces no code or hardware release [S-0036]. - Licensing designs. RAND and O'Gara et al. present designs and open questions, not prototypes [S-0057] [S-0006]. - Embedded off-switch. Petrie's area estimate comes from gate counts, and the paper reports no built or measured hardware [S-3160]. A public SystemVerilog proof of concept gates an example Int8 adder in simulation. Its README lists omitted production features, including constant-time arithmetic and redundant blocks [S-3161]. - Development effort. CNAS estimates that adequate hardware security for on-chip mechanisms, particularly tamper evidence and tamper-proofing, could take as little as 18 months, and up to 4 years, of technical effort by leading firms [S-0056]. Limitations The weaknesses the sources recognize are: - the security of the trigger or license path [S-0036] [S-0006]; - physical, firmware and supply-chain attacks on the enforcing hardware [S-0057]; - sensitivity that varies by architecture, including non-monotonic L2 behaviour [S-0036]; - workloads adapting to the throttle [S-0036]. Ma et al. ran their experiments only on A100-like configurations, and expect the trends to hold on newer GPUs [S-0036]. Secure boot and voltage-glitching risks for license-checking firmware are discussed under Hardware-enabled guarantees (flexHEG) and guarantee processors [S-1204]. Ma et al. present their throttles as mechanisms for GPU vendors to adopt [S-0036]. An August 2025 NVIDIA blog post states that NVIDIA GPUs \"do not and should not have kill switches and backdoors\". It distinguishes optional software features controlled by the user from a kill switch hardwired into a chip [S-3162].", "referenced_by": [ { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "C-0001", "title": "Compute stock is at most a declared amount", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" }, { "id": "K-0007", "title": "Hardware-enabled mechanism (HEM)", "url": "https://trustbutveri.fyi/concepts/hardware-enabled-mechanism/" }, { "id": "O-0207", "title": "Center for a New American Security", "url": "https://trustbutveri.fyi/organizations/center-for-a-new-american-security/" }, { "id": "O-0200", "title": "RAND", "url": "https://trustbutveri.fyi/organizations/rand/" } ] }, { "id": "M-0015", "slug": "memory-wiping-and-secure-erasure", "title": "Memory wiping and proofs of secure erasure", "aliases": [ "Proofs of secure erasure (PoSE)", "Periodic memory wipes", "Forced memorization" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Overwriting a device's memory in a way a verifier can check, so that data from earlier, undeclared work cannot persist in memory the wipe reaches.", "summary": "Memory wiping verifiably overwrites a device's memory, so that data from earlier work cannot persist where the wipe reaches. In AI verification, periodic wipes would help show that only verified outputs carry forward. The technique builds on proofs of secure erasure: the device fills its memory with data derived from a verifier's seed, then answers timed questions about it. The theory is peer-reviewed. Scaling from fill benchmarks on CPUs, GPUs and disks, Amodo estimates about 43 minutes to wipe the memory of one GB200 tray and about 2.5 hours for an NVL72 rack's storage. As of September 2026 no end-to-end wipe with timed challenges has been published for data-centre hardware. The main obstacles are wipe time and memory the wipe cannot reach, such as drive-controller DRAM. The largest known weakness is that the guarantee depends on ruling out outside help during challenges, which fast data-centre networks make harder.", "technical": "- **Origin.** Perito and Tsudik introduced proofs of secure erasure (PoSE) for embedded devices with bounded memory and a small ROM [[S-1304]].\n- **Bursuc et al.'s model.** The adversary is a distant, memory-unbounded helper A0 and a local device A1 bounded to memory M. The protocol has an initialization phase (fill memory), r timed challenge-response rounds, and a verification phase that accepts only if the answers are correct and each round-trip time is at most Δ [[S-0032]].\n- **Variants.** An unconditional variant fills memory with random bits sent by the verifier. A graph-based variant sends only a seed: the prover computes labels of a depth-robust graph with a hash function and stores the output labels, and a construction with in-place labelling needs only about the output size plus O(w) memory. A lightweight variant relaxes depth-robustness to a small constant for speed [[S-0032]].\n- **Prototype.** Bursuc et al. ran their prototype on a standard desktop computer and simulated the erasure of 32 KB, which took 0.25 s; compiled for a 32-bit architecture, the program is 3.4 KB [[S-0032]].\n- **Amodo's implementation.** It follows Bursuc et al.: node labels are L(n) = H(n ∥ L(p1) ∥ … ∥ L(pk)), only \"robust\" labels are stored and challengeable, and the scheme is secure if q < γ, where q is the number of hashes a cheater can compute within the round-trip time and γ is the minimum number of hashes needed to recompute a robust label [[S-1302]].\n- **Amodo's parameters and throughput.** An assumed RTT of 1 ms; γ = 65,536 for host RAM and 32,768 for GPU HBM; Dual-AES-PRF on CPU and BLAKE3 on GPU. The measured throughputs, 126.3 MiB/s for 120 GB of RAM and 244.5 MiB/s for 140 GB of HBM, scale to 2,565 s for the memory sizes in a GB200 tray [[S-1302]].\n- **Disk path.** A later optimization parallelized label generation across GPUs and reached 6 min 41 s per TB with 6 GPUs on one fast drive; it needs 25 GiB of working memory, which is left unattested if used on HBM [[S-1303]]. The NVL72 storage estimate assumes 30.72 TB of drives per tray and an estimated 840 MiB/s per B200 GPU, four GPUs per tray [[S-1303]]. Code for this path (CUDA graph labeller, multi-GPU disk-wipe benchmark with sample verification, wipe-time calculator) is public under the MIT licence [[S-1321]].", "category": "isolation-architecture", "secondary_categories": [ "cryptographic-computational" ], "verifies": [ { "claim": "C-0004", "role": "supporting", "note": "Periodic wipes are proposed so that only verified inference outputs persist (S-0067); Amodo frames wipes as verifying completeness of declared workloads (S-1302)." } ], "threat_model": "adversarial", "adversarial_evaluation": "analysis", "hardware_requirement": "none", "prover_cooperation": "required", "confidentiality": "preserving", "depends_on": [ { "target": "M-0016", "note": "The fill is checked by timed challenges, which must exclude outside help." }, { "target": "M-0013", "note": "Monitored links, or physical disconnection, are needed to rule out remote storage during challenges." } ], "readiness": { "assessment": true, "level": "R1", "scope": "showing that no data from earlier work persists in memory the wipe reaches", "rubric_version": "1.1", "rationale": "The protocols are peer-reviewed and the AI use is proposed, but no public run covers the timed challenge phase on data-centre hardware.\n\n- **R1** met: the AI 2040 plan proposes periodic memory wipes on inference units so that only verified outputs persist [[S-0067]]. Peer-reviewed protocols define proofs of secure erasure and their assumptions [[S-1304]] [[S-0032]]. The MIRI overview specifies filling memory with incompressible data and challenging random samples [[S-0018]].\n- **R2** not met: complete protocols, with a separate verifier, have been reported end to end only on microcontrollers erasing 2–8 KB [[S-3240]], and Bursuc et al. simulated a 32 KB erasure on a desktop computer [[S-0032]]. Amodo has run a Bursuc-style erasure on a Raspberry Pi 5 and benchmarked label generation on H200 and H100 GPUs, CPUs and NVMe drives [[S-1302]] [[S-1303]]. Its public code for the GPU-accelerated disk-wiping path runs on realistic hardware [[S-1321]]. But the repository excludes \"the verifier, the RAM and GPU-HBM session code\" [[S-1321]], the notes report fill throughput rather than timed challenge rounds [[S-1302]] [[S-1303]], and the GB200 figures are estimates scaled from component measurements [[S-1302]] [[S-1303]]. The fill alone does not make an erasure verifiable, so this code does not meet R2. The mechanism's implementations, [[I-0011]] and [[I-0012]], are proposed architectures at R1.\n\nConfidence is medium: the rating turns on the judgment that fill-path code without the challenge phase is not a working implementation of the mechanism.", "evidence": [ "S-0067", "S-1304", "S-0032", "S-0018", "S-1302", "S-1303", "S-1321", "S-3240" ], "next_level_gaps": [ "A public end-to-end wipe of an accelerator server, covering fill and timed challenges by a separate verifier, for host memory, HBM and storage.", "Coverage of memory the wipe cannot reach: drive-controller DRAM, firmware stores, NICs, DPUs and switches, and the algorithm's own working memory.", "Adversarial evaluation in a data-centre setting, including remote-memory (RDMA) help during challenges." ], "confidence": "medium", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Memory the wipe cannot reach", "kind": "open-question", "severity": "significant", "status": "open", "description": "Amodo's inventory of a GB200 system lists many memory stores beyond GPU HBM and host DRAM. It notes that SSD controller DRAM sits on a private bus that host commands cannot read or write, and that its optimized algorithm leaves 25 GiB of HBM unattested. It also asks how switch memory could be wiped.", "sources": [ "S-1302", "S-1303" ], "response": null }, { "assessment": true, "title": "Outside help during challenges", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Classic proofs of secure erasure assume the device is isolated during the protocol. Bursuc et al. relax this to a bound on how close a helper can be, enforced by round-trip times. In data centres, remote memory access has round trips of about 1–2 µs, against about 70–200 ns for local DRAM. The MIRI overview therefore says verification depends on ruling out RDMA by latency or physical disconnection.", "sources": [ "S-0032", "S-0018" ], "response": null }, { "assessment": true, "title": "Gap between erased and total memory", "kind": "theoretical-argument", "severity": "minor", "status": "open", "description": "Bursuc et al. note that memory left between the erased region and the device's full memory could hold data, and that their bounds are tighter only against a restricted adversary.", "sources": [ "S-0032" ], "response": null } ], "blockers": [ { "text": "Wipes take time: tens of minutes for a pod's volatile memory and hours for SSDs, displacing work.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-0018", "S-1302", "S-1303" ] }, { "text": "Timed challenges must exclude remote memory and other helpers.", "theme": "coverage-hidden-compute", "blocked_by": "M-0016", "sources": [ "S-0018", "S-0032" ] }, { "text": "All memory stores in a system must be inventoried and wiped at the same time.", "theme": "coverage-hidden-compute", "blocked_by": null, "sources": [ "S-1302" ] } ], "challenge_themes": [ "coverage-hidden-compute", "performance-compatibility", "protocol-soundness", "adversarial-validation" ], "organizations": [ "O-0101", "O-0201", "O-0202" ], "people": [], "sources": [ { "source": "S-0067", "supports": "periodic memory wipes on inference units via forced memorization; purpose", "locator": "inference-only retrofitting proposal; verification overview" }, { "source": "S-1304", "supports": "origin of proofs of secure erasure; bounded-memory model; applications", "locator": "abstract" }, { "source": "S-0032", "supports": "PoSE with relaxed isolation; adversary model; graph constructions; prototype; stated gap", "locator": "§1–§8; implementation section; conclusion" }, { "source": "S-0018", "supports": "memory-occupation fill and challenge; fill times; RDMA caveat and latencies", "locator": "§5.1.2" }, { "source": "S-1302", "supports": "PoSE implementation, security condition, parameters, measured throughputs, GB200 estimates, memory inventory, open questions", "locator": "whole note (corrected version)" }, { "source": "S-1303", "supports": "multi-GPU label generation, per-TB wipe rate, NVL72 storage estimate and its assumptions, SSD DRAM and scratch limits, unprototyped multi-pass fix, remaining integration", "locator": "whole note" }, { "source": "S-1321", "supports": "public disk-wiping code; scope and exclusions; LBA overwrite limits", "locator": "README" }, { "source": "S-3240", "supports": "end-to-end comparison of seven erasure protocols on IoT microcontrollers; public code; findings", "locator": "abstract; experimental setup; conclusion" } ], "concepts": [ "K-0012", "K-0016", "K-0001", "K-0002", "K-0018" ], "complements": [ "M-0001", "M-0014", "M-0024" ], "alternatives": [], "type": "mechanism", "implementations": [ { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" } ], "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/", "source_file": "content/mechanisms/memory-wiping-and-secure-erasure.md", "flags_all": [], "body_markdown": "## How it works\n\nProofs of secure erasure (PoSE), introduced by Perito and Tsudik for low-cost embedded devices, rely on the bounded memory of such devices [[S-1304]]. A verifier confirms that a device has erased its memory by having it fill that memory with data the verifier can check [[S-0032]]. Perito and Tsudik found this \"particularly well-suited\" to secure code update and to secure memory or storage erasure [[S-1304]].\n\nBursuc et al.'s protocols have three phases [[S-0032]]:\n\n1. **Fill.** The prover fills its memory. In the simplest version the verifier sends random data; in graph-based versions it sends a short seed, and the device computes a large set of hash-chained labels from it [[S-0032]].\n2. **Challenge.** The verifier asks for randomly chosen blocks, several times, and times each answer [[S-0032]].\n3. **Decide.** The verifier accepts only if every answer is correct and arrived within the time bound [[S-0032]].\n\nThe graph construction makes it slow to recompute any missing label, so a device that tried to keep other data must fail the deadline [[S-0032]] [[S-1302]].\n\nIn AI verification, the AI 2040 plan proposes \"periodic memory wiping on the inference units (which we think can be done through forced memorization)\" [[S-0067]]. The MIRI overview describes the same step: incompressible data is loaded until the device is full, and then random samples are challenged [[S-0018]]. Amodo generates the fill on the device from a verifier seed, following Bursuc et al. [[S-1302]].\n\n## What it establishes\nAmodo describes the goal as a verifier ensuring \"the prover has fully overwritten its memory, with no information persisted across the wipe\" [[S-1302]]. Amodo says wipes help to verify the \"completeness\" of declared workloads [[S-1302]]. In the AI 2040 plan they help ensure that the only persisting results are verified inference outputs [[S-0067]]; see [[C-0004]].\n\nWipes limit what can carry over between verification cycles, not what runs between them. The AI 2040 plan therefore pairs them with network taps and partial recomputation ([[M-0013]], [[M-0001]]) [[S-0067]]. Amodo notes that all stores must be wiped at the same time, because otherwise \"an attacker could simply move the data between the disks\" [[S-1302]].\n\n## Threat model\n- **Bounded memory.** Perito and Tsudik assume bounded memory and a small read-only memory [[S-1304]].\n- **Distant helpers.** Earlier software protocols assumed the device was isolated during the protocol [[S-0032]]. Bursuc et al. relax this: helpers may exist but must be far enough away that their round-trip times exceed the deadline [[S-0032]].\n- **Round-trip time.** Amodo assumes a 1 ms round-trip time, \"expected to be a safe overestimate\" [[S-1302]].\n- **No remote memory.** The MIRI overview says verification depends on ruling out remote memory access, \"either via response latency or physical disconnection\" [[S-0018]].\n\nTimed challenges are covered in [[M-0016]].\n\n## Evidence\n- **Protocols.** The core protocols are formally analysed [[S-1304]] [[S-0032]]. Bursuc et al. ran their prototype on a standard desktop computer and simulated the erasure of 32 KB, which took 0.25 seconds [[S-0032]].\n- **Microcontroller comparison.** Gil-Pons, Mauw and Trujillo-Rasua ran seven erasure protocols end to end on three IoT microcontrollers, erasing 2–8 KB with a laptop verifier over Bluetooth, and published their code [[S-3240]]. They found the protocols feasible, although slow devices may take several seconds, and that no protocol was best in every setting [[S-3240]].\n- **First Amodo test.** Amodo ran the algorithm on a Raspberry Pi 5, erasing its RAM and a connected SSD [[S-1302]].\n- **Amodo's July 2026 analysis.** Scaling measured GPU and CPU throughputs, with an assumed 1 ms round-trip time, gave 2,565 seconds (43 minutes) for the RAM and HBM of a GB200 tray [[S-1302]]. Drive writes slowed sharply after about 12 GiB, so Amodo estimated over 24 hours for a GB200 system with 15 TB of storage [[S-1302]]. Challenge-phase calculations were still to come [[S-1302]].\n- **Amodo's September 2026 update.** Amodo attributed the slowdown to consumer drives' fast write cache running out [[S-1303]]. Six GPUs wiping one fast drive reached 6 minutes 41 seconds per TB [[S-1303]]. Amodo now expects \"~2.5 hours to wipe an NVL72 system's storage\", using a B200 label rate estimated from its GPU benchmarks, and reports that erasure speed, not drive speed, is the bottleneck [[S-1303]]. Integrating the disk wipe into a full-system wipe, and memory wiping into a full verification scheme, is still to come [[S-1303]].\n- **Public code.** Amodo has published code for the disk-wiping path, which fills every sector of an NVMe drive with graph labels generated on NVIDIA GPUs [[S-1321]]. The repository excludes the verifier and the RAM and GPU-HBM session code [[S-1321]].\n\n## Limitations\n- **Unreachable memory.** SSD controller DRAM sits on a private bus that host commands cannot access [[S-1303]]. Amodo's code notes that overwriting by logical block address cannot reach over-provisioned or remapped blocks either [[S-1321]]. The optimized algorithm's 25 GiB of working memory is \"supposedly wiped but isn't attested\" [[S-1303]]. Amodo suggests fixes, such as wiping the HBM in several passes, but has not prototyped them [[S-1303]]. Amodo lists many other stores in a GB200 system and asks how network-switch memory could be wiped [[S-1302]].\n- **Outside help.** In data centres, remote memory round trips of about 1–2 µs compare with about 70–200 ns for local DRAM, which the MIRI overview puts at roughly a tenfold timing margin [[S-0018]]. Evasion by pre-staging data is closed only by unpredictable, capacity-filling challenges [[S-0018]].\n- **Downtime.** Filling a pod's volatile memory takes tens of minutes, and SSDs take hours [[S-0018]].\n- **Residual gap.** Memory between the erased region and full capacity could hold data [[S-0032]]. Bursuc et al. say that closing this gap needs future work [[S-0032]].", "body_text": "How it works Proofs of secure erasure (PoSE), introduced by Perito and Tsudik for low-cost embedded devices, rely on the bounded memory of such devices [S-1304]. A verifier confirms that a device has erased its memory by having it fill that memory with data the verifier can check [S-0032]. Perito and Tsudik found this \"particularly well-suited\" to secure code update and to secure memory or storage erasure [S-1304]. Bursuc et al.'s protocols have three phases [S-0032]: 1. Fill. The prover fills its memory. In the simplest version the verifier sends random data; in graph-based versions it sends a short seed, and the device computes a large set of hash-chained labels from it [S-0032]. 2. Challenge. The verifier asks for randomly chosen blocks, several times, and times each answer [S-0032]. 3. Decide. The verifier accepts only if every answer is correct and arrived within the time bound [S-0032]. The graph construction makes it slow to recompute any missing label, so a device that tried to keep other data must fail the deadline [S-0032] [S-1302]. In AI verification, the AI 2040 plan proposes \"periodic memory wiping on the inference units (which we think can be done through forced memorization)\" [S-0067]. The MIRI overview describes the same step: incompressible data is loaded until the device is full, and then random samples are challenged [S-0018]. Amodo generates the fill on the device from a verifier seed, following Bursuc et al. [S-1302]. What it establishes Amodo describes the goal as a verifier ensuring \"the prover has fully overwritten its memory, with no information persisted across the wipe\" [S-1302]. Amodo says wipes help to verify the \"completeness\" of declared workloads [S-1302]. In the AI 2040 plan they help ensure that the only persisting results are verified inference outputs [S-0067]; see This compute runs inference, not training. Wipes limit what can carry over between verification cycles, not what runs between them. The AI 2040 plan therefore pairs them with network taps and partial recomputation (Network taps and certifiers, Sampled inference recomputation) [S-0067]. Amodo notes that all stores must be wiped at the same time, because otherwise \"an attacker could simply move the data between the disks\" [S-1302]. Threat model - Bounded memory. Perito and Tsudik assume bounded memory and a small read-only memory [S-1304]. - Distant helpers. Earlier software protocols assumed the device was isolated during the protocol [S-0032]. Bursuc et al. relax this: helpers may exist but must be far enough away that their round-trip times exceed the deadline [S-0032]. - Round-trip time. Amodo assumes a 1 ms round-trip time, \"expected to be a safe overestimate\" [S-1302]. - No remote memory. The MIRI overview says verification depends on ruling out remote memory access, \"either via response latency or physical disconnection\" [S-0018]. Timed challenges are covered in Timed challenge-response and memory-occupation challenges. Evidence - Protocols. The core protocols are formally analysed [S-1304] [S-0032]. Bursuc et al. ran their prototype on a standard desktop computer and simulated the erasure of 32 KB, which took 0.25 seconds [S-0032]. - Microcontroller comparison. Gil-Pons, Mauw and Trujillo-Rasua ran seven erasure protocols end to end on three IoT microcontrollers, erasing 2–8 KB with a laptop verifier over Bluetooth, and published their code [S-3240]. They found the protocols feasible, although slow devices may take several seconds, and that no protocol was best in every setting [S-3240]. - First Amodo test. Amodo ran the algorithm on a Raspberry Pi 5, erasing its RAM and a connected SSD [S-1302]. - Amodo's July 2026 analysis. Scaling measured GPU and CPU throughputs, with an assumed 1 ms round-trip time, gave 2,565 seconds (43 minutes) for the RAM and HBM of a GB200 tray [S-1302]. Drive writes slowed sharply after about 12 GiB, so Amodo estimated over 24 hours for a GB200 system with 15 TB of storage [S-1302]. Challenge-phase calculations were still to come [S-1302]. - Amodo's September 2026 update. Amodo attributed the slowdown to consumer drives' fast write cache running out [S-1303]. Six GPUs wiping one fast drive reached 6 minutes 41 seconds per TB [S-1303]. Amodo now expects \"~2.5 hours to wipe an NVL72 system's storage\", using a B200 label rate estimated from its GPU benchmarks, and reports that erasure speed, not drive speed, is the bottleneck [S-1303]. Integrating the disk wipe into a full-system wipe, and memory wiping into a full verification scheme, is still to come [S-1303]. - Public code. Amodo has published code for the disk-wiping path, which fills every sector of an NVMe drive with graph labels generated on NVIDIA GPUs [S-1321]. The repository excludes the verifier and the RAM and GPU-HBM session code [S-1321]. Limitations - Unreachable memory. SSD controller DRAM sits on a private bus that host commands cannot access [S-1303]. Amodo's code notes that overwriting by logical block address cannot reach over-provisioned or remapped blocks either [S-1321]. The optimized algorithm's 25 GiB of working memory is \"supposedly wiped but isn't attested\" [S-1303]. Amodo suggests fixes, such as wiping the HBM in several passes, but has not prototyped them [S-1303]. Amodo lists many other stores in a GB200 system and asks how network-switch memory could be wiped [S-1302]. - Outside help. In data centres, remote memory round trips of about 1–2 µs compare with about 70–200 ns for local DRAM, which the MIRI overview puts at roughly a tenfold timing margin [S-0018]. Evasion by pre-staging data is closed only by unpredictable, capacity-filling challenges [S-0018]. - Downtime. Filling a pod's volatile memory takes tens of minutes, and SSDs take hours [S-0018]. - Residual gap. Memory between the erased region and full capacity could hold data [S-0032]. Bursuc et al. say that closing this gap needs future work [S-0032].", "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "K-0003", "title": "Positive and negative claims", "url": "https://trustbutveri.fyi/concepts/positive-and-negative-claims/" }, { "id": "K-0012", "title": "Proof of space", "url": "https://trustbutveri.fyi/concepts/proof-of-space/" }, { "id": "K-0016", "title": "Undeclared compute", "url": "https://trustbutveri.fyi/concepts/undeclared-compute/" }, { "id": "O-0201", "title": "AI Futures Project", "url": "https://trustbutveri.fyi/organizations/ai-futures-project/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ] }, { "id": "M-0012", "slug": "model-identity-attestation", "title": "Model identity attestation", "aliases": [ "Model integrity verification", "Proving which model is served", "Weight commitment and attestation", "Model provenance attestation" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "one_liner": "Establishes that responses come from a specific, committed set of model weights, using enclave measurements or recomputation of sampled outputs.", "summary": "Model identity attestation shows users, auditors and regulators that a provider is serving the model weights it committed to. There are two routes. First, an attestation from a trusted execution environment shows that measured software enforced a hash commitment to the weights while the model ran. Second, a verifier holding the declared weights recomputes a sample of logged outputs, which can detect weights smuggled out in responses. Tinfoil reports running the enclave route commercially with its open-source Modelwrap tool. For unpublished weights, the commitment alone shows only that the same weights are served each time. Research prototypes bind evaluations to the same hash, so results apply to the served model. The enclave route inherits the weaknesses of TEEs, including published attacks that forge Intel TDX and AMD SEV-SNP attestations. Recomputation, tested on models of up to 30 billion parameters, needs trusted logging and randomness, and must tolerate numerical nondeterminism.", "technical": "- **Modelwrap build (Tinfoil's description).** The tool downloads a pinned model revision, normalizes its directory structure and builds an EROFS image. It computes a dm-verity root hash with veritysetup. The Merkle tree gives a 32-byte commitment to, for example, 140 GB of weights [[S-0013]].\n- **Binding and enforcement.** The root hash is passed on the kernel command line, which the enclave measurement covers. dm-verity then checks every block read against it and returns an error on any mismatch [[S-0013]].\n- **Reported costs.** Tinfoil reports a hash-tree overhead of about 0.8% of image size, and build times from 5 s for a 549 MB model to 13 min 25 s for a 554 GB model. Cold-cache loading takes about 80% longer, and inference is unaffected once the weights are in GPU memory [[S-0013]].\n- **PAL\\*M inference attestation.** PAL\\*M sets the Intel TDX REPORTDATA field to the concatenation of the operation, a challenge, and hashes of the inputs and outputs. It supports single-prompt and multi-turn inference attestations. Across three models of 3.8 to 8 billion parameters on an H100, PAL\\*M's added time was 3.8–11.4% of total run time for multi-turn sessions and 45.5–66.4% for single prompts, so a single attested prompt took 1.8 to 3 times as long as without PAL\\*M [[S-0012]].\n- **Fixed-seed sampling likelihood.** Rinberg et al. score the likelihood that each token was honestly sampled from the claimed model under a known seed, with estimators for inverse-probability-transform and Gumbel-max samplers. On a 30B mixture-of-experts model under benign prompt traffic, the check limits exfiltratable information to under 0.5% at a false-positive rate under 0.01%, a slowdown of more than 200 times for the adversary [[S-0015]]. An independent study found that an adversary who controls the prompts roughly doubles leakage per token, reducing the slowdown to 60–118 times [[S-1507]].", "category": "cryptographic-computational", "secondary_categories": [ "on-chip" ], "verifies": [ { "claim": "C-0005", "role": "primary", "note": "Core purpose: responses come from the declared weights." }, { "claim": "C-0006", "role": "supporting", "note": "Links an attested evaluation to the model later served (S-0009)." }, { "claim": "C-0009", "role": "supporting", "note": "The recomputation variant limits steganographic weight exfiltration through outputs (S-0015)." } ], "threat_model": "semi-trusted", "adversarial_evaluation": "independent-red-team", "hardware_requirement": "existing-features", "prover_cooperation": "required", "confidentiality": "partial", "depends_on": [ { "target": "M-0008", "note": "The enclave variant relies on TEE and GPU attestation." }, { "target": "M-0001", "note": "The recomputation variant is a form of sampled inference recomputation." } ], "readiness": { "assessment": true, "level": "R3", "scope": "showing users that a service runs the declared model weights", "rubric_version": "1.1", "rationale": "Tinfoil reports running the enclave route in a production service, but the independent attacks published so far left flaws open, including a critical flaw in the trusted execution environments that route relies on.\n\n- **R1** met: designs and assumptions are published for both routes [[S-0013]] [[S-0012]] [[S-0015]].\n- **R2** met through [[I-0006|Tinfoil's Modelwrap chain]]. Its code is open source [[S-1209]], it runs on NVIDIA H100, H200 or B200 GPUs with AMD SEV-SNP or Intel TDX (provider-reported) [[S-1206]], and it has been reported on models of up to 554 GB [[S-0013]]. Rinberg et al. publish code and results on models from 3B to 30B parameters [[S-0015]].\n- **R3** met on the provider's account: Tinfoil reports offering the feature in a production service [[S-1207]] [[S-1208]].\n- **R4** not met, because the independent evaluations that exist left flaws open. TEE.fail used physical access to forge the Intel TDX attestation that the enclave route relies on. By pairing the forgeries with relayed H100 attestations, it made a vLLM proxy running outside TEE protection pass both the TDX and the NVIDIA confidential-computing checks [[S-1202]]. That flaw is critical and open. Kezins, at Delft University of Technology, attacked the exfiltration bound of Rinberg et al.'s recomputation check with chosen prompts. On a 30B mixture-of-experts model this raised leakage from 0.119 to 0.286 bits per token. The attack widens the covert channel and does not target the check that outputs match the declared model [[S-1507]].", "evidence": [ "S-0013", "S-0012", "S-0015", "S-1206", "S-1207", "S-1208", "S-1209", "S-1202", "S-1507" ], "next_level_gaps": [ "Independent security evaluation of a deployed model-identity scheme that leaves no critical flaw open.", "Resistance of the enclave variant to physical attackers (see [[M-0008]]).", "Third-party verification for private models beyond consistency across requests.", "Tooling for audit-time checking of transparency records." ], "confidence": "medium", "assessed_by": [ "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Underlying attestation can be forged or relayed", "kind": "demonstrated-attack", "severity": "critical", "status": "open", "description": "The enclave variant is only as sound as the attestation. With physical access and root privileges, TEE.fail extracted a per-CPU Intel attestation signing key and forged TDX attestations. Pairing the forgeries with genuine H100 attestations relayed from a rented server, the researchers made a workload outside TEE protection pass both checks [[S-1202]]. The authors report that Intel and AMD treat interposer attacks as outside their threat models. They advise users to run confidential workloads only on physically secure servers [[S-1202]]. Other independent research forged AMD SEV-SNP attestations. Battering RAM did so with a DDR4 memory interposer, and RMPocalypse did so from a malicious hypervisor without physical access [[S-1210]] [[S-1212]]. AMD reports firmware fixes for RMPocalypse [[S-1213]]. Tinfoil's documentation lists physical attacks among the limitations of enclaves [[S-1206]].", "sources": [ "S-1202", "S-1210", "S-1212", "S-1213", "S-1206" ], "response": null }, { "assessment": true, "title": "Launch-state attestation does not by itself cover weights loaded later", "kind": "theoretical-argument", "severity": "significant", "status": "mitigated", "description": "Attestation measures launch state, and weights are read from disk after boot. A signature checked at load time does not stop a malicious hypervisor from altering the disk afterwards [[S-0013]]. Tinfoil reports mitigating this with dm-verity checks on every read [[S-0013]]. Unmeasured runtime configuration remains a general risk [[S-0014]].", "sources": [ "S-0013", "S-0014" ], "response": null }, { "assessment": true, "title": "For private models, a user can confirm consistency but not content", "kind": "open-question", "severity": "significant", "status": "open", "description": "When weights are not published, users can check that the same root hash is served each time, but not what the model is [[S-0013]]. Pairing the hash with an attested evaluation, as in Attestable Audits, is one proposed remedy [[S-0009]].", "sources": [ "S-0013", "S-0009" ], "response": null }, { "assessment": true, "title": "Recomputation depends on trusted logging and randomness, and its tolerance leaves a covert channel", "kind": "demonstrated-attack", "severity": "significant", "status": "open", "description": "The recomputation variant assumes that every input, output and seed is logged correctly, and that the attacker can neither predict nor manipulate which messages are sampled for verification. Legitimate nondeterminism concentrates at a few token positions, and slow leaks within the tolerated slack remain possible [[S-0015]]. An independent study showed that an adversary who controls the prompts roughly doubles the bits leaked per token, reducing the exfiltration slowdown from 146–254 times under benign prompts to 60–118 times. The attack targets the exfiltration bound, not the check that outputs match the declared model [[S-1507]].", "sources": [ "S-0015", "S-1507" ], "response": null } ], "blockers": [ { "text": "Attestation that resists physical attackers, for the enclave variant.", "theme": "hardware-trust", "blocked_by": "M-0008", "sources": [ "S-1202" ] }, { "text": "Numerical nondeterminism limits how tightly recomputation can pin down the model and sampling.", "theme": "protocol-soundness", "blocked_by": "M-0002", "sources": [ "S-0015" ] }, { "text": "The recomputation variant needs the verifier to hold the declared weights.", "theme": "access-governance", "blocked_by": null, "sources": [ "S-0015" ] } ], "challenge_themes": [ "evidence-binding", "hardware-trust", "protocol-soundness", "access-governance" ], "organizations": [ "O-0141", "O-0142", "O-0202" ], "people": [], "sources": [ { "source": "S-0013", "supports": "Modelwrap design, launch-state problem, signing comparison, private models, overheads (provider-reported)", "locator": "sections on the challenge, the three phases, performance, private models" }, { "source": "S-0012", "supports": "inference attestation binding hashes to TDX report; overheads", "locator": "§4, Table 6" }, { "source": "S-0009", "supports": "audit-to-inference model-hash binding; prototype evaluation of the audit step", "locator": "§3, Algorithms 1-3, §5" }, { "source": "S-0015", "supports": "recomputation-based verification, assumptions, results, limitations, code", "locator": "Abstract; §1, §3.3-3.4, §4.2, §5-§7" }, { "source": "S-1507", "supports": "independent prompt-control attack that widens the exfiltration bound; 0.119 to 0.286 bits per token on the 30B MoE model", "locator": "Abstract; Table 1" }, { "source": "S-0014", "supports": "measurement incompleteness; hashing-scheme warning" }, { "source": "S-3121", "supports": "Trail of Bits audit of WhatsApp Private Processing: environment variables loaded after the measurement (TOB-WAPI-13) and Meta's fix" }, { "source": "S-3124", "supports": "the review's finding that CVMs could be compromised through environment-variable injection" }, { "source": "S-1202", "supports": "Intel TDX attestation forgery and H100 attestation relay to a vLLM proxy outside TEE protection", "locator": "Abstract; §1.1, §8.3" }, { "source": "S-1210", "supports": "SEV-SNP attestation breach with a DDR4 interposer (Battering RAM)", "locator": "Abstract; site FAQ" }, { "source": "S-1212", "supports": "software-only SEV-SNP attestation forgery by a malicious hypervisor (RMPocalypse)", "locator": "Abstract; site" }, { "source": "S-1213", "supports": "AMD firmware fixes for RMPocalypse (vendor-reported)", "locator": "Mitigation tables" }, { "source": "S-1206", "supports": "Tinfoil hardware, trust model and documented limitations (provider-reported)" }, { "source": "S-1207", "supports": "measured boot chain, Sigstore publication, production model volumes (provider-reported)" }, { "source": "S-1208", "supports": "production deployment; no supported audit-time tool (provider-reported)", "locator": "In-band vs. out-of-band verification" }, { "source": "S-1209", "supports": "open-source implementation, release v0.3.0" }, { "source": "S-1800", "supports": "Apple PCC: integrity protection of code and model assets; attestation against a public transparency log (provider-reported)" }, { "source": "S-1810", "supports": "Gensyn's statement that Verde shows outputs came from the declared model and data (provider-reported)" }, { "source": "S-1809", "supports": "Verde's guarantee holds if at least one compute provider is honest", "locator": "Abstract" } ], "concepts": [ "K-0024", "K-0004", "K-0006", "K-0009", "K-0008", "K-0022", "K-0019" ], "complements": [ "M-0002", "M-0023" ], "alternatives": [ "M-0004" ], "type": "mechanism", "implementations": [ { "id": "I-0007", "title": "Attestable Audits", "url": "https://trustbutveri.fyi/implementations/attestable-audits/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" } ], "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/", "source_file": "content/mechanisms/model-identity-attestation.md", "flags_all": [ "provider-reported", "critical-flaw-open" ], "body_markdown": "## How it works\n\nTwo routes exist, and they can be combined.\n\n### Commit and attest\n\nThis route runs inside a TEE ([[M-0008]]). It closes a gap: attestation \"measures launch state, not runtime state\", while weights are read from disk after boot [[S-0013]]. Three designs bind the weights to the attestation:\n\n- **[[I-0006|Tinfoil's Modelwrap]]** packs the weights into a read-only image and computes a Merkle-tree root hash. It places that hash on the kernel command line, which the enclave measurement covers, and the kernel's dm-verity layer checks every block read against it [[S-0013]]. Clients compare the attestation's measurements with values that Tinfoil's build pipeline publishes to the Sigstore transparency log [[S-1207]]. For a public model, anyone can rebuild the image from the same Hugging Face commit and compare root hashes [[S-0013]].\n- **PAL\\*M** puts hashes of each operation's inputs, including the model, and of its outputs into the Intel TDX report, for single prompts and multi-turn sessions [[S-0012]].\n- **[[I-0007|Attestable Audits]]** records the model hash when an audit runs. At inference time it checks that the served model matches, then returns each response with an attestation that links model, prompt, output and audit result [[S-0009]].\n\n### Recompute and compare\n\nThis route uses [[M-0001|sampled recomputation]]. A verifier that holds the declared weights scores logged outputs by the likelihood that each token was sampled from that model under a known seed. The estimators allow for legitimate numerical nondeterminism [[S-0015]]. Rinberg et al. frame the check as a way to catch weights hidden steganographically in responses, and it also shows whether outputs are consistent with the declared model [[S-0015]]. Deterministic inference is covered in [[M-0002]].\n\n## What it establishes\nThe two routes establish different things:\n\n- **Commit and attest** can show that the bytes served match a commitment [[S-0013]].\n- **Recompute and compare** can show that logged outputs are consistent with the declared model's sampling procedure [[S-0015]].\n\nNeither shows what a private model can do, everything else that shapes behaviour, or properties that a weak hashing scheme leaves out:\n\n- With unpublished weights, a user can confirm only that the same weights are served each time [[S-0013]]. An attested evaluation bound to the same hash can close part of that gap [[S-0009]].\n- Tinfoil measures the firmware, kernel, initrd and a configuration file as well as the weights [[S-1207]]. Gloria Z warns that unmeasured runtime flags can undermine integrity [[S-0014]]. In a pre-launch audit of WhatsApp's TEE-based Private Processing, Trail of Bits found such a gap: configuration files with environment variables were loaded after the measurement, so an insider could have loaded malicious code without invalidating the attestation. Meta fixed it before launch [[S-3121]] [[S-3124]].\n- Gloria Z also warns that a scheme that sequences values loosely, or checks the weights too early in loading, could let two models with significantly different properties share a hash \"without breaking the hash function itself\" [[S-0014]].\n\n## Threat model\nThe enclave route trusts the hardware vendor and treats the host software as hostile. Tinfoil states \"You trust the hardware manufacturer\", but not the hypervisor or host OS [[S-1206]]. PAL\\*M excludes side channels and physical attacks [[S-0012]].\n\nThe recomputation route assumes the following [[S-0015]]:\n\n- the verification code is trusted;\n- every input, output and seed is logged correctly;\n- the attacker can neither predict nor manipulate which messages are sampled for verification;\n- the verifier sees prompts and responses and holds the weights.\n\n## Evidence\n- **[[I-0006|Tinfoil]].** Modelwrap is published under an MIT license [[S-1209]]. Tinfoil reports a storage overhead of 0.8%, build times of up to 13 min 25 s for a 554 GB model, and slower cold loading but no inference slowdown [[S-0013]].\n- **PAL\\*M.** It reports inference attestation on an H100 across three models of 3.8 to 8 billion parameters. Its added time was 3.8–11.4% of total run time in multi-turn sessions and 45.5–66.4% for single prompts. The authors plan to release the code after peer review [[S-0012]].\n- **[[I-0007|Attestable Audits]].** Its protocol binds inference to the audited model hash. The reported evaluation covers the audit step, for a 4-bit Llama-3.1-8B on CPU-only AWS Nitro Enclaves [[S-0009]].\n- **Rinberg et al.** They tested Llama-3.1-8B, Llama-3.2-3B and two Qwen mixture-of-experts models, and publish their code. On the 30B model under benign prompt traffic, the detector cut exfiltratable information to under 0.5% at a false-positive rate under 0.01% [[S-0015]]. An independent study found that prompt control roughly doubles leakage per token [[S-1507]].\n- **[[I-0013|Apple Private Cloud Compute]].** Apple reports that devices send requests only to servers that attest to software in a public transparency log, and that model assets share the code's integrity protection [[S-1800]].\n- **[[I-0015|Verde]].** Gensyn reports that its refereed-delegation system shows a delegated output came from the declared model and data, by re-running disputed operations with bitwise-reproducible kernels [[S-1810]]. The guarantee holds if at least one of the compute providers is honest [[S-1809]].\n\n## Limitations\n- **Inherited TEE attacks.** Using physical access, TEE.fail forged Intel TDX attestations and, by pairing them with relayed H100 attestations, passed a workload outside TEE protection [[S-1202]]. Battering RAM and RMPocalypse forged AMD SEV-SNP attestations, the second without physical access [[S-1210]] [[S-1212]]. Tinfoil's documentation acknowledges physical-access and side-channel risks [[S-1206]].\n- **Audit-time checking.** Tinfoil does not provide a supported tool for querying its transparency records at audit time [[S-1208]].\n- **Recomputation slack.** Nondeterminism concentrates at a few token positions, and very slow leaks remain possible [[S-0015]]. An adversary who controls prompts widens the leak [[S-1507]].\n- **Recomputation dependencies.** The check depends on complete and trusted logging, and on the verifier holding the weights [[S-0015]].", "body_text": "How it works Two routes exist, and they can be combined. Commit and attest This route runs inside a TEE (TEE remote attestation for AI workloads). It closes a gap: attestation \"measures launch state, not runtime state\", while weights are read from disk after boot [S-0013]. Three designs bind the weights to the attestation: - Tinfoil's Modelwrap packs the weights into a read-only image and computes a Merkle-tree root hash. It places that hash on the kernel command line, which the enclave measurement covers, and the kernel's dm-verity layer checks every block read against it [S-0013]. Clients compare the attestation's measurements with values that Tinfoil's build pipeline publishes to the Sigstore transparency log [S-1207]. For a public model, anyone can rebuild the image from the same Hugging Face commit and compare root hashes [S-0013]. - PAL*M puts hashes of each operation's inputs, including the model, and of its outputs into the Intel TDX report, for single prompts and multi-turn sessions [S-0012]. - Attestable Audits records the model hash when an audit runs. At inference time it checks that the served model matches, then returns each response with an attestation that links model, prompt, output and audit result [S-0009]. Recompute and compare This route uses sampled recomputation. A verifier that holds the declared weights scores logged outputs by the likelihood that each token was sampled from that model under a known seed. The estimators allow for legitimate numerical nondeterminism [S-0015]. Rinberg et al. frame the check as a way to catch weights hidden steganographically in responses, and it also shows whether outputs are consistent with the declared model [S-0015]. Deterministic inference is covered in Deterministic and bit-exact inference. What it establishes The two routes establish different things: - Commit and attest can show that the bytes served match a commitment [S-0013]. - Recompute and compare can show that logged outputs are consistent with the declared model's sampling procedure [S-0015]. Neither shows what a private model can do, everything else that shapes behaviour, or properties that a weak hashing scheme leaves out: - With unpublished weights, a user can confirm only that the same weights are served each time [S-0013]. An attested evaluation bound to the same hash can close part of that gap [S-0009]. - Tinfoil measures the firmware, kernel, initrd and a configuration file as well as the weights [S-1207]. Gloria Z warns that unmeasured runtime flags can undermine integrity [S-0014]. In a pre-launch audit of WhatsApp's TEE-based Private Processing, Trail of Bits found such a gap: configuration files with environment variables were loaded after the measurement, so an insider could have loaded malicious code without invalidating the attestation. Meta fixed it before launch [S-3121] [S-3124]. - Gloria Z also warns that a scheme that sequences values loosely, or checks the weights too early in loading, could let two models with significantly different properties share a hash \"without breaking the hash function itself\" [S-0014]. Threat model The enclave route trusts the hardware vendor and treats the host software as hostile. Tinfoil states \"You trust the hardware manufacturer\", but not the hypervisor or host OS [S-1206]. PAL*M excludes side channels and physical attacks [S-0012]. The recomputation route assumes the following [S-0015]: - the verification code is trusted; - every input, output and seed is logged correctly; - the attacker can neither predict nor manipulate which messages are sampled for verification; - the verifier sees prompts and responses and holds the weights. Evidence - Tinfoil. Modelwrap is published under an MIT license [S-1209]. Tinfoil reports a storage overhead of 0.8%, build times of up to 13 min 25 s for a 554 GB model, and slower cold loading but no inference slowdown [S-0013]. - PAL*M. It reports inference attestation on an H100 across three models of 3.8 to 8 billion parameters. Its added time was 3.8–11.4% of total run time in multi-turn sessions and 45.5–66.4% for single prompts. The authors plan to release the code after peer review [S-0012]. - Attestable Audits. Its protocol binds inference to the audited model hash. The reported evaluation covers the audit step, for a 4-bit Llama-3.1-8B on CPU-only AWS Nitro Enclaves [S-0009]. - Rinberg et al. They tested Llama-3.1-8B, Llama-3.2-3B and two Qwen mixture-of-experts models, and publish their code. On the 30B model under benign prompt traffic, the detector cut exfiltratable information to under 0.5% at a false-positive rate under 0.01% [S-0015]. An independent study found that prompt control roughly doubles leakage per token [S-1507]. - Apple Private Cloud Compute. Apple reports that devices send requests only to servers that attest to software in a public transparency log, and that model assets share the code's integrity protection [S-1800]. - Verde. Gensyn reports that its refereed-delegation system shows a delegated output came from the declared model and data, by re-running disputed operations with bitwise-reproducible kernels [S-1810]. The guarantee holds if at least one of the compute providers is honest [S-1809]. Limitations - Inherited TEE attacks. Using physical access, TEE.fail forged Intel TDX attestations and, by pairing them with relayed H100 attestations, passed a workload outside TEE protection [S-1202]. Battering RAM and RMPocalypse forged AMD SEV-SNP attestations, the second without physical access [S-1210] [S-1212]. Tinfoil's documentation acknowledges physical-access and side-channel risks [S-1206]. - Audit-time checking. Tinfoil does not provide a supported tool for querying its transparency records at audit time [S-1208]. - Recomputation slack. Nondeterminism concentrates at a few token positions, and very slow leaks remain possible [S-0015]. An adversary who controls prompts widens the leak [S-1507]. - Recomputation dependencies. The check depends on complete and trusted logging, and on the verifier holding the weights [S-0015].", "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0007", "title": "Attestable Audits", "url": "https://trustbutveri.fyi/implementations/attestable-audits/" }, { "id": "I-0010", "title": "RAND secure inference data center (SIDC) design", "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "O-0141", "title": "Tinfoil", "url": "https://trustbutveri.fyi/organizations/tinfoil/" }, { "id": "O-0142", "title": "University of Cambridge", "url": "https://trustbutveri.fyi/organizations/university-of-cambridge/" } ] }, { "id": "M-0013", "slug": "network-taps-and-certifiers", "title": "Network taps and certifiers", "aliases": [ "Secure network taps", "Cluster I/O fingerprinting", "Secure Gateway Device" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Devices on a cluster's network links that copy and hash all traffic, so a verifier can later check sampled records against declared work.", "summary": "Network taps copy the traffic on a data centre's cables. In verification proposals, a tap or \"secure gateway\" hashes everything entering and leaving a cluster. The verifier later asks the operator to reveal randomly chosen records, which are recomputed and checked. The aim is to show that outputs came from declared work, and that results of undeclared work cannot leave covertly through the tapped links. Designs, cost estimates and covert-channel analyses are public. Amodo's tests have copied traffic from 1–10 Gbps links without hashing it. As of September 2026 no tap has been demonstrated at production link rates or red-teamed. The main obstacles are hashing minimum-size frames at line rate on 400G-class links, and tapping copper or encrypted fabrics. The largest known weakness is covert signalling in model outputs: one design estimates about 40 Mbit/s for a 200,000-GPU inference cluster, even with replay checks, unless inference becomes deterministic.", "technical": "Secure Gateway Device (SeGaDev) [[S-1300]]:\n\n- **Placement.** The device sits on the north-south boundary, between TLS termination at the frontend gateway and the internal fabric. If traffic is end-to-end encrypted internally, the device hashes ciphertext and the prover stores session keys [[S-1300]].\n- **Pipeline.** (1) Sanitization by an active warden, which replaces exploitable header fields (ephemeral source ports, TCP initial sequence numbers, session IDs) with pseudorandom values, keeps connection-tracking tables, and releases buffered packets on a fixed schedule to suppress timing and analogue modulation; (2) hashing of a configurable group of packets (the \"hashing boundary\") over payloads only; (3) transmission of hashes alone to the verifier on a dedicated fibre, with a passive splitter giving the prover a copy and a prover-controlled \"timing warden\" re-timing the hash stream [[S-1300]].\n- **Shared randomness and timestamps.** Shared pseudorandom values come from a commit-then-reveal coin flip between prover- and verifier-supplied generators, precomputed in bulk and consumed in FIFO order. Timestamps are taken by both parties on receipt of hashes, not by the tap [[S-1300]] [[S-0018]].\n- **Audit.** The verifier selects stored hashes; the prover returns preimages (prompt, output, token start index, token count) to a verification facility that replays the computation and compares hashes. With k random challenges, a misreported fraction p is detected with probability 1 − (1 − p)^k; k = 3,200 detects 0.1% misreporting at 96% confidence [[S-1300]].\n- **Storage.** A pessimistic upper bound (100k Hopper GPUs at 2,000 tokens/s each, one token per streaming frame) is about 200 PB of SHA-256 hashes and 1,260 PB of prover-held plaintext per year, about €22.5 million in drives [[S-1300]].\n- **Dictionary attacks.** Against verifier dictionary attacks on hashes of short outputs, the paper recommends at least 5 tokens per hashed template, or at least 10 if the attacker filters for likely tokens [[S-1300]].\n- **Residual covert channels.** A 10 ms re-timing window at 100 Gbps holds up to about 625k packets, so packet-count modulation must be fixed per connection per window. Output nondeterminism leaves about 0.1 bit per token even with seed-synchronized replay, which gives about 40 Mbit/s for 200k GPUs at 2,000 tokens/GPU/s. Feigned hardware failures give about 1 kbit/day [[S-1300]].\n- **Line-rate hashing (Amodo).** On an AMD EPYC 9354P with DPDK, all tested algorithms except BLAKE3 reached the receive baseline at 1500-byte frames once enough threads were used; at 64-byte frames none reached line rate within the CPU's physical core count. Offloading to the tested DPU did not reach the target. SipHash-1-3-128 was the leading candidate, and FPGA or ASIC hashing is expected to be needed for some plans [[S-1311]].\n- **Passive optics (Amodo).** Amodo computes a 1.45 dB link margin, which it calls tight, for an example 53 GBaud link; a 95/5 split costs about 0.25 dB on the live leg and about 13 dB on the monitor leg [[S-1310]].", "category": "off-chip-devices", "secondary_categories": [ "cryptographic-computational" ], "verifies": [ { "claim": "C-0004", "role": "primary", "note": "Supplies the committed record of cluster I/O that recomputation checks against declared inference (S-0067, S-1300)." }, { "claim": "C-0009", "role": "supporting", "note": "Aims to make covert exfiltration of results through tapped links infeasible (S-1300)." }, { "claim": "C-0005", "role": "supporting", "note": "Replaying challenged records with the declared model checks which model produced outputs (S-1300, S-0067)." } ], "threat_model": "adversarial", "adversarial_evaluation": "analysis", "hardware_requirement": "retrofit-device", "prover_cooperation": "required", "confidentiality": "partial", "depends_on": [ { "target": "M-0001", "note": "Challenged records are checked by recomputing them." }, { "target": "M-0002", "note": "Bit-exact replay is the main proposed way to remove covert capacity in model outputs." }, { "target": "M-0017", "note": "Taps and gateway devices must be physically protected and the facility monitored so all traffic passes through them." }, { "target": "M-0022", "note": "Radio, power-line and thermal channels bypass the tapped links." } ], "readiness": { "assessment": true, "level": "R1", "scope": "committing a complete record of cluster traffic, so declared inference can be checked", "rubric_version": "1.1", "rationale": "The Secure Gateway Device and related designs are public, but no tapped prototype has run at production link rates or with an active warden.\n\n- **R1** met: the Secure Gateway Device paper specifies the architecture, its goal (no covert exfiltration of results through tapped wires), its assumptions (physical monitoring, prover-held plaintext) and covert-channel estimates [[S-1300]]. Feasibility and cost analyses cover north-south, storage and compute-fabric taps [[S-0031]]. The MIRI overview places taps in a full reference architecture [[S-0018]], and the AI 2040 plan relies on them [[S-0067]].\n- **R2** not met: Amodo reports a passive fibre tap on a 1 Gbps lab link, with all 800 test inferences tapped and verified during a throughput stress test that flooded the link with non-whitelisted traffic [[S-1312]]. That result is provider-reported on a blog, at link rates far below production frontend links, with no hashing of traffic contents and no test against an adversary [[S-1312]]. Amodo says the passive tap must become active to close side channels [[S-1312]]. No code is linked, and Amodo's public prototype code captures traffic with a software sidecar, not the tap [[S-1007]]. The most mature Implementation, SASH's [[I-0008|confidential network logger]], has public code, but it runs a toy model through an inline software proxy that computes no hashes [[S-1319]] [[S-1320]]. [[I-0011]] and [[I-0012]] are proposed architectures assessed R1. The Secure Gateway Device paper states that empirical validation and red-teaming are still required [[S-1300]].\n\nConfidence is medium: the Amodo test uses real passive tap hardware, but at 1 Gbps, as a stress test rather than against an adversary, and without the hashed record the designs rely on.", "evidence": [ "S-1300", "S-0031", "S-0018", "S-0067", "S-1312", "S-1007", "S-1319", "S-1320" ], "next_level_gaps": [ "An end-to-end tapped prototype with public code, or with results described in enough detail for an independent team to repeat.", "Hashing and tapping demonstrated at production frontend link rates (400 Gbps class), including minimum-size frames.", "A built active warden or Secure Gateway Device, with measured residual covert bandwidth.", "Red-teaming of tap bypass, covert channels and physical security." ], "confidence": "medium", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Output nondeterminism leaves covert capacity", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Hashing cannot remove information hidden in the outputs themselves. The Secure Gateway Device paper estimates that about 0.1 bit per token remains even with seed-synchronized replay checks. For a 200k-GPU inference cluster at full load (2,000 tokens per GPU per second), that is about 40 Mbit/s of covert egress, enough to move a 1 TB model in under three days. The paper names this the core remaining challenge and points to deterministic replay or active scrubbing of hardware-induced entropy [[S-1300]]. An independent study found that an adversary who chooses the prompts roughly doubles the bits leaked per token under Gumbel-based inference verification [[S-1507]]; see [[M-0024]].", "sources": [ "S-1300", "S-1507" ], "response": null }, { "assessment": true, "title": "Some links cannot be passively tapped", "kind": "open-question", "severity": "significant", "status": "open", "description": "Cankaya notes that copper-connected scale-up domains (for example NVL72 racks and TPU v7 cubes) are much harder to tap than fibre, and that optical budgets make passive taps impractical on 400GBASE-SR8 multimode links [[S-0031]]. Amodo found no taps advertised for 53 GBaud links as of May 2026 [[S-1310]].", "sources": [ "S-0031", "S-1310" ], "response": null }, { "assessment": true, "title": "Encrypted fabrics hide plaintext from both parties", "kind": "open-question", "severity": "significant", "status": "open", "description": "Cankaya notes that with TEE-protected sessions whose keys are ephemeral and managed inside the TEE, neither the operator nor the manufacturer can recover session keys after the session, so tapped traffic could not be opened for recomputation. For other encrypted fabrics, the operator can retain keys [[S-0031]].", "sources": [ "S-0031" ], "response": null }, { "assessment": true, "title": "Residual side channels in simple passive setups", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Amodo's analysis of its own tapped prototype lists unvalidated header fields, timing of permitted traffic and variation in response formatting as residual channels, and concludes that the passive tap must be replaced by an active one [[S-1312]].", "sources": [ "S-1312" ], "response": null }, { "assessment": true, "title": "Completeness rests on physical monitoring left out of scope", "kind": "open-question", "severity": "significant", "status": "open", "description": "The Secure Gateway Device paper assumes the facility is physically monitored, and states that the whole architecture depends on the device being the only communication channel. It names radio emanation, power-line signalling and thermal channels as covert channels beyond that scope [[S-1300]].", "sources": [ "S-1300" ], "response": null }, { "assessment": true, "title": "Verifier dictionary attacks on hashes", "kind": "theoretical-argument", "severity": "minor", "status": "mitigated", "description": "Hashes of very short outputs could be brute-forced by the verifier. The paper recommends hashing at least 5 tokens together, or at least 10 if the attacker filters for likely tokens [[S-1300]].", "sources": [ "S-1300" ], "response": null } ], "blockers": [ { "text": "No complete verification tap has been demonstrated at production frontend link rates, and on the tested CPU no hash algorithm reached line rate with minimum-size frames.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-1311", "S-1310" ] }, { "text": "Nondeterministic inference leaves covert capacity in outputs that hashing cannot remove.", "theme": "evidence-binding", "blocked_by": "M-0002", "sources": [ "S-1300" ] }, { "text": "Taps and gateway devices need tamper-evident housing and physical monitoring so that traffic cannot bypass them.", "theme": "hardware-trust", "blocked_by": "M-0017", "sources": [ "S-1300", "S-0018" ] }, { "text": "Radio, power-line and thermal channels are not addressed by network-level designs.", "theme": "coverage-hidden-compute", "blocked_by": "M-0022", "sources": [ "S-1300" ] }, { "text": "Red-teaming by specialists is called for but has not been reported.", "theme": "adversarial-validation", "blocked_by": null, "sources": [ "S-1300" ] } ], "challenge_themes": [ "evidence-binding", "coverage-hidden-compute", "performance-compatibility", "hardware-trust", "privacy-leakage", "adversarial-validation" ], "organizations": [ "O-0101", "O-0160", "O-0201", "O-0202", "O-0209" ], "people": [], "sources": [ { "source": "S-1300", "supports": "Secure Gateway Device architecture, goal, assumptions, scope, covert-channel estimates, storage, dictionary attacks, need for validation", "locator": "abstract; §1; §3; §4.1–4.2; §5.1–5.3; §6; appendices A–C" }, { "source": "S-0031", "supports": "tap definition; placement options and cost fractions; copper, multimode and encryption limits", "locator": "sections on north-south, storage and compute-fabric taps; open problems" }, { "source": "S-0018", "supports": "capture-everything principle; hashing over packet groups; failure conditions; perimeter size; commercial precedents; physical monitoring", "locator": "§2b; §3.1–3.2; §5.1.1" }, { "source": "S-0067", "supports": "passive frontend taps feeding a recomputation server", "locator": "inference-only retrofitting proposal" }, { "source": "S-1309", "supports": "first passive optical tap test at about 10 Gbps", "locator": "whole note" }, { "source": "S-1310", "supports": "COTS tap availability by baud rate; optical budget; possible need for a new ASIC", "locator": "BLUF; passive and active tap sections" }, { "source": "S-1311", "supports": "line-rate hashing benchmarks on CPU and DPU", "locator": "results and conclusions" }, { "source": "S-1312", "supports": "tapped 1 Gbps lab prototype, whitelist, flood stress-test result, no hashing of traffic contents, residual side channels, need for active tap", "locator": "setup, results and side-channel sections" }, { "source": "S-1511", "supports": "companion page's 400G component status and unresolved installation and monitoring", "locator": "State of Play: Network taps on the frontend network; Physical security" }, { "source": "S-1007", "supports": "Amodo's public prototype code captures traffic with a software sidecar, not a physical tap", "locator": "README" }, { "source": "S-1319", "supports": "public prototype code of a network-logger demo: inline proxy, no hashing, 270M model", "locator": "README; components/" }, { "source": "S-1320", "supports": "confidential network logger prototype and planned FPGA certificates", "locator": "whole post" }, { "source": "S-1507", "supports": "prompt-controlled attack roughly doubling per-token leakage under Gumbel-based inference verification", "locator": "abstract" } ], "concepts": [ "K-0014", "K-0024", "K-0009", "K-0020", "K-0013", "K-0022", "K-0008", "K-0019", "K-0001", "K-0002" ], "complements": [ "M-0003", "M-0014", "M-0015", "M-0016", "M-0024" ], "alternatives": [ "M-0008" ], "type": "mechanism", "implementations": [ { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "I-0008", "title": "SASH confidential network logger", "url": "https://trustbutveri.fyi/implementations/sash-confidential-network-logger/" } ], "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/", "source_file": "content/mechanisms/network-taps-and-certifiers.md", "flags_all": [], "body_markdown": "## How it works\n\nA network tap is a device that \"mirror[s] network traffic without disrupting normal operation\" [[S-0031]]. Verification proposals use taps to build a record of what crossed a link, which the verifier can check later [[S-0031]] [[S-0018]]. The MIRI system overview states the principle as \"capture everything, challenge and audit random samples after\" [[S-0018]].\n\nIn the most detailed designs, the verifier never receives plaintext [[S-0018]] [[S-1300]]. The tap hashes payloads over groups of packets, without knowing where objects such as prompts begin or end, and sends only hashes out; the operator (the [[K-0001|prover]]) keeps the plaintext [[S-0018]] [[S-1300]]. Later the verifier picks stored hashes at random, the prover supplies the matching data, and a verification facility replays the computation and checks that the hashes match [[S-1300]]. A missing or mismatching preimage is a failure; a replay that does not reproduce points to false metadata or a technical fault [[S-0018]].\n\nThe AI 2040 plan proposes a simpler variant: passive taps on the frontend network copy all input and output traffic to a recomputation server ([[M-0001]], [[M-0003]]) [[S-0067]].\n\nTaps can sit at three levels [[S-0031]]:\n\n- **North-south (frontend) links** to the outside world, at an estimated material cost under 0.01% of the monitored data centre [[S-0031]].\n- **The storage fabric**, which carries model loads, dataset reads and checkpoint saves, at an estimated 0.3–0.5% of data-centre cost [[S-0031]].\n- **The compute (east-west) fabric**, at an estimated 0.2–1.5% of data-centre cost for its optical levels. Volumes there are so high that the note proposes sampling, for example 0.01% of traffic, about 20 Gb/s, through an optical circuit switch [[S-0031]].\n\nFor inference, the monitored perimeter could be as small as pods of a few hundred accelerators; monitoring training needs larger perimeters or taps on the compute and storage fabrics [[S-0018]].\n\nA key aim of the \"Secure Gateway Device\" design is that no processor has to be trusted by both sides [[S-1300]]. It uses passive optical splitters, so that each party can watch the signals it must check, and coin-flip protocols to generate shared random values [[S-1300]]. It also acts as an \"active warden\", rewriting free header fields and re-timing packets to close timing, analogue and header channels that checks on hashes cannot address [[S-1300]].\n\nSASH's planned next prototype would place an FPGA between the cluster and its gateway to issue \"cryptographic certificates attesting to the encrypted traffic it has seen\" [[S-1320]]; see [[I-0008]].\n\n## What it establishes\nThe Secure Gateway Device paper aims to make it \"infeasible to covertly exfiltrate the results of undisclosed workloads in the cluster through the tapped wires\" [[S-1300]]. Combined with replay, it would show that recorded outputs match declared workloads [[S-1300]] [[S-0067]]; see [[C-0004]] and [[C-0009]].\n\nIt does not establish:\n\n- Taps capture flows, not device state, and the MIRI overview treats memory probes as complementary [[S-0018]].\n- A north-south tap does not directly prevent covert workloads, only the exfiltration of their results [[S-1300]]. The paper judges it futile to block every single-bit leak. It argues that the residual risk, covert computation plus low-bandwidth signalling of results, needs other checks such as east-west taps and throughput accounting [[S-1300]].\n- The paper does not decide which workloads should be allowed, or whether declared workloads comply [[S-1300]].\n\n## Threat model\n- The operator and the verifier distrust each other. The operator must not hide information in the traffic, and the verifier must learn nothing beyond hashes [[S-1300]].\n- The facility is physically monitored, so that all traffic passes through the gateway [[S-1300]].\n- The operator stores all hashed data for the agreement's duration [[S-1300]].\n- Replay needs reproducible inference ([[M-0002]]) [[S-0018]] [[S-1300]].\n- The MIRI overview models a covert adversary, deterred when the likelihood of detection times the cost of being caught exceeds the expected gain [[S-0018]].\n\n## Evidence\n- **Designs and analysis.** The Secure Gateway Device paper, Cankaya's feasibility note and the MIRI overview are design studies [[S-1300]] [[S-0031]] [[S-0018]]. The paper states that empirical validation is still required and calls for red-teaming by specialists [[S-1300]].\n- **Commercial precedent.** Taps are used for stock-exchange monitoring, packet capture, defence networks and lawful intercept [[S-0018]]. The MIRI overview says AI use needs changes for a nation-state covert adversary [[S-0018]].\n- **Amodo's first tap test (March 2026).** A passive multimode splitter on a link of about 10 Gbps copied both directions of traffic to a monitoring machine [[S-1309]].\n- **Amodo's tap survey (May 2026).** Commodity taps exist for 26 GBaud links, but none are advertised for 53 GBaud [[S-1310]].\n- **Amodo's hashing benchmarks (July 2026).** On an AMD EPYC server CPU on a 400 Gbps link, all tested hash algorithms except BLAKE3 kept up with 1500-byte frames [[S-1311]]. None reached line rate with 64-byte frames within the CPU's physical core count, and the tested DPU fell short of the 400 Gbps target [[S-1311]].\n- **Amodo's tapped prototype (September 2026).** A passive fibre tap on a 1 Gbps lab link of its inference-verification prototype fed a frame processor, which rebuilt inference requests and checked all other traffic against a whitelist [[S-1312]]. Amodo reports that all 800 test inferences were tapped and verified during a stress test that flooded the link with about 1 Gbps of non-whitelisted traffic [[S-1312]]. The frame processor computes no hashes of the traffic contents [[S-1312]].\n- **Production boundary.** The AI 2040 companion page reports passive optical tap components working at 400G [[S-1511]]. The published inference-verification tap test ran at 1 Gbps, without content hashes or an active warden [[S-1312]]. Neither result demonstrates a complete monitored facility perimeter.\n- **SASH logger.** SASH's public prototype passes every request through an inline software logger and re-runs it on a 270-million-parameter model; it computes no hashes of the traffic [[S-1319]]. See [[I-0008]].\n\n## Limitations\n- **Output steganography.** The paper estimates about 40 Mbit/s of covert egress from output nondeterminism for a 200k-GPU cluster, even with replay checks [[S-1300]]. An attacker who chooses the prompts roughly doubles per-token leakage under Gumbel-based verification [[S-1507]]; see [[M-0002]] and [[M-0024]].\n- **Line rate.** On the tested CPU, no hash algorithm kept up with minimum-size frames within its physical core count, and the tested DPU was insufficient [[S-1311]].\n- **Optics and copper.** Passive taps are impractical on 400GBASE-SR8 multimode links, and copper-connected racks are much harder to tap [[S-0031]]. For 53 and 106 GBaud links, Amodo says a new ASIC may be needed [[S-1310]].\n- **Encryption.** TEE-protected fabrics would hide plaintext from both parties [[S-0031]].\n- **Passive-tap side channels.** Amodo concludes that its passive tap must become active [[S-1312]].\n- **Bypass.** Physical monitoring, and radio, power-line and thermal channels, are outside the network design [[S-1300]]; see [[M-0017]] and [[M-0022]].", "body_text": "How it works A network tap is a device that \"mirror[s] network traffic without disrupting normal operation\" [S-0031]. Verification proposals use taps to build a record of what crossed a link, which the verifier can check later [S-0031] [S-0018]. The MIRI system overview states the principle as \"capture everything, challenge and audit random samples after\" [S-0018]. In the most detailed designs, the verifier never receives plaintext [S-0018] [S-1300]. The tap hashes payloads over groups of packets, without knowing where objects such as prompts begin or end, and sends only hashes out; the operator (the prover) keeps the plaintext [S-0018] [S-1300]. Later the verifier picks stored hashes at random, the prover supplies the matching data, and a verification facility replays the computation and checks that the hashes match [S-1300]. A missing or mismatching preimage is a failure; a replay that does not reproduce points to false metadata or a technical fault [S-0018]. The AI 2040 plan proposes a simpler variant: passive taps on the frontend network copy all input and output traffic to a recomputation server (Sampled inference recomputation, Whole-workload recomputation (reproducible packets)) [S-0067]. Taps can sit at three levels [S-0031]: - North-south (frontend) links to the outside world, at an estimated material cost under 0.01% of the monitored data centre [S-0031]. - The storage fabric, which carries model loads, dataset reads and checkpoint saves, at an estimated 0.3–0.5% of data-centre cost [S-0031]. - The compute (east-west) fabric, at an estimated 0.2–1.5% of data-centre cost for its optical levels. Volumes there are so high that the note proposes sampling, for example 0.01% of traffic, about 20 Gb/s, through an optical circuit switch [S-0031]. For inference, the monitored perimeter could be as small as pods of a few hundred accelerators; monitoring training needs larger perimeters or taps on the compute and storage fabrics [S-0018]. A key aim of the \"Secure Gateway Device\" design is that no processor has to be trusted by both sides [S-1300]. It uses passive optical splitters, so that each party can watch the signals it must check, and coin-flip protocols to generate shared random values [S-1300]. It also acts as an \"active warden\", rewriting free header fields and re-timing packets to close timing, analogue and header channels that checks on hashes cannot address [S-1300]. SASH's planned next prototype would place an FPGA between the cluster and its gateway to issue \"cryptographic certificates attesting to the encrypted traffic it has seen\" [S-1320]; see SASH confidential network logger. What it establishes The Secure Gateway Device paper aims to make it \"infeasible to covertly exfiltrate the results of undisclosed workloads in the cluster through the tapped wires\" [S-1300]. Combined with replay, it would show that recorded outputs match declared workloads [S-1300] [S-0067]; see This compute runs inference, not training and Model weights have not left the facility. It does not establish: - Taps capture flows, not device state, and the MIRI overview treats memory probes as complementary [S-0018]. - A north-south tap does not directly prevent covert workloads, only the exfiltration of their results [S-1300]. The paper judges it futile to block every single-bit leak. It argues that the residual risk, covert computation plus low-bandwidth signalling of results, needs other checks such as east-west taps and throughput accounting [S-1300]. - The paper does not decide which workloads should be allowed, or whether declared workloads comply [S-1300]. Threat model - The operator and the verifier distrust each other. The operator must not hide information in the traffic, and the verifier must learn nothing beyond hashes [S-1300]. - The facility is physically monitored, so that all traffic passes through the gateway [S-1300]. - The operator stores all hashed data for the agreement's duration [S-1300]. - Replay needs reproducible inference (Deterministic and bit-exact inference) [S-0018] [S-1300]. - The MIRI overview models a covert adversary, deterred when the likelihood of detection times the cost of being caught exceeds the expected gain [S-0018]. Evidence - Designs and analysis. The Secure Gateway Device paper, Cankaya's feasibility note and the MIRI overview are design studies [S-1300] [S-0031] [S-0018]. The paper states that empirical validation is still required and calls for red-teaming by specialists [S-1300]. - Commercial precedent. Taps are used for stock-exchange monitoring, packet capture, defence networks and lawful intercept [S-0018]. The MIRI overview says AI use needs changes for a nation-state covert adversary [S-0018]. - Amodo's first tap test (March 2026). A passive multimode splitter on a link of about 10 Gbps copied both directions of traffic to a monitoring machine [S-1309]. - Amodo's tap survey (May 2026). Commodity taps exist for 26 GBaud links, but none are advertised for 53 GBaud [S-1310]. - Amodo's hashing benchmarks (July 2026). On an AMD EPYC server CPU on a 400 Gbps link, all tested hash algorithms except BLAKE3 kept up with 1500-byte frames [S-1311]. None reached line rate with 64-byte frames within the CPU's physical core count, and the tested DPU fell short of the 400 Gbps target [S-1311]. - Amodo's tapped prototype (September 2026). A passive fibre tap on a 1 Gbps lab link of its inference-verification prototype fed a frame processor, which rebuilt inference requests and checked all other traffic against a whitelist [S-1312]. Amodo reports that all 800 test inferences were tapped and verified during a stress test that flooded the link with about 1 Gbps of non-whitelisted traffic [S-1312]. The frame processor computes no hashes of the traffic contents [S-1312]. - Production boundary. The AI 2040 companion page reports passive optical tap components working at 400G [S-1511]. The published inference-verification tap test ran at 1 Gbps, without content hashes or an active warden [S-1312]. Neither result demonstrates a complete monitored facility perimeter. - SASH logger. SASH's public prototype passes every request through an inline software logger and re-runs it on a 270-million-parameter model; it computes no hashes of the traffic [S-1319]. See SASH confidential network logger. Limitations - Output steganography. The paper estimates about 40 Mbit/s of covert egress from output nondeterminism for a 200k-GPU cluster, even with replay checks [S-1300]. An attacker who chooses the prompts roughly doubles per-token leakage under Gumbel-based verification [S-1507]; see Deterministic and bit-exact inference and Bounding unexplained information in outputs. - Line rate. On the tested CPU, no hash algorithm kept up with minimum-size frames within its physical core count, and the tested DPU was insufficient [S-1311]. - Optics and copper. Passive taps are impractical on 400GBASE-SR8 multimode links, and copper-connected racks are much harder to tap [S-0031]. For 53 and 106 GBaud links, Amodo says a new ASIC may be needed [S-1310]. - Encryption. TEE-protected fabrics would hide plaintext from both parties [S-0031]. - Passive-tap side channels. Amodo concludes that its passive tap must become active [S-1312]. - Bypass. Physical monitoring, and radio, power-line and thermal channels, are outside the network design [S-1300]; see Tamper evidence for verifier devices and Side-channel suppression for isolated facilities.", "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" }, { "id": "M-0005", "title": "Zero-knowledge proofs of training constraints", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-training-constraints/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "I-0008", "title": "SASH confidential network logger", "url": "https://trustbutveri.fyi/implementations/sash-confidential-network-logger/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0024", "title": "Cryptographic commitment", "url": "https://trustbutveri.fyi/concepts/cryptographic-commitment/" }, { "id": "K-0019", "title": "Evidence binding", "url": "https://trustbutveri.fyi/concepts/evidence-binding/" }, { "id": "K-0014", "title": "Network tap", "url": "https://trustbutveri.fyi/concepts/network-tap/" }, { "id": "O-0201", "title": "AI Futures Project", "url": "https://trustbutveri.fyi/organizations/ai-futures-project/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" }, { "id": "O-0209", "title": "Hardware AI Governance Lab", "url": "https://trustbutveri.fyi/organizations/oxford-hardware-ai-governance-lab/" }, { "id": "O-0160", "title": "Singapore AI Safety Hub (SASH)", "url": "https://trustbutveri.fyi/organizations/singapore-ai-safety-hub/" }, { "id": "O-0203", "title": "Verifiable Compute Foundation", "url": "https://trustbutveri.fyi/organizations/verifiable-compute-foundation/" } ] }, { "id": "M-0010", "slug": "on-chip-telemetry", "title": "On-chip telemetry from timing, memory and performance counters", "aliases": [ "GPU telemetry", "Performance counters", "NVML telemetry", "Compute metering", "VRAM residency challenges" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Uses timing, memory-residency and performance-counter signals measured on AI accelerators as evidence about which workloads they are running.", "summary": "On-chip telemetry uses signals measured on AI accelerators as evidence about their workloads, for example to tell training from inference or to check that model weights are held in a chip's own memory. Accelerators already track power, clock rates, memory use and operation counts, and challenge programs can time how quickly a chip completes set tasks. Studies on NVIDIA GPUs from the T4 to the B200 show that such signals can distinguish workloads. One classifier spotted training with 98.2% accuracy across its authors' corpus, but caught the hardest unseen disguised workloads only 43–87% of the time after five rounds of hardening. The biggest obstacle is trust in the data: an operator who controls the software stack can forge standard telemetry unless the read path is protected in hardware. The signals give statistical evidence, not proof. Counters can also leak secrets, and NVIDIA disables them in confidential-computing mode for that reason.", "technical": "- **Counter-based classification.** Rahman and Tajdari sample nine always-available NVML counters at 1 Hz: GPU and memory utilization, memory used, power, temperature, SM and memory clocks, and PCIe TX/RX. They extract 166 features over 5–60 s windows, including memory slope and epoch periodicity from an FFT of power [[S-0037]].\n- **Memory-hard proof of work.** In Monfared et al.'s challenge suite, these puzzles expose parallel effort and HBM use [[S-0033]].\n- **Verifiable delay functions.** Based on sequential modular squaring, they expose sequential compute pressure [[S-0033]].\n- **GEMM puzzles.** They target tensor-core throughput; the authors note that over 90% of LLM floating-point operations are GEMMs. Results can be checked with Freivalds' algorithm, subject to floating-point rounding discrepancies [[S-0033]].\n- **VRAM residency test.** It runs bandwidth-bound Argon2id over challenge data. With 60 GB of challenge data on an H100, the response time for data held in HBM and for data in pinned host memory reached over PCIe differed by more than 350 ms [[S-0033]].\n- **Guaranteeable Memory.** A guarantee chiplet beneath the HBM stacks would observe memory traffic directly. The author argues that the HBM standard makes it compatible with multiple leading accelerators [[S-0034]].\n- **Metering targets.** Candidate targets for licensing include floating-point and integer arithmetic, memory, NVLink and PCIe transfer volume, energy and clock cycles [[S-0006]].", "category": "on-chip", "secondary_categories": [ "remote-sensing" ], "verifies": [ { "claim": "C-0004", "role": "primary", "note": "Classifying training versus other workloads from counters (S-0037) or timing (S-0033)." }, { "claim": "C-0003", "role": "primary", "note": "Utilization and residency signals indicate whether declared-idle hardware is busy (S-0033)." }, { "claim": "C-0010", "role": "supporting", "note": "Contention from undeclared co-running workloads shifts challenge timing (S-0033); coverage of other chips needs other mechanisms." }, { "claim": "C-0007", "role": "supporting", "note": "Counters for FLOP, memory and interconnect traffic are proposed as meters for compute accounting (S-0057, S-0006)." } ], "threat_model": "semi-trusted", "adversarial_evaluation": "red-teamed", "hardware_requirement": "existing-features", "prover_cooperation": "partial", "confidentiality": "partial", "depends_on": [ { "target": "M-0008", "note": "A tamper-resistant read path, an authenticated channel and secure boot of the monitoring software are needed for counters to be trustworthy (S-0037)." } ], "readiness": { "assessment": true, "level": "R2", "scope": "a verifier reading workload evidence from GPUs that a hostile operator controls", "rubric_version": "1.1", "rationale": "Detailed results exist on current GPUs against a stated adversary, but the data can be forged in software and no one outside the developers has evaluated the methods.\n\n- **R1** met: primitives, claims and threat models are published [[S-0033]] [[S-0034]] [[S-0037]].\n- **R2** met through reproducible published results. Rahman and Tajdari report end-to-end results on nine NVIDIA GPU models, including H100, H200 and B200, over five monitor-evader rounds against a stated adversary. They state that code for their workloads is in the project repository, but give no repository URL [[S-0037]]; the rubric does not require code for this route. Monfared et al. report measurements on T4 and H100 GPUs with language models of up to 72B parameters [[S-0033]]. No Implementation record realises this mechanism, and the most mature work is the NVML classifier.\n- **R3** not met: no production-grade verification system built on these signals is available, and no source reports a party other than the developers relying on them for a verification decision. NVIDIA's Fleet Intelligence is a production telemetry service, but NVIDIA describes it as a tool for operators to monitor their own fleets [[S-3180]]. Trustworthy collection also needs hardware protections that Rahman and Tajdari list as prerequisites [[S-0037]].\n- **R4** not met: as of September 2026 no independent evaluation has been published.", "evidence": [ "S-0033", "S-0034", "S-0037" ], "next_level_gaps": [ "Use by a party other than the developers for a verification decision.", "Telemetry read paths that the operator cannot forge, such as signed counters from a root of trust or a guarantee processor.", "Calibrated false-positive and false-negative rates, with a detection-theoretic threshold framework.", "Independent red-teaming, including custom-kernel and multi-node evasion." ], "confidence": "medium", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Software-read telemetry can be forged by the operator", "kind": "theoretical-argument", "severity": "critical", "status": "open", "description": "NVML-based classification assumes trustworthy telemetry. Without a tamper-resistant read path, an authenticated telemetry channel and secure boot of the monitoring software, an operator who controls the full software stack could forge counter values [[S-0037]]. Monfared et al. start from the same premise: current GPUs expose little trusted telemetry and can be modified or virtualized [[S-0033]].", "sources": [ "S-0037", "S-0033" ], "response": null }, { "assessment": true, "title": "Adversarially disguised fine-tuning partly evades classification", "kind": "demonstrated-attack", "severity": "significant", "status": "open", "description": "Across 20 evasion strategy families in five monitor-evader rounds, the classifier's accuracy against the most challenging disguised workloads held out from training was 43–87%. White-box LoRA fine-tuning was the only evasion family detected less than half the time. The evaluation covered single nodes, PyTorch-level evasion and NVIDIA hardware [[S-0037]].", "sources": [ "S-0037" ], "response": null }, { "assessment": true, "title": "Timing challenges do not identify the individual chip", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "GEMM and VDF challenges can be answered by identical GPUs elsewhere, and floating-point fingerprints distinguish GPU models, not individual devices. GPU virtualization adds timing leakage that prevents attributing compute use [[S-0033]].", "sources": [ "S-0033" ], "response": null }, { "assessment": true, "title": "Counters leak information about protected workloads", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Performance counters have been used as a side channel against TEEs, for example in CounterSEVeillance [[S-0014]]. NVIDIA disables performance counters in full confidential-computing mode, stating that they could provide an avenue for side-channel attacks [[S-1200]]. Richer counters for verification therefore pull against confidentiality.", "sources": [ "S-0014", "S-1200" ], "response": null }, { "assessment": true, "title": "No quantified error rates or formal thresholds for timing primitives", "kind": "open-question", "severity": "minor", "status": "open", "description": "Monfared et al. state that false-positive and false-negative rates are not quantified and leave hardware-specific formal thresholds to future work [[S-0033]].", "sources": [ "S-0033" ], "response": null } ], "blockers": [ { "text": "Shipping accelerators need a tamper-resistant, authenticated telemetry path.", "theme": "hardware-trust", "blocked_by": "M-0009", "sources": [ "S-0037", "S-0034" ] }, { "text": "Confidential computing disables performance counters, so richer telemetry conflicts with it.", "theme": "privacy-leakage", "blocked_by": null, "sources": [ "S-1200", "S-0014" ] }, { "text": "Continuous challenge puzzles cost power and throughput on production workloads.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-0033" ] }, { "text": "Evaluation has not gone beyond single nodes, framework-level evasion and one vendor's hardware.", "theme": "adversarial-validation", "blocked_by": null, "sources": [ "S-0037" ] } ], "challenge_themes": [ "hardware-trust", "privacy-leakage", "adversarial-validation", "coverage-hidden-compute", "performance-compatibility" ], "organizations": [ "O-0202" ], "people": [], "sources": [ { "source": "S-0033", "supports": "four timing and memory primitives, threat model, T4/H100 results, residency-test conditions, overheads, limitations", "locator": "§3-§6, Figs. 5, 8, 10, 12, 15" }, { "source": "S-0034", "supports": "guarantee chiplet under HBM observing memory traffic; HBM-standard compatibility; independence from the accelerator die", "locator": "Abstract (read via ICML 2025 virtual site; OpenReview PDF not reachable)" }, { "source": "S-0037", "supports": "NVML counter classifier, trust assumptions, GPU models, accuracy and evasion results, code statement, limitations", "locator": "Abstract; threat model; results; limitations" }, { "source": "S-0057", "supports": "existing on-device counters and their use for metering", "locator": "p. 19" }, { "source": "S-0006", "supports": "candidate metering targets", "locator": "§2.2.2, §2.5.2, Table 1" }, { "source": "S-0014", "supports": "counters as side channel; memory-residency and random challenges; completeness of workload declarations" }, { "source": "S-1200", "supports": "performance counters disabled in full CC mode, and NVIDIA's side-channel rationale", "locator": "p. 18" }, { "source": "S-0073", "supports": "nvidia-smi power readings (via NVML) sample only 25% of runtime on A100 and H100; error about ±5% versus NVIDIA's claimed ±5 W", "locator": "Abstract; accuracy findings" }, { "source": "S-3180", "supports": "NVIDIA Fleet Intelligence: general availability, read-only open-source host agent, telemetry collected, signed attestation evidence (provider self-description)", "locator": "blog post" } ], "concepts": [ "K-0025", "K-0023", "K-0013", "K-0016", "K-0020", "K-0007", "K-0018" ], "complements": [ "M-0009", "M-0016", "M-0021", "M-0022" ], "alternatives": [], "type": "mechanism", "implementations": [], "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/", "source_file": "content/mechanisms/on-chip-telemetry.md", "flags_all": [ "critical-flaw-open" ], "body_markdown": "## How it works\n\nThree families of signals are proposed.\n\n**Built-in counters and management telemetry.** RAND notes that GPUs already have on-device counters for floating-point operations, instructions executed, NVLink and PCIe bandwidth, memory accesses and power. These can be reported per streaming multiprocessor or for the whole chip [[S-0057]]. O'Gara et al. list such counters as possible meters for compute accounting and licensing [[S-0006]]. Rahman and Tajdari train a classifier to spot training workloads from nine standard NVML counters sampled once a second. They describe these signals as content-agnostic [[S-0037]].\n\n**Timing and memory challenges.** Monfared et al. have a challenger send puzzles to the GPU and time the answers [[S-0033]]:\n\n- memory-hard proof-of-work puzzles probe parallel effort;\n- verifiable delay functions probe sequential execution;\n- matrix-multiplication puzzles probe the tensor cores;\n- a residency test checks whether challenge data stayed in the GPU's own memory.\n\nCo-running workloads compete for the same units, so contention shows up as shifts in the timing distributions [[S-0033]]. Gloria Z similarly suggests memory-residency and random challenges, which force an operator to keep false accounts consistent across every signal [[S-0014]]. Timed challenge-response in general is covered in [[M-0016]].\n\n**New memory-side hardware.** Guaranteeable Memory would place an open-source \"guarantee chiplet\" beneath the HBM stacks. The chiplet would observe memory traffic and attest to memory snapshots, GPU instructions, or random subsets of claimed computations [[S-0034]]. This connects to the guarantee processors of [[M-0009]].\n\n## What it establishes\nThese signals give statistical indicators of utilisation and workload type. Examples are whether a GPU is training [[S-0037]] and whether data is resident in its memory [[S-0033]]. They bear most directly on [[C-0004]], and more weakly on [[C-0003]] and [[C-0010]].\n\nThey cannot establish:\n\n- **Correct execution.** Monfared et al. state that their measurements are \"not designed to deliver cryptographic proof of correct execution\" [[S-0033]].\n- **Which device answered.** The timing challenges do not identify the individual GPU [[S-0033]].\n- **The content of the computation.** The counters do not access model weights or training data [[S-0037]].\n- **That all activity was declared.** Gloria Z treats telemetry as one partial answer to resource accounting, and judges that it \"seems extremely difficult to verify the completeness of workload declarations\" [[S-0014]].\n\nClassifying workloads from these and other signals is covered in [[M-0021]].\n\n## Threat model\nThe schemes trust different parts of the stack:\n\n- Rahman and Tajdari's NVML classifier assumes the counters are genuine. They list the protections this would need: a tamper-resistant read path, an authenticated channel and secure boot of the monitoring software. They model two evaders: a cloud tenant with user-space control, and a hardware operator who controls the full software stack [[S-0037]].\n- In Monfared et al.'s timing challenges, the most adversarial case lets the host and GPU firmware be modified, virtualized or colluding. Only challenge generation, timing and verification are trusted, and the challenges run on the GPU being measured [[S-0033]].\n- Guaranteeable Memory aims not to rely on any other component, including the main accelerator die [[S-0034]].\n\nConfidentiality is only partial. Counters are content-agnostic [[S-0037]], but they are also a known side channel [[S-0014]]. NVIDIA disables them in full confidential-computing mode because they could provide an avenue for side-channel attacks [[S-1200]], which puts this mechanism in tension with [[M-0008]].\n\n## Evidence\n- **Rahman and Tajdari.** They tested nine NVIDIA GPU models from four architecture generations, including A100, H100, H200 and B200, with up to eight GPUs per node. The classifier reached 98.2% binary accuracy at identifying training across their whole corpus. After five monitor-evader rounds covering 20 evasion strategy families, it reached 43–87% accuracy on the most challenging disguised workloads held out from training. The authors state that code for the workloads is in the project repository, but give no URL [[S-0037]].\n- **Monfared et al.** They measured T4 and H100 GPUs running language models from 1.1B to 72B parameters. With 60 GB of challenge data on an H100, the residency test separated on-device from off-device data by more than 350 ms. Continuous puzzles added noticeable power overhead and cut throughput. The residency test's power and throughput costs were negligible, but by design it occupies a large part of the GPU's memory. The paper mentions no code release [[S-0033]].\n- **Guaranteeable Memory.** It is a workshop proposal with no reported prototype [[S-0034]].\n- **NVIDIA Fleet Intelligence.** NVIDIA reports that this service, generally available since May 2026, uses a read-only host agent to stream GPU power, temperature, performance, health and configuration telemetry to an NVIDIA cloud service. It has released the agent as open source. NVIDIA reports that the agent also collects GPU attestation evidence signed with on-device certificates, but does not say that the telemetry values themselves are signed [[S-3180]].\n\n## Limitations\n- **Forgery.** If the operator controls the software that reads the counters, the counters prove little [[S-0037]].\n- **Disguise.** Adversarial disguise works in part: white-box LoRA fine-tuning was detected less than half the time [[S-0037]].\n- **Outsourcing and virtualization.** Timing challenges can be answered by identical GPUs, and virtualization confounds attribution [[S-0033]].\n- **Checking matrix results.** Floating-point rounding complicates the check [[S-0033]].\n- **Error rates.** The timing primitives have no calibrated error rates [[S-0033]].\n- **Evaluation scope.** The classifier was evaluated only on single nodes, with evasion at the PyTorch level and sampling at about 1 Hz [[S-0037]].\n- **Power sampling.** Yang and colleagues found that on A100 and H100 GPUs the built-in power reading, which nvidia-smi obtains through NVML, samples only 25% of runtime. The GPU can draw very different power in the other 75% without the reading showing it [[S-0073]]. They also found the reading's error to be within about ±5% in most cases, against the ±5 W that NVIDIA claims [[S-0073]].\n- **Leakage.** Richer counters risk leaking secrets [[S-0014]].", "body_text": "How it works Three families of signals are proposed. Built-in counters and management telemetry. RAND notes that GPUs already have on-device counters for floating-point operations, instructions executed, NVLink and PCIe bandwidth, memory accesses and power. These can be reported per streaming multiprocessor or for the whole chip [S-0057]. O'Gara et al. list such counters as possible meters for compute accounting and licensing [S-0006]. Rahman and Tajdari train a classifier to spot training workloads from nine standard NVML counters sampled once a second. They describe these signals as content-agnostic [S-0037]. Timing and memory challenges. Monfared et al. have a challenger send puzzles to the GPU and time the answers [S-0033]: - memory-hard proof-of-work puzzles probe parallel effort; - verifiable delay functions probe sequential execution; - matrix-multiplication puzzles probe the tensor cores; - a residency test checks whether challenge data stayed in the GPU's own memory. Co-running workloads compete for the same units, so contention shows up as shifts in the timing distributions [S-0033]. Gloria Z similarly suggests memory-residency and random challenges, which force an operator to keep false accounts consistent across every signal [S-0014]. Timed challenge-response in general is covered in Timed challenge-response and memory-occupation challenges. New memory-side hardware. Guaranteeable Memory would place an open-source \"guarantee chiplet\" beneath the HBM stacks. The chiplet would observe memory traffic and attest to memory snapshots, GPU instructions, or random subsets of claimed computations [S-0034]. This connects to the guarantee processors of Hardware-enabled guarantees (flexHEG) and guarantee processors. What it establishes These signals give statistical indicators of utilisation and workload type. Examples are whether a GPU is training [S-0037] and whether data is resident in its memory [S-0033]. They bear most directly on This compute runs inference, not training, and more weakly on Declared hardware is idle or shut down and There is no undeclared relevant compute. They cannot establish: - Correct execution. Monfared et al. state that their measurements are \"not designed to deliver cryptographic proof of correct execution\" [S-0033]. - Which device answered. The timing challenges do not identify the individual GPU [S-0033]. - The content of the computation. The counters do not access model weights or training data [S-0037]. - That all activity was declared. Gloria Z treats telemetry as one partial answer to resource accounting, and judges that it \"seems extremely difficult to verify the completeness of workload declarations\" [S-0014]. Classifying workloads from these and other signals is covered in Workload classification from telemetry and side channels. Threat model The schemes trust different parts of the stack: - Rahman and Tajdari's NVML classifier assumes the counters are genuine. They list the protections this would need: a tamper-resistant read path, an authenticated channel and secure boot of the monitoring software. They model two evaders: a cloud tenant with user-space control, and a hardware operator who controls the full software stack [S-0037]. - In Monfared et al.'s timing challenges, the most adversarial case lets the host and GPU firmware be modified, virtualized or colluding. Only challenge generation, timing and verification are trusted, and the challenges run on the GPU being measured [S-0033]. - Guaranteeable Memory aims not to rely on any other component, including the main accelerator die [S-0034]. Confidentiality is only partial. Counters are content-agnostic [S-0037], but they are also a known side channel [S-0014]. NVIDIA disables them in full confidential-computing mode because they could provide an avenue for side-channel attacks [S-1200], which puts this mechanism in tension with TEE remote attestation for AI workloads. Evidence - Rahman and Tajdari. They tested nine NVIDIA GPU models from four architecture generations, including A100, H100, H200 and B200, with up to eight GPUs per node. The classifier reached 98.2% binary accuracy at identifying training across their whole corpus. After five monitor-evader rounds covering 20 evasion strategy families, it reached 43–87% accuracy on the most challenging disguised workloads held out from training. The authors state that code for the workloads is in the project repository, but give no URL [S-0037]. - Monfared et al. They measured T4 and H100 GPUs running language models from 1.1B to 72B parameters. With 60 GB of challenge data on an H100, the residency test separated on-device from off-device data by more than 350 ms. Continuous puzzles added noticeable power overhead and cut throughput. The residency test's power and throughput costs were negligible, but by design it occupies a large part of the GPU's memory. The paper mentions no code release [S-0033]. - Guaranteeable Memory. It is a workshop proposal with no reported prototype [S-0034]. - NVIDIA Fleet Intelligence. NVIDIA reports that this service, generally available since May 2026, uses a read-only host agent to stream GPU power, temperature, performance, health and configuration telemetry to an NVIDIA cloud service. It has released the agent as open source. NVIDIA reports that the agent also collects GPU attestation evidence signed with on-device certificates, but does not say that the telemetry values themselves are signed [S-3180]. Limitations - Forgery. If the operator controls the software that reads the counters, the counters prove little [S-0037]. - Disguise. Adversarial disguise works in part: white-box LoRA fine-tuning was detected less than half the time [S-0037]. - Outsourcing and virtualization. Timing challenges can be answered by identical GPUs, and virtualization confounds attribution [S-0033]. - Checking matrix results. Floating-point rounding complicates the check [S-0033]. - Error rates. The timing primitives have no calibrated error rates [S-0033]. - Evaluation scope. The classifier was evaluated only on single nodes, with evasion at the PyTorch level and sampling at about 1 Hz [S-0037]. - Power sampling. Yang and colleagues found that on A100 and H100 GPUs the built-in power reading, which nvidia-smi obtains through NVML, samples only 25% of runtime. The GPU can draw very different power in the other 75% without the reading showing it [S-0073]. They also found the reading's error to be within about ±5% in most cases, against the ±5 W that NVIDIA claims [S-0073]. - Leakage. Richer counters risk leaking secrets [S-0014].", "referenced_by": [ { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" }, { "id": "K-0007", "title": "Hardware-enabled mechanism (HEM)", "url": "https://trustbutveri.fyi/concepts/hardware-enabled-mechanism/" }, { "id": "K-0025", "title": "Inference and training workloads", "url": "https://trustbutveri.fyi/concepts/inference-and-training-workloads/" }, { "id": "O-0140", "title": "NVIDIA", "url": "https://trustbutveri.fyi/organizations/nvidia/" }, { "id": "O-0200", "title": "RAND", "url": "https://trustbutveri.fyi/organizations/rand/" } ] }, { "id": "M-0006", "slug": "proof-of-learning", "title": "Training-transcript verification (proof-of-learning)", "aliases": [ "Proof-of-learning and training-transcript verification", "proof-of-learning", "PoL", "proof of training transcript", "PoTT", "proof-of-training-data", "PoTD", "optimistic verifiable training" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "A trainer logs checkpoints, data order and settings, so a verifier can re-run sampled training segments and check that the claimed training happened.", "summary": "Proof-of-learning asks a model trainer to keep a transcript of training: weight checkpoints, the data used at each step and the settings. A verifier re-runs a few segments and checks that they reproduce the logged checkpoints. It was proposed to show that compute was really spent on training, and later to check that training runs obeyed rules on compute and data. Later data-focused tests were run on language models of up to 1 billion parameters. A 2024 variant makes small training runs exactly reproducible across GPU types, so an auditor can replicate them. The biggest known weakness is that published attacks produced invalid proofs that passed verification, and the original authors later argued that no provably robust version exists without a better theory of deep-learning optimisation. The biggest obstacle for verification use is that the verifier must see the training data and weights and be able to re-run training.", "category": "cryptographic-computational", "secondary_categories": [], "verifies": [ { "claim": "C-0007", "role": "primary", "note": "Transcript checks for rules on training compute, data and hyperparameters (Shavit; Choi et al.)." } ], "threat_model": "adversarial", "adversarial_evaluation": "independent-red-team", "hardware_requirement": "none", "prover_cooperation": "required", "confidentiality": "revealing", "depends_on": [], "readiness": { "assessment": true, "level": "R2", "scope": "checking from its transcript that a training run followed declared rules", "rubric_version": "1.1", "rationale": "R2 through later peer-reviewed work by Choi et al. and Srivastava et al. The original verification rule is broken, and no independent attack on the later defences has been published.\n\n- **R1** met: Jia et al. define proof-of-learning and its security goal [[S-0028]], and Shavit sets out training-transcript verification for rules on large training runs [[S-0029]].\n- **R2** met through later work, not through the original scheme. The original authors demonstrated spoofs that pass verification across configurations at a fraction of the cost of earlier spoofing strategies [[S-0027]]. An earlier independent attack reported spoofs cheaper than honest training [[S-1109]], which the original authors dispute [[S-0027]]. These breaks invalidate the original demonstration, including the public code, which implements that rule [[S-1111]]. Choi et al. publish peer-reviewed end-to-end experiments against a stated set of spoofing attacks, training GPT-2 (124M) and testing published Pythia checkpoints up to 1B on GPUs [[S-0030]]. The paper links no code; the rubric does not require code for results described in enough detail to repeat. Srivastava et al. replicate ResNet-50 training and GPT-2 fine-tuning bit for bit on three NVIDIA GPU types, against a trainer that may poison data, and release their code [[S-3081]]. The open critical flaws concern the original rule, not the evidence the level rests on.\n- **R3** not met: [[I-0015|Verde]] realises this mechanism for delegated training jobs, but Gensyn states that its public runtime is limited to LLM inference [[S-1812]], so its production use does not count for training verification. As of September 2026 no third party is known to rely on training-transcript checks, and no production-grade implementation has been published.\n- **R4** not met: no independent attack on the post-2023 defences has been published.", "evidence": [ "S-0028", "S-1111", "S-0030", "S-3081", "S-0027", "S-1109" ], "next_level_gaps": [ "Use by a party other than the developer, or a production-grade implementation, at realistic training scale.", "Verification rules with formal robustness arguments, as Fang et al. argue are needed, or an independent red-team of the post-2023 defences." ], "confidence": "medium", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Adversarial-example spoofs pass verification at lower cost than training", "kind": "demonstrated-attack", "severity": "significant", "status": "disputed", "description": "Zhang et al. construct proofs that pass the original verification \"with significantly less cost than generating a proof by the prover\". Their attack uses adversarial-example-style perturbations, and they demonstrate it on CIFAR-10, CIFAR-100 and an ImageNet subset [[S-1109]]. They suggest dynamic thresholds, or proofs built on verifiable computation, as countermeasures [[S-1109]].", "sources": [ "S-1109", "S-0027" ], "response": "Fang et al., who include the original PoL authors, state that the attack assumes the adversary sets the checkpoint interval k, which the verifier should set, and that the verifier can prevent it by using a small k [[S-0027]]." }, { "assessment": true, "title": "Structurally correct spoofs exploit tolerance thresholds and sampled checks", "kind": "demonstrated-attack", "severity": "critical", "status": "open", "description": "Fang et al., including the original proposers, present spoofing strategies that work across PoL configurations at \"a fraction of the cost of previous spoofing strategies\" [[S-0027]]. The strategies exploit the tolerance that verification must allow for hardware noise, and the choice of which updates the verifier re-runs [[S-0027]].", "sources": [ "S-0027" ], "response": null }, { "assessment": true, "title": "No provably robust verification without better optimisation theory", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Fang et al. conclude that \"one cannot develop a provably robust PoL verification mechanism without further understanding of optimization in deep learning\" [[S-0027]]. Shavit notes that the PoL literature \"has been heuristic-based\" [[S-0029]].", "sources": [ "S-0027", "S-0029" ], "response": null }, { "assessment": true, "title": "Rule-compliance transcripts face a stronger adversary", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Shavit argues that proving a training transcript \"appears to be strictly harder than PoL\". The adversary knows the true transcript and may spend extra compute to build a compliant-looking one [[S-0029]].", "sources": [ "S-0029" ], "response": null }, { "assessment": true, "title": "Small data changes and masked hyperparameters may go undetected", "kind": "open-question", "severity": "significant", "status": "open", "description": "Choi et al. state that their protocol cannot yet detect modest data additions, such as inserted backdoors. They note that attacks could be hidden with \"cleverly chosen hyperparameters\", such as a temporarily lower learning rate than reported, and that the protocol does not apply to online or reinforcement learning [[S-0030]].", "sources": [ "S-0030" ], "response": null } ], "blockers": [ { "text": "The verifier must receive the training data, weights and code [[S-0028]] [[S-0030]].", "theme": "privacy-leakage", "blocked_by": null, "sources": [ "S-0028", "S-0030" ] }, { "text": "Transcripts are large: weight checkpoints may each require terabytes [[S-0029]].", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-0029" ] }, { "text": "The verifier must reproduce training segments, which may be infeasible if the prover uses specialised or proprietary hardware [[S-0030]].", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-0030" ] }, { "text": "The noise tolerance needed for honest reproduction is what structurally correct spoofs exploit [[S-0027]].", "theme": "protocol-soundness", "blocked_by": null, "sources": [ "S-0027" ] }, { "text": "Tying transcripts to real chips needs on-chip weight-snapshot logging, chip inspections and a trusted chip-owner directory [[S-0029]].", "theme": "evidence-binding", "blocked_by": null, "sources": [ "S-0029" ] } ], "challenge_themes": [ "protocol-soundness", "adversarial-validation", "privacy-leakage", "performance-compatibility", "evidence-binding" ], "organizations": [], "people": [], "sources": [ { "source": "S-0028", "supports": "PoL definition, transcript contents, verification, security analysis, limits", "locator": "Definition 1; Algorithm 2; §IV–VI; Table I" }, { "source": "S-1111", "supports": "public PoL implementation (train and verify scripts); rationale for the cost of forging", "locator": "README" }, { "source": "S-1109", "supports": "independent demonstrated spoofing attack; countermeasures", "locator": "abstract; §III–V" }, { "source": "S-0027", "supports": "structurally correct and stochastic spoofing; robustness conclusion; response to Zhang et al.", "locator": "abstract; §2.3; attack sections; conclusion" }, { "source": "S-0029", "supports": "training-transcript verification for rules; chip logging; open problems; costs", "locator": "§4; §5; §5.1; §5.2; §6.1; Table 1" }, { "source": "S-0030", "supports": "proof-of-training-data protocol, experiments, defended attacks, costs, limits", "locator": "§3; §4.1–4.4; §6; §7; App. A" }, { "source": "S-3081", "supports": "exact-replication verifiable training: rounding logs, Merkle-tree dispute search, experiments, overheads, 1-of-n auditor assumption, limitations", "locator": "abstract; §3–§5; limitations" }, { "source": "S-1809", "supports": "Verde dispute narrowing for training jobs; RepOps LoRA fine-tuning overhead", "locator": "§3; Table 2" }, { "source": "S-1812", "supports": "Gensyn's REE release limited to reproducible LLM inference (provider-reported)", "locator": "README" }, { "source": "S-0022", "supports": "cryptographic alternative (ZK proofs of SGD)", "locator": "abstract" } ], "concepts": [ "K-0009", "K-0008", "K-0020", "K-0001", "K-0002", "K-0023" ], "complements": [], "alternatives": [ "M-0005" ], "type": "mechanism", "implementations": [ { "id": "I-0015", "title": "Verde and RepOps (Gensyn)", "url": "https://trustbutveri.fyi/implementations/gensyn-verde-repops/" } ], "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/", "source_file": "content/mechanisms/proof-of-learning.md", "flags_all": [ "critical-flaw-open" ], "body_markdown": "## How it works\n\nThree variants are covered here: tolerance-based proof-of-learning, heuristic tests on the training data [[S-0030]], and exact replication with a dispute protocol [[S-3081]] [[S-1809]]. Published spoofing attacks defeat the first [[S-0027]].\n\n**The transcript.** Proof-of-learning (PoL) asks a trainer to record a transcript of training [[S-0028]]. It contains:\n- weight checkpoints saved every k steps;\n- the indices of the data batches used at each step;\n- signatures of the training data;\n- metadata such as hyperparameters, optimiser and architecture [[S-0028]].\n\n**Checking it.** A verifier checks the transcript by re-running training for selected segments. For each epoch, it takes the Q largest weight updates and recomputes the k steps from the earlier checkpoint. It accepts if the result lies within a distance threshold δ of the logged checkpoint [[S-0028]]. The threshold absorbs the numerical noise from hardware and software [[K-0008|nondeterminism]] [[S-0028]].\n\n**Why forging should be costly.** The PoL code repository explains that gradient descent \"accumulates secret information due to its stochasticity\" [[S-1111]]. On this basis, Jia et al. claim a forger must spend \"at least as much work than is needed for gradient descent itself\" [[S-0028]].\n\n**Use for rule verification.** Shavit adapts the idea to verifying rules on large training runs [[S-0029]]:\n1. Chips log weight snapshots.\n2. Inspectors sample chips.\n3. The prover must show a training transcript that explains each logged snapshot.\n4. The verifier re-runs only the segments concerned [[S-0029]].\n\n**Tests on the training data.** Choi et al. add checks aimed at training data [[S-0030]]:\n- the random seed, initialisation and data order are derived from a hash of the dataset;\n- memorisation tests check that training points are fitted best just after their segment;\n- suspicious segments are flagged for retraining [[S-0030]].\n\n**Exact replication.** Srivastava et al. make training bit-for-bit reproducible on different GPU types, so that checkpoints can be compared by hash rather than within a tolerance [[S-3081]]:\n- the trainer trains at higher precision than the target model, rounds intermediate results, and logs the rounding direction for values close to a rounding boundary;\n- an auditor replicates the whole run, following the logged rounding directions, and stores hashes of checkpoints in a Merkle tree;\n- if the two trees differ, a binary search finds the first checkpoint at which training diverged [[S-3081]].\n\n## What it establishes\n**What it can establish.**\n- If verification holds, the transcript is consistent with training that actually took place. This supports claims about ownership and the compute spent [[S-0028]].\n- With data-focused tests, it also supports claims about \"the amount and kind of data and compute used to train the model\" [[S-0030]]. This bears on [[C-0007]].\n\n**What it cannot establish.**\n- **A link to derived models.** PoL cannot connect a model to a surrogate derived from it by extraction [[S-0028]].\n- **Anything without reproducible training.** The approach relies on training being approximately reproducible from the hyperparameters, data order and seed [[S-0029]].\n- **Online or reinforcement learning.** Where the data is not known in advance, these are not covered [[S-0030]] [[S-0029]].\n\n## Threat model\nThe adversary tries to produce a valid-looking transcript for weights it did not train, or for a run that broke the rules, while spending less than honest training [[S-0028]] [[S-0029]].\n\nThe scheme assumes several things:\n- the verifier can re-run training segments on comparable hardware [[S-0030]];\n- honest noise is small compared with genuine updates [[S-0028]];\n- the verifier receives the data and weights, since Jia et al. require that the training data \"be shared with the verifier\" [[S-0028]].\n\nSrivastava et al.'s scheme assumes that at least one of several auditors is honest [[S-3081]].\n\nIn Shavit's framework, three further assumptions tie transcripts to real chips [[S-0029]]:\n- the chips support firmware attestation and logging;\n- they have tamper-evident packaging;\n- a trusted chip-owner directory exists.\n\n## Evidence\n- **Jia et al.** They evaluate PoL with ResNet-20 and ResNet-50 on CIFAR-10 and CIFAR-100 [[S-0028]]. Their code for creating and verifying proofs is public [[S-1111]].\n- **Choi et al.** They train GPT-2 (124M) on about 8.8 billion OpenWebText tokens, and apply their tests to EleutherAI's published Pythia checkpoints of 70M, 410M and 1B parameters, using four A40 GPUs [[S-0030]]. They show detection of attacks from the PoL literature, including gluing, interpolation and data addition or subtraction [[S-0030]]. Sampling 1% of the points in each segment, they estimate that their new memorisation and data-order tests cost \"just 1.3% of the original cost of training\", assuming inference is three times cheaper than training, in addition to retraining selected segments [[S-0030]].\n- **Srivastava et al.** They replicate ResNet-50 (23M) training on CIFAR-10 and GPT-2 (117M) fine-tuning exactly at FP32 across NVIDIA A40, Titan XP and RTX 2080 Ti GPUs. Their prototype adds 20 to 40% to the trainer's time, and the auditor takes 1.3 to 1.7 times as long as training. Their code is public [[S-3081]].\n- **Verde.** Gensyn's [[I-0015|Verde]] narrows disputes between compute providers over a delegated training job to one step, then to one operation, which a referee re-runs [[S-1809]]. The paper reports that its reproducible operators added 126% to LoRA fine-tuning time for Llama-8B on an A100 [[S-1809]]. Gensyn states that its public runtime is limited to LLM inference [[S-1812]].\n- **Shavit.** His framework is a proposal, with estimates of how many chips inspectors would need to sample [[S-0029]].\n\n## Limitations\n**Independent attack.** Zhang et al. of Zhejiang University showed that adversarial-example-style perturbations let an attacker generate a passing proof \"with significantly less cost\" than honest training [[S-1109]]. Fang et al. dispute this. They state that the attack assumes the adversary chooses the checkpoint interval, which the verifier should set, and that a small interval prevents it [[S-0027]].\n\n**Attack by the original authors.** Fang et al., whose authors include all seven original PoL authors, present cheaper spoofs that work across PoL configurations. They find that \"current PoL verification is not robust to adversaries\", and show that the assumptions needed for robust verification reduce to open problems in learning theory [[S-0027]].\n\n**Gaps in later defences.** Choi et al. defend against several known attacks, but note that small-scale data changes and masked hyperparameters remain open [[S-0030]]. In Srivastava et al.'s scheme, the auditor re-runs the whole run, and all parties must trust it. A trainer that knows the auditor's GPU can choose among models that differ only in rounding decisions at the few steps where the auditor is close to a rounding boundary. Distributed training is left to future work [[S-3081]].\n\n**Cost.** Storage is a burden: checkpoints may each take terabytes [[S-0029]]. Shavit identifies cheaper alternatives to retraining-based verification as future work [[S-0029]].\n\n**Alternatives.** Zhang et al. suggest proofs built on verifiable computation as a countermeasure [[S-1109]]. [[M-0005|Zero-knowledge proofs of training]] take that route, proving each training step cryptographically [[S-0022]].", "body_text": "How it works Three variants are covered here: tolerance-based proof-of-learning, heuristic tests on the training data [S-0030], and exact replication with a dispute protocol [S-3081] [S-1809]. Published spoofing attacks defeat the first [S-0027]. The transcript. Proof-of-learning (PoL) asks a trainer to record a transcript of training [S-0028]. It contains: - weight checkpoints saved every k steps; - the indices of the data batches used at each step; - signatures of the training data; - metadata such as hyperparameters, optimiser and architecture [S-0028]. Checking it. A verifier checks the transcript by re-running training for selected segments. For each epoch, it takes the Q largest weight updates and recomputes the k steps from the earlier checkpoint. It accepts if the result lies within a distance threshold δ of the logged checkpoint [S-0028]. The threshold absorbs the numerical noise from hardware and software nondeterminism [S-0028]. Why forging should be costly. The PoL code repository explains that gradient descent \"accumulates secret information due to its stochasticity\" [S-1111]. On this basis, Jia et al. claim a forger must spend \"at least as much work than is needed for gradient descent itself\" [S-0028]. Use for rule verification. Shavit adapts the idea to verifying rules on large training runs [S-0029]: 1. Chips log weight snapshots. 2. Inspectors sample chips. 3. The prover must show a training transcript that explains each logged snapshot. 4. The verifier re-runs only the segments concerned [S-0029]. Tests on the training data. Choi et al. add checks aimed at training data [S-0030]: - the random seed, initialisation and data order are derived from a hash of the dataset; - memorisation tests check that training points are fitted best just after their segment; - suspicious segments are flagged for retraining [S-0030]. Exact replication. Srivastava et al. make training bit-for-bit reproducible on different GPU types, so that checkpoints can be compared by hash rather than within a tolerance [S-3081]: - the trainer trains at higher precision than the target model, rounds intermediate results, and logs the rounding direction for values close to a rounding boundary; - an auditor replicates the whole run, following the logged rounding directions, and stores hashes of checkpoints in a Merkle tree; - if the two trees differ, a binary search finds the first checkpoint at which training diverged [S-3081]. What it establishes What it can establish. - If verification holds, the transcript is consistent with training that actually took place. This supports claims about ownership and the compute spent [S-0028]. - With data-focused tests, it also supports claims about \"the amount and kind of data and compute used to train the model\" [S-0030]. This bears on A training run stayed within declared limits. What it cannot establish. - A link to derived models. PoL cannot connect a model to a surrogate derived from it by extraction [S-0028]. - Anything without reproducible training. The approach relies on training being approximately reproducible from the hyperparameters, data order and seed [S-0029]. - Online or reinforcement learning. Where the data is not known in advance, these are not covered [S-0030] [S-0029]. Threat model The adversary tries to produce a valid-looking transcript for weights it did not train, or for a run that broke the rules, while spending less than honest training [S-0028] [S-0029]. The scheme assumes several things: - the verifier can re-run training segments on comparable hardware [S-0030]; - honest noise is small compared with genuine updates [S-0028]; - the verifier receives the data and weights, since Jia et al. require that the training data \"be shared with the verifier\" [S-0028]. Srivastava et al.'s scheme assumes that at least one of several auditors is honest [S-3081]. In Shavit's framework, three further assumptions tie transcripts to real chips [S-0029]: - the chips support firmware attestation and logging; - they have tamper-evident packaging; - a trusted chip-owner directory exists. Evidence - Jia et al. They evaluate PoL with ResNet-20 and ResNet-50 on CIFAR-10 and CIFAR-100 [S-0028]. Their code for creating and verifying proofs is public [S-1111]. - Choi et al. They train GPT-2 (124M) on about 8.8 billion OpenWebText tokens, and apply their tests to EleutherAI's published Pythia checkpoints of 70M, 410M and 1B parameters, using four A40 GPUs [S-0030]. They show detection of attacks from the PoL literature, including gluing, interpolation and data addition or subtraction [S-0030]. Sampling 1% of the points in each segment, they estimate that their new memorisation and data-order tests cost \"just 1.3% of the original cost of training\", assuming inference is three times cheaper than training, in addition to retraining selected segments [S-0030]. - Srivastava et al. They replicate ResNet-50 (23M) training on CIFAR-10 and GPT-2 (117M) fine-tuning exactly at FP32 across NVIDIA A40, Titan XP and RTX 2080 Ti GPUs. Their prototype adds 20 to 40% to the trainer's time, and the auditor takes 1.3 to 1.7 times as long as training. Their code is public [S-3081]. - Verde. Gensyn's Verde narrows disputes between compute providers over a delegated training job to one step, then to one operation, which a referee re-runs [S-1809]. The paper reports that its reproducible operators added 126% to LoRA fine-tuning time for Llama-8B on an A100 [S-1809]. Gensyn states that its public runtime is limited to LLM inference [S-1812]. - Shavit. His framework is a proposal, with estimates of how many chips inspectors would need to sample [S-0029]. Limitations Independent attack. Zhang et al. of Zhejiang University showed that adversarial-example-style perturbations let an attacker generate a passing proof \"with significantly less cost\" than honest training [S-1109]. Fang et al. dispute this. They state that the attack assumes the adversary chooses the checkpoint interval, which the verifier should set, and that a small interval prevents it [S-0027]. Attack by the original authors. Fang et al., whose authors include all seven original PoL authors, present cheaper spoofs that work across PoL configurations. They find that \"current PoL verification is not robust to adversaries\", and show that the assumptions needed for robust verification reduce to open problems in learning theory [S-0027]. Gaps in later defences. Choi et al. defend against several known attacks, but note that small-scale data changes and masked hyperparameters remain open [S-0030]. In Srivastava et al.'s scheme, the auditor re-runs the whole run, and all parties must trust it. A trainer that knows the auditor's GPU can choose among models that differ only in rounding decisions at the few steps where the auditor is close to a rounding boundary. Distributed training is left to future work [S-3081]. Cost. Storage is a burden: checkpoints may each take terabytes [S-0029]. Shavit identifies cheaper alternatives to retraining-based verification as future work [S-0029]. Alternatives. Zhang et al. suggest proofs built on verifiable computation as a countermeasure [S-1109]. Zero-knowledge proofs of training take that route, proving each training step cryptographically [S-0022].", "referenced_by": [ { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "M-0005", "title": "Zero-knowledge proofs of training constraints", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-training-constraints/" }, { "id": "I-0015", "title": "Verde and RepOps (Gensyn)", "url": "https://trustbutveri.fyi/implementations/gensyn-verde-repops/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0008", "title": "Numerical nondeterminism", "url": "https://trustbutveri.fyi/concepts/numerical-nondeterminism/" }, { "id": "K-0001", "title": "Prover", "url": "https://trustbutveri.fyi/concepts/prover/" }, { "id": "K-0009", "title": "Recomputation", "url": "https://trustbutveri.fyi/concepts/recomputation/" }, { "id": "K-0020", "title": "Sampling and assurance", "url": "https://trustbutveri.fyi/concepts/sampling-and-assurance/" }, { "id": "K-0002", "title": "Verifier", "url": "https://trustbutveri.fyi/concepts/verifier/" } ] }, { "id": "M-0007", "slug": "proofs-of-useful-work", "title": "Proofs of useful work for capacity accounting", "aliases": [ "Proofs of useful work and resource exhaustion", "proof of useful work", "PoUW", "proof-of-work accounting", "resource exhaustion" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "one_liner": "Cryptographic evidence that a given amount of matrix-multiplication work was completed, proposed as one input to accounting for spare capacity on declared hardware.", "summary": "A proof of useful work shows that a given amount of computation, such as the matrix multiplications at the heart of AI models, was completed. The computation can serve a useful task, but the proof does not show that it did. For verification, the idea is to keep declared hardware occupied with agreed, provable work, so that little capacity is left for unmonitored work such as an unauthorised training run. The cryptographic building block exists: a 2025 construction adds asymptotically negligible overhead to large matrix multiplications, and a company runs a public blockchain built on it. On that network, an independent study found that random matrices passed verification. Using it to bound spare capacity is only proposed. The proof covers the computation; the capacity claim separately needs a credible estimate of the actor's available compute. Such proofs cannot find undeclared hardware.", "category": "cryptographic-computational", "secondary_categories": [], "verifies": [ { "claim": "C-0004", "role": "primary", "note": "Bounds the spare capacity of declared hardware that could run unauthorised training; Attestable pairs work accounting with ZK inference proofs." }, { "claim": "C-0003", "role": "supporting", "note": "Keeping declared hardware provably busy with agreed work, as an alternative to showing it idle." }, { "claim": "C-0010", "role": "supporting", "note": "On declared hardware only: bounds capacity left for unmonitored work; cannot find undeclared facilities." } ], "threat_model": "adversarial", "adversarial_evaluation": "analysis", "hardware_requirement": "none", "prover_cooperation": "required", "confidentiality": "partial", "depends_on": [], "readiness": { "assessment": true, "level": "R1", "scope": "bounding the spare capacity of declared hardware that could run training", "rubric_version": "1.1", "rationale": "The scheme is proposed, and the only implementation proves work for blockchain consensus, not that hardware has no spare capacity.\n\n- **R1** met: Attestable describes such a scheme, with its claim and a key assumption, namely a credible estimate of the actor's compute [[S-1102]]. Scher and Thiergart list proof of work, for compute declared to be doing crypto mining, among the ways to verify that known compute is not used for a large training run [[S-0005]]. The underlying proof-of-useful-work construction is publicly specified with its hardness assumptions [[S-1609]].\n- **R2** not met for this use. The most mature implementation, [[I-0004|Pearl]], is assessed R3 only for the narrower claim that GPUs performed matrix-multiplication work. It is built for blockchain consensus, and no public implementation or result uses proofs of useful work to bound the spare capacity of declared hardware [[S-1105]] [[S-1107]].", "evidence": [ "S-1102", "S-0005", "S-1609", "S-1105", "S-1107" ], "next_level_gaps": [ "A public implementation or reproducible end-to-end result that uses proofs of work to bound the spare capacity of declared hardware against a stated adversary.", "A method for the verifier to obtain a credible estimate of the prover's available compute." ], "confidence": "low", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Proves that work was done, not that no capacity remains", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Proof-of-work accounting bounds unmonitored compute only relative to an estimate of what the actor has. Attestable states that the verifier \"needs a credible estimate of the compute available\" to the actor, and that a proof \"cannot discover a datacenter that was never declared\" [[S-1102]].", "sources": [ "S-1102" ], "response": null }, { "assessment": true, "title": "Security rests on new hardness assumptions", "kind": "open-question", "severity": "significant", "status": "open", "description": "Komargodski and Weinstein base security on hardness assumptions about batches of low-rank random linear equations, and list PoUW \"from more standard or well-studied assumptions\" as an open problem [[S-1609]]. Pearl's floating-point variant introduces a further \"quantized-subspace hardness\" assumption [[S-1105]].", "sources": [ "S-1609", "S-1105" ], "response": null }, { "assessment": true, "title": "Known shortcuts let a miner claim somewhat more work than it did", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Pearl's specification lists known mining speedups: crafted inputs, precision shortcuts, seed grinding, work reuse, and faster kernels or hardware. A policy check caps the summands a miner may skip at one-sixteenth of those in a tile [[S-1105]]. For capacity bounding, any gap between work proven and work possible leaves spare capacity.", "sources": [ "S-1105" ], "response": null } ], "blockers": [ { "text": "Bounding spare capacity needs a credible estimate of the compute available to the actor, including third-party access [[S-1102]].", "theme": "capacity-bounds", "blocked_by": null, "sources": [ "S-1102" ] }, { "text": "Proofs of work cannot find facilities that were never declared [[S-1102]].", "theme": "coverage-hidden-compute", "blocked_by": null, "sources": [ "S-1102" ] }, { "text": "As of September 2026 no implementation, demonstration or independent evaluation of proofs of work for capacity bounding has been published.", "theme": "adversarial-validation", "blocked_by": null, "sources": [] } ], "challenge_themes": [ "capacity-bounds", "coverage-hidden-compute", "protocol-soundness", "adversarial-validation" ], "organizations": [ "O-0120", "O-0121", "O-0202" ], "people": [], "sources": [ { "source": "S-1609", "supports": "PoUW construction for arbitrary matrix multiplication; miners choose the matrices; overhead; assumptions; open problems", "locator": "abstract; §2; Remarks 2.1–2.2; §3; §5; §6.5 (ePrint numbering)" }, { "source": "S-1105", "supports": "Pearl FP8 protocol, verifier, known mining speedups, assumption (provider-reported)", "locator": "abstract; §2; §4; §5; §6; §7; App. B" }, { "source": "S-1106", "supports": "Pearl INT scheme, benchmark and launch statement (provider-reported)", "locator": "§3; §4.7; §7 Table 1" }, { "source": "S-1107", "supports": "public Pearl network code", "locator": "README; release v1.2.1" }, { "source": "S-0071", "supports": "independent measurement of Pearl's mainnet: random-matrix mining; verification does not check matrix provenance", "locator": "abstract; measurement and verification sections; conclusion" }, { "source": "S-1102", "supports": "proposal to use proof-of-work accounting to bound unmonitored compute (provider-reported)" }, { "source": "S-1608", "supports": "earlier PoUW for fine-grained complexity problems", "locator": "abstract" }, { "source": "S-0005", "supports": "proof of work to confirm that compute declared as crypto mining is mining, as a way to verify known compute is not used for a large training run; R1 evidence", "locator": "'Proof-of-Work methods for crypto mining', in 'Verifying That Known Compute is Not Being Used for a Large Training Run'" }, { "source": "S-3566", "supports": "independent analysis that the share of declared compute accounted for determines the residual budget for covert RL rollouts", "locator": "§2.5; §4" }, { "source": "S-0018", "supports": "memory filling with incompressible noise as a capacity check", "locator": "§5.1.2" }, { "source": "S-1607", "supports": "proofs of space as the storage analogue of proofs of work", "locator": "abstract" } ], "concepts": [ "K-0011", "K-0012", "K-0016", "K-0023", "K-0025" ], "complements": [ "M-0004", "M-0015" ], "alternatives": [], "type": "mechanism", "implementations": [ { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" } ], "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/", "source_file": "content/mechanisms/proofs-of-useful-work.md", "flags_all": [ "provider-reported" ], "body_markdown": "## How it works\n\n**Proofs of work.** A proof of work gives confidence that some declared amount of computation took place. In cryptocurrency mining, workers hash variants of a message until one hash meets a target, and a verifier can check it quickly [[S-0005]]. Scher and Thiergart note that applying proof of work \"could verify that compute declared to be doing crypto mining is indeed doing it\", as one way to check that known compute is not used for a large training run [[S-0005]].\n\n**Proofs of useful work.** A proof of useful work (PoUW) makes that computation serve another purpose. Ball et al. gave constructions for problems such as Orthogonal Vectors, 3SUM and All-Pairs Shortest Path [[S-1608]]. Komargodski and Weinstein give a PoUW for arbitrary matrix multiplication, which lets the miner choose its own matrices [[S-1609]]. Their protocol works in three steps [[S-1609]]:\n1. Low-rank random \"noise\" matrices are derived from public randomness and added to the inputs.\n2. The noised product is computed in blocks, and the transcript of intermediate blocks is hashed.\n3. A hash below a difficulty threshold is a winning proof.\n\nThe noise can then be subtracted cheaply to recover the useful product [[S-1609]]. Hardness rests on the transcript rather than the output, so choosing trivial matrices gives no shortcut [[S-1609]].\n\n**Pearl's variant.** Pearl's floating-point specification adapts this to FP8 matrix multiplication on GPUs [[S-1105]]:\n- quantising after the noise is added lets output tiles be hashed directly;\n- the verifier \"checks one tile of the matmul, never the whole product\", recomputing it bit for bit [[S-1105]].\n\n**Use for capacity accounting.** Attestable proposes proof-of-work accounting to bound compute available for unmonitored activity [[S-1102]]. In its motivating case, every inference proof from a datacenter is correct, yet the operator could still be using spare capacity \"to run an unauthorized training job\" [[S-1102]]. In its scheme, \"Useful, approved inference contributes to the required work budget\". When demand falls short, \"additional protocol-defined work fills the shortfall\" [[S-1102]].\n\n**Memory analogues.**\n- Proofs of space require dedicating \"a significant amount of disk space\" [[S-1607]].\n- A verification system design proposes filling device memory with incompressible noise and challenging random samples [[S-0018]] (see [[M-0015]]).\n\n## What it establishes\n**What it can establish.**\n- A valid proof of useful work shows that a stated amount of matrix-multiplication work was performed on inputs committed through the noise derivation [[S-1609]] [[S-1105]].\n- If the verifier knows the capacity of declared hardware, a sustained rate of such proofs could bound the capacity left for other work. This is Attestable's proposal [[S-1102]], and it bears on [[C-0004]], [[C-0003]] and [[C-0010]] for declared hardware.\n- The useful-work total must approach the available capacity over the same interval. JoshC argues that more than 95% of computation must be accounted for to constrain one covert reinforcement-learning strategy. That threshold is a scenario estimate, not a measured result [[S-3566]].\n\n**What it cannot establish.**\n- **Undeclared hardware.** Attestable states that a proof \"cannot discover a datacenter that was never declared\" [[S-1102]].\n- **What the work was for.** The proof does not show which AI workload, if any, the matrices came from. In Komargodski and Weinstein's construction the miner chooses the matrices [[S-1609]]. On Pearl's network, an independent study found that verification \"does not check whether the matrices originate from an AI model\" [[S-0071]]. Attestable proposes pairing work accounting with [[M-0004|proofs of inference]] for that [[S-1102]].\n\n## Threat model\nThe adversary is a miner who tries to produce accepted proofs faster than honest work would allow [[S-1609]]. Security holds in the random-oracle model under a \"direct-product\" hardness assumption about intermediate values of multiplying random low-rank matrices [[S-1609]]. Pearl's variant assumes that computing quantised products of noised matrices \"is not substantially easier\" than computing quantised products of matrices with generic random rows [[S-1105]]. For bit-exact verification it reproduces the GPU's arithmetic off the GPU with the technique of Badash et al. [[S-1105]].\n\nFor capacity bounding, the verifier also needs \"a credible estimate of the compute available\" to the actor [[S-1102]].\n\n## Evidence\n- **Theory.** Komargodski and Weinstein prove a multiplicative overhead of 1 + o(1) over naive matrix multiplication [[S-1609]].\n- **[[I-0004|Pearl]].** Pearl publishes the code of a network built on this construction [[S-1107]], and reports that the chain went live when the node code became public [[S-1106]]. It also reports a benchmark on four H200 GPUs. Its \"two-for-one\" variant of Llama 3.3 70B, which re-implements a layer with a new quantisation mechanism, reached up to 18,291.66 tokens per second. The original model's best configuration reached 15,269.81 tokens per second; with the four-way data parallelism that gave the variant its best figure, the original bf16 model ran out of memory [[S-1106]].\n- **Independent measurement of Pearl.** Basu studied Pearl's mainnet in May 2026. String analysis suggests that the dominant mining software, from a third party, contains no inference code and generates matrices from random seeds. Random matrices passed verification in the study's tests [[S-0071]].\n- **Capacity bounding.** As of September 2026 no public result applies proofs of useful work to bounding the capacity of declared AI hardware. Attestable describes its proposal as near-term work [[S-1102]].\n\n## Limitations\n**Verification cost.** Komargodski and Weinstein note that plain verification is \"relatively expensive on the verifier's side\". They suggest that the prover compute a SNARK to lighten it, or a zkSNARK if the matrices must stay private [[S-1609]]. They also note that storing the transcript takes significant memory [[S-1609]].\n\n**Known shortcuts.** Pearl lists known mining speedups: crafted inputs, precision shortcuts, seed grinding and work reuse. It adds checks to limit them [[S-1105]]. It describes faster kernels or hardware as \"not an attack on the protocol\" [[S-1105]].\n\n**Assumptions and scope.** Open problems include PoUW from more standard assumptions, and PoUW for tasks beyond matrix multiplication [[S-1609]].", "body_text": "How it works Proofs of work. A proof of work gives confidence that some declared amount of computation took place. In cryptocurrency mining, workers hash variants of a message until one hash meets a target, and a verifier can check it quickly [S-0005]. Scher and Thiergart note that applying proof of work \"could verify that compute declared to be doing crypto mining is indeed doing it\", as one way to check that known compute is not used for a large training run [S-0005]. Proofs of useful work. A proof of useful work (PoUW) makes that computation serve another purpose. Ball et al. gave constructions for problems such as Orthogonal Vectors, 3SUM and All-Pairs Shortest Path [S-1608]. Komargodski and Weinstein give a PoUW for arbitrary matrix multiplication, which lets the miner choose its own matrices [S-1609]. Their protocol works in three steps [S-1609]: 1. Low-rank random \"noise\" matrices are derived from public randomness and added to the inputs. 2. The noised product is computed in blocks, and the transcript of intermediate blocks is hashed. 3. A hash below a difficulty threshold is a winning proof. The noise can then be subtracted cheaply to recover the useful product [S-1609]. Hardness rests on the transcript rather than the output, so choosing trivial matrices gives no shortcut [S-1609]. Pearl's variant. Pearl's floating-point specification adapts this to FP8 matrix multiplication on GPUs [S-1105]: - quantising after the noise is added lets output tiles be hashed directly; - the verifier \"checks one tile of the matmul, never the whole product\", recomputing it bit for bit [S-1105]. Use for capacity accounting. Attestable proposes proof-of-work accounting to bound compute available for unmonitored activity [S-1102]. In its motivating case, every inference proof from a datacenter is correct, yet the operator could still be using spare capacity \"to run an unauthorized training job\" [S-1102]. In its scheme, \"Useful, approved inference contributes to the required work budget\". When demand falls short, \"additional protocol-defined work fills the shortfall\" [S-1102]. Memory analogues. - Proofs of space require dedicating \"a significant amount of disk space\" [S-1607]. - A verification system design proposes filling device memory with incompressible noise and challenging random samples [S-0018] (see Memory wiping and proofs of secure erasure). What it establishes What it can establish. - A valid proof of useful work shows that a stated amount of matrix-multiplication work was performed on inputs committed through the noise derivation [S-1609] [S-1105]. - If the verifier knows the capacity of declared hardware, a sustained rate of such proofs could bound the capacity left for other work. This is Attestable's proposal [S-1102], and it bears on This compute runs inference, not training, Declared hardware is idle or shut down and There is no undeclared relevant compute for declared hardware. - The useful-work total must approach the available capacity over the same interval. JoshC argues that more than 95% of computation must be accounted for to constrain one covert reinforcement-learning strategy. That threshold is a scenario estimate, not a measured result [S-3566]. What it cannot establish. - Undeclared hardware. Attestable states that a proof \"cannot discover a datacenter that was never declared\" [S-1102]. - What the work was for. The proof does not show which AI workload, if any, the matrices came from. In Komargodski and Weinstein's construction the miner chooses the matrices [S-1609]. On Pearl's network, an independent study found that verification \"does not check whether the matrices originate from an AI model\" [S-0071]. Attestable proposes pairing work accounting with proofs of inference for that [S-1102]. Threat model The adversary is a miner who tries to produce accepted proofs faster than honest work would allow [S-1609]. Security holds in the random-oracle model under a \"direct-product\" hardness assumption about intermediate values of multiplying random low-rank matrices [S-1609]. Pearl's variant assumes that computing quantised products of noised matrices \"is not substantially easier\" than computing quantised products of matrices with generic random rows [S-1105]. For bit-exact verification it reproduces the GPU's arithmetic off the GPU with the technique of Badash et al. [S-1105]. For capacity bounding, the verifier also needs \"a credible estimate of the compute available\" to the actor [S-1102]. Evidence - Theory. Komargodski and Weinstein prove a multiplicative overhead of 1 + o(1) over naive matrix multiplication [S-1609]. - Pearl. Pearl publishes the code of a network built on this construction [S-1107], and reports that the chain went live when the node code became public [S-1106]. It also reports a benchmark on four H200 GPUs. Its \"two-for-one\" variant of Llama 3.3 70B, which re-implements a layer with a new quantisation mechanism, reached up to 18,291.66 tokens per second. The original model's best configuration reached 15,269.81 tokens per second; with the four-way data parallelism that gave the variant its best figure, the original bf16 model ran out of memory [S-1106]. - Independent measurement of Pearl. Basu studied Pearl's mainnet in May 2026. String analysis suggests that the dominant mining software, from a third party, contains no inference code and generates matrices from random seeds. Random matrices passed verification in the study's tests [S-0071]. - Capacity bounding. As of September 2026 no public result applies proofs of useful work to bounding the capacity of declared AI hardware. Attestable describes its proposal as near-term work [S-1102]. Limitations Verification cost. Komargodski and Weinstein note that plain verification is \"relatively expensive on the verifier's side\". They suggest that the prover compute a SNARK to lighten it, or a zkSNARK if the matrices must stay private [S-1609]. They also note that storing the transcript takes significant memory [S-1609]. Known shortcuts. Pearl lists known mining speedups: crafted inputs, precision shortcuts, seed grinding and work reuse. It adds checks to limit them [S-1105]. It describes faster kernels or hardware as \"not an attack on the protocol\" [S-1105]. Assumptions and scope. Open problems include PoUW from more standard assumptions, and PoUW for tasks beyond matrix multiplication [S-1609].", "referenced_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0005", "title": "Attestable zero-knowledge inference prover", "url": "https://trustbutveri.fyi/implementations/attestable-zk-inference/" }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "K-0011", "title": "Proof of (useful) work", "url": "https://trustbutveri.fyi/concepts/proof-of-useful-work/" }, { "id": "O-0120", "title": "Attestable", "url": "https://trustbutveri.fyi/organizations/attestable/" }, { "id": "O-0121", "title": "Pearl Research Labs", "url": "https://trustbutveri.fyi/organizations/pearl-research/" } ] }, { "id": "M-0020", "slug": "remote-detection-of-data-centres", "title": "Remote detection of data centres", "aliases": [ "Satellite monitoring of data centres", "Remote sensing of AI compute facilities", "National technical means for AI compute" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Remote detection locates large data centres and estimates their power capacity without site access, using satellite imagery, heat signatures and public records such as permits.", "summary": "Remote detection of data centres is a set of methods for finding facilities and estimating their size from outside. Large AI data centres need buildings, substations and cooling equipment, and they shed roughly as much heat as the electricity they use. These features can be seen without the operator's cooperation. Analysts already combine satellite imagery with permits and utility filings to track the construction of known large facilities and estimate their power capacity. For verification, the harder task is finding facilities nobody has declared. As of September 2026 no systematic search for such facilities has been published, and automated detection of data centres remains mainly conceptual. Imagery also cannot see inside buildings or count chips. The main weaknesses are concealment, such as disguising a facility as other industry or building it underground, and sites too small to stand out. Verification frameworks treat these signals as supplements to stronger mechanisms.", "category": "remote-sensing", "secondary_categories": [], "verifies": [ { "claim": "C-0010", "role": "primary", "note": "Searches for large facilities that have not been declared." }, { "claim": "C-0001", "role": "supporting", "note": "Estimates the power capacity, and so roughly the compute, of observed facilities." } ], "threat_model": "adversarial", "adversarial_evaluation": "analysis", "hardware_requirement": "none", "prover_cooperation": "not-required", "confidentiality": "preserving", "depends_on": [], "readiness": { "assessment": true, "level": "R1", "scope": "finding undeclared data centres (capacity estimates for known sites would meet at least R2)", "rubric_version": "1.1", "rationale": "R1 for finding undeclared facilities, because no public work shows a systematic search that finds them.\n\n- **R1** met: Halstead and Larsen describe heat, imagery and other detection signals, ways to conceal a facility and the odds of detecting covert ones [[S-1410]]. Baker et al. place satellite imagery among supplementary verification mechanisms [[S-0002]].\n- **R2** not met for this use: Krawec's case studies track known sites [[S-1409]]. Epoch AI reports that its public database finds large facilities mainly through company announcements, news, third-party databases and social media [[S-3301]]. Krawec states that automated data-centre detection \"remains primarily conceptual at present\" [[S-1409]].\n\nThe level is for the primary use ([[C-0010]]). For the supporting use of estimating the capacity of known sites ([[C-0001]]), the public Epoch database, with its published error estimates, would meet at least R2 [[S-3301]] [[S-1411]].", "evidence": [ "S-1410", "S-0002", "S-1409", "S-1411", "S-3301" ], "next_level_gaps": [ "Published end-to-end results on finding previously unknown large facilities over a wide area, with measured miss and false-alarm rates.", "An evaluation against a stated concealment adversary, for example disguised or underground facilities." ], "confidence": "medium", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Facilities can be disguised or hidden", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Halstead and Larsen discuss two ways to hide a facility. One is to disguise it as a legitimate industrial site. The other is to build it underground, with cooling that avoids visible heat plumes. They note that the underground option requires bespoke engineering [[S-1410]].", "sources": [ "S-1410" ], "response": null }, { "assessment": true, "title": "Small sites may not be detectable", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Halstead and Larsen conclude that a sufficiently small covert project could not be ruled out with confidence. In their estimates, the chance of detection is lower for smaller sites [[S-1410]]. Krawec notes that small data centres in existing buildings may lack the distinctive features of large facilities [[S-1409]].", "sources": [ "S-1410", "S-1409" ], "response": null }, { "assessment": true, "title": "Search for unknown sites is undemonstrated", "kind": "open-question", "severity": "significant", "status": "open", "description": "Krawec reports that telling data centres apart from other industrial facilities systematically is difficult. Automating detection would need large amounts of training imagery and a purpose-trained model. In Krawec's words, automated data-centre detection \"remains primarily conceptual at present\" [[S-1409]].", "sources": [ "S-1409" ], "response": null } ], "blockers": [ { "text": "Wide-area, automated detection of data centres is not yet practical and needs large training datasets.", "theme": "coverage-hidden-compute", "blocked_by": null, "sources": [ "S-1409" ] }, { "text": "No measured detection or false-alarm rates for finding undeclared facilities have been published.", "theme": "adversarial-validation", "blocked_by": null, "sources": [ "S-1410", "S-1409" ] }, { "text": "Recent high-resolution imagery is costly, is limited by weather and needs trained analysts.", "theme": "access-governance", "blocked_by": null, "sources": [ "S-1409" ] } ], "challenge_themes": [ "coverage-hidden-compute", "adversarial-validation", "capacity-bounds" ], "organizations": [ "O-0102", "O-0201", "O-0208" ], "people": [], "sources": [ { "source": "S-1409", "supports": "observable features; imagery sources and limits; capacity estimate example; cannot see inside; automated detection conceptual and its data needs; IAEA analogy; future sensors", "locator": "Methodology; Opportunities and Challenges; Case Studies 1-2; Recommendations; Opportunities for Further Research" }, { "source": "S-1411", "supports": "public dataset; cooling-equipment-based capacity method; coverage figures; licence", "locator": "announcement post" }, { "source": "S-3301", "supports": "how sites are found; reported accuracy of IT-power estimates; estimated coverage of global AI computing capacity (provider-reported)", "locator": "Coverage; Cooling model; Analysis" }, { "source": "S-1410", "supports": "heat-balance argument; detection signals; confidence in locating non-concealed data centres; concealment strategies; intuition-based detection probabilities and their conditions; limits for small projects; undeployed chips", "locator": "detection sections; direct observation of AI datacenters; table of intuition-based detection probabilities by site size and number of sites" }, { "source": "S-0002", "supports": "satellite imagery, OSINT, supplier information and financial audits as supplementary mechanisms; national intelligence layer", "locator": "§4.3, §4.4" }, { "source": "S-0007", "supports": "infrared imaging can detect undeclared data centres; power alone cannot separate AI from other HPC", "locator": "§3.1 (M4)" }, { "source": "S-3300", "supports": "AI data centres not visually distinguishable from other data centres (2024); AI compute housed in existing campuses", "locator": "blog post" } ], "concepts": [ "K-0016", "K-0003" ], "complements": [ "M-0019", "M-0021", "M-0018" ], "alternatives": [], "type": "mechanism", "implementations": [], "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/", "source_file": "content/mechanisms/remote-detection-of-data-centres.md", "flags_all": [], "body_markdown": "## How it works\n\nRemote detection infers the existence, size and status of data centres without entering them [[S-1409]]. Electro-optical satellite imagery shows these features [[S-1409]]:\n\n- large data halls;\n- substations and switchyards;\n- on-site gas turbines and backup generators;\n- cooling towers and chillers;\n- construction progress.\n\nAnalysts combine imagery with permits, utility filings, company announcements and other open sources [[S-1409]]. Free imagery comes from archives such as the ESA Copernicus Sentinel and NASA Landsat missions [[S-1409]]. Paid commercial imagery reaches sub-metre resolution, from providers such as [[O-0102|Planet Labs]], Airbus and Vantor [[S-1409]].\n\nCapacity can be estimated from cooling equipment [[S-1411]] [[S-1409]]. Epoch AI's Frontier Data Centers Hub finds chillers and cooling towers in satellite images and checks them against permits and public disclosures [[S-1411]]. From these it infers each facility's power capacity, and then the compute installed, in H100-equivalents [[S-1411]].\n\nHalstead and Larsen treat waste heat as the main obstacle to hiding a facility [[S-1410]]. They note that \"each megawatt of electricity going into a datacenter must be matched with a megawatt going out in some form (mostly as heat)\" [[S-1410]]. They list other signals [[S-1410]]:\n\n- Infrared satellites can see exhaust air and cooling plumes.\n- Radar interferometry can reveal ground movement from excavation.\n- Ground moving-target radar can track logistics vehicles.\n\nAnsari also notes that infrared remote sensing can detect undeclared data centres through their heat signatures [[S-0007]].\n\nVerification frameworks usually give these signals a supporting role [[S-0002]]. Baker et al. list satellite or aerial images, open-source intelligence, information from data-centre suppliers and financial audits as \"less robust mechanisms\" that could supplement the main ones [[S-0002]]. One of their main layers is national intelligence activities, which can draw on human, cyber and signals intelligence [[S-0002]]. Remote detection complements chip accounting ([[M-0019]]) and workload classification ([[M-0021]]).\n\n## What it establishes\nFrom outside, analysts can see construction progress, site layout and power and cooling infrastructure, and can estimate power capacity approximately [[S-1409]] [[S-1411]]. In one of Krawec's case studies, Epoch AI's capacity estimates from cooling equipment ranged from about 200 to 500 MW, against a published expected capacity of 300 MW [[S-1409]]. Imagery can also reveal gaps between announced and observed construction [[S-1409]].\n\nRemote detection has five gaps:\n\n- Electro-optical imagery \"can only view the outside of buildings\", so it gives no chip counts, chip types or actual power consumption [[S-1409]].\n- Capacity figures from cooling equipment are approximations, not measurements of energy use [[S-1409]].\n- Power figures alone cannot distinguish AI training from other high-performance computing [[S-0007]].\n- Heim and Pilz wrote in 2024 that AI data centres had no visual features that set them apart from data centres hosting other compute, and that large companies often place AI compute on existing campuses [[S-3300]].\n- Observing data centres does not help find chips that have been produced but not yet installed [[S-1410]].\n\n## Threat model\nRemote detection assumes that large facilities have physical signatures that are hard to remove, such as buildings, substations, cooling equipment and waste heat [[S-1410]] [[S-1409]]. It uses imagery and public records, so it needs no cooperation from the operator [[S-1409]].\n\nThe adversary may try to hide a facility [[S-1410]]. Halstead and Larsen consider disguising sites as industrial facilities, and underground designs whose cooling avoids visible plumes [[S-1410]].\n\nThe approach also assumes access to recent imagery and skilled analysts. Krawec lists cloud cover, smoke and weather, the cost of recent high-resolution imagery and the need for trained analysts as constraints [[S-1409]].\n\n## Evidence\n- Krawec's May 2026 report for the Federation of American Scientists presents two case studies [[S-1409]]. One tracks a facility's construction over 2024 and 2025 against public announcements [[S-1409]]. The other counts infrastructure at an operating AI facility and checks it against utility records and permits [[S-1409]].\n- Epoch AI's hub covered 13 large US data centres at its launch in November 2025 [[S-1411]]. Epoch estimated that they held about 2.5 million of the roughly 15 million H100-equivalents delivered to customers worldwide in recent years [[S-1411]]. The data are released under a Creative Commons Attribution licence [[S-1411]].\n- By September 2026 the database still mostly covered US sites, and Epoch estimated its coverage of global deployed AI computing capacity at 44% (90% CI 24–81%) [[S-3301]]. Epoch reports that 80% of its IT-power estimates are within a factor of 1.4 of the actual value, and that its estimates were near-perfect in the two cases where it obtained ground-truth cooling capacity [[S-3301]].\n- Halstead and Larsen are \"relatively confident\" that before a deal, intelligence services could locate every AI data centre in the country their scenario examines, other than specifically concealed ones, using methods like those of Epoch AI and commercial analysts [[S-1410]]. They estimate the chance that unilateral detection methods find a covert project within ten years of an international agreement [[S-1410]]. For a single site they give 26% at 10 MW, 35% at 100 MW and 50% at 1 GW [[S-1410]]. For ten sites of each size they give 37%, 67% and 85% [[S-1410]]. They call these estimates highly speculative, based entirely on intuition and qualitative reasoning [[S-1410]].\n- Krawec recommends satellite imagery as one element of verification for future agreements, and points to the IAEA's satellite imagery analysis team for nuclear safeguards [[S-1409]]. The report names thermal infrared, synthetic-aperture radar and multispectral imaging as directions for further research [[S-1409]].\n\n## Limitations\n- Facilities can be disguised as other industry or placed underground [[S-1410]].\n- A sufficiently small covert project could not be ruled out with confidence [[S-1410]]. Small data centres in existing buildings may lack distinctive signatures [[S-1409]].\n- Without a known location, telling data centres apart from other industrial facilities is hard, and automated detection is mainly conceptual [[S-1409]].\n- Chip counts, chip types and actual energy use cannot be observed [[S-1409]].\n- Cooling and power features vary across companies and countries, and not every site generates its own power [[S-1409]].", "body_text": "How it works Remote detection infers the existence, size and status of data centres without entering them [S-1409]. Electro-optical satellite imagery shows these features [S-1409]: - large data halls; - substations and switchyards; - on-site gas turbines and backup generators; - cooling towers and chillers; - construction progress. Analysts combine imagery with permits, utility filings, company announcements and other open sources [S-1409]. Free imagery comes from archives such as the ESA Copernicus Sentinel and NASA Landsat missions [S-1409]. Paid commercial imagery reaches sub-metre resolution, from providers such as Planet Labs, Airbus and Vantor [S-1409]. Capacity can be estimated from cooling equipment [S-1411] [S-1409]. Epoch AI's Frontier Data Centers Hub finds chillers and cooling towers in satellite images and checks them against permits and public disclosures [S-1411]. From these it infers each facility's power capacity, and then the compute installed, in H100-equivalents [S-1411]. Halstead and Larsen treat waste heat as the main obstacle to hiding a facility [S-1410]. They note that \"each megawatt of electricity going into a datacenter must be matched with a megawatt going out in some form (mostly as heat)\" [S-1410]. They list other signals [S-1410]: - Infrared satellites can see exhaust air and cooling plumes. - Radar interferometry can reveal ground movement from excavation. - Ground moving-target radar can track logistics vehicles. Ansari also notes that infrared remote sensing can detect undeclared data centres through their heat signatures [S-0007]. Verification frameworks usually give these signals a supporting role [S-0002]. Baker et al. list satellite or aerial images, open-source intelligence, information from data-centre suppliers and financial audits as \"less robust mechanisms\" that could supplement the main ones [S-0002]. One of their main layers is national intelligence activities, which can draw on human, cyber and signals intelligence [S-0002]. Remote detection complements chip accounting (Chip registries and manufacturing records) and workload classification (Workload classification from telemetry and side channels). What it establishes From outside, analysts can see construction progress, site layout and power and cooling infrastructure, and can estimate power capacity approximately [S-1409] [S-1411]. In one of Krawec's case studies, Epoch AI's capacity estimates from cooling equipment ranged from about 200 to 500 MW, against a published expected capacity of 300 MW [S-1409]. Imagery can also reveal gaps between announced and observed construction [S-1409]. Remote detection has five gaps: - Electro-optical imagery \"can only view the outside of buildings\", so it gives no chip counts, chip types or actual power consumption [S-1409]. - Capacity figures from cooling equipment are approximations, not measurements of energy use [S-1409]. - Power figures alone cannot distinguish AI training from other high-performance computing [S-0007]. - Heim and Pilz wrote in 2024 that AI data centres had no visual features that set them apart from data centres hosting other compute, and that large companies often place AI compute on existing campuses [S-3300]. - Observing data centres does not help find chips that have been produced but not yet installed [S-1410]. Threat model Remote detection assumes that large facilities have physical signatures that are hard to remove, such as buildings, substations, cooling equipment and waste heat [S-1410] [S-1409]. It uses imagery and public records, so it needs no cooperation from the operator [S-1409]. The adversary may try to hide a facility [S-1410]. Halstead and Larsen consider disguising sites as industrial facilities, and underground designs whose cooling avoids visible plumes [S-1410]. The approach also assumes access to recent imagery and skilled analysts. Krawec lists cloud cover, smoke and weather, the cost of recent high-resolution imagery and the need for trained analysts as constraints [S-1409]. Evidence - Krawec's May 2026 report for the Federation of American Scientists presents two case studies [S-1409]. One tracks a facility's construction over 2024 and 2025 against public announcements [S-1409]. The other counts infrastructure at an operating AI facility and checks it against utility records and permits [S-1409]. - Epoch AI's hub covered 13 large US data centres at its launch in November 2025 [S-1411]. Epoch estimated that they held about 2.5 million of the roughly 15 million H100-equivalents delivered to customers worldwide in recent years [S-1411]. The data are released under a Creative Commons Attribution licence [S-1411]. - By September 2026 the database still mostly covered US sites, and Epoch estimated its coverage of global deployed AI computing capacity at 44% (90% CI 24–81%) [S-3301]. Epoch reports that 80% of its IT-power estimates are within a factor of 1.4 of the actual value, and that its estimates were near-perfect in the two cases where it obtained ground-truth cooling capacity [S-3301]. - Halstead and Larsen are \"relatively confident\" that before a deal, intelligence services could locate every AI data centre in the country their scenario examines, other than specifically concealed ones, using methods like those of Epoch AI and commercial analysts [S-1410]. They estimate the chance that unilateral detection methods find a covert project within ten years of an international agreement [S-1410]. For a single site they give 26% at 10 MW, 35% at 100 MW and 50% at 1 GW [S-1410]. For ten sites of each size they give 37%, 67% and 85% [S-1410]. They call these estimates highly speculative, based entirely on intuition and qualitative reasoning [S-1410]. - Krawec recommends satellite imagery as one element of verification for future agreements, and points to the IAEA's satellite imagery analysis team for nuclear safeguards [S-1409]. The report names thermal infrared, synthetic-aperture radar and multispectral imaging as directions for further research [S-1409]. Limitations - Facilities can be disguised as other industry or placed underground [S-1410]. - A sufficiently small covert project could not be ruled out with confidence [S-1410]. Small data centres in existing buildings may lack distinctive signatures [S-1409]. - Without a known location, telling data centres apart from other industrial facilities is hard, and automated detection is mainly conceptual [S-1409]. - Chip counts, chip types and actual energy use cannot be observed [S-1409]. - Cooling and power features vary across companies and countries, and not every site generates its own power [S-1409].", "referenced_by": [ { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" }, { "id": "C-0001", "title": "Compute stock is at most a declared amount", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "K-0016", "title": "Undeclared compute", "url": "https://trustbutveri.fyi/concepts/undeclared-compute/" }, { "id": "O-0201", "title": "AI Futures Project", "url": "https://trustbutveri.fyi/organizations/ai-futures-project/" }, { "id": "O-0208", "title": "Epoch AI", "url": "https://trustbutveri.fyi/organizations/epoch-ai/" }, { "id": "O-0102", "title": "Planet Labs", "url": "https://trustbutveri.fyi/organizations/planet-labs/" }, { "id": "O-0200", "title": "RAND", "url": "https://trustbutveri.fyi/organizations/rand/" } ] }, { "id": "M-0003", "slug": "reproducible-computation-packets", "title": "Whole-workload recomputation (reproducible packets)", "aliases": [ "Reproducible computation packets", "Packet-based verification" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Organizing all AI workloads in a facility into discrete, reproducible units, so that a verifier can recompute a random sample and check each one.", "summary": "The AI 2040 verification plan proposes that every AI workload in a monitored facility be organized into discrete, reproducible \"packets\" that a verifier's recomputation server can see. The verifier re-runs a random sample of packets to check that they were computed as declared. The plan states that smaller packets raise the chance of catching a rogue workload. As of September 2026, no implementation of whole-workload packets has been published. The nearest prototypes recompute single inference requests or, in proof-of-learning, selected training steps. The main obstacles are that workloads and network traffic are not reproducible by default, that the recomputation server must be secured, and that compute outside declared packets is not covered. The plan itself does not verify that spare compute is unused, and notes that non-compliant work might be hidden inside compliant-looking workloads.", "category": "isolation-architecture", "secondary_categories": [ "cryptographic-computational" ], "verifies": [ { "claim": "C-0004", "role": "primary", "note": "Proposed as the correctness check for an inference-only retrofit (S-0067)." }, { "claim": "C-0007", "role": "supporting", "note": "Proposed for later R&D verification by treating training steps as packets (S-0067, S-0017)." } ], "threat_model": "adversarial", "adversarial_evaluation": "none", "hardware_requirement": "retrofit-device", "prover_cooperation": "required", "confidentiality": "partial", "depends_on": [ { "target": "M-0002", "note": "Packets must be reproducible, which needs deterministic execution." }, { "target": "M-0013", "note": "Network taps copy traffic to the recomputation server." } ], "readiness": { "assessment": true, "level": "R1", "scope": "recomputing whole workloads to show a cluster runs only declared inference", "rubric_version": "1.1", "rationale": "The design, its claim and its assumptions are public, but no implementation of whole-workload packets has been built.\n\n- **R1** met: the AI 2040 verification plan describes the design (discrete, reproducible packets visible to a recomputation server, with random partial recomputation), the claim it serves (correct outputs in an inference-only regime, later R&D verification) and its assumptions (reproducibility, an intact recomputation server, physical security) [[S-0067]]. Amodo gives a comparable written design at the level of single training steps [[S-0017]]. The plan is the only Implementation record for this mechanism ([[I-0011]], assessed R1).\n- **R2** not met: no public implementation or end-to-end result organizes whole workloads into reproducible packets. The nearest demonstrations recompute single inference requests [[S-1006]] or, in proof-of-learning, selected training steps ([[M-0006]]) [[S-0027]]. The plan's companion page lists a reproducible inference stack and network reproducibility as not started [[S-1511]], and Amodo rates network reproducibility \"not on track\" [[S-1008]].\n\nConfidence is medium. The design is described only at a high level, but the plan's authors and Amodo both list the reproducible inference stack it needs as not started [[S-1511]] [[S-1008]].", "evidence": [ "S-0067", "S-0017", "S-1006", "S-1008", "S-1511", "S-0027" ], "next_level_gaps": [ "A public implementation, or reproducible end-to-end results, of packet-based recomputation beyond single inference requests, under realistic model scale, hardware or a stated adversary." ], "confidence": "medium", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Spare compute is outside the scheme", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "The plan states that it does not verify that spare compute is not used for unapproved workloads, because this seems very challenging [[S-0067]]. Recomputation checks the correctness of declared work, not its completeness [[S-0017]].", "sources": [ "S-0067", "S-0017" ], "response": null }, { "assessment": true, "title": "Non-compliant work could be encoded inside compliant-looking packets", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "The plan notes that an AI company might try to encode a non-compliant workload inside a workload that looks compliant on the surface [[S-0067]].", "sources": [ "S-0067" ], "response": null } ], "blockers": [ { "text": "Workloads are not reproducible by default, and achieving reproducibility may cost performance.", "theme": "performance-compatibility", "blocked_by": "M-0002", "sources": [ "S-0067" ] }, { "text": "Network packets are not individually reproducible by default; making them so may need considerable software, firmware and hardware work. Amodo rates this 'not on track'.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-1008" ] }, { "text": "All traffic must reach the recomputation server via network taps, and the server's integrity is critical.", "theme": "hardware-trust", "blocked_by": "M-0013", "sources": [ "S-0067", "S-1008" ] }, { "text": "Recomputing training steps needs checkpoints: writing one at every step would cost more than 100% overhead, so Amodo's design needs a spare data-parallel replica that tracks the weights instead.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-0017" ] } ], "challenge_themes": [ "coverage-hidden-compute", "performance-compatibility", "hardware-trust", "protocol-soundness" ], "organizations": [ "O-0101", "O-0201" ], "people": [], "sources": [ { "source": "S-0067", "supports": "packet design, hierarchy of workload steps, recomputation budget example, assumptions and stated gaps", "locator": "2027: Concrete inference-only retrofitting proposal; Feb 2029; Early 2030: Workload Approval, Workload Verification (2034 chart)" }, { "source": "S-0017", "supports": "step-level recomputation design for pre-training; shadow replica instead of per-step checkpoint writes; commitments before audit selection; tolerance comparison; audit rate; correctness vs completeness", "locator": "pre-training scheme; introduction" }, { "source": "S-1006", "supports": "single-request inference recomputation prototype", "locator": "whole note" }, { "source": "S-1008", "supports": "status of reproducible inference stack and network reproducibility", "locator": "status items" }, { "source": "S-1511", "supports": "plan authors' status of a reproducible inference stack and network reproducibility, July 2026", "locator": "reproducible packets" }, { "source": "S-0027", "supports": "proof-of-learning verifiers reproduce a subset of updates within a noise threshold; spoofs exploit the noise tolerance", "locator": "§4.1; §4.2; §6.1" } ], "concepts": [ "K-0009", "K-0014", "K-0016", "K-0020", "K-0025" ], "complements": [ "M-0001", "M-0014" ], "alternatives": [], "type": "mechanism", "implementations": [ { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" } ], "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/", "source_file": "content/mechanisms/reproducible-computation-packets.md", "flags_all": [], "body_markdown": "## How it works\n\nFor an inference-only retrofit, the AI 2040 verification plan proposes passive network taps that copy all input and output traffic to a recomputation server [[S-0067]]. All AI workloads, including inference and agentic tool use, would be organized into \"discrete and reproducible packets, which are transparent to the recomputation server\" [[S-0067]]. The server recomputes random samples of the packets to check that they are correct [[S-0067]]. Reproducibility is required because the correctness checks depend on it, and inference is not reproducible by default [[S-0067]].\n\nFor later verification of R&D, the plan breaks workloads into a hierarchy of steps: full training run, training phases, gradient steps, layer forward and backward passes, GPU kernel calls and individual instructions [[S-0067]]. If each step can be made reproducible (the plan notes that seeded randomness is acceptable), the granularity of evidence collection maps predictably to the level of assurance [[S-0067]]. The same plan would also split data centres into isolated inference units by removing much of the back-end (\"east-west\") networking [[S-0067]]; see [[M-0014]].\n\nThe plan illustrates the approach for 2034 [[S-0067]]:\n\n- a pool of about 33 billion H100-equivalents, its projected build-out for that year;\n- a recomputation budget of 1%;\n- packets of 100 H100-equivalent hours.\n\nIt charts, by the deployment's size, the probability of catching a rogue internal deployment within an hour, a week or a month [[S-0067]]. The plan states that detection improves as packet size decreases [[S-0067]].\n\nAmodo describes a related design for pre-training, in which each training step is the unit [[S-0017]]:\n\n- The prover commits a hash of sampled weight positions before learning whether the step will be audited [[S-0017]].\n- For audited steps, the verifier re-runs the step and accepts if the result agrees with the committed checkpoint within a calibrated tolerance [[S-0017]].\n- Because the verifier's hardware is much slower than the prover's, the audit rate follows from the slowdown: about 0.1% of steps if recomputing a step takes 1,000 times as long as the step itself [[S-0017]].\n\nAmodo reports a prototype that recomputes individual inference requests, which [[M-0001]] covers [[S-1006]].\n\n## What it establishes\nThe plan argues that random partial recomputation can verify all outputs with very high probability while using a small fraction of the original compute [[S-0067]]. Like other recomputation schemes, it checks whether declared workloads were computed correctly, not whether other work ran outside them [[S-0017]]. The plan states that it does not verify that spare compute is left unused for unapproved workloads [[S-0067]].\n\n## Threat model\n- **Recomputation server.** The plan calls its integrity \"an extremely important aspect\" of the retrofit [[S-0067]].\n- **Physical security.** The plan relies on strong physical security to protect the verification measures from tampering [[S-0067]].\n- **Cooperation.** Amodo's schemes assume the prover cooperates and lets the verifier install a recomputation server in its data centre [[S-0017]].\n- **Commit before audit.** In Amodo's training-step design, the prover commits its evidence before it learns which steps will be audited [[S-0017]].\n\n## Evidence\n- **Whole-workload packets.** As of September 2026, no implementation or end-to-end result of organizing whole workloads into reproducible packets has been published.\n- **Inference recomputation.** Amodo reports demonstrations of inference recomputation systems that work around nondeterminism [[S-1008]], including its own single-request prototype [[S-1006]].\n- **Training-step recomputation.** In [[M-0006|proof-of-learning]], a separate line of work, the verifier reproduces a subset of training updates and accepts each one within a noise threshold [[S-0027]].\n- **Reproducibility status.** Amodo rates a reproducible inference stack as not started [[S-1008]]. It rates network reproducibility as not on track, because optimizations throughout the networking stack mean that network packets are not individually reproducible by default [[S-1008]]. It suggests schemes that do not need exact packet replication as a possible alternative [[S-1008]]. The plan's own companion page, updated in July 2026, lists both items as not started [[S-1511]].\n\n## Limitations\n- **Reproducibility cost.** Making inference reproducible \"may involve some cost overhead\" [[S-0067]]. For training, writing a checkpoint at every step would cost more than 100% overhead, which Amodo's design avoids by keeping a spare replica [[S-0017]].\n- **Hidden work.** The plan notes that a company might try to encode a non-compliant workload inside one that looks compliant [[S-0067]]. Work outside declared packets, including on spare compute, is not covered [[S-0067]] [[S-0017]].\n- **Tolerance.** Amodo's training-step design accepts a recomputed step within a calibrated tolerance [[S-0017]]. In proof-of-learning, published spoofs exploit the tolerance that verification must allow for hardware noise [[S-0027]].\n- **Dependencies.** The scheme relies on deterministic execution ([[M-0002]]), complete traffic capture ([[M-0013]]) and a secure recomputation server [[S-0067]].", "body_text": "How it works For an inference-only retrofit, the AI 2040 verification plan proposes passive network taps that copy all input and output traffic to a recomputation server [S-0067]. All AI workloads, including inference and agentic tool use, would be organized into \"discrete and reproducible packets, which are transparent to the recomputation server\" [S-0067]. The server recomputes random samples of the packets to check that they are correct [S-0067]. Reproducibility is required because the correctness checks depend on it, and inference is not reproducible by default [S-0067]. For later verification of R&D, the plan breaks workloads into a hierarchy of steps: full training run, training phases, gradient steps, layer forward and backward passes, GPU kernel calls and individual instructions [S-0067]. If each step can be made reproducible (the plan notes that seeded randomness is acceptable), the granularity of evidence collection maps predictably to the level of assurance [S-0067]. The same plan would also split data centres into isolated inference units by removing much of the back-end (\"east-west\") networking [S-0067]; see Bandwidth limits and compartmentalization. The plan illustrates the approach for 2034 [S-0067]: - a pool of about 33 billion H100-equivalents, its projected build-out for that year; - a recomputation budget of 1%; - packets of 100 H100-equivalent hours. It charts, by the deployment's size, the probability of catching a rogue internal deployment within an hour, a week or a month [S-0067]. The plan states that detection improves as packet size decreases [S-0067]. Amodo describes a related design for pre-training, in which each training step is the unit [S-0017]: - The prover commits a hash of sampled weight positions before learning whether the step will be audited [S-0017]. - For audited steps, the verifier re-runs the step and accepts if the result agrees with the committed checkpoint within a calibrated tolerance [S-0017]. - Because the verifier's hardware is much slower than the prover's, the audit rate follows from the slowdown: about 0.1% of steps if recomputing a step takes 1,000 times as long as the step itself [S-0017]. Amodo reports a prototype that recomputes individual inference requests, which Sampled inference recomputation covers [S-1006]. What it establishes The plan argues that random partial recomputation can verify all outputs with very high probability while using a small fraction of the original compute [S-0067]. Like other recomputation schemes, it checks whether declared workloads were computed correctly, not whether other work ran outside them [S-0017]. The plan states that it does not verify that spare compute is left unused for unapproved workloads [S-0067]. Threat model - Recomputation server. The plan calls its integrity \"an extremely important aspect\" of the retrofit [S-0067]. - Physical security. The plan relies on strong physical security to protect the verification measures from tampering [S-0067]. - Cooperation. Amodo's schemes assume the prover cooperates and lets the verifier install a recomputation server in its data centre [S-0017]. - Commit before audit. In Amodo's training-step design, the prover commits its evidence before it learns which steps will be audited [S-0017]. Evidence - Whole-workload packets. As of September 2026, no implementation or end-to-end result of organizing whole workloads into reproducible packets has been published. - Inference recomputation. Amodo reports demonstrations of inference recomputation systems that work around nondeterminism [S-1008], including its own single-request prototype [S-1006]. - Training-step recomputation. In proof-of-learning, a separate line of work, the verifier reproduces a subset of training updates and accepts each one within a noise threshold [S-0027]. - Reproducibility status. Amodo rates a reproducible inference stack as not started [S-1008]. It rates network reproducibility as not on track, because optimizations throughout the networking stack mean that network packets are not individually reproducible by default [S-1008]. It suggests schemes that do not need exact packet replication as a possible alternative [S-1008]. The plan's own companion page, updated in July 2026, lists both items as not started [S-1511]. Limitations - Reproducibility cost. Making inference reproducible \"may involve some cost overhead\" [S-0067]. For training, writing a checkpoint at every step would cost more than 100% overhead, which Amodo's design avoids by keeping a spare replica [S-0017]. - Hidden work. The plan notes that a company might try to encode a non-compliant workload inside one that looks compliant [S-0067]. Work outside declared packets, including on spare compute, is not covered [S-0067] [S-0017]. - Tolerance. Amodo's training-step design accepts a recomputed step within a calibrated tolerance [S-0017]. In proof-of-learning, published spoofs exploit the tolerance that verification must allow for hardware noise [S-0027]. - Dependencies. The scheme relies on deterministic execution (Deterministic and bit-exact inference), complete traffic capture (Network taps and certifiers) and a secure recomputation server [S-0067].", "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "K-0020", "title": "Sampling and assurance", "url": "https://trustbutveri.fyi/concepts/sampling-and-assurance/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ] }, { "id": "M-0023", "slug": "safeguard-attestation", "title": "Safeguard attestation", "aliases": [ "Proof of guardrail", "Attested safeguards", "Verifiable safeguard execution" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "one_liner": "Hardware-signed evidence that an AI service ran its declared safeguards, such as a guardrail classifier or monitor, when producing a given response.", "summary": "Safeguard attestation aims to let users, auditors or other governments check that an AI service ran the safeguards it declares, such as a safety classifier, filter or usage monitor. Published designs run the safeguard code in a trusted execution environment, which signs a measurement of it with a hash of each input and response. A research prototype with public code does this on AWS Nitro Enclaves for a wrapper that routes an AI agent's traffic through an open-source guardrail. The guardrail model is called through an external API outside the enclave. In September 2026 Tinfoil reported rolling out enclave-run safeguard models with public, attested code. No independent security evaluation or reliance by another party has been documented. The main obstacles are showing that all traffic took the attested path and scaling to frontier GPU clusters. Attestation shows a safeguard ran, not that it works, since guardrails can be jailbroken.", "technical": "- **Proof-of-guardrail protocol.** (1) A wrapper program f bundles the public guardrail g, its configuration and the mediation of all agent inputs and outputs. (2) When f is loaded, the enclave records a measurement m, a hash that depends on the binary of f. (3) The private agent is loaded afterwards as a secret input, so it is not part of m. (4) For a user input x and response r, f returns a document signed with the platform's attestation key that contains m and d = Hash(x, r). (5) The verifier checks the certificate chain against the platform's published root, compares m with the expected measurement of the open-source f, and checks d [[S-1500]].\n- **Prototype costs.** On AWS Nitro Enclaves, the authors report 34% added latency on average over the same agent and guardrail run outside the enclave (24.8–38.0% across the four measured steps), 97.8 ms to generate an attestation and 5.1 ms to verify it. Holding the whole guardrail runtime in enclave memory needs an m5.xlarge instance, which costs 18.5 times as much per hour as a t3.micro [[S-1500]].\n- **PAL\\*M.** It defines single and session inference properties, r = M(M_tok(q)) and its multi-turn form over the chat history. It binds hashes of the query, tokenizer, model and response into an Intel TDX quote, together with an NVIDIA H100 attestation token and a verifier challenge [[S-0012]]. Compared with the same TDX machine without PAL\\*M's measurements, session inference cost 3.8–11.4% more and single-prompt inference 45.5–66.4% more, across Llama-3.1-8B, Gemma-3-4B and Phi-4-Mini [[S-0012]].", "category": "cryptographic-computational", "secondary_categories": [ "on-chip" ], "verifies": [ { "claim": "C-0006", "role": "primary", "note": "Attests that a measured safeguard program (guardrail, filter, monitor) mediated the attested responses; coverage of all traffic is not established." }, { "claim": "C-0005", "role": "supporting", "note": "Property and audit attestations bind responses to a measured model (S-0012, S-0009)." } ], "threat_model": "semi-trusted", "adversarial_evaluation": "analysis", "hardware_requirement": "existing-features", "prover_cooperation": "required", "confidentiality": "partial", "depends_on": [ { "target": "M-0008", "note": "Current designs rely on TEE measurement and remote attestation." }, { "target": "M-0012", "note": "Safeguard evidence is meaningful only when bound to the model actually served." } ], "readiness": { "assessment": true, "level": "R2", "scope": "attesting that a declared safeguard mediated a service's responses", "rubric_version": "1.1", "rationale": "R2, narrowly: one prototype with public code attests a guardrail end to end on cloud enclaves. No independent security evaluation or reliance by another party has been documented.\n\n- **R1** met: designs that state the claim (a response was produced after a specific guardrail ran) and their trust assumptions are published [[S-1500]], alongside designs for attesting inference properties [[S-0012]] and plan-scoped monitoring [[S-1503]].\n- **R2** met through the proof-of-guardrail prototype. Its code is public [[S-1501]], and its authors report end-to-end results on production cloud enclave hardware (AWS Nitro Enclaves) against a stated adversary, a developer who skips or modifies the guardrail [[S-1500]]. PAL\\*M adds attested session inference on Intel TDX with an H100 GPU, but its properties do not cover safeguards [[S-0012]].\n- **R3** not met: no party other than a developer is documented as relying on safeguard attestation, and the proof-of-guardrail code is described as a proof of concept that is not production-ready [[S-1501]]. Tinfoil reports that it is rolling out enclave-run safeguards in its chat service and can prove to an auditor or regulator that they are running, but as of mid-September 2026 the rollout was still under way [[S-3362]].\n- **R4** not met: no independent audit or red-team has been published, and the monitoring prototype has not been stress-tested by a counterparty [[S-1504]].\n\nConfidence is low because the demonstration is far from a frontier serving stack. It reaches both its guardrail model and the agent's backend model through external APIs, attests only the responses for which attestation is offered, and runs on CPU enclaves [[S-1500]]. Its README states that the enclave does not yet restrict the agent's command execution, which could be used to bypass the guardrail [[S-1501]]. Intel TDX attestations have also been forged by attackers with physical access to the memory bus [[S-1202]] [[S-3126]].", "evidence": [ "S-1500", "S-1501", "S-3362", "S-0012", "S-1503", "S-1504", "S-1202", "S-3126" ], "next_level_gaps": [ "Reliance by a party other than the developer on safeguard attestations for a verification decision, or a production-grade system that is generally available.", "An independent public evaluation (audit, red-team or peer-reviewed security analysis) of a safeguard-attestation system.", "A demonstration in which the safeguard model itself runs inside the attested boundary on GPU hardware at a realistic serving scale.", "A published way to show that all of a service's traffic, not only attested responses, passed through the attested safeguard path." ], "confidence": "low", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Attestation shows a safeguard ran, not that it is effective", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Proof of guardrail ensures that the guardrail executed, but the guardrail can still err or be jailbroken. Because the guardrail must be open source, a malicious developer can attack it with jailbreaks while still presenting a valid proof. In the authors' evaluation, Llama Guard 3 reached an F1 score of 0.56 on the unsafe class of the ToxicChat dataset. The authors state that proof of guardrail should not be interpreted or advertised as proof of safety.", "sources": [ "S-1500" ], "response": null }, { "assessment": true, "title": "Selective attestation leaves traffic uncovered", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Attestations are issued per response. In the prototype, the agent offers them when it receives high-stakes questions, so nothing shows that unattested traffic went through the same path. PAL\\*M's authors note that a prover could cherry-pick favourable executions, and suggest verifier-published nonces or requesting only session-level proofs. A governance analysis notes that auditors also need assurance that all activity is accounted for, since a host could start a second confidential virtual machine that bypasses monitoring.", "sources": [ "S-1500", "S-0012", "S-0014" ], "response": null }, { "assessment": true, "title": "Measurements may omit behaviour-relevant configuration or runtime changes", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Every component that influences inference behaviour must be covered by the launch measurement, including feature flags, environment variables and invocation arguments. A launch measurement also does not show that a program keeps running as measured if the kernel is later compromised.", "sources": [ "S-0014" ], "response": null }, { "assessment": true, "title": "Components outside the attested boundary", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "In the proof-of-guardrail experiments, the guardrail model and the agent's backend model were both reached through external APIs, and the authors leave the decision to trust those APIs to the verifier. The measured wrapper must also have no vulnerability that lets the unmeasured agent bypass the guardrail, for example by executing arbitrary commands inside the enclave. The code's README states that the enclave does not currently restrict the agent's arbitrary command execution, which could be used to bypass guardrails.", "sources": [ "S-1500", "S-1501" ], "response": null }, { "assessment": true, "title": "Memory-bus interposition extracts attestation keys and forges attestations", "kind": "demonstrated-attack", "severity": "significant", "status": "open", "description": "With physical access to a server's DDR5 memory bus and equipment costing under $1,000, researchers extracted a per-CPU Intel attestation provisioning key and forged TDX attestations. Against AMD SEV-SNP the same attack recovered a signing key used inside the virtual machine, not an AMD attestation key. By pairing forged TDX attestations with genuine H100 attestations relayed from rented hardware, they made a workload without TEE protection appear to run under GPU confidential computing. Intel, AMD and NVIDIA acknowledged the findings, and the researchers report that Intel and AMD consider interposer attacks out of scope. A second team forged attestation reports on an up-to-date Intel TDX platform with an active DDR5 interposer, DDRop. Other attacks have forged SEV-SNP attestation: Battering RAM with an interposer on DDR4 servers, and RMPocalypse from a malicious hypervisor on platforms without AMD's firmware fix. PAL\\*M places physical attacks outside its threat model. A governance analysis calls such key extraction \"relatively low-hanging fruit\" in an international treaty setting.", "sources": [ "S-1202", "S-3126", "S-1210", "S-1212", "S-1213", "S-0012", "S-0014" ], "response": null } ], "blockers": [ { "text": "No published design shows that all of a provider's traffic passes through the attested safeguard path; current evidence covers individual attested responses.", "theme": "coverage-hidden-compute", "blocked_by": null, "sources": [ "S-1500", "S-0014" ] }, { "text": "Frontier model inference typically needs several GPUs, GPU confidential computing is less mature than CPU support, and CPU inference, which an enclave prototype had to use, ran about 100 times slower than GPU inference.", "theme": "performance-compatibility", "blocked_by": "M-0008", "sources": [ "S-0014", "S-0009" ] }, { "text": "Trust rests on a small number of hardware vendors, and a per-CPU Intel attestation key has been extracted by physical attack.", "theme": "hardware-trust", "blocked_by": "M-0008", "sources": [ "S-0014", "S-1202" ] }, { "text": "Safeguard evidence must be bound to the model actually served, which depends on model-identity attestation.", "theme": "evidence-binding", "blocked_by": "M-0012", "sources": [ "S-0009", "S-0013" ] }, { "text": "No independent red-team or audit of a safeguard-attestation system has been published, and the available prototypes are described by their authors as proofs of concept that have not been stress-tested by a counterparty.", "theme": "adversarial-validation", "blocked_by": null, "sources": [ "S-1501", "S-1504" ] } ], "challenge_themes": [ "coverage-hidden-compute", "evidence-binding", "hardware-trust", "performance-compatibility", "adversarial-validation" ], "organizations": [ "O-0142", "O-0202", "O-0141" ], "people": [], "sources": [ { "source": "S-1500", "supports": "problem statement; protocol; threat model and trust assumptions; implementation and overheads; tamper tests; guardrail accuracy on ToxicChat; jailbreak risk; proof-of-safety caveat; external APIs; wrapper-bypass risk; selective attestation", "locator": "abstract; §3; §4.1; Tables 1-3; Appendix A" }, { "source": "S-1501", "supports": "public code; proof-of-concept status; stated limitation on agent command execution", "locator": "README, including Limitations" }, { "source": "S-3362", "supports": "Tinfoil's enclave-run safeguard pipeline: models, what leaves the enclave, open-source and attested code, rollout status", "locator": "whole post (updated 2026-09-16)" }, { "source": "S-0012", "supports": "inference property definitions; TDX+H100 implementation; overheads and baseline; threat model; cherry-picking discussion", "locator": "abstract; §3.2; §4.3.4 (Defs. 7-8); §4.4; Table 6; Appendix A" }, { "source": "S-0009", "supports": "inference protocol linking model, audit result, prompt and response; CPU-only enclave prototype; CPU versus GPU cost and slowdown", "locator": "§3 (Inference protocol); §5; Table 2" }, { "source": "S-1503", "supports": "verifiably-scoped monitoring protocol", "locator": "abstract" }, { "source": "S-1504", "supports": "reference implementation in Tinfoil confidential VMs; stated limitations", "locator": "reference implementation; limitations" }, { "source": "S-0014", "supports": "policy adherence as a verification property; measurement completeness; runtime state; second-CVM completeness gap; vendor root of trust; treaty threat model; GPU TEE maturity and multi-GPU inference", "locator": "deployment integrity; hardware auditability; resource accounting; physical attack surface" }, { "source": "S-0003", "supports": "configuration drift, such as swapping safety classifiers or relaxing filter thresholds", "locator": "§5.2" }, { "source": "S-0018", "supports": "verifier-side compliance screening of re-executed records", "locator": "§3.2.2; §5.2.3" }, { "source": "S-1502", "supports": "black-box statistical test for system-prompt use; its prompt-protection setting", "locator": "abstract; §3.2" }, { "source": "S-1202", "supports": "physical key extraction from TDX and signing-key extraction in SEV-SNP; forged attestations against NVIDIA GPU confidential computing; cost; vendor acknowledgement and positions", "locator": "project site summary; paper abstract and disclosure" }, { "source": "S-3126", "supports": "DDRop forges attestation reports on an up-to-date Intel TDX platform with an active DDR5 interposer", "locator": "site summary" }, { "source": "S-1210", "supports": "Battering RAM forges SEV-SNP attestation with a DDR4 interposer" }, { "source": "S-1212", "supports": "RMPocalypse forges SEV-SNP attestation from a malicious hypervisor" }, { "source": "S-1213", "supports": "AMD firmware fixes for RMPocalypse (CVE-2025-0033)" }, { "source": "S-0013", "supports": "binding model weights to enclave attestation", "locator": "whole post" } ], "concepts": [ "K-0001", "K-0002", "K-0004", "K-0005", "K-0006", "K-0009", "K-0018", "K-0019", "K-0024" ], "complements": [ "M-0025" ], "alternatives": [ "M-0001" ], "type": "mechanism", "implementations": [], "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/", "source_file": "content/mechanisms/safeguard-attestation.md", "flags_all": [ "provider-reported" ], "body_markdown": "## How it works\n\nUsers of AI services often have to rely on a developer's word about how safety is enforced, so safety measures could be falsely advertised [[S-1500]]. A paper on frontier AI auditing notes that swapping in a different safety classifier or relaxing a filter threshold can materially change how easily a system is misused [[S-0003]]. A governance analysis of trusted execution environments (TEEs) lists \"policy adherence\" as a property a verifier needs: a workload should show that it uses the approved model or enforces access controls and safeguards [[S-0014]].\n\nThe published designs follow one pattern: run the safeguard inside a TEE and have the hardware sign what ran [[S-1500]] [[S-0012]] [[S-1503]].\n\n- **Proof of guardrail.** A wrapper program containing a public guardrail and its configuration mediates all of an agent's inputs and outputs inside the enclave [[S-1500]]. When a user asks for it, or the agent offers it, the enclave signs a document containing a hash of the wrapper and a hash of the input and response [[S-1500]] [[S-1501]]. A user compares the wrapper hash with the published open-source build and checks the signature offline, while the developer's agent stays private [[S-1500]].\n- **Property attestation.** PAL\\*M runs inference in a confidential virtual machine with a GPU, and attests that a response came from a specific model and tokenizer on a given query or chat history [[S-0012]]. Its defined inputs are the query, model and tokenizer; it has no separate safeguard input [[S-0012]].\n- **Audit-to-deployment binding.** [[I-0007|Attestable Audits]] returns, with each response, an attestation that links the model, its earlier audit result, the prompt and the response [[S-0009]]. Binding weights to attestation is covered in [[M-0012]].\n- **Scoped monitoring.** The monitoring party and the monitored party co-sign a plan, which an open-source classifier runs inside an attested TEE [[S-1503]]. Queries outside the plan's scope are never evaluated [[S-1503]].\n\nTwo approaches do not use TEEs:\n\n- In a low-trust system design, the verifier re-executes sampled records and screens them itself for compliance, for example for blacklisted uses [[S-0018]]. See [[M-0001]] and [[I-0012]].\n- Prompt Detective compares output distributions to judge whether a given system prompt was used [[S-1502]]. It was developed to detect reuse of proprietary prompts, not to audit declared safeguards [[S-1502]].\n\n## What it establishes\nIt can establish:\n\n- that the declared guardrail code ran for an attested response, without revealing the developer's agent [[S-1500]];\n- that tampering is visible: in the authors' tests, modified guardrail code, modified attestation bytes and modified responses were all detected [[S-1500]];\n- in scoped monitoring, that the monitor evaluated only what the co-signed plan allows [[S-1503]].\n\nIt cannot establish:\n\n- that the safeguard works, since guardrails make errors and can be jailbroken [[S-1500]];\n- that all traffic was covered, since attestations are per response and the prototype offers them for high-stakes questions [[S-1500]];\n- anything the measurement leaves out, such as configuration, flags or later changes at runtime [[S-0014]].\n\n## Threat model\n- Proof of guardrail assumes a developer who might skip or modify the guardrail [[S-1500]]. It trusts the cloud provider's enclave hypervisor to measure code and protect keys, and it needs an open-source guardrail so that verifiers know the expected measurement [[S-1500]].\n- PAL\\*M assumes an adversary who controls the host, hypervisor and disk [[S-0012]]. It trusts the Intel TDX module and the H100 GPU [[S-0012]]. It treats side-channel defences as orthogonal and places physical attacks, such as memory-bus interposition, out of scope [[S-0012]].\n- A governance analysis contrasts the confidential-computing adversary, a dishonest operator, with a treaty adversary: a state with physical access to data centres and leverage over hardware vendors [[S-0014]].\n\n## Evidence\n- **Proof of guardrail.** The authors implemented it for OpenClaw agents on AWS Nitro Enclaves, with Llama Guard 3 for content safety and a fact-checking tool [[S-1500]]. They report 34% added latency on average compared with running outside the enclave [[S-1500]]. The code is public; its README calls it a proof of concept that is not production-ready [[S-1501]].\n- **Tinfoil safeguards.** Tinfoil reports that it is rolling out safeguard models, gpt-oss-safeguard with Kimi-K3 reviewing flagged cases, inside its secure enclaves for Tinfoil Chat [[S-3362]]. It states that the whole pipeline is open source and attested, so that it can prove to an auditor, a regulator or anyone else that the safeguards are running and what the policy is [[S-3362]]. Only the fact of a violation, with the account and conversation ID, leaves the enclave [[S-3362]].\n- **PAL\\*M.** It is implemented on Intel TDX with an NVIDIA H100 [[S-0012]]. The authors report 3.8–11.4% overhead for session inference over the same machine without PAL\\*M, and they modelled the protocol formally with the Tamarin prover [[S-0012]].\n- **Attestable Audits.** A prototype ran a 4-bit 8-billion-parameter model in CPU-only AWS Nitro Enclaves at 1.84 tokens per second [[S-0009]].\n- **Scoped monitoring.** A reference implementation runs in Tinfoil confidential virtual machines [[S-1504]]. Its authors state that user data and plan execution in the demo are not actually secure, and that the system has not been stress-tested by a counterparty [[S-1504]].\n\n## Limitations\n- **Effectiveness gap.** An attested guardrail can still be jailbroken [[S-1500]].\n- **Selective attestation.** Provers might attest only favourable executions [[S-0012]] [[S-1500]].\n- **External APIs.** The prototype called its guardrail model and backend model through external APIs [[S-1500]].\n- **Hardware attacks.** With physical access to the memory bus, researchers forged Intel TDX attestations and, by pairing them with relayed H100 attestations, made a workload outside TEE protection pass as GPU confidential computing [[S-1202]]. A second team forged TDX attestation reports with an active interposer [[S-3126]]. Other attacks forged AMD SEV-SNP attestations, one through a DDR4 interposer and one from software alone on platforms without AMD's fix [[S-1210]] [[S-1212]].\n- **Scale.** Frontier model inference typically needs several GPUs [[S-0014]]. CPU inference, which the enclave prototype had to use, cost 21.7 times as much per token as GPU inference and ran about 100 times slower [[S-0009]].\n\nFor confidential workflows that combine several parties' private inputs, see [[M-0025]].", "body_text": "How it works Users of AI services often have to rely on a developer's word about how safety is enforced, so safety measures could be falsely advertised [S-1500]. A paper on frontier AI auditing notes that swapping in a different safety classifier or relaxing a filter threshold can materially change how easily a system is misused [S-0003]. A governance analysis of trusted execution environments (TEEs) lists \"policy adherence\" as a property a verifier needs: a workload should show that it uses the approved model or enforces access controls and safeguards [S-0014]. The published designs follow one pattern: run the safeguard inside a TEE and have the hardware sign what ran [S-1500] [S-0012] [S-1503]. - Proof of guardrail. A wrapper program containing a public guardrail and its configuration mediates all of an agent's inputs and outputs inside the enclave [S-1500]. When a user asks for it, or the agent offers it, the enclave signs a document containing a hash of the wrapper and a hash of the input and response [S-1500] [S-1501]. A user compares the wrapper hash with the published open-source build and checks the signature offline, while the developer's agent stays private [S-1500]. - Property attestation. PAL*M runs inference in a confidential virtual machine with a GPU, and attests that a response came from a specific model and tokenizer on a given query or chat history [S-0012]. Its defined inputs are the query, model and tokenizer; it has no separate safeguard input [S-0012]. - Audit-to-deployment binding. Attestable Audits returns, with each response, an attestation that links the model, its earlier audit result, the prompt and the response [S-0009]. Binding weights to attestation is covered in Model identity attestation. - Scoped monitoring. The monitoring party and the monitored party co-sign a plan, which an open-source classifier runs inside an attested TEE [S-1503]. Queries outside the plan's scope are never evaluated [S-1503]. Two approaches do not use TEEs: - In a low-trust system design, the verifier re-executes sampled records and screens them itself for compliance, for example for blacklisted uses [S-0018]. See Sampled inference recomputation and Low-trust AI compute verification system overview. - Prompt Detective compares output distributions to judge whether a given system prompt was used [S-1502]. It was developed to detect reuse of proprietary prompts, not to audit declared safeguards [S-1502]. What it establishes It can establish: - that the declared guardrail code ran for an attested response, without revealing the developer's agent [S-1500]; - that tampering is visible: in the authors' tests, modified guardrail code, modified attestation bytes and modified responses were all detected [S-1500]; - in scoped monitoring, that the monitor evaluated only what the co-signed plan allows [S-1503]. It cannot establish: - that the safeguard works, since guardrails make errors and can be jailbroken [S-1500]; - that all traffic was covered, since attestations are per response and the prototype offers them for high-stakes questions [S-1500]; - anything the measurement leaves out, such as configuration, flags or later changes at runtime [S-0014]. Threat model - Proof of guardrail assumes a developer who might skip or modify the guardrail [S-1500]. It trusts the cloud provider's enclave hypervisor to measure code and protect keys, and it needs an open-source guardrail so that verifiers know the expected measurement [S-1500]. - PAL*M assumes an adversary who controls the host, hypervisor and disk [S-0012]. It trusts the Intel TDX module and the H100 GPU [S-0012]. It treats side-channel defences as orthogonal and places physical attacks, such as memory-bus interposition, out of scope [S-0012]. - A governance analysis contrasts the confidential-computing adversary, a dishonest operator, with a treaty adversary: a state with physical access to data centres and leverage over hardware vendors [S-0014]. Evidence - Proof of guardrail. The authors implemented it for OpenClaw agents on AWS Nitro Enclaves, with Llama Guard 3 for content safety and a fact-checking tool [S-1500]. They report 34% added latency on average compared with running outside the enclave [S-1500]. The code is public; its README calls it a proof of concept that is not production-ready [S-1501]. - Tinfoil safeguards. Tinfoil reports that it is rolling out safeguard models, gpt-oss-safeguard with Kimi-K3 reviewing flagged cases, inside its secure enclaves for Tinfoil Chat [S-3362]. It states that the whole pipeline is open source and attested, so that it can prove to an auditor, a regulator or anyone else that the safeguards are running and what the policy is [S-3362]. Only the fact of a violation, with the account and conversation ID, leaves the enclave [S-3362]. - PAL*M. It is implemented on Intel TDX with an NVIDIA H100 [S-0012]. The authors report 3.8–11.4% overhead for session inference over the same machine without PAL*M, and they modelled the protocol formally with the Tamarin prover [S-0012]. - Attestable Audits. A prototype ran a 4-bit 8-billion-parameter model in CPU-only AWS Nitro Enclaves at 1.84 tokens per second [S-0009]. - Scoped monitoring. A reference implementation runs in Tinfoil confidential virtual machines [S-1504]. Its authors state that user data and plan execution in the demo are not actually secure, and that the system has not been stress-tested by a counterparty [S-1504]. Limitations - Effectiveness gap. An attested guardrail can still be jailbroken [S-1500]. - Selective attestation. Provers might attest only favourable executions [S-0012] [S-1500]. - External APIs. The prototype called its guardrail model and backend model through external APIs [S-1500]. - Hardware attacks. With physical access to the memory bus, researchers forged Intel TDX attestations and, by pairing them with relayed H100 attestations, made a workload outside TEE protection pass as GPU confidential computing [S-1202]. A second team forged TDX attestation reports with an active interposer [S-3126]. Other attacks forged AMD SEV-SNP attestations, one through a DDR4 interposer and one from software alone on platforms without AMD's fix [S-1210] [S-1212]. - Scale. Frontier model inference typically needs several GPUs [S-0014]. CPU inference, which the enclave prototype had to use, cost 21.7 times as much per token as GPU inference and ran about 100 times slower [S-0009]. For confidential workflows that combine several parties' private inputs, see Confidential multi-party verification.", "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" }, { "id": "O-0141", "title": "Tinfoil", "url": "https://trustbutveri.fyi/organizations/tinfoil/" }, { "id": "O-0142", "title": "University of Cambridge", "url": "https://trustbutveri.fyi/organizations/university-of-cambridge/" } ] }, { "id": "M-0001", "slug": "sampled-inference-recomputation", "title": "Sampled inference recomputation", "aliases": [ "Inference recomputation", "Recomputation-based inference verification", "Partial recomputation" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "one_liner": "A verifier re-runs a random sample of an AI provider's logged queries on a trusted copy of the declared model and checks the outputs match.", "summary": "Sampled inference recomputation is a check that an AI provider's recorded outputs came from the model and settings it declared. The provider logs each query and response. A verifier's server re-runs a random sample on a trusted copy of the model. It flags outputs that differ by more than benign numerical noise allows. Checking costs less than generating because the verifier processes each sequence in one pass. Two schemes, TOPLOC and DiFR, have public code. DiFR has reproducible results on A100 and H200 GPUs with models of 8 to 30 billion parameters. TOPLOC's developer has used it to accept or reject work from untrusted computers. No independent red-team of the check has been published. For closed-weights models, the verifier's copy must sit in a trusted, confidential environment. Unrecorded work is never checked. The noise tolerance also leaves a covert channel. An adversary who controls the prompts can widen it.", "technical": "The reference protocol has five steps [[S-0015]]:\n\n1. The serving stack writes every request and response to an append-only log, with the model release, code version and sampling seed [[S-0015]].\n2. An isolated verification server draws a private random sample of logged pairs. Its isolation can range from access controls and containers to a full air gap [[S-0015]].\n3. The server runs one prefill pass over the prompt and claimed output with the trusted model and code. This gives the next-token distribution at every position [[S-0015]] [[S-0016]].\n4. The server scores how far the claimed output diverges from the reference. Token-DiFR uses the clipped Gumbel-Max logit margin under a shared seed. Activation-DiFR uses the distance between random projections of activations [[S-0016]]. TOPLOC counts exponent mismatches and measures mantissa differences in the top-128 last-layer activations [[S-1000]].\n5. Scores beyond a threshold calibrated on trusted hardware are escalated for review [[S-0015]] [[S-0016]].\n\nVerification is cheap because it needs only the prefill pass. Decoding typically runs at 3–5 times lower hardware utilization than prefill [[S-0015]]. Amodo's prototype measured a 2–8 times performance advantage for the verifier with models of 1.5 to 120 billion parameters on H100 and H200 GPUs [[S-1006]].\n\nThe number of samples sets the chance of detection. For a false-output rate p, the chance of catching at least one false output in k independent samples is 1 − (1 − p)^k [[S-0020]]. With 3,000 samples, a 0.1% rate is caught with 95.03% probability [[S-0018]].\n\nRecomputation can also match bit for bit, which makes the check pass/fail. This exact variant needs a known hardware model, deployed weights, parallelism topology, software versions and per-pass batch size, and no atomic operations [[S-0018]] [[S-0020]].", "category": "cryptographic-computational", "secondary_categories": [], "verifies": [ { "claim": "C-0005", "role": "primary", "note": "Checks that sampled recorded outputs are consistent with the declared model, precision and sampling settings." }, { "claim": "C-0009", "role": "supporting", "note": "Bounds how much information can be hidden steganographically in checked outputs. It is not a stand-alone defence against weight exfiltration (S-0015)." }, { "claim": "C-0004", "role": "supporting", "note": "Proposed as the correctness check in inference-only retrofit plans. Completeness needs other mechanisms (S-0067, S-0017)." } ], "threat_model": "adversarial", "adversarial_evaluation": "analysis", "hardware_requirement": "none", "prover_cooperation": "required", "confidentiality": "partial", "depends_on": [ { "target": "M-0013", "note": "Network taps or trusted logging supply the recorded inputs and outputs that are sampled." } ], "readiness": { "assessment": true, "level": "R3", "scope": "checking that recorded outputs came from the declared model and settings", "rubric_version": "1.1", "rationale": "[[I-0001|TOPLOC]] has been used in production by its developer, but no independent security audit or red-team of TOPLOC has been published.\n\n- **R1** met: the design, the claim it verifies and its trust assumptions are published. They include a formal security game for steganographic weight exfiltration [[S-0015]].\n- **R2** met through [[I-0002|DiFR]] and [[I-0001|TOPLOC]]. DiFR has MIT-licensed code with a vLLM integration [[S-1005]]. It has reproducible results on A100 and H200 GPUs with models of 8 to 30 billion parameters [[S-0016]]. TOPLOC is an installable package [[S-1001]] with peer-reviewed results [[S-1000]]. Rinberg et al. evaluate the method against a stated adversary who controls the inference server [[S-0015]].\n- **R3** met through TOPLOC: it is publicly available [[S-1001]], and Prime Intellect, its developer, reports using it in two 2025 runs to accept or reject work from untrusted nodes [[S-1003]] [[S-3000]]. No other party is documented relying on sampled recomputation for a verification decision.\n- **R4** not met: as of September 2026 no independent audit, red-team or peer-reviewed security analysis of the consistency check has been published. Amodo lists recomputation red-teaming as not started [[S-1008]]. Amodo's tests of Token-DiFR are independent of its authors but used two fine-tuned models, which by Amodo's account cover only a fraction of possible attacks [[S-3001]]. The one independent attack study targets the weight-exfiltration bound built on this mechanism [[S-1507]]. That bound is a supporting use, assessed under [[M-0024]].\n\nConfidence is low: R3 rests on the developer's own reports of TOPLOC use, the latest from July 2025 [[S-3000]].", "evidence": [ "S-0015", "S-0016", "S-1000", "S-1001", "S-1003", "S-1005", "S-1006", "S-1008", "S-1507", "S-3000", "S-3001" ], "next_level_gaps": [ "An independent audit, red-team or peer-reviewed security analysis of a recomputation consistency check against an adaptive adversary." ], "confidence": "low", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Tolerance for numerical noise leaves a covert channel", "kind": "demonstrated-attack", "severity": "significant", "status": "open", "description": "Schemes that accept approximate matches can put an upper bound on an adversary's covert bandwidth, but they cannot close the channel [[S-0020]]. The weight-exfiltration detector cut exfiltratable information to under 0.5%, not to zero, on a 30-billion-parameter mixture-of-experts model under benign prompt traffic [[S-0015]]. Its authors called the channel's size under adversarial prompts an open empirical question [[S-0015]]. An independent study showed that an adversary who controls the prompts roughly doubles the bits leaked per token. That cuts the slowdown from over 200 times to 60–118 times [[S-1507]]. The attack widens the exfiltration bound. It does not target the check that outputs match the declared model [[S-1507]].", "sources": [ "S-0020", "S-0015", "S-1507" ], "response": null }, { "assessment": true, "title": "Only recorded traffic is checked", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Recomputation checks that recorded, declared workloads are correct. It cannot show that the record is complete. The published schemes do not cover hidden workloads run on the same compute, or substituted work [[S-0017]]. Rinberg et al. say their exfiltration-detection scheme cannot stand alone [[S-0015]].", "sources": [ "S-0017", "S-0015" ], "response": null }, { "assessment": true, "title": "Some inference optimizations are not covered", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "TOPLOC's authors state that it cannot detect speculative decoding in which a cheaper model does the decoding. They did not test whether it distinguishes types of key-value (KV) cache compression [[S-1000]]. DiFR was evaluated only on sampling from a single model. Its authors sketch an extension to one speculative-decoding algorithm but do not test it [[S-0016]].", "sources": [ "S-1000", "S-0016" ], "response": null }, { "assessment": true, "title": "Mixed hardware widens the honest baseline", "kind": "open-question", "severity": "minor", "status": "open", "description": "When honest reference runs span different GPU types, the spread of benign scores grows. In DiFR's tests on Qwen3-30B-A3B, mixing A100 and H200 runs made the smallest deviation tested harder to separate from honest behaviour. That deviation was a temperature raised by 0.1 [[S-0016]].", "sources": [ "S-0016" ], "response": null } ], "blockers": [ { "text": "In tap-based retrofit designs, recording all inference traffic needs network taps and recomputation servers that can ingest it, in the worst case one recomputation-server network interface per inference front-end interface.", "theme": "coverage-hidden-compute", "blocked_by": "M-0013", "sources": [ "S-1008", "S-0067" ] }, { "text": "In retrofit designs, the recomputation server must sit inside the prover's data centre, possibly under the prover's physical control, and still be protected from a compromised provider, which Amodo rates 'not on track'.", "theme": "hardware-trust", "blocked_by": null, "sources": [ "S-1008", "S-0015" ] }, { "text": "No independent red-team of a recomputation consistency check has been published (the one independent attack study targets the weight-exfiltration bound), and Amodo rates recomputation red-teaming 'not started'.", "theme": "adversarial-validation", "blocked_by": null, "sources": [ "S-1008", "S-1507" ] }, { "text": "Tolerance-based checks need calibration on trusted hardware and exact knowledge of the provider's sampling procedure, and in one prototype a sampling-implementation mismatch produced large spurious differences.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-0016", "S-1006" ] }, { "text": "The verifier needs the model weights, so checking a closed-weights model requires a trusted, confidential recomputation environment, which the retrofit designs place inside the prover's facility.", "theme": "privacy-leakage", "blocked_by": null, "sources": [ "S-0016", "S-0017", "S-0018" ] } ], "challenge_themes": [ "coverage-hidden-compute", "hardware-trust", "adversarial-validation", "protocol-soundness", "performance-compatibility", "privacy-leakage" ], "organizations": [ "O-0100", "O-0101", "O-0201", "O-0202" ], "people": [], "sources": [ { "source": "S-0015", "supports": "reference architecture; trust assumptions, including external control of user queries; benign evaluation prompts; adversarial prompts an open question; prefill-only cost; exfiltration results; stand-alone limitation", "locator": "abstract; §4.2; §5; §5.3; §6.1; §8; Fig. 7" }, { "source": "S-0016", "supports": "Token-DiFR and Activation-DiFR; benign nondeterminism; detection results; weights and sampling requirements; limitations and speculative-decoding sketch", "locator": "abstract; §3.3; §5; §5.1; §7.2; §7.4; Appendix F" }, { "source": "S-0017", "supports": "TOPLOC and Token-DiFR as example schemes; recomputation server in prover's data centre; correctness vs completeness", "locator": "introduction; inference schemes" }, { "source": "S-0018", "supports": "bit-exact replay metadata; evaluation in auditing environment; sampling statistics; attribution problem", "locator": "§2b; §3.2.2; §5.2.2; Appendix A1" }, { "source": "S-0020", "supports": "statistical schemes bound but do not close covert bandwidth; detection probability; bit-exact pass/fail", "locator": "abstract; §1" }, { "source": "S-0067", "supports": "network taps feeding a recomputation server; partial recomputation of random samples", "locator": "Concrete inference-only retrofitting proposal" }, { "source": "S-1000", "supports": "TOPLOC mechanism, results and stated limitations, including untested KV-cache compression", "locator": "abstract; §4; §5; §6.1-6.4" }, { "source": "S-1001", "supports": "public TOPLOC implementation", "locator": "README; release v0.1.6" }, { "source": "S-1003", "supports": "developer use of TOPLOC to validate untrusted workers", "locator": "§2.3; §2.4.2" }, { "source": "S-1005", "supports": "public DiFR implementation with vLLM integration", "locator": "README" }, { "source": "S-1006", "supports": "prototype architecture, scale and verifier advantage", "locator": "whole note; Fig. 3" }, { "source": "S-1007", "supports": "prototype code and workflow; DiFR module is a modified copy of the difr library", "locator": "README" }, { "source": "S-3000", "supports": "provider-reported TOPLOC v2 use in SYNTHETIC-2; 1,253 GPUs; false-positive rate over 4 million samples", "locator": "verification section; GPU section" }, { "source": "S-3001", "supports": "Amodo's first prototype reuses the DiFR library; tests limited to a fraction of possible attacks", "locator": "setup; limitations" }, { "source": "S-1008", "supports": "status of recomputation algorithms, capture, server security and red-teaming", "locator": "status items" }, { "source": "S-1507", "supports": "independent prompt-control attack on the weight-exfiltration detector; scope limited to the exfiltration bound", "locator": "abstract; results" } ], "concepts": [ "K-0001", "K-0002", "K-0008", "K-0009", "K-0014", "K-0018", "K-0020", "K-0022" ], "complements": [ "M-0002", "M-0003", "M-0012" ], "alternatives": [ "M-0004" ], "type": "mechanism", "implementations": [ { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "I-0008", "title": "SASH confidential network logger", "url": "https://trustbutveri.fyi/implementations/sash-confidential-network-logger/" }, { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" } ], "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/", "source_file": "content/mechanisms/sampled-inference-recomputation.md", "flags_all": [ "provider-reported" ], "body_markdown": "## How it works\n\nThe provider records its inference traffic, and a verifier re-runs a random sample of it. The provider's serving system logs each request and response with the exact model release, code version and random seed used [[S-0015]]. Proposals for verifying international agreements use passive [[M-0013|network taps]] on the front-end network to copy all input and output traffic to a recomputation server [[S-0067]]. A verification server then re-runs a random subset of the logged inferences with a trusted copy of the model and code [[S-0015]].\n\nChecking is cheaper than generating [[S-0015]]. The verifier feeds the prompt and the claimed output through the model as one sequence. This gives the model's predictions for every position at once [[S-0016]]. Generation must produce tokens one at a time, which typically uses hardware 3–5 times less efficiently than the single pass [[S-0015]].\n\nThe comparison must tolerate numerical noise. Re-running the same inference often gives slightly different results, because floating-point arithmetic is not associative [[S-0016]] [[S-0017]]. Published schemes handle this in three ways:\n\n- [[I-0001|TOPLOC]] compares activation fingerprints. The provider records a compact encoding of the largest last-layer activations. The verifier accepts if its recomputed values differ by less than preset thresholds [[S-1000]] [[S-0017]].\n- Token-DiFR, part of [[I-0002|DiFR]], checks tokens under a shared sampling seed. The verifier scores how far each claimed token departs from the token the reference model would have chosen [[S-0016]].\n- [[M-0002|Exact replay]] needs no tolerance. Recomputation can match bit for bit if the verifier knows the hardware model, deployed weights, parallelism layout, software versions and batch size [[S-0018]] [[S-0020]].\n\nA modest random sample gives statistical assurance about the whole recorded stream [[S-0018]] [[S-0067]]. For example, if 0.1% of outputs are false, checking 3,000 random samples finds at least one with 95.03% probability [[S-0018]].\n\n## What it establishes\n\nSampled recomputation can establish two things:\n\n- It can show that recorded outputs are consistent with the declared model and settings. In DiFR's experiments, recomputation detected 4-bit quantization, a quantized key-value cache, a wrong temperature, a wrong seed and simulated sampling bugs [[S-0016]]. TOPLOC detected changes to the model, the prompt and the numerical precision [[S-1000]].\n- It can limit how much information a compromised server can hide in its responses [[S-0015]].\n\nIt cannot establish three things:\n\n- It checks only recorded workloads, so it cannot show that no other workloads ran unrecorded [[S-0017]]. Rinberg et al. say their weight-exfiltration scheme cannot be a defence on its own [[S-0015]].\n- Schemes that tolerate small differences can bound an adversary's covert bandwidth but cannot close it [[S-0020]].\n- Checks based only on activations cannot verify that tokens were sampled correctly [[S-0016]].\n\n## Threat model\n\nThe adversary controls the inference server [[S-0015]]. Rinberg et al.'s exfiltration study makes these trust assumptions, among others [[S-0015]]:\n\n- There are no side channels.\n- The verification code is trusted.\n- The log is secure and append-only.\n- Input and output messages are paired.\n- The adversary can neither see nor manipulate the sample selection.\n\nThe threat model lets external adversaries fully control user queries, but the published bound was measured on benign prompts from a public chat dataset [[S-0015]]. An independent study found that an adversary who controls the prompts can widen it [[S-1507]]. The verification server's isolation can range up to a full air gap [[S-0015]].\n\nThe prover must cooperate. In Amodo's example schemes for AI agreements, it must let the verifier install a recomputation server in its data centre [[S-0017]]. The verifier must hold the model weights and know the provider's sampling procedure [[S-0016]]. Tolerance thresholds come from calibration runs on trusted hardware [[S-0016]]. One low-trust system design treats physical security of the installed devices as a prerequisite [[S-0018]].\n\n## Evidence\n\n- **DiFR** detected 4-bit quantization with an AUC above 0.999 within 300 output tokens in its authors' tests [[S-0016]]. They tested Llama 3.1 8B, Qwen3-8B and Qwen3-30B-A3B on A100 and H200 GPUs, with and without tensor parallelism [[S-0016]]. The code is public [[S-1005]].\n- **TOPLOC** had no false positives or negatives in the evaluations reported in its peer-reviewed paper. The hardware tests used A100 and RTX 4090 GPUs [[S-1000]]. Prime Intellect reports using TOPLOC to verify work from untrusted inference workers in a decentralized training run. It evicted nodes whose work failed the checks [[S-1003]]. In a later data-generation run on 1,253 GPUs, it reports a false-positive rate of 0.000925% (37 slashes) over 4 million samples [[S-3000]].\n- **Rinberg et al.** tested weight-exfiltration detection on open-weight models of 3 to 30 billion parameters [[S-0015]]. On a 30-billion-parameter mixture-of-experts model under benign prompt traffic, they report that their detector cut exfiltratable information to under 0.5% at a false-positive rate below 0.01%. That slows an adversary more than 200-fold [[S-0015]]. An independent study found that choosing the prompts roughly doubles the leakage per token and cuts the slowdown to 60–118 times [[S-1507]].\n- **Amodo** built prototypes around the DiFR library [[S-3001]] [[S-1007]]. The second version adds a traffic-capturing proxy and a ledger on Kubernetes [[S-1006]] [[S-1007]]. It tested models of 1.5 to 120 billion parameters on H100 and H200 GPUs and reported a 2–8 times performance advantage for the verifier [[S-1006]]. Amodo also reports that TOPLOC and DiFR are well described and are now being tested on relevant hardware and models [[S-1008]].\n\n## Limitations\n\n- Tolerance margins leave a small covert channel [[S-0020]] [[S-0015]]. An adversary who controls the prompts can widen it [[S-1507]].\n- Benign differences between A100 and H200 deployments made the smallest tested deviation harder to detect [[S-0016]].\n- DiFR's authors show that tuning the sampling temperature fools a naive cross-entropy check. Token-DiFR still detects the change [[S-0016]].\n- TOPLOC's authors state it cannot detect speculative decoding with a cheaper model [[S-1000]]. Amodo expects recomputation algorithms to need updating as model architectures and inference techniques change [[S-1008]].\n- TOPLOC's authors name spoofed last-layer activations and deliberately \"unstable\" prompts as possible attacks, which remain untested [[S-1000]].\n- A mismatch does not reveal whether evasion, a hardware fault or an evaluation error caused it [[S-0018]].\n- Amodo rates recomputation red-teaming as not started, and security of the recomputation server as not on track [[S-1008]].\n\n[[M-0004|Zero-knowledge proofs of inference]] are an alternative. They need deterministic computation as a precondition [[S-0020]].", "body_text": "How it works The provider records its inference traffic, and a verifier re-runs a random sample of it. The provider's serving system logs each request and response with the exact model release, code version and random seed used [S-0015]. Proposals for verifying international agreements use passive network taps on the front-end network to copy all input and output traffic to a recomputation server [S-0067]. A verification server then re-runs a random subset of the logged inferences with a trusted copy of the model and code [S-0015]. Checking is cheaper than generating [S-0015]. The verifier feeds the prompt and the claimed output through the model as one sequence. This gives the model's predictions for every position at once [S-0016]. Generation must produce tokens one at a time, which typically uses hardware 3–5 times less efficiently than the single pass [S-0015]. The comparison must tolerate numerical noise. Re-running the same inference often gives slightly different results, because floating-point arithmetic is not associative [S-0016] [S-0017]. Published schemes handle this in three ways: - TOPLOC compares activation fingerprints. The provider records a compact encoding of the largest last-layer activations. The verifier accepts if its recomputed values differ by less than preset thresholds [S-1000] [S-0017]. - Token-DiFR, part of DiFR, checks tokens under a shared sampling seed. The verifier scores how far each claimed token departs from the token the reference model would have chosen [S-0016]. - Exact replay needs no tolerance. Recomputation can match bit for bit if the verifier knows the hardware model, deployed weights, parallelism layout, software versions and batch size [S-0018] [S-0020]. A modest random sample gives statistical assurance about the whole recorded stream [S-0018] [S-0067]. For example, if 0.1% of outputs are false, checking 3,000 random samples finds at least one with 95.03% probability [S-0018]. What it establishes Sampled recomputation can establish two things: - It can show that recorded outputs are consistent with the declared model and settings. In DiFR's experiments, recomputation detected 4-bit quantization, a quantized key-value cache, a wrong temperature, a wrong seed and simulated sampling bugs [S-0016]. TOPLOC detected changes to the model, the prompt and the numerical precision [S-1000]. - It can limit how much information a compromised server can hide in its responses [S-0015]. It cannot establish three things: - It checks only recorded workloads, so it cannot show that no other workloads ran unrecorded [S-0017]. Rinberg et al. say their weight-exfiltration scheme cannot be a defence on its own [S-0015]. - Schemes that tolerate small differences can bound an adversary's covert bandwidth but cannot close it [S-0020]. - Checks based only on activations cannot verify that tokens were sampled correctly [S-0016]. Threat model The adversary controls the inference server [S-0015]. Rinberg et al.'s exfiltration study makes these trust assumptions, among others [S-0015]: - There are no side channels. - The verification code is trusted. - The log is secure and append-only. - Input and output messages are paired. - The adversary can neither see nor manipulate the sample selection. The threat model lets external adversaries fully control user queries, but the published bound was measured on benign prompts from a public chat dataset [S-0015]. An independent study found that an adversary who controls the prompts can widen it [S-1507]. The verification server's isolation can range up to a full air gap [S-0015]. The prover must cooperate. In Amodo's example schemes for AI agreements, it must let the verifier install a recomputation server in its data centre [S-0017]. The verifier must hold the model weights and know the provider's sampling procedure [S-0016]. Tolerance thresholds come from calibration runs on trusted hardware [S-0016]. One low-trust system design treats physical security of the installed devices as a prerequisite [S-0018]. Evidence - DiFR detected 4-bit quantization with an AUC above 0.999 within 300 output tokens in its authors' tests [S-0016]. They tested Llama 3.1 8B, Qwen3-8B and Qwen3-30B-A3B on A100 and H200 GPUs, with and without tensor parallelism [S-0016]. The code is public [S-1005]. - TOPLOC had no false positives or negatives in the evaluations reported in its peer-reviewed paper. The hardware tests used A100 and RTX 4090 GPUs [S-1000]. Prime Intellect reports using TOPLOC to verify work from untrusted inference workers in a decentralized training run. It evicted nodes whose work failed the checks [S-1003]. In a later data-generation run on 1,253 GPUs, it reports a false-positive rate of 0.000925% (37 slashes) over 4 million samples [S-3000]. - Rinberg et al. tested weight-exfiltration detection on open-weight models of 3 to 30 billion parameters [S-0015]. On a 30-billion-parameter mixture-of-experts model under benign prompt traffic, they report that their detector cut exfiltratable information to under 0.5% at a false-positive rate below 0.01%. That slows an adversary more than 200-fold [S-0015]. An independent study found that choosing the prompts roughly doubles the leakage per token and cuts the slowdown to 60–118 times [S-1507]. - Amodo built prototypes around the DiFR library [S-3001] [S-1007]. The second version adds a traffic-capturing proxy and a ledger on Kubernetes [S-1006] [S-1007]. It tested models of 1.5 to 120 billion parameters on H100 and H200 GPUs and reported a 2–8 times performance advantage for the verifier [S-1006]. Amodo also reports that TOPLOC and DiFR are well described and are now being tested on relevant hardware and models [S-1008]. Limitations - Tolerance margins leave a small covert channel [S-0020] [S-0015]. An adversary who controls the prompts can widen it [S-1507]. - Benign differences between A100 and H200 deployments made the smallest tested deviation harder to detect [S-0016]. - DiFR's authors show that tuning the sampling temperature fools a naive cross-entropy check. Token-DiFR still detects the change [S-0016]. - TOPLOC's authors state it cannot detect speculative decoding with a cheaper model [S-1000]. Amodo expects recomputation algorithms to need updating as model architectures and inference techniques change [S-1008]. - TOPLOC's authors name spoofed last-layer activations and deliberately \"unstable\" prompts as possible attacks, which remain untested [S-1000]. - A mismatch does not reveal whether evasion, a hardware fault or an evaluation error caused it [S-0018]. - Amodo rates recomputation red-teaming as not started, and security of the recomputation server as not on track [S-1008]. Zero-knowledge proofs of inference are an alternative. They need deterministic computation as a precondition [S-0020].", "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "I-0016", "title": "Batch-invariant inference kernels (Thinking Machines)", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "I-0008", "title": "SASH confidential network logger", "url": "https://trustbutveri.fyi/implementations/sash-confidential-network-logger/" }, { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0008", "title": "Numerical nondeterminism", "url": "https://trustbutveri.fyi/concepts/numerical-nondeterminism/" }, { "id": "K-0009", "title": "Recomputation", "url": "https://trustbutveri.fyi/concepts/recomputation/" }, { "id": "O-0201", "title": "AI Futures Project", "url": "https://trustbutveri.fyi/organizations/ai-futures-project/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" }, { "id": "O-0100", "title": "Prime Intellect", "url": "https://trustbutveri.fyi/organizations/prime-intellect/" }, { "id": "O-0160", "title": "Singapore AI Safety Hub (SASH)", "url": "https://trustbutveri.fyi/organizations/singapore-ai-safety-hub/" } ] }, { "id": "M-0022", "slug": "side-channel-suppression", "title": "Side-channel suppression for isolated facilities", "aliases": [ "Covert-channel suppression", "TEMPEST-style shielding for verification", "Retrofitted side-channel defences" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Shielding, filtering, jamming and inspecting an AI facility to limit hidden physical communication around monitored network links.", "summary": "Side-channel suppression would retrofit an AI facility with metal enclosures, filters, jamming, vibration isolation and inspections to limit communication outside monitored network links. Verification designs that watch every official link out of an AI cluster depend on it, because data could otherwise travel by radio emissions, sound, vibration or signals on power lines. One design aims to push hidden-channel capacity down to a few kilobits per second; whether that rate is tolerable depends on the prohibited workload and the other controls. The design estimates the cost at about 0.1–0.5% of hardware cost. Its author calls it a first-pass study. As of September 2026 no build has been prototyped, measured or red-teamed; that validation gap is the main obstacle. The biggest known weaknesses are hardware implanted in the supply chain, which inspection may miss, and the difficulty of keeping shielding intact around airflow, cabling and optical links.", "technical": "- **Capacity target.** Cankaya frames suppression with the Shannon-Hartley limit C = B log2(1 + SNR). Defences either lower the signal power or raise the noise floor. Holding capacity to 1 kbit/s over 1 GHz of bandwidth needs an SNR of about −62 dB or less [[S-0038]].\n- **Shielding.** Enclosures built to NSA specification 94-106 are cited at −100 dB of attenuation from 100 MHz to 10 GHz [[S-0038]]. Cable conduits should have at least two right-angle turns, and each turn gives an estimated 15–25 dB of diffraction loss at ultrasonic wavelengths. Air vents should not face each other [[S-0038]].\n- **Jamming and monitoring.** Radio monitoring can run behind jamming using continuous or interrupted look-through, and a commodity-radio demonstration of continuous look-through is cited at 39 dB of suppression [[S-0038]].\n- **Cost.** Each 8-rack scalable unit has 64 power cables (\"whips\") to filter. Cankaya's bill of materials for one scalable unit totals $35,000–$150,000, about 0.1–0.5% of the hardware cost at an assumed $4 million per rack [[S-0038]].", "category": "off-chip-devices", "secondary_categories": [ "isolation-architecture" ], "verifies": [ { "claim": "C-0008", "role": "primary", "note": "Bounds the capacity of physical covert channels out of an enclosure, so that monitored links carry all significant traffic." }, { "claim": "C-0009", "role": "supporting", "note": "Supports arguments that weights cannot leave by unmonitored physical routes." }, { "claim": "C-0004", "role": "supporting", "note": "Inference-only designs count on suppressing unmonitored physical channels so that all significant traffic passes the taps (S-0018)." } ], "threat_model": "adversarial", "adversarial_evaluation": "analysis", "hardware_requirement": "retrofit-device", "prover_cooperation": "partial", "confidentiality": "preserving", "depends_on": [], "readiness": { "assessment": true, "level": "R1", "scope": "bounding physical covert channels out of a verified enclosure", "rubric_version": "1.1", "rationale": "One public design study gives costs and assumptions. Nothing has been built or measured.\n\n- **R1** met: Cankaya publicly describes a design with its goal (bounding covert capacity around a verified enclosure to a tolerable rate), the channel classes it addresses, defences, cost estimates and assumptions [[S-0038]].\n- **R2** not met: Cankaya describes the work as a two-week research sprint that is far from conclusive, and calls for prototyping and red-teaming [[S-0038]]. Components such as shielded enclosures and commercial power-line filters exist as products [[S-0038]], but as of September 2026 no integrated build for an AI facility, or measurement of one, has been published. The mechanism's implementations, [[I-0010]], [[I-0011]] and [[I-0012]], are proposed architectures at R1.\n\nConfidence is medium: a single tier C source carries the design, but its author states plainly that nothing has been prototyped.", "evidence": [ "S-0038" ], "next_level_gaps": [ "A prototype enclosure for at least one AI rack or scalable unit, with measured attenuation for each channel class.", "A red-team exercise against the prototype by a stated adversary.", "Validated costs for filters, jamming and optical conversion at production scale." ], "confidence": "medium", "assessed_by": [ "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Supply-chain implants may evade inspection", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Cankaya identifies malicious hardware embedded deep in purchased components as a residual risk that visual inspection and disassembly may not catch. He notes that radiographic examination under high-security standards could mitigate it [[S-0038]].", "sources": [ "S-0038" ], "response": null }, { "assessment": true, "title": "Openings for airflow, power and optics weaken shielding", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Cankaya notes that keeping attenuation high while passing high-power airflow, cabling and optical links adds complexity beyond existing shielded-enclosure specifications [[S-0038]].", "sources": [ "S-0038" ], "response": null }, { "assessment": true, "title": "Inspection assumptions may not hold", "kind": "open-question", "severity": "significant", "status": "open", "description": "The design's statistical argument assumes that visual or disassembly inspection catches every flaw that is present in a sampled unit. Cankaya is unsure whether destructive teardowns are defence-dominant or offence-dominant [[S-0038]].", "sources": [ "S-0038" ], "response": null } ], "blockers": [ { "text": "No prototype or red-team exists; the design is a first-pass viability study.", "theme": "adversarial-validation", "blocked_by": null, "sources": [ "S-0038" ] }, { "text": "Volume costs of TEMPEST-grade power-line filters are uncertain, because existing products are mostly made to order.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-0038" ] } ], "challenge_themes": [ "adversarial-validation", "capacity-bounds", "performance-compatibility", "hardware-trust" ], "organizations": [ "O-0202" ], "people": [], "sources": [ { "source": "S-0038", "supports": "setting, threat framing, channel classes, Shannon-Hartley framing, defences, attenuation figures, costs, assumptions, residual risks", "locator": "whole post; bill-of-materials table; residual-risk discussion" }, { "source": "S-0018", "supports": "side-channel suppression as part of a low-trust inference verification design; covert side-channel bandwidth target", "locator": "§5.3.1" }, { "source": "S-0043", "supports": "EM side channel leaks DNN parameters on edge GPUs", "locator": "Abstract" }, { "source": "S-0044", "supports": "EM leakage from GPU Tensor Cores, including at 100 cm through glass", "locator": "Abstract" }, { "source": "S-0046", "supports": "RAPL power side channel recovers DNN architectures; 99.75% Levenshtein-distance accuracy", "locator": "Abstract" }, { "source": "S-0007", "supports": "side-channel attacks on on-chip security implementations within reach of commercial tooling", "locator": "§4.3" }, { "source": "S-3566", "supports": "independent public analysis of residual low-bandwidth paths and covert RL-training strategies", "locator": "§2.3; §3.4; §4" } ], "concepts": [ "K-0013", "K-0014", "K-0017", "K-0021", "K-0022" ], "complements": [ "M-0013", "M-0014", "M-0017", "M-0024" ], "alternatives": [], "type": "mechanism", "implementations": [ { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "I-0010", "title": "RAND secure inference data center (SIDC) design", "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/" } ], "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/", "source_file": "content/mechanisms/side-channel-suppression.md", "flags_all": [], "body_markdown": "## How it works\n\nSeveral verification designs place checks on the official links of an AI cluster, for example network taps between accelerators ([[M-0013]]) [[S-0038]]. Cankaya points out that these checks only hold if the facility cannot move significant data by some other physical route [[S-0038]]. Side-channel suppression retrofits an untrusted facility so that any such route has negligible capacity [[S-0038]].\n\nCankaya surveys these classes of physical channel [[S-0038]]:\n\n- electromagnetic radiation, through the air and through optical fibre;\n- airborne acoustics;\n- acoustic signals carried through solids and through coolant water;\n- signals conducted along power lines and water columns;\n- physical access, such as covert devices.\n\nInstead of eliminating every channel, the design accepts a small residual rate, in the single-digit kilobits per second, to keep defence costs manageable [[S-0038]]. It reasons from information theory: a channel's capacity depends on its bandwidth and its signal-to-noise ratio, so defences either weaken the signal or raise the noise [[S-0038]].\nThis is a proposed target, not a measured ceiling. JoshC argues that residual output bandwidth could carry selected reinforcement-learning rollouts for covert training elsewhere [[S-3566]]. That is a different channel from the physical routes Cankaya studies; neither analysis tests whether the proposed physical-channel target is sufficient for a complete verification system [[S-0038]] [[S-3566]].\n\nThe proposed defences, grouped by channel, are [[S-0038]]:\n\n- **Electromagnetic.** Sheet-metal enclosures with bent cable conduits, metal barriers between racks, directional radio jamming, and radio sensors that monitor during or between jamming.\n- **Power lines.** Commercial TEMPEST-grade filters.\n- **Optical fibre.** Optical-electrical-optical conversion at the enclosure boundary, and inspection for undeclared optical components.\n- **Acoustic.** Enclosures and air vents that do not face each other, plus air gaps and vibration isolation between and under racks.\n- **Inspection.** Guards, unannounced inspections and destructive teardowns looking for hidden antennae.\n\n## What it establishes\nIf it works as designed, suppression bounds the capacity of unmonitored physical channels out of an enclosure [[S-0038]]. Other checks can then treat the monitored links as the only significant path for data ([[M-0014]], [[M-0024]]) [[S-0038]].\n\nSuppression does not show what the computation is, since it only makes the other verification layers harder to bypass [[S-0038]]. It also does not bring leakage to zero, because the design deliberately tolerates single-digit kilobits per second [[S-0038]].\n\nSide-channel attacks on verification hardware itself are a separate concern [[S-0007]]. Ansari notes that differential power analysis and electromagnetic attacks on compact cryptographic implementations are within reach of commercially available tooling [[S-0007]].\n\n## Threat model\n- **Adversary.** Cankaya assumes a well-resourced, nation-state adversary constrained only by physics [[S-0038]]. He argues that verification favours the defender because \"the attacker loses if caught even once\" [[S-0038]].\n- **Physical security.** The facility is guarded or monitored, with no unmonitored access and unannounced inspections, possibly by teams drawn from several organizations [[S-0038]].\n- **Inspection.** Inspections of sampled units catch any flaw present, which allows statistical bounds from sampling [[S-0038]].\n\n## Evidence\nPeer-reviewed attacks show that ML hardware leaks information through physical side channels:\n\n- **BarraCUDA** used correlation electromagnetic analysis to recover parameters of convolutional networks running on NVIDIA Jetson devices [[S-0043]].\n- **Kraken** extracted parameters from GPU Tensor Core units, and showed that GPU electromagnetic radiation leaks even 100 cm away through a glass obstacle [[S-0044]].\n- **DeepTheft** recovered the structure of DNN models on general-purpose processors through the RAPL power interface, reporting 99.75% Levenshtein-distance accuracy [[S-0046]].\n\nThese attacks show leakage, not deliberate covert signalling [[S-0043]] [[S-0044]] [[S-0046]]. Cankaya also surveys published covert-channel demonstrations across the channel classes above [[S-0038]].\n\nFor suppression itself, the evidence is one paper design [[S-0038]]. It estimates $35,000–$150,000 per 8-rack scalable unit, about 0.1–0.5% of hardware cost at an assumed $4 million per rack [[S-0038]]. Cankaya describes it as a first-pass viability study and asks for prototypes and adversarial feedback [[S-0038]].\n\n## Limitations\n- **Untested design.** It has not been prototyped or red-teamed [[S-0038]].\n- **Supply-chain implants.** Hardware implanted in purchased components may evade inspection [[S-0038]].\n- **Openings.** Airflow, power cabling and optical links complicate shielding [[S-0038]].\n- **Inspection limits.** It is unclear whether destructive teardown favours defender or attacker [[S-0038]].\n- **Cost uncertainty.** Volume costs of power-line filters are hard to estimate, because existing products are mostly made to order [[S-0038]]. For pre-training, optical conversion would need hundreds of high-performance transceivers per pod, which Cankaya puts within single-digit percent of the cost of the monitored servers [[S-0038]].\n- **Deterrence.** Part of the defence rests on psychological deterrence, created by keeping the sensors' exact capabilities unknown to the prover [[S-0038]].", "body_text": "How it works Several verification designs place checks on the official links of an AI cluster, for example network taps between accelerators (Network taps and certifiers) [S-0038]. Cankaya points out that these checks only hold if the facility cannot move significant data by some other physical route [S-0038]. Side-channel suppression retrofits an untrusted facility so that any such route has negligible capacity [S-0038]. Cankaya surveys these classes of physical channel [S-0038]: - electromagnetic radiation, through the air and through optical fibre; - airborne acoustics; - acoustic signals carried through solids and through coolant water; - signals conducted along power lines and water columns; - physical access, such as covert devices. Instead of eliminating every channel, the design accepts a small residual rate, in the single-digit kilobits per second, to keep defence costs manageable [S-0038]. It reasons from information theory: a channel's capacity depends on its bandwidth and its signal-to-noise ratio, so defences either weaken the signal or raise the noise [S-0038]. This is a proposed target, not a measured ceiling. JoshC argues that residual output bandwidth could carry selected reinforcement-learning rollouts for covert training elsewhere [S-3566]. That is a different channel from the physical routes Cankaya studies; neither analysis tests whether the proposed physical-channel target is sufficient for a complete verification system [S-0038] [S-3566]. The proposed defences, grouped by channel, are [S-0038]: - Electromagnetic. Sheet-metal enclosures with bent cable conduits, metal barriers between racks, directional radio jamming, and radio sensors that monitor during or between jamming. - Power lines. Commercial TEMPEST-grade filters. - Optical fibre. Optical-electrical-optical conversion at the enclosure boundary, and inspection for undeclared optical components. - Acoustic. Enclosures and air vents that do not face each other, plus air gaps and vibration isolation between and under racks. - Inspection. Guards, unannounced inspections and destructive teardowns looking for hidden antennae. What it establishes If it works as designed, suppression bounds the capacity of unmonitored physical channels out of an enclosure [S-0038]. Other checks can then treat the monitored links as the only significant path for data (Bandwidth limits and compartmentalization, Bounding unexplained information in outputs) [S-0038]. Suppression does not show what the computation is, since it only makes the other verification layers harder to bypass [S-0038]. It also does not bring leakage to zero, because the design deliberately tolerates single-digit kilobits per second [S-0038]. Side-channel attacks on verification hardware itself are a separate concern [S-0007]. Ansari notes that differential power analysis and electromagnetic attacks on compact cryptographic implementations are within reach of commercially available tooling [S-0007]. Threat model - Adversary. Cankaya assumes a well-resourced, nation-state adversary constrained only by physics [S-0038]. He argues that verification favours the defender because \"the attacker loses if caught even once\" [S-0038]. - Physical security. The facility is guarded or monitored, with no unmonitored access and unannounced inspections, possibly by teams drawn from several organizations [S-0038]. - Inspection. Inspections of sampled units catch any flaw present, which allows statistical bounds from sampling [S-0038]. Evidence Peer-reviewed attacks show that ML hardware leaks information through physical side channels: - BarraCUDA used correlation electromagnetic analysis to recover parameters of convolutional networks running on NVIDIA Jetson devices [S-0043]. - Kraken extracted parameters from GPU Tensor Core units, and showed that GPU electromagnetic radiation leaks even 100 cm away through a glass obstacle [S-0044]. - DeepTheft recovered the structure of DNN models on general-purpose processors through the RAPL power interface, reporting 99.75% Levenshtein-distance accuracy [S-0046]. These attacks show leakage, not deliberate covert signalling [S-0043] [S-0044] [S-0046]. Cankaya also surveys published covert-channel demonstrations across the channel classes above [S-0038]. For suppression itself, the evidence is one paper design [S-0038]. It estimates $35,000–$150,000 per 8-rack scalable unit, about 0.1–0.5% of hardware cost at an assumed $4 million per rack [S-0038]. Cankaya describes it as a first-pass viability study and asks for prototypes and adversarial feedback [S-0038]. Limitations - Untested design. It has not been prototyped or red-teamed [S-0038]. - Supply-chain implants. Hardware implanted in purchased components may evade inspection [S-0038]. - Openings. Airflow, power cabling and optical links complicate shielding [S-0038]. - Inspection limits. It is unclear whether destructive teardown favours defender or attacker [S-0038]. - Cost uncertainty. Volume costs of power-line filters are hard to estimate, because existing products are mostly made to order [S-0038]. For pre-training, optical conversion would need hundreds of high-performance transceivers per pod, which Cankaya puts within single-digit percent of the cost of the monitored servers [S-0038]. - Deterrence. Part of the defence rests on psychological deterrence, created by keeping the sensors' exact capabilities unknown to the prover [S-0038].", "referenced_by": [ { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "I-0010", "title": "RAND secure inference data center (SIDC) design", "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0017", "title": "Compartmentalization", "url": "https://trustbutveri.fyi/concepts/compartmentalization/" }, { "id": "K-0013", "title": "Side channel", "url": "https://trustbutveri.fyi/concepts/side-channel/" }, { "id": "K-0022", "title": "Weight exfiltration", "url": "https://trustbutveri.fyi/concepts/weight-exfiltration/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" } ] }, { "id": "M-0017", "slug": "tamper-evidence-for-verifier-devices", "title": "Tamper evidence for verifier devices", "aliases": [ "Tamper-evident enclosures", "Tamper-respondent enclosures", "Tamper-indicating enclosures", "Anti-tamper sensing" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Enclosures, seals and sensors that make physical interference with verification hardware either visible or self-defeating.", "summary": "Tamper evidence for verification hardware uses enclosures, seals and sensors to make physical interference visible, or to destroy the hardware's secrets when it happens. The devices, such as network taps, gateways and recomputation servers, would sit in facilities the checked party controls. The tamper-respondent IBM 4765 module was validated at FIPS 140-2 Level 4. Nuclear safeguards and arms control use tamper-indicating enclosures. Peer-reviewed prototypes detect probing through capacitive covers, radio waves inside server cases, or on-chip impedance sensing. As of September 2026 none has been built or evaluated for AI verifier hardware, and the MIRI overview lists retrofittable, mass-manufacturable enclosures for side-channel defence as an open problem. The main obstacles are scale, batteries and inspection burden. The largest known weakness is that seals are often defeated with simple methods: a 1996 Los Alamos study defeated all 94 seals it examined. Any enclosure is only as good as its inspection protocol.", "technical": "- **IBM 4765.** FIPS 140-2 Level 4 overall and for physical security, validated in 2011. NIST later moved the certificate to its historical list after a random-number-generator transition [[S-3260]]. A tamper-respondent matrix monitors for intrusion and adverse conditions, with moderate and severe out-of-range temperature and voltage sensing. A hard tamper triggers active wiping of secrets: the high-speed-erase battery-backed RAM holding core secrets is erased within microseconds, and the module is zeroized and left inoperable [[S-0050]].\n- **Batteryless PUF cover.** A 140 mm × 140 mm flexPCB cover with 16 × 16 electrodes (256 sensor nodes, 128 used differentially) and 100 µm line and space, checked by an STM32F303 microcontroller. After open- and short-circuit checks pass, a key is derived from the differential capacitances, about 5.2 bits of entropy per node with 10x oversampling (about 665 bits in theory). The attacker model assumes penetrations of at least 300 µm; single 0.3 mm holes made key reconstruction infeasible; a full-cover measurement with 10x oversampling takes 384 ms; power is about 0.6 W; testing spanned −20 °C to +60 °C [[S-1315]].\n- **Anti-Tamper Radio.** Two wideband antennas inside a metal case, measured with a vector network analyzer (2–9 GHz) or ultra-wideband transceivers (2.496–7.488 GHz). In a Dell PowerEdge 2850 over 10 days, 40 mm insertions of 1 mm needles were reliably detected; in an empty aluminium box, 16 mm insertions of needles as thin as 0.1 mm were detected. UWB transceivers cost less than US$5 [[S-0052]].\n- **ImpedanceVerif.** An embedded network analyzer on an FPGA measures the frequency response of the power distribution network, and Wasserstein distance is used as the detection statistic [[S-0051]].", "category": "off-chip-devices", "secondary_categories": [ "isolation-architecture" ], "verifies": [ { "claim": "C-0004", "role": "supporting", "note": "Protects the integrity of taps, gateways and recomputation hardware used for inference-only verification (S-0018, S-0067)." }, { "claim": "C-0008", "role": "supporting", "note": "Protects network devices that enforce or monitor bandwidth boundaries (S-0018)." } ], "threat_model": "adversarial", "adversarial_evaluation": "red-teamed", "hardware_requirement": "retrofit-device", "prover_cooperation": "partial", "confidentiality": "preserving", "depends_on": [], "readiness": { "assessment": true, "level": "R2", "scope": "detecting tampering with verifier devices inside facilities the checked party controls", "rubric_version": "1.1", "rationale": "Peer-reviewed tamper-detection results exist under stated adversaries, one in a running server, but no enclosure has been built or evaluated for AI verifier devices.\n\n- **R1** met: enclosure designs with stated attacker models are published [[S-1315]] [[S-0052]], and the MIRI overview describes their role in protecting verification hardware in a host-controlled facility [[S-0018]].\n- **R2** met: published end-to-end results exist under a stated adversary. Anti-Tamper Radio reliably detected needle insertions in a running 19-inch server over a 10-day experiment [[S-0052]]. Immler et al. report statistics over 115 batteryless covers, plus physical attacks against a stated 300 µm penetration model and environmental tests [[S-1315]]. On-chip impedance sensing detected board- and package-level tampering on commercial FPGA kits [[S-0051]]. All three are peer-reviewed; no public code or design files are cited for them.\n- **R3** not met for this use: production-grade tamper-respondent modules exist [[S-0050]], PHYSEC markets a radio-based tamper sensor for infrastructure enclosures [[S-3262]], and tamper-indicating enclosures are used in safeguards and arms control [[S-1316]], but none has been built for AI verifier devices such as optical taps, FPGA gateways or recomputation servers. The MIRI overview says it is less established what defences \"can be retrofitted at a massive scale to prevent bypassing of network taps\" [[S-0018]]. The mechanism's only implementation, [[I-0011]], is a proposed architecture at R1.\n- **R4** not met: the IBM 4765 was validated at FIPS 140-2 Level 4 [[S-0050]], but no enclosure has been independently evaluated on AI verifier devices.\n\nConfidence is medium, because how far the server-scale and HSM results transfer to AI verifier hardware is a judgment call.", "evidence": [ "S-0052", "S-1315", "S-0051", "S-0050", "S-3262", "S-1316", "S-0018" ], "next_level_gaps": [ "An enclosure or sensing design built for AI verifier devices (taps, gateways, recomputation servers) and deployable at data-centre scale.", "An independent public evaluation (red team or certification) of such an enclosure in the AI verification setting.", "Inspection protocols suited to host-controlled AI facilities." ], "confidence": "medium", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Seals are often defeated with simple methods", "kind": "demonstrated-attack", "severity": "significant", "status": "open", "description": "In 1996 a Los Alamos vulnerability assessment defeated all 94 security seals it examined, with 132 defeats in total, using rapid, inexpensive, low-tech methods. It found that seal cost did not predict security. In 2001 Johnston reported that high-tech seals are often easier to defeat than low-tech ones.", "sources": [ "S-1317", "S-1318" ], "response": null }, { "assessment": true, "title": "Security depends on inspection protocols", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Johnston argues that a seal is no better than the protocols for using it, and that inspectors are usually given little useful information on how to detect tampering. The Sandia survey notes that larger enclosures are hard to inspect fully and that sensor data must be authenticated.", "sources": [ "S-1318", "S-1316" ], "response": null }, { "assessment": true, "title": "Attack classes outside published models", "kind": "open-question", "severity": "significant", "status": "open", "description": "The authors of the batteryless cover say they cannot assess chemical-solvent attacks, which exceed their expertise, and deem cover removal impractical. Anti-Tamper Radio's reference can drift as the environment or measurement system ages; the authors suggest gradually renewing the reference. A 2025 follow-up by some of the same authors shows, by emulation on measured channel data, that an attacker who knows the reference channel and the needle's effect on it could inject a signal that cancels the change caused by a needle insertion. It proposes a reconfigurable intelligent surface that randomizes the channel as a countermeasure.", "sources": [ "S-1315", "S-0052", "S-3261" ], "response": null } ], "blockers": [ { "text": "No tamper-evident enclosure has been designed for AI verifier hardware at retrofit scale.", "theme": "hardware-trust", "blocked_by": null, "sources": [ "S-0018" ] }, { "text": "Battery-backed designs add bulk, limit operating temperature (+10 °C to +35 °C for the IBM 4765) and complicate transport.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-1315" ] }, { "text": "Active monitoring needs power, and visual inspection of large enclosures faces access limits.", "theme": "access-governance", "blocked_by": null, "sources": [ "S-1316" ] }, { "text": "No evaluation has been published in the AI verification setting.", "theme": "adversarial-validation", "blocked_by": null, "sources": [ "S-0018" ] } ], "challenge_themes": [ "hardware-trust", "adversarial-validation", "access-governance", "performance-compatibility" ], "organizations": [], "people": [], "sources": [ { "source": "S-0050", "supports": "FIPS 140-2 Level 4; tamper-respondent matrix; sensed conditions; zeroization", "locator": "Table 1; §2 physical security; §8.1 Table 9" }, { "source": "S-3260", "supports": "IBM 4765 validation date and historical certificate status", "locator": "certificate page" }, { "source": "S-0049", "supports": "existence of a review spanning battery-backed to PUF-based enclosures", "locator": "title and abstract (full text not read)" }, { "source": "S-1315", "supports": "batteryless PUF cover design, attacker model, results, stated limitations, drawbacks of battery-backed enclosures", "locator": "abstract; §2.1; §3.1; §8" }, { "source": "S-0052", "supports": "Anti-Tamper Radio concept, threat model, server experiment, costs, limitations", "locator": "abstract; §III–§VI" }, { "source": "S-3261", "supports": "signal-injection (compensation) attack on Anti-Tamper Radio under a known-reference model; RIS countermeasure", "locator": "abstract; §3.1; §4.2.1" }, { "source": "S-3262", "supports": "PHYSEC reports a commercial anti-tamper radio sensor for infrastructure enclosures", "locator": "product page" }, { "source": "S-0051", "supports": "on-chip impedance sensing for tamper detection", "locator": "abstract" }, { "source": "S-1316", "supports": "tamper-indicating enclosures in verification regimes; approaches; limitations", "locator": "abstract; survey sections" }, { "source": "S-1317", "supports": "seal vulnerability assessment results; mean defeat time for one practised person", "locator": "abstract; results; Table 2" }, { "source": "S-1318", "supports": "high-tech vs low-tech seals; role of protocols and inspector training", "locator": "main text" }, { "source": "S-0018", "supports": "enclosure needs in low-trust AI verification; open question on retrofittable enclosures for side-channel defence; inspections", "locator": "§2b; §4.2; §5.3.1" }, { "source": "S-0067", "supports": "tamper-evident enclosures among physical security measures", "locator": "physical security discussion" }, { "source": "S-0035", "supports": "flexHEG secure enclosure for physical tamper protection", "locator": "abstract" } ], "concepts": [ "K-0015", "K-0005", "K-0018", "K-0002" ], "complements": [ "M-0013", "M-0014", "M-0022", "M-0009" ], "alternatives": [], "type": "mechanism", "implementations": [ { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" } ], "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/", "source_file": "content/mechanisms/tamper-evidence-for-verifier-devices.md", "flags_all": [], "body_markdown": "## How it works\n\nVerification hardware placed in a facility run by the party being checked needs protection against physical interference [[S-0018]]. The MIRI system overview relies on monitoring of the facility and on occasional random inspections of analog components and anti-tamper seals [[S-0018]]. Obermaier and Immler review enclosures from battery-backed monitoring to PUF-based designs [[S-0049]]. Published approaches fall into four groups:\n\n- **Tamper-respondent modules.** The IBM 4765 coprocessor uses a \"protective, tamper-respondent matrix to monitor for intrusion and adverse physical conditions\" [[S-0050]]. On a hard tamper, it actively erases its core secrets within microseconds [[S-0050]].\n- **Batteryless tamper-resistant covers.** Immler et al. wrap the protected system in a flexible circuit-board cover with a fine electrode mesh [[S-1315]]. A key is derived from the mesh's capacitances, which act as a physical unclonable function (PUF), and decrypts the system's sensitive data only if the cover is intact [[S-1315]].\n- **System-level sensing.** Anti-Tamper Radio monitors how radio waves propagate inside a metal case, so inserted objects change the measured response [[S-0052]]. ImpedanceVerif uses on-chip network analyzers on FPGAs to detect changes in a board's power distribution network, \"without any modifications to the system\" [[S-0051]].\n- **Tamper-indicating enclosures.** In safeguards and arms control, these leave physical evidence of attack [[S-1316]]. They cover the case where an adversary bypasses a sealed opening, for example by drilling through a side [[S-1316]].\n\nIn AI proposals, flexHEG pairs a guarantee processor with \"a secure enclosure providing physical tamper protection\" [[S-0035]]; see [[M-0009]]. The AI 2040 plan lists tamper-evident enclosures among physical security measures [[S-0067]]. The MIRI overview notes that standards exist for secure facilities (SCIFs) [[S-0018]]. What is less established, it says, is \"what defenses can be retrofitted at a massive scale to prevent bypassing of network taps\" [[S-0018]].\n\n## What it establishes\nEnclosures can show that a device was physically accessed [[S-1316]], or can make a breach destroy the keys or data an attacker wants [[S-0050]] [[S-1315]]. They protect the integrity of other mechanisms, such as taps ([[M-0013]]) and bandwidth shapers ([[M-0014]]), rather than proving a claim on their own [[S-0018]]. Johnston argues that \"a seal is no better than the protocols for using it\" [[S-1318]].\n\n## Threat model\n- **Cover.** Immler et al. \"assume penetrations to be at least 300µm in diameter\" [[S-1315]]. They deem cover removal impractical and do not assess chemical solvents [[S-1315]].\n- **Radio sensing.** Anti-Tamper Radio defines success for the attacker as inserting a needle into a sensitive region without detection [[S-0052]].\n- **AI facilities.** The host controls the facility, so the MIRI overview combines enclosures with continuous monitoring and random inspection [[S-0018]].\n\n## Evidence\n- **IBM 4765.** Validated at FIPS 140-2 Level 4, overall and for physical security [[S-0050]]. NIST later moved the 2011 certificate to its historical list after a random-number-generator transition [[S-3260]].\n- **Batteryless covers.** Immler et al. report statistics over 115 flexPCB covers, physical attacks and environmental testing, aiming at a physical security level comparable to FIPS 140-2 Level 3 [[S-1315]].\n- **Anti-Tamper Radio.** In a running 19-inch server over 10 days, it reliably detected 40 mm insertions of 1 mm needles [[S-0052]]. PHYSEC markets a product, PHYSEC SEAL, and reports that its anti-tamper radio sensor detects drilling or housing deformation in enclosures such as control cabinets [[S-3262]].\n- **ImpedanceVerif.** It detected tampering on commercial FPGA development kits, including the proximity of contactless EM probes and a slightly polished chip package [[S-0051]].\n- **Tamper-indicating enclosures.** They support IAEA safeguards and arms-control chain-of-custody regimes [[S-1316]].\n\nAs of September 2026 no enclosure for AI verifier hardware has been evaluated in public. The MIRI overview lists \"tamper-evident, rapidly mass-manufacturable and retrofittable enclosures\" as an open question for side-channel defence [[S-0018]].\n\n## Limitations\n- **Seal defeats.** In 1996 a Los Alamos team defeated all 94 seals it examined, with 132 defeats using low-tech methods; one practised person needed 4.3 minutes on average [[S-1317]]. Johnston reports that \"high-tech seals are often easier to defeat than low-tech seals\" [[S-1318]].\n- **Batteries.** Battery-backed designs add bulk, limit the operating temperature range and fail when discharged [[S-1315]].\n- **Inspection and power.** Visual methods on large enclosures face access limits, active approaches need power, and sensor data must be authenticated [[S-1316]].\n- **Drift.** Anti-Tamper Radio's reference measurement can drift as the environment or the measurement system ages, which the authors suggest handling by gradually renewing the reference [[S-0052]].", "body_text": "How it works Verification hardware placed in a facility run by the party being checked needs protection against physical interference [S-0018]. The MIRI system overview relies on monitoring of the facility and on occasional random inspections of analog components and anti-tamper seals [S-0018]. Obermaier and Immler review enclosures from battery-backed monitoring to PUF-based designs [S-0049]. Published approaches fall into four groups: - Tamper-respondent modules. The IBM 4765 coprocessor uses a \"protective, tamper-respondent matrix to monitor for intrusion and adverse physical conditions\" [S-0050]. On a hard tamper, it actively erases its core secrets within microseconds [S-0050]. - Batteryless tamper-resistant covers. Immler et al. wrap the protected system in a flexible circuit-board cover with a fine electrode mesh [S-1315]. A key is derived from the mesh's capacitances, which act as a physical unclonable function (PUF), and decrypts the system's sensitive data only if the cover is intact [S-1315]. - System-level sensing. Anti-Tamper Radio monitors how radio waves propagate inside a metal case, so inserted objects change the measured response [S-0052]. ImpedanceVerif uses on-chip network analyzers on FPGAs to detect changes in a board's power distribution network, \"without any modifications to the system\" [S-0051]. - Tamper-indicating enclosures. In safeguards and arms control, these leave physical evidence of attack [S-1316]. They cover the case where an adversary bypasses a sealed opening, for example by drilling through a side [S-1316]. In AI proposals, flexHEG pairs a guarantee processor with \"a secure enclosure providing physical tamper protection\" [S-0035]; see Hardware-enabled guarantees (flexHEG) and guarantee processors. The AI 2040 plan lists tamper-evident enclosures among physical security measures [S-0067]. The MIRI overview notes that standards exist for secure facilities (SCIFs) [S-0018]. What is less established, it says, is \"what defenses can be retrofitted at a massive scale to prevent bypassing of network taps\" [S-0018]. What it establishes Enclosures can show that a device was physically accessed [S-1316], or can make a breach destroy the keys or data an attacker wants [S-0050] [S-1315]. They protect the integrity of other mechanisms, such as taps (Network taps and certifiers) and bandwidth shapers (Bandwidth limits and compartmentalization), rather than proving a claim on their own [S-0018]. Johnston argues that \"a seal is no better than the protocols for using it\" [S-1318]. Threat model - Cover. Immler et al. \"assume penetrations to be at least 300µm in diameter\" [S-1315]. They deem cover removal impractical and do not assess chemical solvents [S-1315]. - Radio sensing. Anti-Tamper Radio defines success for the attacker as inserting a needle into a sensitive region without detection [S-0052]. - AI facilities. The host controls the facility, so the MIRI overview combines enclosures with continuous monitoring and random inspection [S-0018]. Evidence - IBM 4765. Validated at FIPS 140-2 Level 4, overall and for physical security [S-0050]. NIST later moved the 2011 certificate to its historical list after a random-number-generator transition [S-3260]. - Batteryless covers. Immler et al. report statistics over 115 flexPCB covers, physical attacks and environmental testing, aiming at a physical security level comparable to FIPS 140-2 Level 3 [S-1315]. - Anti-Tamper Radio. In a running 19-inch server over 10 days, it reliably detected 40 mm insertions of 1 mm needles [S-0052]. PHYSEC markets a product, PHYSEC SEAL, and reports that its anti-tamper radio sensor detects drilling or housing deformation in enclosures such as control cabinets [S-3262]. - ImpedanceVerif. It detected tampering on commercial FPGA development kits, including the proximity of contactless EM probes and a slightly polished chip package [S-0051]. - Tamper-indicating enclosures. They support IAEA safeguards and arms-control chain-of-custody regimes [S-1316]. As of September 2026 no enclosure for AI verifier hardware has been evaluated in public. The MIRI overview lists \"tamper-evident, rapidly mass-manufacturable and retrofittable enclosures\" as an open question for side-channel defence [S-0018]. Limitations - Seal defeats. In 1996 a Los Alamos team defeated all 94 seals it examined, with 132 defeats using low-tech methods; one practised person needed 4.3 minutes on average [S-1317]. Johnston reports that \"high-tech seals are often easier to defeat than low-tech seals\" [S-1318]. - Batteries. Battery-backed designs add bulk, limit the operating temperature range and fail when discharged [S-1315]. - Inspection and power. Visual methods on large enclosures face access limits, active approaches need power, and sensor data must be authenticated [S-1316]. - Drift. Anti-Tamper Radio's reference measurement can drift as the environment or the measurement system ages, which the authors suggest handling by gradually renewing the reference [S-0052].", "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "I-0008", "title": "SASH confidential network logger", "url": "https://trustbutveri.fyi/implementations/sash-confidential-network-logger/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "K-0015", "title": "Tamper evidence and tamper resistance", "url": "https://trustbutveri.fyi/concepts/tamper-evidence/" } ] }, { "id": "M-0008", "slug": "tee-remote-attestation", "title": "TEE remote attestation for AI workloads", "aliases": [ "Confidential computing", "GPU confidential computing", "Remote attestation", "Confidential VM attestation" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "one_liner": "Trusted execution environments (TEEs) in CPUs and GPUs sign reports of loaded software, so a remote party can check which code ran an AI workload.", "summary": "Trusted execution environment (TEE) remote attestation lets a remote party check which software processed an AI workload. A TEE isolates a workload from the machine's operator and signs a report of the code and configuration it loaded. NVIDIA's Hopper and Blackwell GPUs support this paired with Intel TDX or AMD SEV-SNP confidential virtual machines. Research prototypes use it to show which model produced a result without revealing weights. Tinfoil, Apple and Meta report using it in production AI services. It is not ready for verification between rival states, where the party being checked holds the hardware. Chip vendors treat sophisticated physical attacks as out of scope. Independent researchers with memory-bus devices costing under $50 to under $1000 forged Intel and AMD attestations, and software-only attacks forged AMD attestations on platforms without AMD's firmware fixes. An attestation covers one deployment at launch, and trust rests on a few hardware vendors.", "technical": "NVIDIA describes the GPU attestation chain as follows. An on-die root of trust verifies the identity key fused into the GPU and permits only NVIDIA-signed firmware at boot. The driver then opens an SPDM session with the GPU, using a Diffie-Hellman exchange to set up a session key [[S-1200]]. The GPU presents a device identity certificate signed with a device-unique ECC-384 key, which chains to the NVIDIA certificate authority. It then returns a signed set of measurements [[S-1201]].\n\nOther details of NVIDIA's design:\n\n- Traffic between the confidential VM and the GPU is protected with AES-GCM through encrypted bounce buffers. NVIDIA states that in future, hosts with TDISP/IDE-compatible CPUs and Blackwell B100/B200 GPUs can use inline encryption instead [[S-1200]].\n- Hopper's protected-PCIe mode passes all eight GPUs of an HGX node to one confidential VM, but NVLink traffic between them stays unencrypted. Blackwell also encrypts NVLink, for up to eight GPUs per confidential VM [[S-1200]].\n- Performance counters are disabled in full CC-On mode. They are available only in a CC-DevTools development mode [[S-1200]].\n- At launch, NVIDIA reported that H100 compute and HBM bandwidth were at par with non-confidential mode. CPU–GPU transfers were limited to roughly 4 GB/s by CPU encryption [[S-1201]].\n\nThree systems bind application data to a report:\n\n- **PAL\\*M** sets the Intel TDX REPORTDATA field to the concatenation of the operation, a verifier challenge and hashes of the inputs and outputs. It models the protocol in the Tamarin prover [[S-0012]].\n- **Attestable Audits** publishes attestations that bind the model hash, the audit code and data, and the result to a transparency log [[S-0009]].\n- **Tinfoil** puts a dm-verity root hash of the weights on the measured kernel command line [[S-0013]].", "category": "on-chip", "secondary_categories": [ "cryptographic-computational" ], "verifies": [ { "claim": "C-0005", "role": "primary", "note": "Attests the software stack that produced responses, and the model too when paired with a weight commitment (see M-0012)." }, { "claim": "C-0006", "role": "supporting", "note": "Can attest that measured policy software, such as filters and logging, wrapped the model (S-0014). The evaluation variant is I-0007." }, { "claim": "C-0004", "role": "supporting", "note": "Attests a declared inference deployment, but not that the same chips ran no other workloads (S-0014)." }, { "claim": "C-0007", "role": "supporting", "note": "PAL*M attests single-node training and fine-tuning operations (S-0012). Distributed training is left open." } ], "threat_model": "semi-trusted", "adversarial_evaluation": "independent-red-team", "hardware_requirement": "existing-features", "prover_cooperation": "required", "confidentiality": "preserving", "depends_on": [], "readiness": { "assessment": true, "level": "R3", "scope": "showing which software ran to a party that distrusts the operator holding the hardware", "rubric_version": "1.1", "rationale": "Tinfoil relies on the attestation of commercial GPU confidential computing in production, but the independent evaluations left critical flaws open.\n\n- **R1** met: designs with stated claims and assumptions are published for audits, property attestation and policy enforcement [[S-0009]] [[S-0012]] [[S-0014]].\n- **R2** met through [[I-0006|Tinfoil's model-identity chain]]. It is an open-source production deployment on NVIDIA H100, H200 or B200 GPUs with AMD SEV-SNP or Intel TDX (provider-reported) [[S-1206]] [[S-1208]] [[S-1209]]. [[I-0007|Attestable Audits]] and PAL\\*M add end-to-end results against stated adversaries [[S-0009]] [[S-0012]].\n- **R3** met through reliance by another party. Tinfoil, which did not build the TEEs, relies on the attestation in production: its service checks each GPU's attestation at boot and does not start if the check fails [[S-1207]] [[S-1208]]. [[I-0013|Apple Private Cloud Compute]] is another production deployment (provider-reported) [[S-1800]]. As context, GPU confidential computing is a documented product feature [[S-1200]], and NVIDIA reports that Azure's confidential H100 virtual machines became generally available in 2024 [[S-1815]].\n- **R4** not met. The independent public evaluations of the underlying TEEs (TEE.fail, DDRop, Battering RAM, WireTap, RMPocalypse and Fabricked) all found critical flaws, and the four that need physical access remain open. TEE.fail used physical access, root privileges and under $1000 of equipment to extract a CPU's Intel provisioning certification key, which anchors SGX and TDX attestation, and forge TDX attestations. Paired with relayed H100 attestations, the forgeries let a workload outside TEE protection pass both checks [[S-1202]]. DDRop forged TDX attestation reports on an up-to-date platform with a DDR5 interposer costing under $200 [[S-3126]]. On DDR4 servers, Battering RAM and WireTap forged SGX attestations with interposers costing under $50 and under $1000, and Battering RAM also broke SEV-SNP attestation [[S-1210]] [[S-1211]]. These physical attacks defeat the main claim when the prover controls the hardware. NVIDIA lists sophisticated physical attacks as out of scope [[S-1200]]. Intel and AMD treat interposer and other physical attacks on memory as out of scope, according to the researchers [[S-1202]] [[S-3126]] [[S-1210]] [[S-1211]]. RMPocalypse and Fabricked forged SEV-SNP attestations from malicious host software, with no physical access [[S-1212]] [[S-3127]]. AMD reports firmware fixes for both [[S-1213]] [[S-3128]]. An independent analysis of NVIDIA's GPU confidential computing found residual metadata and timing leaks but reported no attestation break [[S-3129]]. Trail of Bits' pre-launch audit of WhatsApp's deployment found high-severity implementation flaws that Meta fixed, and notes that SEV-SNP does not fully protect against advanced physical attacks [[S-3121]] [[S-3124]]. The breaks do not invalidate the R2 and R3 evidence, which concerns working implementations and production use.", "evidence": [ "S-0009", "S-0012", "S-0014", "S-1200", "S-1815", "S-1202", "S-1206", "S-1207", "S-1208", "S-1209", "S-1800", "S-1210", "S-1211", "S-1212", "S-1213", "S-3121", "S-3124", "S-3126", "S-3127", "S-3128", "S-3129" ], "next_level_gaps": [ "Attestation that survives an attacker who physically holds the hardware, for example through memory integrity and freshness protection or tamper-responsive enclosures, confirmed by independent red-teaming.", "GPU attestation cryptographically bound to the specific confidential VM it serves.", "Coverage of whole-chip and multi-node activity, beyond a single deployment.", "Roots of trust and key provenance that rival parties accept, beyond one vendor's certificate authority.", "Measurement of runtime configuration as well as launch state." ], "confidence": "medium", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "DDR5 memory-bus interposers forge Intel TDX attestations and break SEV-SNP protections (TEE.fail, DDRop)", "kind": "demonstrated-attack", "severity": "critical", "status": "open", "description": "Independent researchers placed an interposer, built for under $1000, on the DDR5 memory bus of servers running Intel TDX and AMD SEV-SNP. Server TEEs encrypt memory deterministically, without integrity or freshness protection, and the researchers exploited this to recover secrets. The attack needs physical access and root privileges [[S-1202]].\n\n- On Intel, they extracted the provisioning certification key from a machine that Intel's service rated fully up to date. This per-CPU key signs the keys used in SGX and TDX attestation. With it they forged SGX and TDX attestations [[S-1202]].\n- On AMD SEV-SNP with ciphertext hiding enabled, they recovered an ECDSA private key used by OpenSSL inside the virtual machine. It was not an AMD attestation key [[S-1202]]. Other independent attacks did break SEV-SNP attestation. Battering RAM did so with a DDR4 interposer, and RMPocalypse and Fabricked from malicious host software [[S-1210]] [[S-1212]] [[S-3127]].\n\nA second team, from KU Leuven, ETH Zurich, Durham University and Google, built DDRop, an active DDR5 interposer with a bill of materials of $159. It silently drops memory writes, which memory encryption without freshness protection cannot detect. With brief physical access and control of the host software and BIOS, the researchers forced trust domains into debug mode and forged attestation reports on an up-to-date Intel TDX platform. The same primitive breaks the integrity of Scalable SGX and SEV-SNP, though the authors report no SEV-SNP attestation forgery [[S-3126]].\n\nThe TEE.fail authors report that Intel and AMD consider interposer attacks out of scope, which leaves physical security as the only mitigation [[S-1202]]. The DDRop authors report the same position, and that both vendors issued security advisories on disclosure in September 2026 [[S-3126]]. PAL\\*M lists this attack class as out of its scope [[S-0012]], and Tinfoil's documentation acknowledges it [[S-1206]]. Gloria Z calls key extraction through bus interposition \"relatively low-hanging fruit\" in an international treaty scenario [[S-0014]].", "sources": [ "S-1202", "S-3126", "S-0012", "S-1206", "S-0014", "S-1210", "S-1212", "S-3127" ], "response": null }, { "assessment": true, "title": "DDR4 memory-bus interposers forge SGX and SEV-SNP attestation (Battering RAM, WireTap)", "kind": "demonstrated-attack", "severity": "critical", "status": "open", "description": "Two independent teams broke server TEE attestation on DDR4 memory with interposers they built themselves. Both attacks need physical access to install the device and root privileges on the host [[S-1210]] [[S-1211]].\n\n- Battering RAM, by researchers at KU Leuven and the Universities of Birmingham and Durham, uses an interposer with a bill of materials of $47.62. It creates memory aliases at runtime, which bypasses the boot-time alias checks that AMD and Intel introduced against static aliasing attacks such as BadRAM. On Intel Scalable SGX it gained arbitrary read and write access to enclave plaintext and extracted SGX's platform provisioning key, which lets an attacker forge attestation certificates for arbitrary quoting enclaves. On up-to-date AMD SEV-SNP servers it captured the launch digests of genuine VMs and replayed them into modified VMs, so that backdoored VMs pass attestation [[S-1210]].\n- WireTap, by researchers at Purdue University and Georgia Tech, uses an interposer built for under $1000 that records DDR4 bus traffic. On a Xeon Scalable server in fully trusted status it recovered the ECDSA attestation key of SGX's Quoting Enclave in 45 minutes and forged SGX quotes. The authors then showed end-to-end attacks on SGX-based blockchain deployments [[S-1211]].\n\nBoth attacks are limited to DDR4 systems. The Battering RAM authors state that all commercial TDX machines use DDR5, and the WireTap authors state that 4th and 5th generation Xeon Scalable processors need DDR5 and are not affected by their current work [[S-1210]] [[S-1211]]. According to the Battering RAM authors, Intel and AMD acknowledged the findings but consider physical attacks on DRAM out of scope for their current products [[S-1210]]. The WireTap authors report that Intel considers their attack outside the SGX threat model, and that there is no mitigation besides running servers in secure physical environments [[S-1211]].", "sources": [ "S-1210", "S-1211" ], "response": null }, { "assessment": true, "title": "Software-only forgery of SEV-SNP attestation (RMPocalypse, Fabricked)", "kind": "demonstrated-attack", "severity": "critical", "status": "mitigated", "description": "Researchers at ETH Zurich showed that a malicious hypervisor can corrupt the Reverse Map Table (RMP) while SEV-SNP initialises it. SEV-SNP uses the RMP to store security metadata for every DRAM page, and a single 8-byte overwrite leaves the whole table compromised. The attack needs no physical access. The authors confirmed it on Zen 3, Zen 4 and Zen 5 processors and state that it affects all AMD processors that support SEV-SNP. They demonstrated forged attestation values, debugging enabled on production confidential VMs, reads and writes of encrypted VM memory, and replay of VM register state [[S-1212]].\n\nAMD assigned CVE-2025-0033 [[S-1212]]. Its bulletin rates the issue medium severity and reports SEV firmware, microcode or platform firmware updates for every affected EPYC server and embedded series, with release dates from June 2025 to February 2026 [[S-1213]].\n\nIn Fabricked, researchers from the same ETH Zurich group showed that a host controlling the hypervisor and UEFI firmware can misconfigure the Infinity Fabric interconnect so that the AMD Secure Processor initialises SEV-SNP incorrectly. On a Zen 5 EPYC processor this gave arbitrary reads and writes in the victim VM and forged attestation reports [[S-3127]]. AMD assigned CVE-2025-54510, rates it medium severity and reports platform firmware updates for its EPYC 7003, 8004, 9004 and 9005 server series, released in November and December 2025 [[S-3128]].", "sources": [ "S-1212", "S-1213", "S-3127", "S-3128" ], "response": null }, { "assessment": true, "title": "H100 attestation not bound to a specific confidential VM", "kind": "demonstrated-attack", "severity": "significant", "status": "open", "description": "The TEE.fail authors fetched genuine H100 confidential-computing attestations from a rented server running their TDX VM. They combined these with forged TDX quotes. A proxy running outside any TEE then passed both the TDX and the GPU attestation checks. The authors attribute this to NVIDIA not binding the H100 to the identities of specific VMs. Their site states more generally that NVIDIA's attestation reports are not bound to a specific confidential VM or CPU. Intel, AMD, NVIDIA and the affected deployments acknowledged the findings, according to the authors, and the affected deployments were working on mitigations. The attack does not target NVIDIA's confidential-computing components directly, so the authors state that there are no mitigations on the NVIDIA side [[S-1202]].", "sources": [ "S-1202" ], "response": null }, { "assessment": true, "title": "Side channels and other attacks by the host on CPU and GPU TEEs", "kind": "demonstrated-attack", "severity": "significant", "status": "open", "description": "PAL\\*M and Attestable Audits cite published side-channel, single-stepping, interrupt-injection and memory-aliasing attacks on Intel TDX and AMD SEV, including T-Time, TDXploit, CIPHER-LEAKS, Heckler and BadRAM. PAL\\*M treats them as out of scope [[S-0012]]. Attestable Audits proposes revoking vulnerable enclave images [[S-0009]]. Gloria Z notes that performance counters have themselves been used as a side channel, for example in CounterSEVeillance [[S-0014]]. New attacks of this kind continue to appear. In StackWarp, researchers at CISPA showed that a malicious hypervisor can shift the stack pointer of an SEV-SNP guest on AMD Zen 1 to Zen 5 processors with simultaneous multithreading enabled, which fully breaks the guest's integrity. AMD released microcode patches [[S-3123]]. On the GPU side, an independent analysis of NVIDIA's confidential computing by IBM Research and Ohio State University found that bulk command and data transfers are protected, but some metadata, timing behaviour and coordination signals remain in unprotected shared memory. The authors report that these can reveal computational behaviour and in some cases allow manipulation of operations. They disclosed the findings to NVIDIA [[S-3129]].", "sources": [ "S-0012", "S-0009", "S-0014", "S-3123", "S-3129" ], "response": null }, { "assessment": true, "title": "Attestation covers launch state, and measurements can be incomplete", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Attestation measures launch state, not runtime state. Data loaded later, such as model weights, must be bound separately [[S-0013]]. Gloria Z argues that gaps in measuring feature flags, environment variables and invocation arguments are \"perhaps the most likely failure mode\". She also warns that a badly designed hashing scheme could let two models with significantly different properties share a hash \"without breaking the hash function itself\" [[S-0014]]. Independent reviews of production systems have found such gaps. In WhatsApp's deployment, Trail of Bits found environment variables and ACPI tables loaded outside the measurement, and rated both high severity. Meta fixed them [[S-3121]] [[S-3124]]. On an Apple PCC node running in Apple's research environment, a researcher reports that tampered configuration files left the attestation unchanged [[S-1804]].", "sources": [ "S-0013", "S-0014", "S-3121", "S-3124", "S-1804" ], "response": null }, { "assessment": true, "title": "Deployment-level attestation does not cover the whole chip", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "An attestation shows what one confidential VM runs. It does not show what else the hypervisor runs on the same hardware. Gloria Z calls the difference between deployment-level attestation and chip-wide monitoring \"the gaping hole in this plan\" [[S-0014]]. This matters most for negative claims such as the absence of training.", "sources": [ "S-0014" ], "response": null }, { "assessment": true, "title": "Root of trust concentrated in a few hardware vendors", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "The root of trust is the certificate authorities of a small number of vendors (AMD, Intel and NVIDIA), which generate the keys and fuse them onto the chips. Gloria Z notes that whoever has access to a hardware key, or can certify one, can in principle produce valid reports for arbitrary measurements without the physical chip [[S-0014]]. Attestable Audits notes that the approach holds only \"as long as the vendor of the secure hardware is trusted\" [[S-0009]]. A 2026 preprint reports that a host with root control and the ability to rewrite platform flash can downgrade an AMD EPYC Milan processor to legacy security-processor firmware and extract the hardware root seed from which SEV-SNP attestation keys are derived. The authors state that this lets them forge attestation reports for any firmware version [[S-3130]]. AMD describes the firmware-loader flaw the attack starts from as a legacy attack mitigated in 2021 [[S-3131]].", "sources": [ "S-0014", "S-0009", "S-3130", "S-3131" ], "response": null } ], "blockers": [ { "text": "Vendor threat models exclude sophisticated physical attacks, but in international verification the prover holds the hardware.", "theme": "hardware-trust", "blocked_by": "M-0009", "sources": [ "S-1200", "S-1204", "S-1202", "S-3126", "S-1210", "S-1211" ] }, { "text": "Negative claims such as \"no undeclared training\" need chip-wide accounting of all workloads, which attestation does not provide.", "theme": "coverage-hidden-compute", "blocked_by": "M-0010", "sources": [ "S-0014" ] }, { "text": "Multi-GPU and multi-node coverage is incomplete, because Hopper leaves NVLink traffic unencrypted and NVIDIA's April 2026 release notes list no multi-node confidential mode.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-1200", "S-3122" ] }, { "text": "Rival parties have not agreed on trust roots and key provenance they would accept.", "theme": "access-governance", "blocked_by": null, "sources": [ "S-0014" ] }, { "text": "CPU-only enclaves are costly for large models, because in the Attestable Audits prototype CPU inference cost 21.7 times as much per token as GPU inference and the enclave roughly doubled the CPU cost.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-0009" ] } ], "challenge_themes": [ "hardware-trust", "evidence-binding", "coverage-hidden-compute", "adversarial-validation", "performance-compatibility" ], "organizations": [ "O-0140", "O-0141", "O-0142", "O-0202", "O-0206" ], "people": [], "sources": [ { "source": "S-1200", "supports": "GPU CC architecture, CPU-TEE pairing, multi-GPU modes, threat model scope, performance counters in CC mode and NVIDIA's side-channel reason", "locator": "pp. 6-18" }, { "source": "S-1201", "supports": "H100 root of trust, device identity key, attestation report, launch performance", "locator": "blog, sections on root of trust and performance" }, { "source": "S-1202", "supports": "TEE design aim; memory-bus interposition attack; Intel PCK extraction and forged SGX/TDX attestations; SEV-SNP OpenSSL key recovery; H100 attestation relay; disclosure and vendor positions", "locator": "Abstract; §1.1-1.2; §3; §8.3; §10.2; site FAQ" }, { "source": "S-1210", "supports": "Battering RAM: DDR4 interposer cost; Scalable SGX plaintext access and provisioning-key extraction; SEV-SNP attestation breach by launch-digest replay; DDR4-only scope; Intel and AMD positions", "locator": "Abstract; §1 contributions; site FAQ" }, { "source": "S-1211", "supports": "WireTap: DDR4 interposer cost; extraction of the SGX Quoting Enclave attestation key and forged quotes; DDR5 Xeons not affected; Intel position and mitigation", "locator": "Abstract; site FAQ" }, { "source": "S-1212", "supports": "RMPocalypse: software-only RMP corruption by a malicious hypervisor; forged SEV-SNP attestation, debug, memory access and register replay; affected Zen generations; CVE", "locator": "Abstract; site; responsible disclosure" }, { "source": "S-1213", "supports": "AMD's severity rating and firmware mitigations for CVE-2025-0033 (vendor-reported)", "locator": "Mitigation tables; revision history" }, { "source": "S-0009", "supports": "audit protocol, AWS Nitro prototype, overheads, vendor trust, cited TEE attacks and image revocation", "locator": "§2, §3, §5 Table 2, §7" }, { "source": "S-0012", "supports": "property attestation on TDX + H100, REPORTDATA binding, overheads, threat model exclusions, code status", "locator": "Abstract; §3-§6, Tables 2-6" }, { "source": "S-0013", "supports": "launch-state vs runtime; dm-verity weight binding", "locator": "sections on the challenge and the three phases" }, { "source": "S-0014", "supports": "international threat model, vendor root of trust, measurement incompleteness, hashing-scheme warning, deployment vs chip-wide gap, process-level enforcement, counter side channels" }, { "source": "S-0010", "supports": "device-only NPU TEE with task attestation; overheads; threat model", "locator": "Abstract; threat model; evaluation" }, { "source": "S-0006", "supports": "open question on TEEs for verifying AI training at scale", "locator": "§2.2.4" }, { "source": "S-0056", "supports": "existing technologies need hardening for adversarial settings", "locator": "Key findings" }, { "source": "S-0057", "supports": "physical-access key extraction caveat for encrypted memory", "locator": "p. 20" }, { "source": "S-1203", "supports": "FLI and Mithril SGX proof-of-concept and its stated limitations" }, { "source": "S-1204", "supports": "TEE-backed software vs physical access; early-access CC lacked multi-node", "locator": "sections on software/TEE options" }, { "source": "S-1206", "supports": "Tinfoil hardware, trust model, report signing keys, documented limitations (provider-reported)", "locator": "Supported hardware; Trust model; Limitations" }, { "source": "S-1207", "supports": "memory encryption against host software, reproducible builds and transparency log, client verification, boot-time GPU attestation check (provider-reported)" }, { "source": "S-1208", "supports": "production deployment of Tinfoil's inference enclaves (provider-reported)", "locator": "In-band vs. out-of-band verification" }, { "source": "S-1209", "supports": "public open-source implementation (v0.3.0)" }, { "source": "S-1815", "supports": "general availability of Azure confidential VMs with H100 GPUs (vendor-reported)" }, { "source": "S-1816", "supports": "Google Private AI Compute: remote attestation to a sealed TPU environment (provider-reported)" }, { "source": "S-1800", "supports": "Apple PCC: devices send requests only to nodes attesting to software in a public transparency log (provider-reported)" }, { "source": "S-1802", "supports": "Apple PCC extended to Google Cloud on NVIDIA confidential computing and Intel TDX (provider-reported)" }, { "source": "S-1804", "supports": "independent finding that a PCC node with tampered configuration passed attestation; fix" }, { "source": "S-1817", "supports": "Anthropic's confidential inference design sketch (provider-reported)" }, { "source": "S-3120", "supports": "WhatsApp Private Processing hardware and client attestation checks (provider-reported)" }, { "source": "S-3121", "supports": "Trail of Bits audit of WhatsApp Private Processing: finding counts, unmeasured configuration data, missing freshness, fixes, SEV-SNP physical-attack caveat" }, { "source": "S-3124", "supports": "Trail of Bits' finding counts by severity and titles of the eight high-severity findings, including unmeasured environment variables and ACPI tables and unverified GPU attestation" }, { "source": "S-3122", "supports": "confidential modes generally available in NVIDIA's R595 release (April 2026); no multi-node mode listed (vendor-reported)" }, { "source": "S-3123", "supports": "StackWarp: software-only integrity break of SEV-SNP guests on Zen 1-5; AMD microcode patches" }, { "source": "S-3126", "supports": "DDRop: active DDR5 interposer cost; debug-mode forcing and forged attestation reports on up-to-date TDX; SGX and SEV-SNP integrity breaks; vendor positions and advisories", "locator": "Abstract; threat model; case studies; site FAQ" }, { "source": "S-3127", "supports": "Fabricked: software-only Infinity Fabric misconfiguration; arbitrary read and write and forged SEV-SNP attestation on Zen 5", "locator": "Abstract; evaluation" }, { "source": "S-3128", "supports": "AMD's severity rating and firmware mitigations for CVE-2025-54510 (vendor-reported)", "locator": "Summary; mitigation tables" }, { "source": "S-3129", "supports": "independent security analysis of NVIDIA GPU confidential computing: residual metadata, timing and coordination leaks; disclosure to NVIDIA", "locator": "Abstract; conclusion" }, { "source": "S-3130", "supports": "software-only extraction of the SEV-SNP VCEK root seed on EPYC Milan via firmware downgrade; forged reports for any firmware version", "locator": "Abstract; contributions; disclosure" }, { "source": "S-3131", "supports": "AMD's view of MilanLaunchy as a legacy attack mitigated in 2021 (vendor-reported)", "locator": "Summary" } ], "concepts": [ "K-0006", "K-0004", "K-0005", "K-0013", "K-0015", "K-0018", "K-0019", "K-0024", "K-0001", "K-0002" ], "complements": [ "M-0009", "M-0010", "M-0012", "M-0023", "M-0025" ], "alternatives": [ "M-0001", "M-0004" ], "type": "mechanism", "implementations": [ { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/" }, { "id": "I-0007", "title": "Attestable Audits", "url": "https://trustbutveri.fyi/implementations/attestable-audits/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" } ], "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/", "source_file": "content/mechanisms/tee-remote-attestation.md", "flags_all": [ "provider-reported", "critical-flaw-open" ], "body_markdown": "## How it works\n\nA trusted execution environment (TEE) is a hardware-enforced boundary around a computation. TEEs aim to protect confidentiality and integrity even from an attacker who controls all of the host's software [[S-1202]]. In a confidential virtual machine (CVM), the CPU encrypts the VM's memory so that the hypervisor and other privileged software cannot read or tamper with it [[S-1207]].\n\nThe hardware measures each piece of software before it runs. It signs a report of those measurements with keys that chain to the manufacturer [[S-1206]]. A remote verifier checks the signature chain back to the vendor's root certificate. It then compares the measurements with expected values, which may come from reproducible builds published to a transparency log [[S-1207]].\n\nNVIDIA extends attestation to its GPUs. In NVIDIA's description, an on-die root of trust checks the GPU's fused identity key and permits only signed firmware. The driver then opens an authenticated session with the GPU and obtains a signed attestation report [[S-1200]] [[S-1201]]. The GPU must be paired with a CPU TEE, either Intel TDX or AMD SEV, and NVIDIA recommends SEV-SNP [[S-1200]].\n\nOther designs attest the accelerator alone. GuardAIn gives a discrete neural processing unit (NPU) measured boot and task attestation without relying on a CPU TEE [[S-0010]].\n\nSeveral verification schemes build on these reports:\n\n- **[[I-0007|Attestable Audits]]** has the model provider and the auditor each send encrypted inputs to an enclave. The enclave runs the audit and publishes an attestation that links the model hash, the audit and the result. Later inference is checked against the audited hash [[S-0009]].\n- **PAL\\*M** attests dataset, training, fine-tuning, evaluation and inference operations on Intel TDX with an H100 [[S-0012]].\n- **[[I-0006|Tinfoil]]** binds the served weights to the attestation, as described under [[M-0012]] [[S-0013]].\n- **Gloria Z** proposes running auditable, reproducibly built software inside a CVM to enforce access control, audit logging and safety classifiers [[S-0014]].\n- **The Future of Life Institute (FLI) and Mithril Security** built an Intel SGX proof of concept for leasing a model with protected weights, usage telemetry and an off-switch [[S-1203]].\n\n## What it establishes\nAn attestation can show two things, provided the hardware is sound and the vendor's keys are not compromised:\n\n- A specific, measured software stack produced a given output from inputs identified by hash [[S-0012]].\n- The stack ran on genuine vendor hardware in confidential mode [[S-1207]].\n\nThis supports positive claims, such as which model answered ([[C-0005]]) or which policy code wrapped it ([[C-0006]]).\n\nAn attestation has three gaps:\n\n- It reflects the launch state, so it does not cover weights or other data loaded afterwards. These must be bound separately [[S-0013]].\n- It does not cover anything that was never measured. Gloria Z calls gaps in measuring feature flags, environment variables and invocation arguments \"perhaps the most likely failure mode\" [[S-0014]].\n- It covers one deployment, not everything the chip does [[S-0014]]. Chip-level signals that bear on negative claims, such as that no training happened, are covered in [[M-0010]] and [[M-0021]].\n\n## Threat model\nTEE designs treat the operator's software as hostile, including the host operating system, the hypervisor and the cloud stack [[S-1206]] [[S-0012]]. They trust the hardware and its vendor [[S-1206]] [[S-0009]]. Each vendor and research design lists attacks it does not defend against:\n\n- **NVIDIA** lists software, basic physical, rollback, cryptographic and replay attacks as in scope. Sophisticated physical attacks and denial of service are out of scope [[S-1200]].\n- **Intel and AMD** consider memory-bus interposer attacks and other physical attacks on memory out of scope, according to the TEE.fail, DDRop, Battering RAM and WireTap authors [[S-1202]] [[S-3126]] [[S-1210]] [[S-1211]].\n- **PAL\\*M** excludes side channels and physical attacks such as memory-bus interposition or swapping GPUs [[S-0012]].\n- **GuardAIn** excludes side channels and denial of service [[S-0010]].\n\nInternational verification faces a stronger adversary. Gloria Z describes it as a nation-state that may have advance physical access to data centres and leverage over hardware vendors [[S-0014]]. Other analysts also doubt that TEEs hold up against physical access:\n\n- **The flexHEG authors** judge TEE-backed software \"probably not secure against sophisticated actors if they have unrestricted physical access\" [[S-1204]].\n- **The Center for a New American Security (CNAS)** says existing features need hardening before they can be relied on in adversarial settings [[S-0056]].\n- **RAND** notes that an attacker with physical access may be able to extract both the keys and the protected data [[S-0057]].\n\n## Evidence\n- **NVIDIA** documents confidential computing for Hopper and Blackwell GPUs [[S-1200]].\n- **Tinfoil** reports support for H100, H200 and B200 GPUs [[S-1206]] and a production inference deployment [[S-1208]]. Its model-identity tool is open source [[S-1209]].\n- **PAL\\*M** reports under 11% overhead for common operations on Intel TDX with an H100. For inference attestation across three models, total observed overhead was 3.8–11.4% for multi-turn sessions and 45.5–66.4% for single prompts. Its code is \"to be released after peer review\" [[S-0012]].\n- **Attestable Audits** ran its prototype on CPU-only AWS Nitro Enclaves with a 4-bit Llama-3.1-8B. CPU inference cost 21.7 times as much per token as GPU inference, and the enclave roughly doubled the CPU cost. The authors expect a production version on confidential-computing GPUs to have \"an overhead as small as 5×\" [[S-0009]].\n- **GuardAIn** reports under 0.1% inference overhead for Llama variants on a Huawei Ascend 910A [[S-0010]].\n- **FLI and Mithril** call their SGX prototype \"not necessarily deployable as is\", because of performance and hardware attacks that need mitigation [[S-1203]].\n- **Public clouds** sell confidential GPU virtual machines. NVIDIA reports that Azure's NCC H100 v5 confidential VMs became generally available in two regions in September 2024 [[S-1815]]. Google reports that its Private AI Compute, launched in November 2025, uses remote attestation and encryption to connect users' devices to Gemini models in a sealed environment on its TPUs [[S-1816]].\n- **[[I-0013|Apple Private Cloud Compute]]** sends each request only to servers that attest to software listed in a public transparency log, according to Apple [[S-1800]]. In June 2026 Apple announced an extension to Google Cloud on NVIDIA confidential computing and Intel TDX [[S-1802]]. An independent researcher reports that a node in Apple's research environment passed its attestation check with tampered configuration files, until Apple fixed the bug that allowed the tampering [[S-1804]].\n- **Meta** reports that WhatsApp's Private Processing runs AI requests in confidential VMs on AMD SEV-SNP with NVIDIA Hopper GPUs, and that clients check each attestation against a third-party transparency log [[S-3120]]. In a pre-launch audit, Trail of Bits reports finding 28 issues, eight of them high severity, including configuration data loaded outside the measurement, attestations with no freshness guarantee and GPU attestation that was not verified [[S-3121]] [[S-3124]]. Meta fixed 16 and partly addressed four before launch [[S-3121]].\n- **Anthropic**, with Pattern Labs, sketches a confidential inference design in which a small model loader with an attested boot decrypts data only inside a trusted environment before passing it to the accelerator. It aims to protect both user data and model weights, and Anthropic calls the work early [[S-1817]].\n\n## Limitations\n- A physical attacker can forge attestations. In TEE.fail, independent researchers with physical access, root privileges and equipment costing under $1000 extracted a per-CPU Intel key that certifies attestation keys from an up-to-date machine, and forged TDX attestations. They paired the forgeries with genuine H100 attestations relayed from rented hardware, and a workload outside TEE protection passed both checks [[S-1202]]. A second team forged TDX attestation reports with an active DDR5 interposer costing under $200 [[S-3126]]. On DDR4 servers, Battering RAM and WireTap forged SGX attestations with interposers costing under $50 and under $1000, and Battering RAM also broke AMD SEV-SNP attestation [[S-1210]] [[S-1211]].\n- A software attacker has forged attestations too. In RMPocalypse, a malicious hypervisor faked SEV-SNP attestation on Zen 3, Zen 4 and Zen 5 processors without physical access [[S-1212]]. Fabricked did the same on Zen 5 from the hypervisor and UEFI firmware, by misconfiguring the processor's interconnect [[S-3127]]. AMD reports firmware fixes for both [[S-1213]] [[S-3128]].\n- Side channels and other attacks by the host remain. PAL\\*M and Attestable Audits cite earlier side-channel, interrupt-injection and memory-aliasing attacks on CPU TEEs [[S-0012]] [[S-0009]], and new ones such as StackWarp continue to appear [[S-3123]]. An independent analysis of NVIDIA's GPU confidential computing found metadata and timing leaks in unprotected shared memory [[S-3129]]. NVIDIA disables performance counters in confidential mode because they could provide an avenue for side-channel attacks [[S-1200]]. Gloria Z notes that counters have leaked secrets from TEEs [[S-0014]].\n- Coverage of large systems is incomplete. Hopper does not encrypt NVLink [[S-1200]]. NVIDIA's early-access release lacked multi-node support in mid-2025 [[S-1204]], and its April 2026 release notes list confidential modes only for GPUs passed through to a confidential VM, with no multi-node mode [[S-3122]]. O'Gara et al. ask what changes would be needed to use TEEs for verifying AI training across multi-node, multi-GPU systems at scale [[S-0006]].\n- The root of trust rests with a few vendors (AMD, Intel and NVIDIA) [[S-0014]].", "body_text": "How it works A trusted execution environment (TEE) is a hardware-enforced boundary around a computation. TEEs aim to protect confidentiality and integrity even from an attacker who controls all of the host's software [S-1202]. In a confidential virtual machine (CVM), the CPU encrypts the VM's memory so that the hypervisor and other privileged software cannot read or tamper with it [S-1207]. The hardware measures each piece of software before it runs. It signs a report of those measurements with keys that chain to the manufacturer [S-1206]. A remote verifier checks the signature chain back to the vendor's root certificate. It then compares the measurements with expected values, which may come from reproducible builds published to a transparency log [S-1207]. NVIDIA extends attestation to its GPUs. In NVIDIA's description, an on-die root of trust checks the GPU's fused identity key and permits only signed firmware. The driver then opens an authenticated session with the GPU and obtains a signed attestation report [S-1200] [S-1201]. The GPU must be paired with a CPU TEE, either Intel TDX or AMD SEV, and NVIDIA recommends SEV-SNP [S-1200]. Other designs attest the accelerator alone. GuardAIn gives a discrete neural processing unit (NPU) measured boot and task attestation without relying on a CPU TEE [S-0010]. Several verification schemes build on these reports: - Attestable Audits has the model provider and the auditor each send encrypted inputs to an enclave. The enclave runs the audit and publishes an attestation that links the model hash, the audit and the result. Later inference is checked against the audited hash [S-0009]. - PAL*M attests dataset, training, fine-tuning, evaluation and inference operations on Intel TDX with an H100 [S-0012]. - Tinfoil binds the served weights to the attestation, as described under Model identity attestation [S-0013]. - Gloria Z proposes running auditable, reproducibly built software inside a CVM to enforce access control, audit logging and safety classifiers [S-0014]. - The Future of Life Institute (FLI) and Mithril Security built an Intel SGX proof of concept for leasing a model with protected weights, usage telemetry and an off-switch [S-1203]. What it establishes An attestation can show two things, provided the hardware is sound and the vendor's keys are not compromised: - A specific, measured software stack produced a given output from inputs identified by hash [S-0012]. - The stack ran on genuine vendor hardware in confidential mode [S-1207]. This supports positive claims, such as which model answered (The declared model is the one being served) or which policy code wrapped it (Declared safeguards were applied during inference). An attestation has three gaps: - It reflects the launch state, so it does not cover weights or other data loaded afterwards. These must be bound separately [S-0013]. - It does not cover anything that was never measured. Gloria Z calls gaps in measuring feature flags, environment variables and invocation arguments \"perhaps the most likely failure mode\" [S-0014]. - It covers one deployment, not everything the chip does [S-0014]. Chip-level signals that bear on negative claims, such as that no training happened, are covered in On-chip telemetry from timing, memory and performance counters and Workload classification from telemetry and side channels. Threat model TEE designs treat the operator's software as hostile, including the host operating system, the hypervisor and the cloud stack [S-1206] [S-0012]. They trust the hardware and its vendor [S-1206] [S-0009]. Each vendor and research design lists attacks it does not defend against: - NVIDIA lists software, basic physical, rollback, cryptographic and replay attacks as in scope. Sophisticated physical attacks and denial of service are out of scope [S-1200]. - Intel and AMD consider memory-bus interposer attacks and other physical attacks on memory out of scope, according to the TEE.fail, DDRop, Battering RAM and WireTap authors [S-1202] [S-3126] [S-1210] [S-1211]. - PAL*M excludes side channels and physical attacks such as memory-bus interposition or swapping GPUs [S-0012]. - GuardAIn excludes side channels and denial of service [S-0010]. International verification faces a stronger adversary. Gloria Z describes it as a nation-state that may have advance physical access to data centres and leverage over hardware vendors [S-0014]. Other analysts also doubt that TEEs hold up against physical access: - The flexHEG authors judge TEE-backed software \"probably not secure against sophisticated actors if they have unrestricted physical access\" [S-1204]. - The Center for a New American Security (CNAS) says existing features need hardening before they can be relied on in adversarial settings [S-0056]. - RAND notes that an attacker with physical access may be able to extract both the keys and the protected data [S-0057]. Evidence - NVIDIA documents confidential computing for Hopper and Blackwell GPUs [S-1200]. - Tinfoil reports support for H100, H200 and B200 GPUs [S-1206] and a production inference deployment [S-1208]. Its model-identity tool is open source [S-1209]. - PAL*M reports under 11% overhead for common operations on Intel TDX with an H100. For inference attestation across three models, total observed overhead was 3.8–11.4% for multi-turn sessions and 45.5–66.4% for single prompts. Its code is \"to be released after peer review\" [S-0012]. - Attestable Audits ran its prototype on CPU-only AWS Nitro Enclaves with a 4-bit Llama-3.1-8B. CPU inference cost 21.7 times as much per token as GPU inference, and the enclave roughly doubled the CPU cost. The authors expect a production version on confidential-computing GPUs to have \"an overhead as small as 5×\" [S-0009]. - GuardAIn reports under 0.1% inference overhead for Llama variants on a Huawei Ascend 910A [S-0010]. - FLI and Mithril call their SGX prototype \"not necessarily deployable as is\", because of performance and hardware attacks that need mitigation [S-1203]. - Public clouds sell confidential GPU virtual machines. NVIDIA reports that Azure's NCC H100 v5 confidential VMs became generally available in two regions in September 2024 [S-1815]. Google reports that its Private AI Compute, launched in November 2025, uses remote attestation and encryption to connect users' devices to Gemini models in a sealed environment on its TPUs [S-1816]. - Apple Private Cloud Compute sends each request only to servers that attest to software listed in a public transparency log, according to Apple [S-1800]. In June 2026 Apple announced an extension to Google Cloud on NVIDIA confidential computing and Intel TDX [S-1802]. An independent researcher reports that a node in Apple's research environment passed its attestation check with tampered configuration files, until Apple fixed the bug that allowed the tampering [S-1804]. - Meta reports that WhatsApp's Private Processing runs AI requests in confidential VMs on AMD SEV-SNP with NVIDIA Hopper GPUs, and that clients check each attestation against a third-party transparency log [S-3120]. In a pre-launch audit, Trail of Bits reports finding 28 issues, eight of them high severity, including configuration data loaded outside the measurement, attestations with no freshness guarantee and GPU attestation that was not verified [S-3121] [S-3124]. Meta fixed 16 and partly addressed four before launch [S-3121]. - Anthropic, with Pattern Labs, sketches a confidential inference design in which a small model loader with an attested boot decrypts data only inside a trusted environment before passing it to the accelerator. It aims to protect both user data and model weights, and Anthropic calls the work early [S-1817]. Limitations - A physical attacker can forge attestations. In TEE.fail, independent researchers with physical access, root privileges and equipment costing under $1000 extracted a per-CPU Intel key that certifies attestation keys from an up-to-date machine, and forged TDX attestations. They paired the forgeries with genuine H100 attestations relayed from rented hardware, and a workload outside TEE protection passed both checks [S-1202]. A second team forged TDX attestation reports with an active DDR5 interposer costing under $200 [S-3126]. On DDR4 servers, Battering RAM and WireTap forged SGX attestations with interposers costing under $50 and under $1000, and Battering RAM also broke AMD SEV-SNP attestation [S-1210] [S-1211]. - A software attacker has forged attestations too. In RMPocalypse, a malicious hypervisor faked SEV-SNP attestation on Zen 3, Zen 4 and Zen 5 processors without physical access [S-1212]. Fabricked did the same on Zen 5 from the hypervisor and UEFI firmware, by misconfiguring the processor's interconnect [S-3127]. AMD reports firmware fixes for both [S-1213] [S-3128]. - Side channels and other attacks by the host remain. PAL*M and Attestable Audits cite earlier side-channel, interrupt-injection and memory-aliasing attacks on CPU TEEs [S-0012] [S-0009], and new ones such as StackWarp continue to appear [S-3123]. An independent analysis of NVIDIA's GPU confidential computing found metadata and timing leaks in unprotected shared memory [S-3129]. NVIDIA disables performance counters in confidential mode because they could provide an avenue for side-channel attacks [S-1200]. Gloria Z notes that counters have leaked secrets from TEEs [S-0014]. - Coverage of large systems is incomplete. Hopper does not encrypt NVLink [S-1200]. NVIDIA's early-access release lacked multi-node support in mid-2025 [S-1204], and its April 2026 release notes list confidential modes only for GPUs passed through to a confidential VM, with no multi-node mode [S-3122]. O'Gara et al. ask what changes would be needed to use TEEs for verifying AI training across multi-node, multi-GPU systems at scale [S-0006]. - The root of trust rests with a few vendors (AMD, Intel and NVIDIA) [S-0014].", "referenced_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/" }, { "id": "I-0007", "title": "Attestable Audits", "url": "https://trustbutveri.fyi/implementations/attestable-audits/" }, { "id": "I-0009", "title": "Lucid sovereignty (location) certificates", "url": "https://trustbutveri.fyi/implementations/lucid-location-certificates/" }, { "id": "I-0017", "title": "PySyft double-blind evaluations", "url": "https://trustbutveri.fyi/implementations/pysyft-double-blind-evaluations/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" }, { "id": "K-0019", "title": "Evidence binding", "url": "https://trustbutveri.fyi/concepts/evidence-binding/" }, { "id": "K-0001", "title": "Prover", "url": "https://trustbutveri.fyi/concepts/prover/" }, { "id": "K-0004", "title": "Remote attestation", "url": "https://trustbutveri.fyi/concepts/remote-attestation/" }, { "id": "K-0005", "title": "Root of trust", "url": "https://trustbutveri.fyi/concepts/root-of-trust/" }, { "id": "K-0006", "title": "Trusted execution environment (TEE)", "url": "https://trustbutveri.fyi/concepts/trusted-execution-environment/" }, { "id": "O-0207", "title": "Center for a New American Security", "url": "https://trustbutveri.fyi/organizations/center-for-a-new-american-security/" }, { "id": "O-0206", "title": "Future of Life Institute", "url": "https://trustbutveri.fyi/organizations/future-of-life-institute/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" }, { "id": "O-0140", "title": "NVIDIA", "url": "https://trustbutveri.fyi/organizations/nvidia/" }, { "id": "O-0141", "title": "Tinfoil", "url": "https://trustbutveri.fyi/organizations/tinfoil/" }, { "id": "O-0142", "title": "University of Cambridge", "url": "https://trustbutveri.fyi/organizations/university-of-cambridge/" }, { "id": "O-0203", "title": "Verifiable Compute Foundation", "url": "https://trustbutveri.fyi/organizations/verifiable-compute-foundation/" } ] }, { "id": "M-0016", "slug": "timed-challenge-response", "title": "Timed challenge-response and memory-occupation challenges", "aliases": [ "Memory challenges", "Memory-occupation challenges", "Software-based attestation", "Proof-of-space-style challenges" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "A verifier sends unpredictable questions that a device can answer in time only if it holds specified data, or dedicates specified resources, locally.", "summary": "In a timed challenge, a verifier sends an unpredictable question that a device can answer in time only if it holds certain data locally or dedicates certain resources to the answer. Filling a device's memory with verifier-chosen data and then checking random pieces can bound how much spare memory it has. The approach builds on software-based attestation of embedded devices and on proofs of space. AI-specific public work is thin: one design overview, one study on T4 and H100 GPUs showing that response times reveal co-running models and whether data sits in GPU memory, and memory-wiping designs that use such challenges. As of September 2026, no network-level test across data-centre servers has been reported. The main obstacle is excluding outside help, such as fast remote memory. Timing-based attestation of embedded devices has been broken in practice, and its attackers judged it \"very difficult, if not impossible, to design correctly\".", "category": "cryptographic-computational", "secondary_categories": [ "accounting-provenance" ], "verifies": [ { "claim": "C-0003", "role": "primary", "note": "Compute and memory probes can reveal whether a GPU is engaged in other work (S-0033)." }, { "claim": "C-0004", "role": "supporting", "note": "Listed as an alternative inference-verification direction that may not need a hardware retrofit (S-0067); bounds spare memory (S-0018)." }, { "claim": "C-0002", "role": "supporting", "note": "Speed-of-light bounds on signed challenge round trips underlie delay-based location checks; see M-0018." } ], "threat_model": "adversarial", "adversarial_evaluation": "analysis", "hardware_requirement": "none", "prover_cooperation": "required", "confidentiality": "preserving", "depends_on": [ { "target": "M-0014", "note": "Excluding remote memory during challenges may need physical disconnection or isolation of the device group." } ], "readiness": { "assessment": true, "level": "R2", "scope": "detecting whether a GPU is doing other work", "rubric_version": "1.1", "rationale": "R2 for detecting whether a GPU is doing other work: published experiments on T4 and H100 GPUs show that challenge response times reveal co-running models and data residency, but only on single GPUs, and no challenge that bounds free memory across servers has been shown.\n\n- **R1** met: the MIRI overview describes memory challenges for verifying the presence of information and the absence of free memory, with timing figures and assumptions [[S-0018]]. The AI 2040 plan names memory-challenge verification as a possible direction [[S-0067]]. The underlying primitives (timed attestation, proofs of space and proofs of secure erasure) are peer-reviewed [[S-1307]] [[S-1607]] [[S-1304]] [[S-0032]].\n- **R2** met through reproducible published results. Monfared et al. describe four verifier-issued challenge probes with their parameters and sample counts, under a threat model in which host and GPU firmware \"may be modified, virtualized, or colluding\" [[S-0033]]. On T4 and H100 GPUs, solve times rise when language models run alongside, and a VRAM-residency challenge separates data in GPU memory from data in host memory by more than 350 ms [[S-0033]]. No code is linked, which the rubric does not require. SAGE shows timed software attestation on A100 GPUs for trusted execution, not for detecting other work [[S-1306]]. The mechanism's only implementation, [[I-0012]], is a proposed architecture at R1, so the level rests on the research results.\n- **R3** not met: no production-grade challenge tool is publicly documented, and no source reports a party other than the authors relying on such challenges for a verification decision. For bounding spare memory ([[C-0004]]), the MIRI overview states that, to its author's knowledge, a network-level timing probe of memory contents between servers \"has not yet been demonstrated\" [[S-0018]].\n\nConfidence is medium: the results come from one group's single-GPU experiments without quantified error rates.", "evidence": [ "S-0018", "S-0067", "S-0033", "S-1306", "S-1307", "S-1607", "S-1304", "S-0032" ], "next_level_gaps": [ "Production-grade challenge tooling, or use by a party other than the developers for a verification decision.", "A network-level challenge that bounds free memory across accelerator servers, with public code or measurements described in enough detail to repeat.", "Quantified false-positive and false-negative rates under adversarial conditions.", "Evaluation against known attack classes on timed attestation, such as compression and relocation." ], "confidence": "medium", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Timing-based software attestation has been broken in practice", "kind": "demonstrated-attack", "severity": "significant", "status": "open", "description": "Castelluccia et al. implemented two generic attacks, one based on a return-oriented rootkit and one on code compression, together with specific attacks on SWATT and ICE-based schemes, on commodity sensor nodes. They conclude that secure time-based attestation is \"very difficult, if not impossible, to design correctly\". The attacks target embedded schemes, not AI accelerators.", "sources": [ "S-1308" ], "response": null }, { "assessment": true, "title": "Remote memory narrows the timing margin", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Data-centre remote memory access returns in about 1–2 µs, against about 70–200 ns for local DRAM. The MIRI overview says verification of memory saturation depends on ruling out remote access by latency or physical disconnection. It adds that pre-staging data is ruled out only by unpredictable, capacity-filling challenges.", "sources": [ "S-0018" ], "response": null }, { "assessment": true, "title": "Error rates not quantified", "kind": "open-question", "severity": "minor", "status": "open", "description": "Monfared et al. show separable timing distributions but do not define thresholds or statistical tests, so false-positive and false-negative rates are not quantified.", "sources": [ "S-0033" ], "response": null } ], "blockers": [ { "text": "No network-level memory challenge across data-centre servers has been demonstrated.", "theme": "adversarial-validation", "blocked_by": null, "sources": [ "S-0018" ] }, { "text": "Challenges that fill memory displace workloads; filling a pod's volatile memory takes tens of minutes and SSDs take hours.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-0018", "S-0033" ] }, { "text": "Outside help, such as remote memory, must be excluded during challenges.", "theme": "coverage-hidden-compute", "blocked_by": "M-0014", "sources": [ "S-0018" ] } ], "challenge_themes": [ "capacity-bounds", "coverage-hidden-compute", "protocol-soundness", "adversarial-validation", "performance-compatibility" ], "organizations": [ "O-0101", "O-0202" ], "people": [], "sources": [ { "source": "S-0018", "supports": "memory challenging for presence and absence; response-time domains; latencies; fill times; RDMA caveat; pre-staging; not yet demonstrated at network level", "locator": "§5.1.2" }, { "source": "S-0067", "supports": "memory-challenge verification and low-latency challenges as alternative directions", "locator": "verification overview; other promising directions" }, { "source": "S-0033", "supports": "PoW, VDF, GEMM and VRAM-residency probes; contention results on T4 and H100; H100 residency result; threat model; overhead; FP/FN caveat", "locator": "abstract; §4–§6; limitations" }, { "source": "S-1306", "supports": "software-based attestation on A100 GPUs", "locator": "abstract" }, { "source": "S-1307", "supports": "timed checksum attestation; verifier knowledge requirements", "locator": "abstract; design sections" }, { "source": "S-1308", "supports": "demonstrated attacks on timed attestation; conclusions", "locator": "abstract; §3–§5" }, { "source": "S-1607", "supports": "proofs of space: dedicating disk space instead of computation; construction", "locator": "abstract" }, { "source": "S-0032", "supports": "timed rounds with a round-trip bound in PoSE; peer-reviewed timed PoSE", "locator": "§3" }, { "source": "S-1302", "supports": "challenge phase of a PoSE implementation; 1 ms RTT assumption", "locator": "protocol section" }, { "source": "S-1304", "supports": "peer-reviewed proofs of secure erasure; weaknesses of timed software attestation as motivation", "locator": "abstract" } ], "concepts": [ "K-0012", "K-0011", "K-0016", "K-0001", "K-0002", "K-0018", "K-0020" ], "complements": [ "M-0015", "M-0013" ], "alternatives": [ "M-0010" ], "type": "mechanism", "implementations": [ { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" } ], "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/", "source_file": "content/mechanisms/timed-challenge-response.md", "flags_all": [], "body_markdown": "## How it works\n\nA timed challenge asks a device something it can answer correctly, within a deadline, only if it holds certain data locally or dedicates certain resources to the answer [[S-1307]] [[S-0018]] [[S-0033]]. Three families of primitives exist:\n\n- In timed attestation, as in SWATT, the verifier sends a random seed, and the device computes a checksum over its memory in a pseudorandom order [[S-1307]]. If memory has been altered, the extra work shows up as a detectable delay [[S-1307]].\n- In a proof of space, a prover shows that it dedicates \"a significant amount of disk space as opposed to computation\" [[S-1607]]. Constructions use graphs that are hard to pebble, and Merkle hash trees [[S-1607]].\n- Some proofs of secure erasure fill memory and then run timed rounds, accepting only answers that arrive within a round-trip bound [[S-0032]] [[S-1302]]; see [[M-0015]].\n\nFor AI facilities, the MIRI system overview describes two uses of memory challenges: \"A) verifying presence of information B) verifying the absence of free memory/storage\" [[S-0018]]. Response time is the main evidence, because answering from another device is measurably slower [[S-0018]]. The overview cites average NVMe read latencies of a few microseconds against about 100 ns for DRAM [[S-0018]]. For the absence case, incompressible data is loaded until capacity is full, and random samples are challenged [[S-0018]].\n\nMonfared et al. propose four GPU probes [[S-0033]]:\n\n- memory-hard puzzles inspired by proof of work, which reveal parallel effort;\n- verifiable-delay-function workloads, which measure sequential execution;\n- matrix-multiplication timing, which reflects tensor-core performance;\n- a VRAM-residency test.\n\nIn the residency test, a large challenge dataset is loaded into GPU memory [[S-0033]]. At random times, fresh challenges require keyed, memory-hard computation over it, and response times show whether it is still resident [[S-0033]].\n\n## What it establishes\nA capacity-filling challenge that is answered on time bounds how much free memory a device had at that moment [[S-0018]]. Monfared et al. argue that timing variations can expose GPU engagement, complementing other accountability mechanisms [[S-0033]]; see [[C-0003]]. The AI 2040 plan says memory-challenge verification \"has the promise of potentially also not requiring a hardware retrofit, but seems likely to face some difficulties with verifying completeness\" [[S-0067]].\n\nA presence check does not show that nothing else is stored [[S-0018]]. The overview notes that data could be pre-staged into local memory before a challenge, which only an unpredictable, capacity-filling challenge rules out [[S-0018]].\n\n## Threat model\n- **Known device.** SWATT's verifier must know the device's clock speed, instruction set, memory architecture and expected memory contents [[S-1307]].\n- **Cryptographic model.** Proofs of space are proved secure in the random oracle model, with one additional mild assumption [[S-1607]].\n- **Compromised stack.** Monfared et al. consider host and GPU firmware that \"may be modified, virtualized, or colluding\" [[S-0033]].\n- **No remote helpers.** In data centres, remote memory access is the key helper to exclude [[S-0018]]. Its round trips of about 1–2 µs compare with about 70–200 ns for local DRAM, and the overview says verification \"depends on the ability to rule out RDMA, either via response latency or physical disconnection\" [[S-0018]].\n\n## Evidence\n- **GPU contention.** Monfared et al. report that proof-of-work-style and verifiable-delay challenges on a T4, and matrix-multiplication challenges on an H100, take longer when language models run alongside, and that larger models cause larger delays [[S-0033]].\n- **GPU memory residency.** On an H100 with a 60 GB challenge dataset, Monfared et al. report that the gap between memory-resident and host-resident responses \"exceeds 350 ms, making them trivial to distinguish\" [[S-0033]]. Their experiments ran on NVIDIA Turing and Hopper GPUs, and they do not quantify error rates [[S-0033]].\n- **GPU attestation.** SAGE, a peer-reviewed software-based attestation mechanism for A100 GPUs, is reported by its authors to be \"already practical today\" for trustworthy execution without special hardware support [[S-1306]].\n- **Memory wiping.** Amodo's wiping design includes a timed challenge phase with an assumed 1 ms round trip, but its July 2026 analysis left the challenge-phase calculations for later [[S-1302]].\n- **Across servers.** The MIRI overview states that, to its author's knowledge, distinguishing memory contents between servers with a network-level timing probe \"has not yet been demonstrated\" [[S-0018]].\n\n## Limitations\n- **Broken embedded schemes.** Castelluccia et al. implemented attacks based on a return-oriented rootkit and on code compression, together with specific attacks on SWATT and ICE-based schemes [[S-1308]]. They conclude that secure time-based attestation is \"very difficult, if not impossible, to design correctly\" [[S-1308]]. Perito and Tsudik cite such weaknesses as motivation for proofs of secure erasure [[S-1304]].\n- **Coverage.** Castelluccia et al. argue that \"all memories (RAM, ROM, EEPROM) have to be attested\" [[S-1308]].\n- **Overhead.** The VRAM-residency test \"intentionally incurs substantial memory overhead\" [[S-0033]], and filling a pod's volatile memory takes tens of minutes [[S-0018]].\n- **Unquantified error rates.** Monfared et al. do not define thresholds or statistical tests [[S-0033]].", "body_text": "How it works A timed challenge asks a device something it can answer correctly, within a deadline, only if it holds certain data locally or dedicates certain resources to the answer [S-1307] [S-0018] [S-0033]. Three families of primitives exist: - In timed attestation, as in SWATT, the verifier sends a random seed, and the device computes a checksum over its memory in a pseudorandom order [S-1307]. If memory has been altered, the extra work shows up as a detectable delay [S-1307]. - In a proof of space, a prover shows that it dedicates \"a significant amount of disk space as opposed to computation\" [S-1607]. Constructions use graphs that are hard to pebble, and Merkle hash trees [S-1607]. - Some proofs of secure erasure fill memory and then run timed rounds, accepting only answers that arrive within a round-trip bound [S-0032] [S-1302]; see Memory wiping and proofs of secure erasure. For AI facilities, the MIRI system overview describes two uses of memory challenges: \"A) verifying presence of information B) verifying the absence of free memory/storage\" [S-0018]. Response time is the main evidence, because answering from another device is measurably slower [S-0018]. The overview cites average NVMe read latencies of a few microseconds against about 100 ns for DRAM [S-0018]. For the absence case, incompressible data is loaded until capacity is full, and random samples are challenged [S-0018]. Monfared et al. propose four GPU probes [S-0033]: - memory-hard puzzles inspired by proof of work, which reveal parallel effort; - verifiable-delay-function workloads, which measure sequential execution; - matrix-multiplication timing, which reflects tensor-core performance; - a VRAM-residency test. In the residency test, a large challenge dataset is loaded into GPU memory [S-0033]. At random times, fresh challenges require keyed, memory-hard computation over it, and response times show whether it is still resident [S-0033]. What it establishes A capacity-filling challenge that is answered on time bounds how much free memory a device had at that moment [S-0018]. Monfared et al. argue that timing variations can expose GPU engagement, complementing other accountability mechanisms [S-0033]; see Declared hardware is idle or shut down. The AI 2040 plan says memory-challenge verification \"has the promise of potentially also not requiring a hardware retrofit, but seems likely to face some difficulties with verifying completeness\" [S-0067]. A presence check does not show that nothing else is stored [S-0018]. The overview notes that data could be pre-staged into local memory before a challenge, which only an unpredictable, capacity-filling challenge rules out [S-0018]. Threat model - Known device. SWATT's verifier must know the device's clock speed, instruction set, memory architecture and expected memory contents [S-1307]. - Cryptographic model. Proofs of space are proved secure in the random oracle model, with one additional mild assumption [S-1607]. - Compromised stack. Monfared et al. consider host and GPU firmware that \"may be modified, virtualized, or colluding\" [S-0033]. - No remote helpers. In data centres, remote memory access is the key helper to exclude [S-0018]. Its round trips of about 1–2 µs compare with about 70–200 ns for local DRAM, and the overview says verification \"depends on the ability to rule out RDMA, either via response latency or physical disconnection\" [S-0018]. Evidence - GPU contention. Monfared et al. report that proof-of-work-style and verifiable-delay challenges on a T4, and matrix-multiplication challenges on an H100, take longer when language models run alongside, and that larger models cause larger delays [S-0033]. - GPU memory residency. On an H100 with a 60 GB challenge dataset, Monfared et al. report that the gap between memory-resident and host-resident responses \"exceeds 350 ms, making them trivial to distinguish\" [S-0033]. Their experiments ran on NVIDIA Turing and Hopper GPUs, and they do not quantify error rates [S-0033]. - GPU attestation. SAGE, a peer-reviewed software-based attestation mechanism for A100 GPUs, is reported by its authors to be \"already practical today\" for trustworthy execution without special hardware support [S-1306]. - Memory wiping. Amodo's wiping design includes a timed challenge phase with an assumed 1 ms round trip, but its July 2026 analysis left the challenge-phase calculations for later [S-1302]. - Across servers. The MIRI overview states that, to its author's knowledge, distinguishing memory contents between servers with a network-level timing probe \"has not yet been demonstrated\" [S-0018]. Limitations - Broken embedded schemes. Castelluccia et al. implemented attacks based on a return-oriented rootkit and on code compression, together with specific attacks on SWATT and ICE-based schemes [S-1308]. They conclude that secure time-based attestation is \"very difficult, if not impossible, to design correctly\" [S-1308]. Perito and Tsudik cite such weaknesses as motivation for proofs of secure erasure [S-1304]. - Coverage. Castelluccia et al. argue that \"all memories (RAM, ROM, EEPROM) have to be attested\" [S-1308]. - Overhead. The VRAM-residency test \"intentionally incurs substantial memory overhead\" [S-0033], and filling a pod's volatile memory takes tens of minutes [S-0018]. - Unquantified error rates. Monfared et al. do not define thresholds or statistical tests [S-0033].", "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" }, { "id": "K-0012", "title": "Proof of space", "url": "https://trustbutveri.fyi/concepts/proof-of-space/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" } ] }, { "id": "M-0021", "slug": "workload-classification-from-telemetry", "title": "Workload classification from telemetry and side channels", "aliases": [ "Workload classification", "Training detection from GPU telemetry", "Power-based workload identification", "Training-versus-inference classification" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Telling whether chips are training, serving or doing non-AI work from GPU counters or power draw, signals that do not read weights or data.", "summary": "Workload classification infers what a chip is doing, such as training a model, serving it or running non-AI code. It uses signals that do not read weights or data: software counters such as power and utilization, or power draw measured outside the chip. Two 2026 preprints report high accuracy on their own test sets: 98.2% from software counters across nine NVIDIA GPU models, and 97% from an external probe on one H200. In the hardest cases, evasion strategies the classifier had not seen cut detection to 43–87% and 48–88% respectively; an added rule raised the second to at least 98%. The main obstacle is trust in the signal: software counters are read by software the operator controls, so they can be forged unless hardware protects the reading. External probes avoid this but have been tested on one GPU. Results cover single machines, and no independent evaluation exists.", "technical": "- **NVML signals and corpus.** Rahman and Tajdari sample nine NVML signals at about 1 Hz: GPU and memory utilization, memory used, power, temperature, SM and memory clocks, and PCIe transmit and receive bandwidth [[S-0037]]. Their corpus has 1,404 runs of 162 workloads, about 117 GPU-hours, on 9 NVIDIA GPU models across the Ampere, Ada, Hopper and Blackwell generations, with 1 to 8 GPUs per configuration; 421 of the runs are evasion attempts [[S-0037]].\n- **NVML classifier.** A two-stage random forest, first ML versus non-ML and then training versus inference, classifies 30 s windows [[S-0037]]. Before adversarial hardening, five-fold cross-validation grouped by run (26,496 windows from 1,396 runs) gives 98.21% ± 0.84% accuracy, 97.39% precision, 98.90% recall and a 3.06% false-positive rate [[S-0037]].\n- **NVML evasion.** The 20 evasion strategy families fall into six classes: utilization manipulation, memory reduction, workload mimicry, training disguised as inference, multi-GPU evasion and white-box attacks [[S-0037]]. In the fifth round the evader had white-box access to the previous round's classifier. After hardening, strategies seen in training were detected 100% of the time and held-out white-box strategies 43–87% of the time [[S-0037]].\n- **External power probe.** Gargiulo and Kulp clamp a Rogowski current probe around the positive conductors of an NVIDIA H200 NVL's PCIe auxiliary power supply and digitize at a nominal 10 MHz in 5 s windows [[S-1412]]. Their dataset has 930 genuine and 680 adversarial recordings over 17 open model families and 25 non-AI workloads [[S-1412]]. On genuine recordings of model families unseen in training, they report 97% accuracy and a macro-averaged F1 of 0.955 across training, inference and non-AI classes [[S-1412]].\n- **Power-probe evasion.** A detector hardened against evasion, with the tested strategy held out, caught training at least 99% of the time for three of four strategies. It caught diluted low-rank adaptation (LoRA) 48–88% of the time, rising to at least 98% with an added rule [[S-1412]]. The four strategies cost 28.1% to 69.3% of throughput [[S-1412]]. The authors state that these attacks are not a comprehensive evaluation against adversarial behaviour [[S-1412]].", "category": "remote-sensing", "secondary_categories": [ "on-chip", "off-chip-devices" ], "verifies": [ { "claim": "C-0004", "role": "primary", "note": "Classifies observed activity as training, inference or non-ML work." }, { "claim": "C-0007", "role": "supporting", "note": "Can flag training on hardware declared for other uses; does not measure training size by itself." } ], "threat_model": "adversarial", "adversarial_evaluation": "red-teamed", "hardware_requirement": "existing-features", "prover_cooperation": "partial", "confidentiality": "partial", "depends_on": [ { "target": "M-0010", "note": "Classifiers that use software-read counters need a tamper-resistant, authenticated path for on-chip telemetry." } ], "readiness": { "assessment": true, "level": "R2", "scope": "telling training from inference and other work, against an operator disguising workloads", "rubric_version": "1.1", "rationale": "Two preprints publish end-to-end results on current GPUs against stated, iterated adversaries; nobody independent has evaluated them.\n\n- **R1** met: the claim, signals and threat model are publicly described [[S-0037]] [[S-1412]] [[S-0007]].\n- **R2** met: Rahman and Tajdari evaluate 9 current NVIDIA GPU models against 20 evasion strategy families over five rounds of monitor–evader iteration, with method and setup described in enough detail to repeat [[S-0037]]. They state that their workload source code is in the project repository, but the paper gives no link [[S-0037]]. Gargiulo and Kulp classify workloads on an H200 from externally measured power, test four evasion strategies and report releasing their recordings as a dataset [[S-1412]].\n- **R3** not met: no party other than the developers is reported to rely on these classifiers for a verification decision, and no production-grade system is available.\n- **R4** not met: no independent adversarial evaluation has been published.\n\nConfidence is medium: both results come from single machines, and the location of Rahman and Tajdari's code is not given.", "evidence": [ "S-0037", "S-1412" ], "next_level_gaps": [ "Reliance by a verifier other than the developers, or a production-grade, available system.", "An independent red-team or peer-reviewed security analysis.", "Results at multi-node cluster scale and across hardware vendors.", "A tamper-resistant, authenticated telemetry path (see [[M-0010]]), or physical sensing validated across devices." ], "confidence": "medium", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Software-read counters can be forged by a privileged operator", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Rahman and Tajdari state that without hardware-enabled guarantees, an adversary with software privilege can return forged counter values. Their results assume a tamper-resistant read path and an authenticated telemetry channel [[S-0037]]. Gargiulo and Kulp note that on-chip counters are read by software under the operator's control, so values could be reported without being measured, or replayed from an authorized workload while a hidden one runs [[S-1412]]. Against an operator who controls the full stack, forgery defeats classification from software counters, the signal source that [[M-0010]] covers. It does not affect the external-probe route, which Gargiulo and Kulp argue can in principle be observed without the operator's cooperation [[S-1412]].", "sources": [ "S-0037", "S-1412" ], "response": null }, { "assessment": true, "title": "Unseen evasion strategies still reduce detection", "kind": "demonstrated-attack", "severity": "significant", "status": "open", "description": "Rahman and Tajdari gave the evader white-box access to the previous round's classifier in their fifth round. After hardening against the other strategies of that round, their classifier detected the held-out white-box strategies only 43–87% of the time, against 100% for strategies it had been trained on [[S-0037]]. In Gargiulo and Kulp's power-trace study, a hardened detector that had not seen diluted LoRA caught it 48–88% of the time. An added rule raised detection to at least 98%, and the strategy cost the evader 33.8–69.3% of throughput [[S-1412]].", "sources": [ "S-0037", "S-1412" ], "response": null }, { "assessment": true, "title": "The training-inference boundary may blur", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Ansari argues that as inference-time compute scaling and fine-tuning become more capable, the training-inference distinction may cease to be a reliable governance boundary [[S-0007]].", "sources": [ "S-0007" ], "response": null }, { "assessment": true, "title": "Limited scale and hardware coverage", "kind": "open-question", "severity": "significant", "status": "open", "description": "Rahman and Tajdari's experiments are single-node, with up to 8 GPUs, and cover NVIDIA hardware only [[S-0037]]. Gargiulo and Kulp's corpus was collected on a single H200 NVL, and generalization across devices remains to be established [[S-1412]].", "sources": [ "S-0037", "S-1412" ], "response": null } ], "blockers": [ { "text": "Software telemetry is trustworthy only if on-chip counters are read over a path the operator cannot tamper with.", "theme": "hardware-trust", "blocked_by": "M-0010", "sources": [ "S-0037", "S-1412" ] }, { "text": "No independent red-team or third-party reliance has been reported.", "theme": "adversarial-validation", "blocked_by": null, "sources": [ "S-0037", "S-1412" ] }, { "text": "Results do not yet cover multi-node clusters, other vendors or multi-tenant serving.", "theme": "coverage-hidden-compute", "blocked_by": null, "sources": [ "S-0037", "S-1412" ] } ], "challenge_themes": [ "hardware-trust", "adversarial-validation", "coverage-hidden-compute", "evidence-binding" ], "organizations": [ "O-0202", "O-0210", "O-0101" ], "people": [], "sources": [ { "source": "S-0037", "supports": "NVML-based classifier, corpus, cross-validated accuracy, evasion families and rounds, hardened detection of unseen strategies, threat model, trust assumption, code statement, limitations", "locator": "Abstract; §2.1, §2.2, §4.1-4.3, §5.1-5.2 and Table 5, §6.5; App. F" }, { "source": "S-1412", "supports": "external power-probe classifier, accuracy on unseen model families, evasion strategies, hardened detection and costs, dataset release, NVML spoofing argument, limitations", "locator": "Abstract; §2-4; limitations" }, { "source": "S-0007", "supports": "workload-classification and power-monitoring feasibility; training-inference boundary", "locator": "§3.1 (M2, M4); §4.6" }, { "source": "S-0048", "supports": "power delivery hierarchy filters signals; low-level monitoring harder to spoof", "locator": "whole note" }, { "source": "S-0041", "supports": "measured training power of an 8-GPU H100 node", "locator": "Abstract" }, { "source": "S-0042", "supports": "input data changes GEMM power draw", "locator": "Abstract" }, { "source": "S-0040", "supports": "precedent: counter-based detection of covert cryptomining", "locator": "Abstract; evaluation" }, { "source": "S-0039", "supports": "precedent: operator-accessible hardware signals for workload-agnostic anomaly detection", "locator": "Abstract; §3, §4.1" }, { "source": "S-0046", "supports": "power traces can leak model architecture", "locator": "Abstract" }, { "source": "S-0059", "supports": "compute-structuring detection: per-workload classification, aggregation of a customer's sequential or data-exchanging workloads against thresholds; analysis only; failure mode with very low data exchange", "locator": "threat models; Algorithms 1–2; limitations" }, { "source": "S-0073", "supports": "nvidia-smi power readings (via NVML) sample only 25% of runtime on A100 and H100; error about ±5% versus NVIDIA's claimed ±5 W", "locator": "Abstract; accuracy findings" } ], "concepts": [ "K-0025", "K-0013", "K-0007", "K-0018" ], "complements": [ "M-0013", "M-0020" ], "alternatives": [], "type": "mechanism", "implementations": [], "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/", "source_file": "content/mechanisms/workload-classification-from-telemetry.md", "flags_all": [], "body_markdown": "## How it works\n\nWorkload classification infers what a chip is doing from signals that do not reveal the content of the computation [[S-0037]] [[S-1412]]. Typical labels are training, inference and non-AI work [[S-0037]] [[S-1412]]. Ansari lists it among monitoring mechanisms [[S-0007]]. It uses cluster-level signals such as power and network bandwidth, and node-level signals such as accelerator utilization and memory bandwidth [[S-0007]].\n\nThere are two main signal sources:\n\n- Rahman and Tajdari use software telemetry from NVIDIA's NVML counters, including utilization, memory, power, temperature, clocks and PCIe traffic, sampled about once a second [[S-0037]]. They describe these as zero-overhead, content-agnostic signals that do not access weights, data or hyperparameters [[S-0037]]. A random-forest pipeline first separates ML from non-ML work, then training from inference [[S-0037]].\n- Gargiulo and Kulp read a physical side channel with a current probe clamped on a GPU's auxiliary power supply, and classify five-second power recordings sampled at 10 MHz [[S-1412]]. They argue that, unlike NVML telemetry, this channel can in principle be observed without the operator's cooperation [[S-1412]].\n\nWhere power is measured matters [[S-0048]]. Amodo Design describes each stage of data-centre power delivery, from the voltage regulator to the grid, as a low-pass filter: the further up the chain one measures, the less compute-level detail survives [[S-0048]]. Amodo argues that low-level power monitoring could be more expensive to spoof [[S-0048]].\n\nSimilar methods have precedents outside AI governance [[S-0040]] [[S-0039]]. Gangwal et al. detected covert cryptocurrency mining from CPU hardware performance counters with 99.9–100% binary accuracy on two test systems [[S-0040]]. Chen et al.'s Reveal detects system anomalies in ML clusters from hardware signals that operators can read, without knowledge of user workloads [[S-0039]].\n\n## What it establishes\nThe authors report that, on the hardware tested, classifiers separate training from inference and non-ML work with high accuracy [[S-0037]] [[S-1412]]. Rahman and Tajdari report 98.2% across their own corpus of NVML telemetry [[S-0037]]. Gargiulo and Kulp report 97% from external power traces on one H200, for model families unseen in training [[S-1412]]. Both figures fall for evasion strategies the classifier has not seen (see Evidence to date). The results support checks that hardware declared for inference or non-AI use is not training ([[C-0004]]) [[S-0037]] [[S-1412]].\n\nWhat it cannot show:\n\n- The papers classify workload type only, not how much training occurred or which model was trained [[S-0037]] [[S-1412]].\n- Ansari's April 2026 feasibility taxonomy noted that power monitoring alone cannot separate AI training from other high-performance computing without other signals, and that no published study had quantified false-positive rates for power-based training detection [[S-0007]].\n- Power draw depends on more than workload type: Gregersen et al. found that changing only the input data can alter GPU power in matrix multiplication by nearly 40% [[S-0042]]. Measured draw can also differ from rated power: Latif et al. recorded a peak of about 8.4 kW on an 8-GPU H100 node during training, 18% below its rated 10.2 kW, with GPUs near full utilization [[S-0041]].\n\nThe signals can also reveal more than the workload class. Gao et al. recovered the architectures of models running on general-purpose processors from software-read power measurements, with 99.75% accuracy by their Levenshtein-distance measure [[S-0046]].\n\n## Threat model\nRahman and Tajdari consider two evaders [[S-0037]]:\n\n- **A cloud tenant** who controls only user-space software.\n- **A hardware operator** who controls the full stack and wants to hide unauthorized training from a regulator.\n\nTheir results depend on a tamper-resistant NVML read path and an authenticated telemetry channel, which they treat as minimum hardware-enabled guarantees ([[M-0010]]) [[S-0037]]. In their fifth round they give the evader white-box access to the classifier, which they describe as a disaster scenario for governance [[S-0037]]. Gargiulo and Kulp treat the operator as adversarial and able to reshape the computation, but not able to control the external probe [[S-1412]].\n\n## Evidence\n- **Software telemetry.** Rahman and Tajdari ran five rounds of monitor–evader iteration over 20 evasion strategy families on 9 GPU models spanning four architecture generations [[S-0037]]. They report 98.2% accuracy across their corpus [[S-0037]]. After hardening, strategies seen in training were always detected, but white-box strategies held out from training were detected only 43–87% of the time [[S-0037]].\n- **External power.** Gargiulo and Kulp report 97% accuracy on held-out model families from a probe on one H200 NVL [[S-1412]]. Their hardened detector caught three of four held-out evasion strategies at least 99% of the time, and diluted LoRA 48–88% of the time, or at least 98% with an added rule [[S-1412]]. They report releasing their 1,610 recordings as a dataset, and state that their attacks are not a comprehensive adversarial evaluation [[S-1412]].\n- **Earlier assessment.** Before these papers, Ansari rated workload classification as deployable in cooperative settings and near-term in adversarial settings [[S-0007]].\n\n## Limitations\n- **Forged or replayed telemetry.** A privileged operator can forge or replay software-read counters unless hardware protects the read path [[S-0037]] [[S-1412]].\n- **Unseen evasion strategies.** Detection falls to 43–87% for held-out white-box strategies on NVML telemetry [[S-0037]], and to 48–88% for diluted LoRA on external power before an added rule [[S-1412]]. In Gargiulo and Kulp's tests the evasion strategies cost 28.1–69.3% of throughput [[S-1412]].\n- **Scale.** Results are single-node and NVIDIA-only [[S-0037]], or from one H200 [[S-1412]].\n- **Power sampling.** Yang and colleagues found that on A100 and H100 GPUs the built-in power reading, which nvidia-smi obtains through NVML, samples only 25% of runtime. The GPU can draw very different power in the other 75% without the reading showing it [[S-0073]]. They also found the reading's error to be within about ±5% in most cases, against the ±5 W that NVIDIA claims [[S-0073]].\n- **Distributed training.** Ansari reports that classification accuracy degrades when training is distributed across providers [[S-0007]]. Seferis and Fist propose that cloud providers classify each workload, then add up the compute of one customer's workloads that run in sequence or exchange data above a bandwidth threshold, and flag totals above a regulatory threshold [[S-0059]]. Their proposal is an analysis without empirical tests, and they note that it fails if decentralized training becomes possible with very little data exchange [[S-0059]].\n- **Blurring categories.** The training-inference distinction may lose governance value [[S-0007]].\n- **Spoofed power monitoring.** Amodo expects that power monitoring can likely be spoofed, but still sees value in it as a complement to other verification systems [[S-0048]].", "body_text": "How it works Workload classification infers what a chip is doing from signals that do not reveal the content of the computation [S-0037] [S-1412]. Typical labels are training, inference and non-AI work [S-0037] [S-1412]. Ansari lists it among monitoring mechanisms [S-0007]. It uses cluster-level signals such as power and network bandwidth, and node-level signals such as accelerator utilization and memory bandwidth [S-0007]. There are two main signal sources: - Rahman and Tajdari use software telemetry from NVIDIA's NVML counters, including utilization, memory, power, temperature, clocks and PCIe traffic, sampled about once a second [S-0037]. They describe these as zero-overhead, content-agnostic signals that do not access weights, data or hyperparameters [S-0037]. A random-forest pipeline first separates ML from non-ML work, then training from inference [S-0037]. - Gargiulo and Kulp read a physical side channel with a current probe clamped on a GPU's auxiliary power supply, and classify five-second power recordings sampled at 10 MHz [S-1412]. They argue that, unlike NVML telemetry, this channel can in principle be observed without the operator's cooperation [S-1412]. Where power is measured matters [S-0048]. Amodo Design describes each stage of data-centre power delivery, from the voltage regulator to the grid, as a low-pass filter: the further up the chain one measures, the less compute-level detail survives [S-0048]. Amodo argues that low-level power monitoring could be more expensive to spoof [S-0048]. Similar methods have precedents outside AI governance [S-0040] [S-0039]. Gangwal et al. detected covert cryptocurrency mining from CPU hardware performance counters with 99.9–100% binary accuracy on two test systems [S-0040]. Chen et al.'s Reveal detects system anomalies in ML clusters from hardware signals that operators can read, without knowledge of user workloads [S-0039]. What it establishes The authors report that, on the hardware tested, classifiers separate training from inference and non-ML work with high accuracy [S-0037] [S-1412]. Rahman and Tajdari report 98.2% across their own corpus of NVML telemetry [S-0037]. Gargiulo and Kulp report 97% from external power traces on one H200, for model families unseen in training [S-1412]. Both figures fall for evasion strategies the classifier has not seen (see Evidence to date). The results support checks that hardware declared for inference or non-AI use is not training (This compute runs inference, not training) [S-0037] [S-1412]. What it cannot show: - The papers classify workload type only, not how much training occurred or which model was trained [S-0037] [S-1412]. - Ansari's April 2026 feasibility taxonomy noted that power monitoring alone cannot separate AI training from other high-performance computing without other signals, and that no published study had quantified false-positive rates for power-based training detection [S-0007]. - Power draw depends on more than workload type: Gregersen et al. found that changing only the input data can alter GPU power in matrix multiplication by nearly 40% [S-0042]. Measured draw can also differ from rated power: Latif et al. recorded a peak of about 8.4 kW on an 8-GPU H100 node during training, 18% below its rated 10.2 kW, with GPUs near full utilization [S-0041]. The signals can also reveal more than the workload class. Gao et al. recovered the architectures of models running on general-purpose processors from software-read power measurements, with 99.75% accuracy by their Levenshtein-distance measure [S-0046]. Threat model Rahman and Tajdari consider two evaders [S-0037]: - A cloud tenant who controls only user-space software. - A hardware operator who controls the full stack and wants to hide unauthorized training from a regulator. Their results depend on a tamper-resistant NVML read path and an authenticated telemetry channel, which they treat as minimum hardware-enabled guarantees (On-chip telemetry from timing, memory and performance counters) [S-0037]. In their fifth round they give the evader white-box access to the classifier, which they describe as a disaster scenario for governance [S-0037]. Gargiulo and Kulp treat the operator as adversarial and able to reshape the computation, but not able to control the external probe [S-1412]. Evidence - Software telemetry. Rahman and Tajdari ran five rounds of monitor–evader iteration over 20 evasion strategy families on 9 GPU models spanning four architecture generations [S-0037]. They report 98.2% accuracy across their corpus [S-0037]. After hardening, strategies seen in training were always detected, but white-box strategies held out from training were detected only 43–87% of the time [S-0037]. - External power. Gargiulo and Kulp report 97% accuracy on held-out model families from a probe on one H200 NVL [S-1412]. Their hardened detector caught three of four held-out evasion strategies at least 99% of the time, and diluted LoRA 48–88% of the time, or at least 98% with an added rule [S-1412]. They report releasing their 1,610 recordings as a dataset, and state that their attacks are not a comprehensive adversarial evaluation [S-1412]. - Earlier assessment. Before these papers, Ansari rated workload classification as deployable in cooperative settings and near-term in adversarial settings [S-0007]. Limitations - Forged or replayed telemetry. A privileged operator can forge or replay software-read counters unless hardware protects the read path [S-0037] [S-1412]. - Unseen evasion strategies. Detection falls to 43–87% for held-out white-box strategies on NVML telemetry [S-0037], and to 48–88% for diluted LoRA on external power before an added rule [S-1412]. In Gargiulo and Kulp's tests the evasion strategies cost 28.1–69.3% of throughput [S-1412]. - Scale. Results are single-node and NVIDIA-only [S-0037], or from one H200 [S-1412]. - Power sampling. Yang and colleagues found that on A100 and H100 GPUs the built-in power reading, which nvidia-smi obtains through NVML, samples only 25% of runtime. The GPU can draw very different power in the other 75% without the reading showing it [S-0073]. They also found the reading's error to be within about ±5% in most cases, against the ±5 W that NVIDIA claims [S-0073]. - Distributed training. Ansari reports that classification accuracy degrades when training is distributed across providers [S-0007]. Seferis and Fist propose that cloud providers classify each workload, then add up the compute of one customer's workloads that run in sequence or exchange data above a bandwidth threshold, and flag totals above a regulatory threshold [S-0059]. Their proposal is an analysis without empirical tests, and they note that it fails if decentralized training becomes possible with very little data exchange [S-0059]. - Blurring categories. The training-inference distinction may lose governance value [S-0007]. - Spoofed power monitoring. Amodo expects that power monitoring can likely be spoofed, but still sees value in it as a complement to other verification systems [S-0048].", "referenced_by": [ { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "K-0025", "title": "Inference and training workloads", "url": "https://trustbutveri.fyi/concepts/inference-and-training-workloads/" }, { "id": "K-0013", "title": "Side channel", "url": "https://trustbutveri.fyi/concepts/side-channel/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" }, { "id": "O-0203", "title": "Verifiable Compute Foundation", "url": "https://trustbutveri.fyi/organizations/verifiable-compute-foundation/" } ] }, { "id": "M-0004", "slug": "zk-proofs-of-inference", "title": "Zero-knowledge proofs of inference", "aliases": [ "ZKML inference proofs", "verifiable inference with zkSNARKs" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "A prover produces a cryptographic proof that an output came from running a committed model on a given input, without revealing the weights.", "summary": "Zero-knowledge proofs of inference let an AI developer show that a committed model computed an output from a given input, without disclosing the weights. A verifier checks a small proof in seconds. The peer-reviewed system zkLLM proved one 2,048-token forward pass of a 13-billion-parameter model in about 13 minutes on one A100 GPU. Its zero-knowledge guarantee assumes a verifier that follows the protocol, and its public research code is unaudited. A company, Attestable, reports proving a 31-billion-parameter model at 53 tokens per second for one 16,000-token sequence on one H100, without a paper or code. The main obstacle is cost. The main weaknesses are coverage and fidelity: a proof covers only the computation proven, and that computation is a fixed-point approximation of the model. Proofs also do not show how much computation produced an output. An independent audit of one proving library, ezkl, found high-severity soundness bugs, since fixed.", "technical": "Systems encode a network as an arithmetic circuit, or as sumcheck and lookup relations, over a finite field, with tensors scaled to fixed point [[S-0021]] [[S-0023]].\n\n- **ZKML (EuroSys 2024)** compiles models to halo2 circuits with KZG commitments (trusted setup) or IPA commitments (transparent), and optimises circuit layout. A distilled GPT-2 (81.3M parameters) took 3,651.67 s to prove with KZG, 18.70 s to verify and a 28,128-byte proof on a 128-vCPU, 1 TB machine [[S-0021]].\n- **ezkl** also targets halo2 with KZG. South et al. proved a 250,000-parameter nanoGPT in 2,781 s, verified it in 2.69 s and needed a 219 GB proving key [[S-0024]]. A Trail of Bits audit of ezkl (January 2025, commit bdcba5c) reported 34 findings, 8 of high severity; three high-severity findings were circuit soundness issues, and all high-severity findings were resolved at the March 2025 fix review [[S-0070]].\n- **zkLLM (CCS 2024)** uses sumcheck-based arguments, a parallel lookup argument (tlookup) for non-arithmetic tensor operations, an attention protocol (zkAttn) and Hyrax commitments over BLS12-381, with values scaled by 2^16. On one A100 40 GB GPU with 2,048-token inputs, LLaMa-2-13B needed 986 s for the one-time weight commitment, 803 s to prove, 188 kB of proof, 3.95 s to verify and 23.1 GB of memory. Perplexity on C4 moved from 6.520 to 6.528 [[S-0023]].\n- **zkGPT (USENIX Security 2025)** combines the GKR protocol, Lasso lookups and Hyrax commitments, and is made non-interactive with Fiat–Shamir. On a 16-core Xeon server with 200 GB of memory, it proved GPT-2 inference in 21.8 s with 32 threads, with a 101 KB proof verified in 0.35 s. Its code is archived on Zenodo [[S-3060]].\n- **NanoZK (ICICS 2026)** proves each transformer layer separately with Halo2/IPA over the Pallas curve, uses 16-bit lookup tables for softmax, GELU and normalisation, and chains layer boundaries with SHA-256 commitments. It reports 3.2 to 3.7 KB per sub-circuit (attention or MLP) proof, about 83 KB in total for 12 layers. It measured full-block proofs on CPU up to width 128 and, assuming a GPU speedup, projects about 68 s per block at GPT-2 width, or about 14 minutes to prove a 12-layer GPT-2 sequentially [[S-0068]].\n- **Attestable** reports a prover whose security rests only on hash functions, at 100-bit security, with 8-bit integer matrix multiplications. On one H100 it reports 4.35 to 7.92 MiB proofs, 157 to 648 ms CPU verification, and 53 tokens per second for one 16K-token sequence of a 31B Gemma model [[S-1101]].", "category": "cryptographic-computational", "secondary_categories": [], "verifies": [ { "claim": "C-0005", "role": "primary", "note": "Binds each proven output to committed weights and a public architecture." }, { "claim": "C-0004", "role": "supporting", "note": "Attestable proposes using proofs to show accounted workloads used an approved, unchanged model." }, { "claim": "C-0006", "role": "supporting", "note": "Attestable proposes that a proof could show an agreed input classifier was applied; South et al. prove evaluation results." } ], "threat_model": "adversarial", "adversarial_evaluation": "independent-red-team", "hardware_requirement": "none", "prover_cooperation": "required", "confidentiality": "partial", "depends_on": [], "readiness": { "assessment": true, "level": "R2", "scope": "proving each output came from committed weights, against a prover who cheats", "rubric_version": "1.1", "rationale": "R2 through [[I-0003|zkLLM]], which has public, artifact-evaluated code and peer-reviewed results at 13 billion parameters; no implementation is yet production-grade for language models.\n\n- **R1** met: ZKML, zkLLM and NanoZK describe the claim (an output equals the committed model applied to an input) and their assumptions [[S-0021]] [[S-0023]] [[S-0068]].\n- **R2** met through zkLLM. Its code is public and received CCS 2024 artifact-evaluation badges [[S-1108]]. Its published end-to-end results use a 13-billion-parameter model on a data-centre GPU, against a stated adversary: a cheating polynomial-time prover [[S-0023]].\n- **R3** not met for this use. zkLLM's README says the code is not ready for industrial applications [[S-1108]]. Attestable's results come without public code, paper or reproducible artifacts [[S-1101]]. The ezkl library is public, and Trail of Bits reports that other projects use its verifier contracts in production [[S-0070]]. The language model South et al. prove with ezkl is a 250,000-parameter nanoGPT, and the largest model in their results is a VAE decoder of about 1.07 million parameters [[S-0024]]. Both are far below the scale this use concerns. Lagrange calls DeepProve \"production-ready\" [[S-1807]], but its repository has no releases, and its reported benchmarks are GPT-2 and Gemma 3 at 512 tokens [[S-1808]]. No source documents a party other than a developer relying on any of these systems for a verification decision about a language model.\n- **R4** not met. Only ezkl has an independent audit: it left no high-severity finding unresolved [[S-0070]]. As of September 2026 no independent audit or red-team of zkLLM has been published. One independent analysis, tested on a small transformer, shows that valid proofs of LLM inference do not bind the computation spent, so a much smaller model can pass as the declared one [[S-1112]].", "evidence": [ "S-0023", "S-1108", "S-0021", "S-0068", "S-1101", "S-0024", "S-0070", "S-1807", "S-1808", "S-1112" ], "next_level_gaps": [ "An implementation that is production-grade and available for language models, or relied on by a party other than its developer for a verification decision.", "An independent public security evaluation (audit, red-team or peer-reviewed analysis) of a proof system that handles language models." ], "confidence": "medium", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "The proof covers a fixed-point approximation, not the floating-point model", "kind": "open-question", "severity": "significant", "status": "open", "description": "Current ZK inference systems prove a quantised version of the network. zkLLM scales values by 2^16 and reports small perplexity changes [[S-0023]]. Attestable reports quantising matrix multiplications to 8-bit integers while proving other operations in floating point [[S-1101]]. A verifier therefore learns about the proof-friendly variant, and must separately accept that this variant is the declared model. Trail of Bits built a ResNet-18 backdoor that is dormant in the full-precision model and active after ezkl's quantisation; whether it persists through proving was left for further investigation [[S-0070]]. A verification system design calls floating-point emulation in ZKPs an open problem [[S-0018]].", "sources": [ "S-0023", "S-1101", "S-0070", "S-0018" ], "response": null }, { "assessment": true, "title": "A proof speaks only for the computations that were proven", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Attestable writes that \"a proof of some computation is not a proof of all computation\", and that a proof cannot discover a datacenter that was never declared [[S-1102]]. Proofs of inference do not by themselves show that no other workload ran on the same or other hardware.", "sources": [ "S-1102" ], "response": null }, { "assessment": true, "title": "The model architecture is disclosed", "kind": "theoretical-argument", "severity": "minor", "status": "open", "description": "ZKML \"requires that the model architecture (but not weights) is revealed\" [[S-0021]], and zkLLM assumes a publicly known model structure [[S-0023]]. Architecture can be commercially sensitive.", "sources": [ "S-0021", "S-0023" ], "response": null }, { "assessment": true, "title": "Proofs do not bind computational effort (Hollow-LLM)", "kind": "demonstrated-attack", "severity": "significant", "status": "open", "description": "Researchers at the University of Southern California show that a proof of inference certifies that an output is consistent with committed weights under the declared architecture, but not how much computation produced it [[S-1112]]. In their Hollow-LLM attack, a provider keeps the declared architecture and parameter count but commits to \"ghost weights\". Some layers pass their inputs through unchanged, and wide layers carry the signal in a small subspace, so a much smaller inner model does the real work. The ghost weights satisfy the verification circuit and yield valid proofs [[S-1112]].\n\nThe authors ran the attack with the proof procedure of zkGPT, a separate ZK inference system, on a 6-layer, 512-dimensional transformer declared as up to 12 layers and 1,024 dimensions. Outputs were identical to the inner model's, and serving cost stayed at the inner model's level. An honest model of the declared size cost 2.4 times as much to prefill and 3.1 times as much to decode. Proving cost still grew with the declared architecture [[S-1112]].\n\nThe authors note that results may be served before any proof, with the provider building the witness only when a call is selected for audit. They describe their constructions as \"compatible with state-of-the-art zkLLM pipelines\", and state that the attack does not imply a flaw in the proof system itself. They propose challenge-based audits and ablation tests, which raise the cost of cheating but give no guarantee [[S-1112]].", "sources": [ "S-1112" ], "response": null } ], "blockers": [ { "text": "Proving takes about 12 to 13 minutes per 2,048-token forward pass of a 13B model on one A100 [[S-0023]], and a verification system design calls the overhead heavy [[S-0018]].", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-0023", "S-0018" ] }, { "text": "ZKML and zkLLM prove fixed-point arithmetic [[S-0021]] [[S-0023]], and floating-point emulation in ZKPs is described as an open problem [[S-0018]].", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-0021", "S-0023", "S-0018" ] }, { "text": "zkLLM's code is unaudited, interactive and archived [[S-1108]]; the one audited ZK inference library, ezkl, had high-severity circuit soundness bugs before its fixes [[S-0070]].", "theme": "adversarial-validation", "blocked_by": null, "sources": [ "S-1108", "S-0070" ] }, { "text": "Showing that proven inference was the only work done needs a compute-accounting mechanism such as proof-of-work accounting, which is only proposed [[S-1102]].", "theme": "coverage-hidden-compute", "blocked_by": "M-0007", "sources": [ "S-1102" ] } ], "challenge_themes": [ "performance-compatibility", "coverage-hidden-compute", "evidence-binding", "adversarial-validation", "privacy-leakage" ], "organizations": [ "O-0120", "O-0122" ], "people": [], "sources": [ { "source": "S-0023", "supports": "zkLLM design, threat model, security theorems, overheads, fixed-point effects", "locator": "abstract; §3.6; §4–5; §7.2 Theorems 7.3–7.4; §8 Table 1; §9" }, { "source": "S-1108", "supports": "zkLLM code availability, artifact badges and README caveats", "locator": "README; Zenodo record" }, { "source": "S-0021", "supports": "ZKML design, halo2 backends, GPT-2 overheads, limitations", "locator": "§3; §4.1; §4.4; §9 Tables 5–7" }, { "source": "S-0068", "supports": "NanoZK layerwise proofs, threat model, proof sizes, partial audits", "locator": "Definition 1; §5; §6; Table 8; App. A.4" }, { "source": "S-0024", "supports": "verifiable evaluation attestations with ezkl; public inputs and outputs; costs of small models", "locator": "abstract; §5; §6.1 Table 1" }, { "source": "S-0070", "supports": "independent audit of ezkl: circuit soundness findings, quantisation-activated backdoor, production use, fix review", "locator": "Executive Summary; findings TOB-EZKL-4 to 6 and 17; App. D" }, { "source": "S-1100", "supports": "definition and categorisation of ZKML; main implementation bottlenecks", "locator": "abstract; §III; Table VI" }, { "source": "S-1101", "supports": "Attestable's reported prover, statement proven, performance and limits (provider-reported)" }, { "source": "S-1102", "supports": "Attestable's coverage argument and pacing proposal (provider-reported)" }, { "source": "S-1103", "supports": "Attestable's proposal to prove randomly sampled outputs (provider-reported)" }, { "source": "S-0018", "supports": "ZKPs as a 'tentative plan B' in a verification system; overhead assessment; floating-point gap", "locator": "§5.2.4" }, { "source": "S-1112", "supports": "independent Hollow-LLM analysis: proofs do not bind computational effort; ghost-weight constructions; zkGPT-based experiment and cost results; countermeasures", "locator": "Abstract; §I contributions; §V Table 2; §VI" }, { "source": "S-3060", "supports": "zkGPT design, non-interactive proofs, GPT-2 proving and verification figures, public code", "locator": "abstract; §3; §6 Table 3" }, { "source": "S-1807", "supports": "Lagrange's reported proof of full GPT-2 inference and 'production-ready' description (provider-reported)" }, { "source": "S-1808", "supports": "DeepProve public code, licence and reported GPT-2 proving figures (provider-reported)", "locator": "README" } ], "concepts": [ "K-0010", "K-0024", "K-0001", "K-0002", "K-0020", "K-0016", "K-0018" ], "complements": [ "M-0007" ], "alternatives": [ "M-0001", "M-0008" ], "type": "mechanism", "implementations": [ { "id": "I-0005", "title": "Attestable zero-knowledge inference prover", "url": "https://trustbutveri.fyi/implementations/attestable-zk-inference/" }, { "id": "I-0014", "title": "EZKL", "url": "https://trustbutveri.fyi/implementations/ezkl/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "I-0003", "title": "zkLLM", "url": "https://trustbutveri.fyi/implementations/zkllm/" } ], "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/", "source_file": "content/mechanisms/zk-proofs-of-inference.md", "flags_all": [], "body_markdown": "## How it works\n\nA zero-knowledge proof of inference lets a [[K-0001|prover]] convince a [[K-0002|verifier]] that an output came from a specific model on a specific input, without revealing the model's weights [[S-1100]]. A survey describes ZKPs as letting one party certify that a training, testing or inference result \"was produced by the claimed computation without revealing sensitive data or proprietary model parameters\" [[S-1100]].\n\nThe protocol has two steps:\n1. The prover publishes a [[K-0024|cryptographic commitment]] to the weights once. For LLaMa-2-13B, zkLLM's commitment took 986 seconds [[S-0023]].\n2. For each query, the prover proves that the output equals the committed model applied to the input [[S-0023]] [[S-0021]]. The verifier checks the proof without the weights and without rerunning the model [[S-0023]].\n\nThe weights are hidden, and the input and output are known to the verifier. In zkLLM's setting the verifier submits the prompt and receives the output with its proof [[S-0023]]. South et al. likewise keep weights private and inputs and outputs public [[S-0024]].\n\nSystems turn the network's operations into arithmetic over a finite field, with tensors as fixed-point numbers [[S-0021]] [[S-0023]]. Softmax, GELU and normalisation need special handling:\n- zkLLM uses a parallel lookup argument (tlookup) and an attention-specific protocol (zkAttn) [[S-0023]].\n- NanoZK uses 16-bit lookup tables [[S-0068]].\n\nProof systems differ:\n- ZKML compiles models to halo2 circuits with either KZG commitments, which need a trusted setup, or transparent IPA commitments [[S-0021]].\n- zkLLM uses sumcheck-based protocols with Hyrax commitments [[S-0023]].\n- zkGPT uses the GKR protocol with Lasso lookups and Hyrax commitments, made non-interactive with Fiat–Shamir [[S-3060]].\n- NanoZK proves each transformer layer separately and links the layers with a chain of SHA-256 commitments [[S-0068]].\n- Attestable reports a prover whose security rests \"only on the security of cryptographic hash functions\" [[S-1101]].\n\nProofs over many inputs can be packaged into attestations that a model with fixed private weights reaches a stated benchmark score [[S-0024]].\n\n## What it establishes\n**What it can establish.**\n- A valid proof shows that the output follows from the committed weights, the public architecture and the input [[S-0023]] [[S-0021]]. This bears directly on [[C-0005]].\n- The weights stay confidential against a verifier that follows the protocol. zkLLM's zero-knowledge theorem shows that such a verifier's view reveals nothing about the weights beyond the output [[S-0023]].\n- Attestable proposes proofs that an accounted workload \"used an approved, unchanged model rather than a newly trained replacement\", which bears on [[C-0004]] [[S-1102]].\n\n**What it cannot establish.**\n- **Which model the commitment is.** The proof does not tie the committed weights to an outside identity such as \"the declared model\". Proven benchmark results over the committed weights are one way to link them [[S-0024]]. Committed weights can also be hollow. Weights with the declared architecture and parameter count can route the work through a much smaller model and still yield valid proofs [[S-1112]].\n- **Work that was not proven.** Attestable notes that \"a proof of some computation is not a proof of all computation\", and that a proof cannot find an undeclared datacenter [[S-1102]].\n- **Every output, when only some are proven.** Assurance from sampling is probabilistic. Attestable proposes selecting outputs at random for proof after they have been committed [[S-1103]]. NanoZK bounds the chance of detecting tampering when auditors check a subset of layers [[S-0068]].\n- **The floating-point model.** What is proven is a fixed-point approximation of the model [[S-0023]]. Attestable reports 8-bit integer matrix multiplications [[S-1101]].\n- **A hidden architecture.** ZKML requires the architecture to be revealed [[S-0021]], and zkLLM assumes it is publicly known [[S-0023]].\n\n## Threat model\nSoundness rests on computational assumptions against a cheating prover:\n- zkLLM proves its lookup argument sound: a cheating probabilistic polynomial-time prover succeeds only with negligible probability. The rest of its protocol applies sumcheck and proofs of opening for commitments whose binding rests on discrete-log hardness [[S-0023]].\n- NanoZK assumes a polynomial-time adversary that controls the inference infrastructure and may substitute models or fabricate outputs. It relies on discrete-log hardness over the Pallas curve and SHA-256 collision resistance [[S-0068]].\n- Attestable reports 100-bit security, and post-quantum security because it avoids public-key cryptography [[S-1101]].\n\nConfidentiality rests on further assumptions:\n- zkLLM assumes a semi-honest verifier, which \"accurately reports the outcome of the proof verification\" but tries to learn the weights. Its zero-knowledge theorem also assumes zero-knowledge variants of the sumcheck protocols [[S-0023]].\n- NanoZK leaves side channels, denial of service and hiding the input from the provider out of scope [[S-0068]].\n- Attestable treats physical-access and metadata channels as separate threats [[S-1103]].\n\n## Evidence\n- **ZKML.** It proved a distilled 81.3-million-parameter GPT-2 in 3,651.67 seconds on a 128-vCPU, 1 TB machine. This was the largest model its authors could prove with under 1 TB of RAM [[S-0021]].\n- **South et al.** They used [[I-0014|ezkl]] to prove evaluations of small models. A 250,000-parameter nanoGPT took 2,781 seconds to prove and needed a 219 GB proving key [[S-0024]].\n- **[[I-0003|zkLLM]].** It proved OPT models up to 13B and LLaMa-2 models of 7B and 13B, each proof covering one 2,048-token forward pass on one A100 GPU. LLaMa-2-13B took 803 seconds to prove and 3.95 seconds to verify, with a 188 kB proof [[S-0023]]. Its code is public [[S-1108]].\n- **zkGPT.** It proved GPT-2 inference in 21.8 seconds with 32 threads on a 16-core CPU server, with a 101 KB non-interactive proof that verifies in 0.35 seconds. Its code is public [[S-3060]].\n- **NanoZK.** It reports proofs of 3.2 to 3.7 KB per attention or MLP sub-circuit, about 83 KB in total for 12 layers. It measured full transformer blocks on CPU only up to width 128. Assuming a GPU speedup, it projects that a 12-layer GPT-2 would take about 14 minutes to prove sequentially [[S-0068]].\n- **[[I-0005|Attestable]].** It reports proving a 31-billion-parameter Gemma model at 53 tokens per second for one 16K-token sequence on one H100 GPU, with 4.35 to 7.92 MiB proofs and 157 to 648 ms CPU verification. No code or paper accompanies these results [[S-1101]].\n- **Lagrange DeepProve.** Lagrange reports a zero-knowledge proof of full GPT-2 inference [[S-1807]]. Its public repository, under Lagrange's own licence, reports proving 512 tokens of GPT-2 in 7.6 minutes on a 24-core CPU server with 504 GB of memory, with a 10.7 MiB proof that verifies in 1.3 seconds [[S-1808]].\n- **Independent audit of ezkl.** Trail of Bits reviewed the ezkl library in January 2025, at its developer's request. It reported three high-severity circuit soundness issues and four ways to bypass data attestation in ezkl's smart contracts. Its fix review found every high-severity finding resolved [[S-0070]].\n- **Hollow-LLM attack.** Researchers at the University of Southern California ran a ghost-weight attack through zkGPT's proof procedure on a 6-layer, 512-dimensional transformer declared as up to 12 layers and 1,024 dimensions. Outputs were unchanged and serving cost stayed at the small model's level [[S-1112]].\n\nA verification system design for AI agreements lists ZKPs as a \"tentative plan B\" that, if they mature, could \"remove the need for secure computing hardware setups\" [[S-0018]].\n\n## Limitations\n**Cost.**\n- zkLLM needs about 12 to 13 minutes per 2,048-token forward pass at 13B scale on one A100 [[S-0023]].\n- The verification system design calls the overhead \"heavy\" [[S-0018]].\n- The survey names \"limited circuit expressiveness, high proving cost, and deployment complexity\" as the main implementation bottlenecks [[S-1100]].\n\n**Expressiveness.** ZKML does not support branching or variable-length loops, so language models need fixed-length inputs [[S-0021]]. Floating-point emulation remains open [[S-0018]]. Attestable reports a 16K-token context limit [[S-1101]].\n\n**Implementation soundness.** In ezkl, Trail of Bits found circuits with missing constraints that \"would allow a malicious prover to convince a verifier of incorrect calculations\"; these were fixed [[S-0070]]. The zkLLM README says its code \"has NOT undergone security auditing and is NOT ready for industrial applications\". It also says prover and verifier run side by side, and that a deployment would need Fiat–Shamir to be non-interactive [[S-1108]].\n\n**Quantisation.** Trail of Bits also built a ResNet-18 backdoor that is dormant in the full-precision model and active after ezkl's quantisation. Whether it persists through proving was left for further investigation [[S-0070]].\n\n**Coverage.** A proof covers only the outputs proven [[S-1102]]. For accounting of other work, see [[M-0007]]. A proof also does not show how much computation produced an output. In the Hollow-LLM attack, weights with the declared architecture let a much smaller model do the work, and the proofs remain valid [[S-1112]].", "body_text": "How it works A zero-knowledge proof of inference lets a prover convince a verifier that an output came from a specific model on a specific input, without revealing the model's weights [S-1100]. A survey describes ZKPs as letting one party certify that a training, testing or inference result \"was produced by the claimed computation without revealing sensitive data or proprietary model parameters\" [S-1100]. The protocol has two steps: 1. The prover publishes a cryptographic commitment to the weights once. For LLaMa-2-13B, zkLLM's commitment took 986 seconds [S-0023]. 2. For each query, the prover proves that the output equals the committed model applied to the input [S-0023] [S-0021]. The verifier checks the proof without the weights and without rerunning the model [S-0023]. The weights are hidden, and the input and output are known to the verifier. In zkLLM's setting the verifier submits the prompt and receives the output with its proof [S-0023]. South et al. likewise keep weights private and inputs and outputs public [S-0024]. Systems turn the network's operations into arithmetic over a finite field, with tensors as fixed-point numbers [S-0021] [S-0023]. Softmax, GELU and normalisation need special handling: - zkLLM uses a parallel lookup argument (tlookup) and an attention-specific protocol (zkAttn) [S-0023]. - NanoZK uses 16-bit lookup tables [S-0068]. Proof systems differ: - ZKML compiles models to halo2 circuits with either KZG commitments, which need a trusted setup, or transparent IPA commitments [S-0021]. - zkLLM uses sumcheck-based protocols with Hyrax commitments [S-0023]. - zkGPT uses the GKR protocol with Lasso lookups and Hyrax commitments, made non-interactive with Fiat–Shamir [S-3060]. - NanoZK proves each transformer layer separately and links the layers with a chain of SHA-256 commitments [S-0068]. - Attestable reports a prover whose security rests \"only on the security of cryptographic hash functions\" [S-1101]. Proofs over many inputs can be packaged into attestations that a model with fixed private weights reaches a stated benchmark score [S-0024]. What it establishes What it can establish. - A valid proof shows that the output follows from the committed weights, the public architecture and the input [S-0023] [S-0021]. This bears directly on The declared model is the one being served. - The weights stay confidential against a verifier that follows the protocol. zkLLM's zero-knowledge theorem shows that such a verifier's view reveals nothing about the weights beyond the output [S-0023]. - Attestable proposes proofs that an accounted workload \"used an approved, unchanged model rather than a newly trained replacement\", which bears on This compute runs inference, not training [S-1102]. What it cannot establish. - Which model the commitment is. The proof does not tie the committed weights to an outside identity such as \"the declared model\". Proven benchmark results over the committed weights are one way to link them [S-0024]. Committed weights can also be hollow. Weights with the declared architecture and parameter count can route the work through a much smaller model and still yield valid proofs [S-1112]. - Work that was not proven. Attestable notes that \"a proof of some computation is not a proof of all computation\", and that a proof cannot find an undeclared datacenter [S-1102]. - Every output, when only some are proven. Assurance from sampling is probabilistic. Attestable proposes selecting outputs at random for proof after they have been committed [S-1103]. NanoZK bounds the chance of detecting tampering when auditors check a subset of layers [S-0068]. - The floating-point model. What is proven is a fixed-point approximation of the model [S-0023]. Attestable reports 8-bit integer matrix multiplications [S-1101]. - A hidden architecture. ZKML requires the architecture to be revealed [S-0021], and zkLLM assumes it is publicly known [S-0023]. Threat model Soundness rests on computational assumptions against a cheating prover: - zkLLM proves its lookup argument sound: a cheating probabilistic polynomial-time prover succeeds only with negligible probability. The rest of its protocol applies sumcheck and proofs of opening for commitments whose binding rests on discrete-log hardness [S-0023]. - NanoZK assumes a polynomial-time adversary that controls the inference infrastructure and may substitute models or fabricate outputs. It relies on discrete-log hardness over the Pallas curve and SHA-256 collision resistance [S-0068]. - Attestable reports 100-bit security, and post-quantum security because it avoids public-key cryptography [S-1101]. Confidentiality rests on further assumptions: - zkLLM assumes a semi-honest verifier, which \"accurately reports the outcome of the proof verification\" but tries to learn the weights. Its zero-knowledge theorem also assumes zero-knowledge variants of the sumcheck protocols [S-0023]. - NanoZK leaves side channels, denial of service and hiding the input from the provider out of scope [S-0068]. - Attestable treats physical-access and metadata channels as separate threats [S-1103]. Evidence - ZKML. It proved a distilled 81.3-million-parameter GPT-2 in 3,651.67 seconds on a 128-vCPU, 1 TB machine. This was the largest model its authors could prove with under 1 TB of RAM [S-0021]. - South et al. They used ezkl to prove evaluations of small models. A 250,000-parameter nanoGPT took 2,781 seconds to prove and needed a 219 GB proving key [S-0024]. - zkLLM. It proved OPT models up to 13B and LLaMa-2 models of 7B and 13B, each proof covering one 2,048-token forward pass on one A100 GPU. LLaMa-2-13B took 803 seconds to prove and 3.95 seconds to verify, with a 188 kB proof [S-0023]. Its code is public [S-1108]. - zkGPT. It proved GPT-2 inference in 21.8 seconds with 32 threads on a 16-core CPU server, with a 101 KB non-interactive proof that verifies in 0.35 seconds. Its code is public [S-3060]. - NanoZK. It reports proofs of 3.2 to 3.7 KB per attention or MLP sub-circuit, about 83 KB in total for 12 layers. It measured full transformer blocks on CPU only up to width 128. Assuming a GPU speedup, it projects that a 12-layer GPT-2 would take about 14 minutes to prove sequentially [S-0068]. - Attestable. It reports proving a 31-billion-parameter Gemma model at 53 tokens per second for one 16K-token sequence on one H100 GPU, with 4.35 to 7.92 MiB proofs and 157 to 648 ms CPU verification. No code or paper accompanies these results [S-1101]. - Lagrange DeepProve. Lagrange reports a zero-knowledge proof of full GPT-2 inference [S-1807]. Its public repository, under Lagrange's own licence, reports proving 512 tokens of GPT-2 in 7.6 minutes on a 24-core CPU server with 504 GB of memory, with a 10.7 MiB proof that verifies in 1.3 seconds [S-1808]. - Independent audit of ezkl. Trail of Bits reviewed the ezkl library in January 2025, at its developer's request. It reported three high-severity circuit soundness issues and four ways to bypass data attestation in ezkl's smart contracts. Its fix review found every high-severity finding resolved [S-0070]. - Hollow-LLM attack. Researchers at the University of Southern California ran a ghost-weight attack through zkGPT's proof procedure on a 6-layer, 512-dimensional transformer declared as up to 12 layers and 1,024 dimensions. Outputs were unchanged and serving cost stayed at the small model's level [S-1112]. A verification system design for AI agreements lists ZKPs as a \"tentative plan B\" that, if they mature, could \"remove the need for secure computing hardware setups\" [S-0018]. Limitations Cost. - zkLLM needs about 12 to 13 minutes per 2,048-token forward pass at 13B scale on one A100 [S-0023]. - The verification system design calls the overhead \"heavy\" [S-0018]. - The survey names \"limited circuit expressiveness, high proving cost, and deployment complexity\" as the main implementation bottlenecks [S-1100]. Expressiveness. ZKML does not support branching or variable-length loops, so language models need fixed-length inputs [S-0021]. Floating-point emulation remains open [S-0018]. Attestable reports a 16K-token context limit [S-1101]. Implementation soundness. In ezkl, Trail of Bits found circuits with missing constraints that \"would allow a malicious prover to convince a verifier of incorrect calculations\"; these were fixed [S-0070]. The zkLLM README says its code \"has NOT undergone security auditing and is NOT ready for industrial applications\". It also says prover and verifier run side by side, and that a deployment would need Fiat–Shamir to be non-interactive [S-1108]. Quantisation. Trail of Bits also built a ResNet-18 backdoor that is dormant in the full-precision model and active after ezkl's quantisation. Whether it persists through proving was left for further investigation [S-0070]. Coverage. A proof covers only the outputs proven [S-1102]. For accounting of other work, see Proofs of useful work for capacity accounting. A proof also does not show how much computation produced an output. In the Hollow-LLM attack, weights with the declared architecture let a much smaller model do the work, and the proofs remain valid [S-1112].", "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0005", "title": "Attestable zero-knowledge inference prover", "url": "https://trustbutveri.fyi/implementations/attestable-zk-inference/" }, { "id": "I-0014", "title": "EZKL", "url": "https://trustbutveri.fyi/implementations/ezkl/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "I-0003", "title": "zkLLM", "url": "https://trustbutveri.fyi/implementations/zkllm/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "K-0024", "title": "Cryptographic commitment", "url": "https://trustbutveri.fyi/concepts/cryptographic-commitment/" }, { "id": "K-0001", "title": "Prover", "url": "https://trustbutveri.fyi/concepts/prover/" }, { "id": "K-0010", "title": "Zero-knowledge proof", "url": "https://trustbutveri.fyi/concepts/zero-knowledge-proof/" }, { "id": "O-0120", "title": "Attestable", "url": "https://trustbutveri.fyi/organizations/attestable/" }, { "id": "O-0122", "title": "University of Waterloo", "url": "https://trustbutveri.fyi/organizations/university-of-waterloo/" }, { "id": "O-0213", "title": "Zkonduit", "url": "https://trustbutveri.fyi/organizations/zkonduit/" } ] }, { "id": "M-0005", "slug": "zk-proofs-of-training-constraints", "title": "Zero-knowledge proofs of training constraints", "aliases": [ "zero-knowledge proofs of training", "zkPoT", "verifiable training" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Cryptographic proofs that a training run followed a committed dataset, procedure and rules, checkable without revealing the model or the data.", "summary": "Zero-knowledge proofs of training would let an AI developer prove that a model was trained as declared, on a committed dataset and within agreed rules such as a compute limit, without showing the weights or the data. Peer-reviewed systems have done this for small models. Kaizen proves training iterations of a 10-million-parameter image model. ZkAudit proves single training steps of small image and recommender models, and estimates the cost of full runs. A 2025 preprint proves single fine-tuning steps of 13-billion-parameter language models. A 2026 preprint argues that frontier-scale verification is feasible by proving only randomly challenged training steps, anchored by network observations. It is unbuilt, and its overheads are estimates. The biggest obstacle is cost: each proven training step takes minutes. The biggest known weakness of the frontier design is that sparse challenges give probabilistic detection, and its network anchor cannot see traffic inside a server.", "category": "cryptographic-computational", "secondary_categories": [], "verifies": [ { "claim": "C-0007", "role": "primary", "note": "Proves training followed a committed specification and data; the frontier design adds compute-threshold attestations." } ], "threat_model": "adversarial", "adversarial_evaluation": "analysis", "hardware_requirement": "none", "prover_cooperation": "required", "confidentiality": "partial", "depends_on": [ { "target": "M-0013", "note": "The frontier-scale design by Peigné et al. anchors its commitments with an auditor-controlled network tap or an attested SmartNIC." } ], "readiness": { "assessment": true, "level": "R2", "scope": "proving a training run followed a committed specification and data", "rubric_version": "1.1", "rationale": "Peer-reviewed end-to-end results exist for small models against a stated adversary, and public code proves single fine-tuning steps of 13-billion-parameter language models. The frontier-scale design is unbuilt.\n\n- **R1** met: Kaizen [[S-1110]] and ZkAudit [[S-0022]] define proofs of correct training on committed data. Peigné et al. describe a frontier design with stated claims, trust anchors and open problems [[S-0025]].\n- **R2** met through reproducible published end-to-end results and a public working implementation. Kaizen and ZkAudit are peer-reviewed, specify the protocol, setup and parameters, and state a cheating prover as the adversary. Kaizen measures proving per training iteration of a 10-million-parameter VGG-11, with recursive aggregation implemented [[S-1110]]. ZkAudit proves single SGD steps of MobileNet v2 and recommender models on AWS g4dn.8xlarge instances, and estimates the cost of proving full training runs [[S-0022]]. Kaizen links no code [[S-1110]], and ZkAudit links only an anonymised review repository [[S-0022]]; the rubric does not require code on this route. VeriLoRA, a preprint with public code, proves one LoRA fine-tuning step on a single sample for LLaMA and OPT models of 3 to 13 billion parameters [[S-3080]]. All of these results are far below frontier training.\n- **R3** not met: as of September 2026 no deployment or reliance by a third party has been published. The frontier design is unimplemented, and its authors present its costs as estimates, with target values \"not yet measured\" [[S-0025]].\n- **R4** not met: no independent evaluation has been published.", "evidence": [ "S-1110", "S-0022", "S-3080", "S-0025" ], "next_level_gaps": [ "Any use by a party other than the developer, or a production-grade, available implementation.", "An independent public security evaluation of a proof-of-training system.", "For the frontier use: an implementation of challenge-based step proofs at realistic model and cluster scale." ], "confidence": "medium", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Sparse challenge-based auditing gives probabilistic detection only", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "In the frontier design, proofs are generated only for training steps the auditor challenges after the hash chain is frozen. The authors describe this as \"detection-grade, not universal\": the verifier \"cannot make universal claims about every step\", but can detect deviations with high probability when sampling occurs [[S-0025]].", "sources": [ "S-0025" ], "response": null }, { "assessment": true, "title": "The network anchor misses traffic inside a server", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "The frontier design observes traffic between nodes only, so \"intra-node NVLink is invisible\". Its attested-SmartNIC tier is weaker than a physical tap against firmware or supply-chain adversaries [[S-0025]].", "sources": [ "S-0025" ], "response": null }, { "assessment": true, "title": "Proven training uses fixed-point arithmetic", "kind": "open-question", "severity": "significant", "status": "open", "description": "Kaizen and ZkAudit prove training in fixed point [[S-1110]] [[S-0022]]. ZkAudit reports accuracy 0.5 to 0.7 points below fp32 on three image datasets [[S-0022]]. The frontier design proposes native floating-point precompiles, and lists the algebraic reductions needed to verify floating-point matrix multiplication as an open problem [[S-0025]].", "sources": [ "S-1110", "S-0022", "S-0025" ], "response": null }, { "assessment": true, "title": "A proof binds committed data but does not vet it", "kind": "open-question", "severity": "minor", "status": "open", "description": "ZkAudit notes that it does not protect against data poisoning, and that it reveals the model architecture [[S-0022]]. Whether committed data obeys a rule needs a separate proven audit function [[S-0022]].", "sources": [ "S-0022" ], "response": null } ], "blockers": [ { "text": "Proving costs minutes per training step even for small models: 15 minutes per VGG-11 iteration [[S-1110]] and 47.5 to 328.3 seconds per single-image MobileNet v2 SGD step [[S-0022]].", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-1110", "S-0022" ] }, { "text": "The frontier design is unbuilt and lists 13 open problems, including zero-knowledge proofs of backpropagation and deterministic attention backward passes with low overhead [[S-0025]].", "theme": "protocol-soundness", "blocked_by": null, "sources": [ "S-0025" ] }, { "text": "The frontier design needs deterministic training; current deterministic tensor-parallel all-reduce is reported to lose 64 to 89% of bandwidth [[S-0025]].", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-0025" ] }, { "text": "The frontier design needs an open-hardware network tap at line rate, listed as an open problem [[S-0025]].", "theme": "hardware-trust", "blocked_by": "M-0013", "sources": [ "S-0025" ] }, { "text": "Mixture-of-experts, reinforcement-learning post-training and multi-site training are not yet covered [[S-0025]].", "theme": "coverage-hidden-compute", "blocked_by": null, "sources": [ "S-0025" ] } ], "challenge_themes": [ "performance-compatibility", "protocol-soundness", "hardware-trust", "coverage-hidden-compute", "evidence-binding" ], "organizations": [], "people": [], "sources": [ { "source": "S-1110", "supports": "Kaizen zkPoT definition, techniques, threat model and costs", "locator": "abstract; §1.2; evaluation" }, { "source": "S-0022", "supports": "ZkAudit training proofs, costs, accuracy, limitations, code link", "locator": "abstract; §3; §4–6; evaluation tables; §8" }, { "source": "S-3080", "supports": "VeriLoRA zero-knowledge proofs of single LoRA fine-tuning steps for 3B–13B language models; hardware; costs; public code", "locator": "abstract; evaluation" }, { "source": "S-0025", "supports": "frontier-scale design, trust anchors, overhead estimates, open problems", "locator": "abstract; §3.2; MOD. 1–4; Tables 1–2; App. A; App. G.5" }, { "source": "S-1100", "supports": "categorisation of verifiable training; survey-reported costs of other systems", "locator": "§III-A1; Table IV" }, { "source": "S-0023", "supports": "zkLLM authors' view of extending ZKPs to LLM training", "locator": "§9" } ], "concepts": [ "K-0010", "K-0024", "K-0014", "K-0020", "K-0008", "K-0023" ], "complements": [], "alternatives": [ "M-0006" ], "type": "mechanism", "implementations": [], "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-training-constraints/", "source_file": "content/mechanisms/zk-proofs-of-training-constraints.md", "flags_all": [], "body_markdown": "## How it works\n\nA zero-knowledge proof of training (zkPoT) lets a party prove \"that they have correctly trained a committed model based on a committed dataset\", revealing nothing further about the model or the data [[S-1110]]. A survey calls this verifiable training: certifying that updated parameters result from the declared procedure [[S-1100]].\n\n**Small-scale protocols.** Published systems follow three steps:\n1. The prover commits to the dataset [[S-1110]] [[S-0022]].\n2. It fixes the data order using verified randomness [[S-0022]].\n3. It proves each gradient-descent step: the forward pass, the backward pass and the weight update [[S-0022]].\n\nKaizen adds three techniques [[S-1110]]:\n- sumcheck-based proofs specialised for gradient descent;\n- recursive composition, so that proof size and verifier time do not grow with the number of iterations;\n- aggregatable polynomial commitments.\n\nZkAudit adds proofs of arbitrary audit functions over the committed data and weights [[S-0022]].\n\n**A frontier-scale design.** Peigné et al. propose a design for frontier training with four parts [[S-0025]]:\n- **Commitment.** Before training, the trainer publishes a hashed commitment that combines the architecture specification, a Merkle root of the dataset and the root of the initial weights [[S-0025]].\n- **Hash chain.** During training, the GPUs compute Merkle roots of per-layer tensors at every step and publish them in a hash chain [[S-0025]].\n- **Network anchor.** An auditor-controlled network tap, or an attested SmartNIC, records hashes of traffic between nodes as an independent anchor [[S-0025]] (see [[M-0013]]).\n- **Challenges.** After the chain is frozen, the auditor challenges sampled steps. The trainer re-executes those steps, and a zero-knowledge virtual machine, with native precompiles mostly for floating-point operations, checks them against the committed roots [[S-0025]].\n\nThe design also includes genesis proofs and \"ex-ante\" attestations that enforce policy invariants, such as compute thresholds [[S-0025]].\n\n## What it establishes\n**What it can establish.**\n- A zkPoT shows that the committed weights resulted from the declared training procedure on the committed dataset [[S-1110]] [[S-0022]]. This bears on [[C-0007]].\n- Peigné et al. aim to verify further claims: that training faithfully executed a pre-committed specification, how much compute it consumed, its training regime and data-content filters [[S-0025]].\n\n**What it cannot establish.**\n- **That the data is acceptable.** A proof binds data to a commitment but does not vet it. ZkAudit does not protect against data poisoning [[S-0022]].\n- **Every step, in the frontier design.** Sparse auditing is \"detection-grade, not universal\" [[S-0025]].\n- **Full confidentiality.** ZkAudit reveals the model architecture [[S-0022]].\n\n## Threat model\nThe small-scale systems assume a cheating prover bounded by cryptographic assumptions, and a public training algorithm and architecture [[S-1110]] [[S-0022]]. The frontier design adds trust anchors [[S-0025]]:\n- deterministic GPU execution;\n- a physical network tap, or secure boot and device identity for the SmartNIC tier;\n- the soundness of the zkVM;\n- a public proof-checker binary.\n\nIt does not cover traffic inside a server, and its SmartNIC tier is weaker against supply-chain adversaries [[S-0025]].\n\n## Evidence\n- **Kaizen (CCS 2024).** It proves training of a 10-million-parameter VGG-11 on CIFAR-10 at batch size 16. The prover takes 15 minutes per iteration; the proof is 1.63 MB and verifies in 130 milliseconds, independent of the number of iterations [[S-1110]]. Its authors report \"24× faster prover time\" than generic recursive proof systems [[S-1110]].\n- **ZkAudit (ICML 2024).** It proved single SGD steps for MobileNet v2 image classifiers and a recommender model on AWS g4dn.8xlarge instances. Proving one step on a single image took 47.5 to 328.3 seconds for MobileNet v2 (1.0, 224), depending on the fixed-point scale factor. The authors estimated, rather than generated, proofs of full training runs, at costs of hundreds to thousands of dollars [[S-0022]].\n- **VeriLoRA (2025 preprint).** It proves one LoRA fine-tuning step, covering the forward pass, backward pass and parameter update, on a single sample, for LLaMA and OPT models of 3 to 13 billion parameters on one A100 GPU. Proving a step takes minutes and verifying it takes seconds, and the code is public [[S-3080]].\n- **Other systems.** The survey lists further verifiable-training systems in its Table IV [[S-1100]].\n- **The frontier design.** Peigné et al. estimate 2 to 10% training-side overhead for a Llama 3.1 405B-scale run, and deployment \"within approximately 36 months\" [[S-0025]]. The paper reports no prototype or measurements of its own, and marks its target values as \"not yet measured\" [[S-0025]].\n\nThe zkLLM authors wrote in 2024 that zero-knowledge proofs of LLM training \"may pose insurmountable challenges\" [[S-0023]].\n\n## Limitations\n**Cost.** A 10-million-parameter model needs minutes of proving per step [[S-1110]]. ZkAudit's authors leave scaling to larger models, such as language models, to future work [[S-0022]]. VeriLoRA reaches 13-billion-parameter language models, but for single-sample steps that update only low-rank adapters [[S-3080]].\n\n**Open problems in the frontier design.** Peigné et al. list 13, including [[S-0025]]:\n- zero-knowledge proofs of backpropagation;\n- deterministic attention backward passes with under 5% overhead;\n- an open-hardware network tap at line rate;\n- a way to tell silent data corruption apart from adversarial deviation;\n- coverage of mixture-of-experts, reinforcement-learning post-training and multi-site training.\n\nThey also report that current deterministic tensor-parallel all-reduce configurations lose 64 to 89% of bandwidth [[S-0025]].\n\n**Attacks.** As of September 2026 no attack on these proof systems has been published.", "body_text": "How it works A zero-knowledge proof of training (zkPoT) lets a party prove \"that they have correctly trained a committed model based on a committed dataset\", revealing nothing further about the model or the data [S-1110]. A survey calls this verifiable training: certifying that updated parameters result from the declared procedure [S-1100]. Small-scale protocols. Published systems follow three steps: 1. The prover commits to the dataset [S-1110] [S-0022]. 2. It fixes the data order using verified randomness [S-0022]. 3. It proves each gradient-descent step: the forward pass, the backward pass and the weight update [S-0022]. Kaizen adds three techniques [S-1110]: - sumcheck-based proofs specialised for gradient descent; - recursive composition, so that proof size and verifier time do not grow with the number of iterations; - aggregatable polynomial commitments. ZkAudit adds proofs of arbitrary audit functions over the committed data and weights [S-0022]. A frontier-scale design. Peigné et al. propose a design for frontier training with four parts [S-0025]: - Commitment. Before training, the trainer publishes a hashed commitment that combines the architecture specification, a Merkle root of the dataset and the root of the initial weights [S-0025]. - Hash chain. During training, the GPUs compute Merkle roots of per-layer tensors at every step and publish them in a hash chain [S-0025]. - Network anchor. An auditor-controlled network tap, or an attested SmartNIC, records hashes of traffic between nodes as an independent anchor [S-0025] (see Network taps and certifiers). - Challenges. After the chain is frozen, the auditor challenges sampled steps. The trainer re-executes those steps, and a zero-knowledge virtual machine, with native precompiles mostly for floating-point operations, checks them against the committed roots [S-0025]. The design also includes genesis proofs and \"ex-ante\" attestations that enforce policy invariants, such as compute thresholds [S-0025]. What it establishes What it can establish. - A zkPoT shows that the committed weights resulted from the declared training procedure on the committed dataset [S-1110] [S-0022]. This bears on A training run stayed within declared limits. - Peigné et al. aim to verify further claims: that training faithfully executed a pre-committed specification, how much compute it consumed, its training regime and data-content filters [S-0025]. What it cannot establish. - That the data is acceptable. A proof binds data to a commitment but does not vet it. ZkAudit does not protect against data poisoning [S-0022]. - Every step, in the frontier design. Sparse auditing is \"detection-grade, not universal\" [S-0025]. - Full confidentiality. ZkAudit reveals the model architecture [S-0022]. Threat model The small-scale systems assume a cheating prover bounded by cryptographic assumptions, and a public training algorithm and architecture [S-1110] [S-0022]. The frontier design adds trust anchors [S-0025]: - deterministic GPU execution; - a physical network tap, or secure boot and device identity for the SmartNIC tier; - the soundness of the zkVM; - a public proof-checker binary. It does not cover traffic inside a server, and its SmartNIC tier is weaker against supply-chain adversaries [S-0025]. Evidence - Kaizen (CCS 2024). It proves training of a 10-million-parameter VGG-11 on CIFAR-10 at batch size 16. The prover takes 15 minutes per iteration; the proof is 1.63 MB and verifies in 130 milliseconds, independent of the number of iterations [S-1110]. Its authors report \"24× faster prover time\" than generic recursive proof systems [S-1110]. - ZkAudit (ICML 2024). It proved single SGD steps for MobileNet v2 image classifiers and a recommender model on AWS g4dn.8xlarge instances. Proving one step on a single image took 47.5 to 328.3 seconds for MobileNet v2 (1.0, 224), depending on the fixed-point scale factor. The authors estimated, rather than generated, proofs of full training runs, at costs of hundreds to thousands of dollars [S-0022]. - VeriLoRA (2025 preprint). It proves one LoRA fine-tuning step, covering the forward pass, backward pass and parameter update, on a single sample, for LLaMA and OPT models of 3 to 13 billion parameters on one A100 GPU. Proving a step takes minutes and verifying it takes seconds, and the code is public [S-3080]. - Other systems. The survey lists further verifiable-training systems in its Table IV [S-1100]. - The frontier design. Peigné et al. estimate 2 to 10% training-side overhead for a Llama 3.1 405B-scale run, and deployment \"within approximately 36 months\" [S-0025]. The paper reports no prototype or measurements of its own, and marks its target values as \"not yet measured\" [S-0025]. The zkLLM authors wrote in 2024 that zero-knowledge proofs of LLM training \"may pose insurmountable challenges\" [S-0023]. Limitations Cost. A 10-million-parameter model needs minutes of proving per step [S-1110]. ZkAudit's authors leave scaling to larger models, such as language models, to future work [S-0022]. VeriLoRA reaches 13-billion-parameter language models, but for single-sample steps that update only low-rank adapters [S-3080]. Open problems in the frontier design. Peigné et al. list 13, including [S-0025]: - zero-knowledge proofs of backpropagation; - deterministic attention backward passes with under 5% overhead; - an open-hardware network tap at line rate; - a way to tell silent data corruption apart from adversarial deviation; - coverage of mixture-of-experts, reinforcement-learning post-training and multi-site training. They also report that current deterministic tensor-parallel all-reduce configurations lose 64 to 89% of bandwidth [S-0025]. Attacks. As of September 2026 no attack on these proof systems has been published.", "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0024", "title": "Cryptographic commitment", "url": "https://trustbutveri.fyi/concepts/cryptographic-commitment/" }, { "id": "K-0010", "title": "Zero-knowledge proof", "url": "https://trustbutveri.fyi/concepts/zero-knowledge-proof/" }, { "id": "O-0122", "title": "University of Waterloo", "url": "https://trustbutveri.fyi/organizations/university-of-waterloo/" } ] }, { "id": "I-0011", "slug": "ai-2040-inference-only-verification-plan", "title": "AI 2040 inference-only verification stack", "aliases": [ "AI 2040 verification plan (inference-only retrofit)", "Inference-only retrofit" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "one_liner": "A proposed retrofit that isolates data-centre inference units, taps their front-end traffic and recomputes random samples to check that only declared inference runs.", "summary": "The AI 2040 plan proposes checking that retrofitted data centres run only inference, not training or experiments. Data centres would be split into inference units by removing much of their back-end networking. Passive network taps would copy all front-end traffic to a recomputation server, which re-runs random samples of workloads organized into reproducible packets. Strong physical security would protect the equipment, and side-channel limits plus periodic memory wiping are meant to stop hidden work from leaving results. As of July 2026, passive optical taps were reported to work at 400G and recomputation schemes were being tested, but the integrated stack had not been built; one team working on it says nothing is past a proof-of-principle prototype. The obstacles are reproducible inference and networking, securing the recomputation server, and scaling side-channel defences. It does not verify that spare compute is unused. The 400G tap claim concerns components, not an installed system.", "technical": "- **Secure network gateways.** The plan names more capable gateways as a possible replacement for the passive taps. Splitters on the input and output paths log data flow into packets, which are randomly sampled and sent to a secure recomputing server. At boot, the gateways would also check that only whitelisted model weights are uploaded to the inference unit [[S-0067]].\n- **Physical security measures.** The plan lists tamper-evident enclosures, security cameras, perimeter controls and air-gapping [[S-0067]].\n- **Other options the plan names.** Zero-knowledge proofs could preserve privacy and need less hardware retrofit, but are \"currently somewhat speculative\" for lack of efficient enough algorithms. Memory-challenge verification might also avoid a hardware retrofit, but \"seems likely to face some difficulties with verifying completeness\". For an initial phase, removing major scale-out interconnect and installing simple sensors might suffice [[S-0067]].", "category": "isolation-architecture", "secondary_categories": [ "off-chip-devices", "cryptographic-computational" ], "verifies": [ { "claim": "C-0004", "role": "primary", "note": "The stack's stated purpose: retrofitted data centres run only inference." }, { "claim": "C-0005", "role": "supporting", "note": "Recomputation checks sampled packets against the declared model." }, { "claim": "C-0008", "role": "supporting", "note": "Removing back-end networking limits communication between inference units." } ], "threat_model": "adversarial", "adversarial_evaluation": "none", "hardware_requirement": "retrofit-device", "prover_cooperation": "required", "confidentiality": "partial", "depends_on": [ { "target": "M-0001", "note": "Correctness rests on sampled recomputation." }, { "target": "M-0003", "note": "Workloads must be organized into reproducible packets." } ], "readiness": { "assessment": true, "level": "R1", "scope": "showing that retrofitted data centres run only inference", "rubric_version": "1.1", "rationale": "The six-layer design, its claim and its assumptions are published, but the integrated stack has not been built.\n\n- **R1** met: the plan publishes the design of the stack, the claim it would verify (only inference runs on retrofitted compute) and its key assumptions, including a trusted recomputation server and strong physical security [[S-0067]].\n- **R2** not met: there is no public implementation or end-to-end result for the integrated stack. The plan's companion page lists a reproducible inference stack as not started and gives no plan yet for scaling side-channel defences [[S-1511]]. A team working on the components says that nothing is past a proof-of-principle prototype [[S-1512]]. Some components are further along: passive optical taps work at 400G, and recomputation schemes such as TOPLOC and DiFR are being tested on relevant hardware [[S-1511]]; see [[M-0001]]. Amodo reports a lab prototype that feeds a passive tap on a 1 Gbps link into DiFR recomputation, which covers two of the six layers [[S-1312]].\n\nConfidence is medium: the design is clearly published, but component status comes from pages that are updated over time [[S-1511]] [[S-1008]].", "evidence": [ "S-0067", "S-1511", "S-1512", "S-1008", "S-1312" ], "next_level_gaps": [ "A public working implementation or reproducible end-to-end results for the integrated stack (isolated inference unit, taps, packetization and recomputation) at realistic scale or against a stated adversary.", "A hardened recomputation server and a method for checking that taps are correctly installed and remain in place.", "Red-teaming of recomputation and of the completeness measures (side channels, memory wiping)." ], "confidence": "medium", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "The recomputation server must be trusted", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "The plan calls the integrity of the recomputation server an extremely important aspect, and its argument that sampling verifies all outputs assumes that the server's computations and outputs can be trusted. The companion page notes that the server sits inside the prover's facility, possibly under the prover's physical control, and that hardening it against integrity attacks needs significant research. Amodo rates recomputation-server security as not on track.", "sources": [ "S-0067", "S-1511", "S-1008" ], "response": null }, { "assessment": true, "title": "Spare compute is not verified", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "The plan states that it does not verify that spare compute is unused for unapproved workloads, because this seems very challenging. It relies instead on side-channel bounds and memory wipes, so that the only results that persist are verified inference outputs.", "sources": [ "S-0067" ], "response": null }, { "assessment": true, "title": "A recomputation family degrades against prompt-controlling adversaries", "kind": "demonstrated-attack", "severity": "significant", "status": "open", "description": "The plan's companion page names DiFR among the recomputation schemes being tested. An independent study found that Gumbel-based inference verification, the family that includes Token-DiFR, leaks roughly twice as many bits per token when the adversary chooses prompts that disrupt the text's structure, across six models of 1 to 32 billion parameters. The slowdown it imposes on a hidden-signalling adversary falls from over 200 times to 60–118 times. The attack weakens the bound on hidden information in outputs, which the plan relies on to keep undeclared results from leaving.", "sources": [ "S-1511", "S-0015", "S-1507", "S-0067" ], "response": null } ], "blockers": [ { "text": "A fully reproducible inference stack needs substantial software and tooling, and per-packet network reproducibility may need considerable software, firmware and possibly hardware work.", "theme": "performance-compatibility", "blocked_by": "M-0003", "sources": [ "S-1511" ] }, { "text": "Passive optical taps work at 400G, but the 800G and 1600G line rates now arriving in data centres are undemonstrated.", "theme": "performance-compatibility", "blocked_by": "M-0013", "sources": [ "S-1511" ] }, { "text": "Checking that taps are correctly installed and stay in place at scale is not a solved problem, and hardening the recomputation server inside the prover's facility needs significant research.", "theme": "hardware-trust", "blocked_by": "M-0017", "sources": [ "S-1511", "S-1008" ] }, { "text": "There is no plan yet for quickly scaling side-channel defences on a frontier cluster; only early theoretical pieces exist.", "theme": "coverage-hidden-compute", "blocked_by": "M-0022", "sources": [ "S-1511" ] }, { "text": "Memory wiping may use existing algorithms, but hardware testing is at an early stage.", "theme": "coverage-hidden-compute", "blocked_by": "M-0015", "sources": [ "S-1511" ] }, { "text": "Robust red-teaming of recomputation schemes has not started, and most algorithm development remains academic.", "theme": "adversarial-validation", "blocked_by": null, "sources": [ "S-1511", "S-1008" ] } ], "challenge_themes": [ "coverage-hidden-compute", "hardware-trust", "performance-compatibility", "adversarial-validation", "protocol-soundness" ], "organizations": [ "O-0101", "O-0201" ], "people": [], "sources": [ { "source": "S-0067", "supports": "the six-layer stack; isolation rationale; passive optical taps; packets and reproducibility; recomputation and its trust assumption; physical security measures; completeness measures; spare-compute scope; gateway variant; other options", "locator": "Summary of the plan; inference-only retrofit description; Appendix reference" }, { "source": "S-1511", "supports": "component status and open problems as of 9 July 2026; recomputation-server hardening", "locator": "network taps; reproducible packets; partial recomputation; physical security; completeness" }, { "source": "S-1512", "supports": "overall maturity statement; description of taps and recomputation", "locator": "introduction" }, { "source": "S-1008", "supports": "status of recomputation server security and red-teaming", "locator": "status items" }, { "source": "S-1312", "supports": "passive tap fitted to a DiFR recomputation prototype on a 1 Gbps lab link; stress-test result; need for an active tap", "locator": "setup, results and side-channel sections" }, { "source": "S-0015", "supports": "Token-DiFR as a Gumbel-Max estimator", "locator": "§6.3" }, { "source": "S-1507", "supports": "prompt-controlling attack on Gumbel-based verification; models tested; bits per token; slowdown factor", "locator": "abstract" } ], "concepts": [ "K-0009", "K-0014", "K-0016", "K-0020", "K-0021", "K-0025" ], "kind": "proposed-architecture", "developer": [ "O-0201" ], "realises": [ "M-0013", "M-0001", "M-0003", "M-0014", "M-0015", "M-0017", "M-0022" ], "homepage": "https://ai-2040.com/supplements/verification-plan", "type": "implementation", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/", "source_file": "content/implementations/ai-2040-inference-only-verification-plan.md", "flags_all": [ "provider-reported" ], "body_markdown": "## What it is\n\nThe AI 2040 verification plan, written by Romeo Dean and published by the AI Futures Project, includes a technical stack for verifying that retrofitted data centres run only inference workloads, not training or experiments [[S-0067]] [[S-1511]]. The plan calls it an inference-only retrofit and lists six layers [[S-0067]]. It is a proposal: the plan's companion page tracks which components exist and which are still open problems [[S-1511]].\n\n## How it works\n\nThe six layers, as the plan lists them [[S-0067]]:\n\n- **Isolated inference units.** Data centres are split into groups of isolated inference units by removing much of their back-end (east–west) networking [[S-0067]]. The plan argues that inference needs much less scale-out interconnect than training, so the overhead is relatively small [[S-0067]]. See [[M-0014]].\n- **Passive taps.** Simple, passive taps on the front-end (north–south) network copy all input and output traffic to a recomputation server [[S-0067]]. The plan favours optical splitters, \"essentially a piece of glass\", because they are probably the easiest to trust [[S-0067]]. See [[M-0013]].\n- **Reproducible packets.** All AI workloads, including agentic tool use, are organized into discrete, reproducible packets that the recomputation server can inspect [[S-0067]]. Inference is not reproducible by default, and achieving it may cost some overhead [[S-0067]]. See [[M-0003]].\n- **Partial recomputation.** The server recomputes random samples of packets to check their correctness [[S-0067]]. The plan argues that this can verify all outputs with very high probability using a small fraction of the original compute, provided the server can be trusted [[S-0067]]. See [[M-0001]].\n- **Physical security.** Strong physical security protects the verification equipment against tampering [[S-0067]]. See [[M-0017]].\n- **Completeness.** Side-channel communication is bounded, and memory is wiped periodically, probably by forced memorization [[S-0067]]. The aim is that the only results that persist are verified inference outputs [[S-0067]]. See [[M-0022]] and [[M-0015]].\n\n## Evidence\nThe plan's companion page, updated in July 2026, and a practitioner status page report the state of each component [[S-1511]] [[S-1008]]:\n\n- **Taps.** The companion page says passive optical taps work at 400G, while 800G and 1600G remain undemonstrated [[S-1511]]. This is component readiness; the published tapped inference-verification prototype used a 1 Gbps lab link [[S-1312]]. Capturing all front-end traffic still needs infrastructure and work on network topologies and sampling [[S-1511]].\n- **Recomputation.** TOPLOC and DiFR are well described and are being tested on relevant hardware and models [[S-1511]]. Both status pages list recomputation red-teaming as not started [[S-1511]] [[S-1008]].\n- **Tapped prototype.** In September 2026 Amodo reported fitting a passive fibre tap to a 1 Gbps lab link of its DiFR-based recomputation prototype [[S-1312]]. All 800 test inferences were tapped and verified while the link was flooded with non-whitelisted traffic [[S-1312]]. Amodo concludes that the passive tap must become active to close the side channels that remain [[S-1312]]. See [[M-0013]].\n- **Reproducibility.** Demonstrations work around non-determinism, but work on a fully reproducible inference stack has not started [[S-1511]].\n- **Completeness.** Early hardware testing of memory wiping is under way [[S-1511]]. For side-channel mitigation, only early theoretical pieces exist [[S-1511]].\n- **Overall.** The Amodo team reports that the core components are being de-risked, but that nothing is past a proof-of-principle prototype [[S-1512]].\n\n## Limitations\n- **Trusted recomputation server.** The assurance argument assumes that the server's computations and outputs can be trusted [[S-0067]]. Amodo rates recomputation-server security as not on track [[S-1008]].\n- **Spare compute.** Idle or spare compute is not directly verified [[S-0067]].\n- **Tap installation.** Checking that taps are correctly installed and stay in place is not yet solved [[S-1511]].\n- **Network reproducibility.** Making individual packets reproducible across the network may need considerable software, firmware and possibly hardware work [[S-1511]].\n- **Recomputation attacks.** When the adversary controls the prompts, Gumbel-based inference verification, the family that includes Token-DiFR [[S-0015]], leaks roughly twice as many bits per token as under benign prompts [[S-1507]]. See [[M-0024]].\n- **Side-channel defence.** There is no plan yet for scaling side-channel defences quickly on a frontier cluster [[S-1511]].", "body_text": "What it is The AI 2040 verification plan, written by Romeo Dean and published by the AI Futures Project, includes a technical stack for verifying that retrofitted data centres run only inference workloads, not training or experiments [S-0067] [S-1511]. The plan calls it an inference-only retrofit and lists six layers [S-0067]. It is a proposal: the plan's companion page tracks which components exist and which are still open problems [S-1511]. How it works The six layers, as the plan lists them [S-0067]: - Isolated inference units. Data centres are split into groups of isolated inference units by removing much of their back-end (east–west) networking [S-0067]. The plan argues that inference needs much less scale-out interconnect than training, so the overhead is relatively small [S-0067]. See Bandwidth limits and compartmentalization. - Passive taps. Simple, passive taps on the front-end (north–south) network copy all input and output traffic to a recomputation server [S-0067]. The plan favours optical splitters, \"essentially a piece of glass\", because they are probably the easiest to trust [S-0067]. See Network taps and certifiers. - Reproducible packets. All AI workloads, including agentic tool use, are organized into discrete, reproducible packets that the recomputation server can inspect [S-0067]. Inference is not reproducible by default, and achieving it may cost some overhead [S-0067]. See Whole-workload recomputation (reproducible packets). - Partial recomputation. The server recomputes random samples of packets to check their correctness [S-0067]. The plan argues that this can verify all outputs with very high probability using a small fraction of the original compute, provided the server can be trusted [S-0067]. See Sampled inference recomputation. - Physical security. Strong physical security protects the verification equipment against tampering [S-0067]. See Tamper evidence for verifier devices. - Completeness. Side-channel communication is bounded, and memory is wiped periodically, probably by forced memorization [S-0067]. The aim is that the only results that persist are verified inference outputs [S-0067]. See Side-channel suppression for isolated facilities and Memory wiping and proofs of secure erasure. Evidence The plan's companion page, updated in July 2026, and a practitioner status page report the state of each component [S-1511] [S-1008]: - Taps. The companion page says passive optical taps work at 400G, while 800G and 1600G remain undemonstrated [S-1511]. This is component readiness; the published tapped inference-verification prototype used a 1 Gbps lab link [S-1312]. Capturing all front-end traffic still needs infrastructure and work on network topologies and sampling [S-1511]. - Recomputation. TOPLOC and DiFR are well described and are being tested on relevant hardware and models [S-1511]. Both status pages list recomputation red-teaming as not started [S-1511] [S-1008]. - Tapped prototype. In September 2026 Amodo reported fitting a passive fibre tap to a 1 Gbps lab link of its DiFR-based recomputation prototype [S-1312]. All 800 test inferences were tapped and verified while the link was flooded with non-whitelisted traffic [S-1312]. Amodo concludes that the passive tap must become active to close the side channels that remain [S-1312]. See Network taps and certifiers. - Reproducibility. Demonstrations work around non-determinism, but work on a fully reproducible inference stack has not started [S-1511]. - Completeness. Early hardware testing of memory wiping is under way [S-1511]. For side-channel mitigation, only early theoretical pieces exist [S-1511]. - Overall. The Amodo team reports that the core components are being de-risked, but that nothing is past a proof-of-principle prototype [S-1512]. Limitations - Trusted recomputation server. The assurance argument assumes that the server's computations and outputs can be trusted [S-0067]. Amodo rates recomputation-server security as not on track [S-1008]. - Spare compute. Idle or spare compute is not directly verified [S-0067]. - Tap installation. Checking that taps are correctly installed and stay in place is not yet solved [S-1511]. - Network reproducibility. Making individual packets reproducible across the network may need considerable software, firmware and possibly hardware work [S-1511]. - Recomputation attacks. When the adversary controls the prompts, Gumbel-based inference verification, the family that includes Token-DiFR [S-0015], leaks roughly twice as many bits per token as under benign prompts [S-1507]. See Bounding unexplained information in outputs. - Side-channel defence. There is no plan yet for scaling side-channel defences quickly on a frontier cluster [S-1511].", "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "O-0201", "title": "AI Futures Project", "url": "https://trustbutveri.fyi/organizations/ai-futures-project/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ] }, { "id": "I-0013", "slug": "apple-private-cloud-compute", "title": "Apple Private Cloud Compute", "aliases": [ "PCC", "Private Cloud Compute" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "one_liner": "Apple's cloud AI inference service, in which user devices send requests only to servers that attest to running software published in a public transparency log.", "summary": "Private Cloud Compute (PCC) is Apple's server system for Apple Intelligence requests that need larger models than a device can run. Apple reports that user devices send a request only to servers that attest to running a software release listed in a public, append-only transparency log. Apple publishes the software images, a virtual research environment that boots them, and part of the source code. Independent researchers found flaws in how PCC authenticates requests, and one reports that a node with tampered configuration files passed attestation until Apple fixed the bug that allowed the tampering. In June 2026 Apple announced an extension of PCC to Google Cloud on Intel TDX and NVIDIA confidential computing. Independent researchers with physical access to such hardware have forged TDX attestations and relayed NVIDIA ones.", "category": "on-chip", "secondary_categories": [ "cryptographic-computational" ], "verifies": [ { "claim": "C-0005", "role": "supporting", "note": "Attests the software release that served a request. Apple reports that model assets share the code's integrity protection (S-1800)." } ], "threat_model": "semi-trusted", "adversarial_evaluation": "independent-red-team", "hardware_requirement": "existing-features", "prover_cooperation": "required", "confidentiality": "preserving", "depends_on": [ { "target": "M-0008", "note": "Relies on hardware attestation: the Secure Enclave on Apple silicon servers, and Intel TDX, NVIDIA confidential computing and Google's Titan chip on Google Cloud." } ], "readiness": { "assessment": true, "level": "R3", "scope": "showing users which software serves their AI requests, not which model", "rubric_version": "1.1", "rationale": "PCC is a production service whose attestation, transparency log and research tools are public, but no independent evaluation has examined its attestation chain as a whole.\n\n- **R1** met: Apple published the design, the claim that devices send data only to nodes that attest to publicly listed software, and a threat model that includes attackers with physical access to a node [[S-1800]].\n- **R2** met: the software images, a research environment that boots them and part of the source code are public [[S-1800]] [[S-1801]]. Independent researchers sent queries to the production service from macOS and iOS clients between December 2025 and March 2026 [[S-1803]].\n- **R3** met: PCC is production-grade and available as the server side of Apple Intelligence (provider-reported) [[S-1800]], and independent researchers have used the production service [[S-1803]]. Its transparency log, binaries and research environment are public [[S-1801]]. The attestation check is made by Apple's own client software, and no other party is documented relying on it for a verification decision.\n- **R4** not met. The two independent public analyses are narrow. A peer-reviewed study reverse-engineered the client and found request-authentication flaws, which Apple addressed only in its documentation. Its subject is request privacy, not attestation [[S-1803]]. A researcher working in Apple's research environment reports that a node with tampered configuration files passed Apple's attestation check, and Apple fixed the path-handling bug behind it [[S-1804]] [[S-1805]]. Neither evaluates the attestation and transparency chain as a whole. The 2026 deployment on Google Cloud uses Intel TDX and NVIDIA confidential computing [[S-1802]]. Independent researchers with physical access forged TDX attestations and paired them with relayed H100 attestations [[S-1202]]. No published work tests whether Apple's use of two independent roots of trust resists that attack.", "evidence": [ "S-1800", "S-1801", "S-1802", "S-1803", "S-1804", "S-1805", "S-1202" ], "next_level_gaps": [ "An independent public evaluation of the attestation and transparency-log chain, including what attestation covers at runtime, that leaves no critical flaw open.", "Evidence that the Google Cloud deployment's attestation resists attackers with physical access to TDX and NVIDIA hardware.", "Reproducible builds, so that published binaries can be checked against published source." ], "confidence": "medium", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Tampered node configuration passed attestation", "kind": "demonstrated-attack", "severity": "significant", "status": "mitigated", "description": "Working in Apple's Virtual Research Environment, an independent researcher used a path traversal in darwin-init, which unpacks software archives when a node boots, to write files as root that survived the node's userspace reboot. The change redirected a logging daemon's telemetry and exposed per-request metadata such as token counts and timings. The researcher reports that Apple's attestation verifier treated the tampered node and a clean one identically. The researcher concludes that attestation appears to measure the installed software but not the writable configuration files that drive daemons at runtime [[S-1804]]. Apple's CVE record describes an attacker in a privileged network position and a fix through improved path validation, in releases from 5E290.3 [[S-1805]].", "sources": [ "S-1804", "S-1805" ], "response": null }, { "assessment": true, "title": "Google Cloud deployment relies on TEEs whose attestations have been forged", "kind": "open-question", "severity": "significant", "status": "open", "description": "Apple reports that PCC on Google Cloud uses NVIDIA confidential computing, Intel CPUs with TDX and Google's Titan chip. For components that could be abused to exfiltrate user data if compromised, Apple reports that attestation is rooted in at least two separate roots of trust from independent vendors [[S-1802]]. With physical access and root privileges, independent researchers forged Intel TDX attestations and, by pairing them with relayed H100 attestations, passed a workload outside TEE protection [[S-1202]]. A second team forged TDX attestation reports on an up-to-date platform with a DDR5 interposer costing under $200 [[S-3126]]. Whether PCC's combination of roots of trust resists such attacks has not been tested in public.", "sources": [ "S-1802", "S-1202", "S-3126" ], "response": null } ], "blockers": [ { "text": "Published binaries cannot be rebuilt from source and carry no symbols, so checking what the attested software does needs reverse engineering.", "theme": "evidence-binding", "blocked_by": null, "sources": [ "S-1803" ] }, { "text": "Outside developers can use PCC only with an entitlement from Apple, which is limited to small developers.", "theme": "access-governance", "blocked_by": null, "sources": [ "S-3125" ] } ], "challenge_themes": [ "evidence-binding", "hardware-trust", "adversarial-validation" ], "organizations": [], "people": [], "sources": [ { "source": "S-1800", "supports": "design goals, Apple silicon servers, signed trust cache, integrity protection of code and model assets, device-side attestation check against the transparency log, publication of images, threat model with physical attackers (provider-reported)" }, { "source": "S-1801", "supports": "Virtual Research Environment, published source components and licence, bounty (provider-reported)" }, { "source": "S-1802", "supports": "Google Cloud deployment with NVIDIA confidential computing, Intel TDX and Titan; two roots of trust; protections ramped up during a summer preview; research mode on live nodes (provider-reported)" }, { "source": "S-1803", "supports": "independent reverse engineering of the client; queries to the production service; no reproducible builds or symbols; request-token flaws; Apple's documentation-only response; no third-party API at the time of the study", "locator": "abstract; §3; limitations" }, { "source": "S-1804", "supports": "independent finding that a tampered node passed attestation; metadata exposure; bounty" }, { "source": "S-1805", "supports": "CVE description, affected versions and fix (vendor-assigned)" }, { "source": "S-1202", "supports": "independent forgery of Intel TDX attestations and H100 attestation relay", "locator": "Abstract; §1.1; §8.3" }, { "source": "S-3125", "supports": "third-party developer access to PCC: eligibility and entitlement (provider-reported)" }, { "source": "S-3126", "supports": "independent forgery of TDX attestation reports with a DDR5 interposer", "locator": "Abstract; case studies" } ], "concepts": [ "K-0004", "K-0005", "K-0006", "K-0019" ], "kind": "product", "developer": [], "realises": [ "M-0008" ], "homepage": "https://security.apple.com/blog/private-cloud-compute/", "repo": "https://github.com/apple/security-pcc", "type": "implementation", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/", "source_file": "content/implementations/apple-private-cloud-compute.md", "flags_all": [ "provider-reported" ], "body_markdown": "## What it is\n\nPrivate Cloud Compute (PCC) is Apple's server system for Apple Intelligence features that need larger foundation models than a device can run [[S-1800]]. Apple built it on custom servers with Apple silicon, the Secure Enclave and Secure Boot [[S-1800]]. Apple states that security researchers must be able to verify that PCC's privacy and security guarantees match its public promises [[S-1800]].\n\nPCC applies [[M-0008|TEE remote attestation]] to show users which software handles their AI requests. It was designed to keep those requests private, and its checks concern software releases [[S-1800]].\n\nIn June 2026 Apple announced that it was extending PCC to Google Cloud systems with NVIDIA GPUs, using NVIDIA confidential computing, Intel CPUs with TDX and Google's Titan chip [[S-1802]]. Apple stated that this deployment would reach its complete set of protections gradually during a summer preview period [[S-1802]].\n\n## How it works\n\nApple describes the chain as follows [[S-1800]]:\n\n1. Every piece of code a node can run must be in a trust cache signed by Apple and loaded by the Secure Enclave, so it cannot change at runtime. Code and model assets use the integrity protection of Apple's Signed System Volume.\n2. Apple publishes the measurements of all code running on PCC in an append-only transparency log, and the corresponding software images within 90 days.\n3. A user's device encrypts each request only to the keys of nodes whose attested measurements match a release in the log.\n\nFor research, Apple publishes a Virtual Research Environment that boots PCC releases in a virtual machine on an Apple silicon Mac, checks the transparency log and runs inference against demonstration models [[S-1801]]. Apple also publishes source code for its attestation and request-handling components under a limited-use licence, and offers bounties of up to $1,000,000 for PCC findings [[S-1801]].\n\nApple's threat model includes an attacker with physical access to a node who can subvert some hardware protections [[S-1800]]. Apple's answer is that an attacker should not be able to target particular users without compromising the whole system. It relies on a hardened hardware supply chain, including imaging each server's components before sealing it, and on routing that does not let an attacker steer chosen users' requests to a compromised node [[S-1800]]. On Google Cloud, Apple reports that attestation for components that could be abused to exfiltrate user data is rooted in at least two separate roots of trust from independent vendors, and that attested keys are held in a separate confidential virtual machine [[S-1802]].\n\n## Evidence\n- Researchers at the Hasso Plattner Institute, TU Darmstadt and IMDEA Networks reverse-engineered PCC's client and sent custom queries to the production service between December 2025 and March 2026 [[S-1803]]. They found that one-time request tokens could be reused, that a signature check was skipped and that token salts let requests be linked. Apple clarified its documentation but did not change the checks [[S-1803]]. The study, peer-reviewed at WiSec 2026, concerns request privacy, not attestation.\n- An independent researcher working in Apple's research environment reports a path-traversal flaw (CVE-2026-20685) that let a node's configuration be altered while its attestation stayed unchanged [[S-1804]] [[S-1805]]. Apple fixed it and paid a $150,000 bounty [[S-1804]].\n- Apple reports that for the Google Cloud deployment it will publish all binaries and give researchers access to live nodes in research mode [[S-1802]].\n\n## Limitations\n- The published checks concern software releases. Apple reports that model assets share the code's integrity protection [[S-1800]], and its research environment runs demonstration models [[S-1801]].\n- PCC ships compiled binaries without reproducible builds or symbols, which its independent analysts say leaves room for differences between the specification and what is shipped [[S-1803]].\n- In the one published attack, the researcher found that attestation appeared to cover installed software but not writable configuration files [[S-1804]].\n- In early 2026 the independent analysts found no PCC API for third parties [[S-1803]]. Apple reports that developers in its App Store Small Business Program with fewer than 2 million first-time downloads can use its models on PCC, once Apple assigns them an entitlement [[S-3125]].\n- The Google Cloud deployment runs on Intel TDX and NVIDIA confidential computing [[S-1802]]. Independent researchers with physical access have forged TDX attestations [[S-1202]] [[S-3126]], and one team relayed H100 attestations to a workload outside TEE protection [[S-1202]].", "body_text": "What it is Private Cloud Compute (PCC) is Apple's server system for Apple Intelligence features that need larger foundation models than a device can run [S-1800]. Apple built it on custom servers with Apple silicon, the Secure Enclave and Secure Boot [S-1800]. Apple states that security researchers must be able to verify that PCC's privacy and security guarantees match its public promises [S-1800]. PCC applies TEE remote attestation to show users which software handles their AI requests. It was designed to keep those requests private, and its checks concern software releases [S-1800]. In June 2026 Apple announced that it was extending PCC to Google Cloud systems with NVIDIA GPUs, using NVIDIA confidential computing, Intel CPUs with TDX and Google's Titan chip [S-1802]. Apple stated that this deployment would reach its complete set of protections gradually during a summer preview period [S-1802]. How it works Apple describes the chain as follows [S-1800]: 1. Every piece of code a node can run must be in a trust cache signed by Apple and loaded by the Secure Enclave, so it cannot change at runtime. Code and model assets use the integrity protection of Apple's Signed System Volume. 2. Apple publishes the measurements of all code running on PCC in an append-only transparency log, and the corresponding software images within 90 days. 3. A user's device encrypts each request only to the keys of nodes whose attested measurements match a release in the log. For research, Apple publishes a Virtual Research Environment that boots PCC releases in a virtual machine on an Apple silicon Mac, checks the transparency log and runs inference against demonstration models [S-1801]. Apple also publishes source code for its attestation and request-handling components under a limited-use licence, and offers bounties of up to $1,000,000 for PCC findings [S-1801]. Apple's threat model includes an attacker with physical access to a node who can subvert some hardware protections [S-1800]. Apple's answer is that an attacker should not be able to target particular users without compromising the whole system. It relies on a hardened hardware supply chain, including imaging each server's components before sealing it, and on routing that does not let an attacker steer chosen users' requests to a compromised node [S-1800]. On Google Cloud, Apple reports that attestation for components that could be abused to exfiltrate user data is rooted in at least two separate roots of trust from independent vendors, and that attested keys are held in a separate confidential virtual machine [S-1802]. Evidence - Researchers at the Hasso Plattner Institute, TU Darmstadt and IMDEA Networks reverse-engineered PCC's client and sent custom queries to the production service between December 2025 and March 2026 [S-1803]. They found that one-time request tokens could be reused, that a signature check was skipped and that token salts let requests be linked. Apple clarified its documentation but did not change the checks [S-1803]. The study, peer-reviewed at WiSec 2026, concerns request privacy, not attestation. - An independent researcher working in Apple's research environment reports a path-traversal flaw (CVE-2026-20685) that let a node's configuration be altered while its attestation stayed unchanged [S-1804] [S-1805]. Apple fixed it and paid a $150,000 bounty [S-1804]. - Apple reports that for the Google Cloud deployment it will publish all binaries and give researchers access to live nodes in research mode [S-1802]. Limitations - The published checks concern software releases. Apple reports that model assets share the code's integrity protection [S-1800], and its research environment runs demonstration models [S-1801]. - PCC ships compiled binaries without reproducible builds or symbols, which its independent analysts say leaves room for differences between the specification and what is shipped [S-1803]. - In the one published attack, the researcher found that attestation appeared to cover installed software but not writable configuration files [S-1804]. - In early 2026 the independent analysts found no PCC API for third parties [S-1803]. Apple reports that developers in its App Store Small Business Program with fewer than 2 million first-time downloads can use its models on PCC, once Apple assigns them an entitlement [S-3125]. - The Google Cloud deployment runs on Intel TDX and NVIDIA confidential computing [S-1802]. Independent researchers with physical access have forged TDX attestations [S-1202] [S-3126], and one team relayed H100 attestations to a workload outside TEE protection [S-1202].", "referenced_by": [ { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ] }, { "id": "I-0007", "slug": "attestable-audits", "title": "Attestable Audits", "aliases": [ "Attestable Audits prototype", "Verifiable AI safety benchmarks in TEEs" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "one_liner": "A research prototype that runs AI safety benchmarks inside a trusted execution environment and publishes attestations binding the model, the audit and the results.", "summary": "Attestable Audits, from University of Cambridge researchers, lets a model provider and an auditor who do not trust each other run a benchmark on a confidential model. Both send their inputs, encrypted, to a trusted execution environment: the provider its weights, the auditor its test code and data. The enclave runs the audit and publishes an attestation that binds the model hash, the audit and the result. Later, users can check that the model answering them is the one that was audited. The authors' prototype on CPU-only AWS Nitro Enclaves ran a 4-bit Llama-3.1-8B on MMLU, XSum and ToxicChat. CPU inference cost 21.7 times as much per token as GPU inference, and the enclave roughly doubled the CPU cost; the authors expect an overhead as small as 5 times on confidential-computing GPUs. No code is linked. The design trusts the TEE vendor, and prompt-based model exfiltration remains an open gap.", "technical": "- **Security goals.** G1 model verifiability, G2 audit verifiability, G3 confidentiality of model IP and audit data, G4 transparency of artifacts, G5 statelessness, and G6 output verifiability [[S-0009]].\n- **Adversaries.** A1 is a network adversary that can intercept, tamper with or spoof messages, with denial of service excluded. A2 is a physical or privileged adversary able to take RAM snapshots, roll back VMs and run side-channel attacks [[S-0009]].\n- **PREPARE.** The enclave generates a KEM key pair and attests its boot image. The provider sends the encrypted model, which the enclave quantizes and hashes. The enclave publishes an attestation linking model and quantized model to a transparency log [[S-0009]].\n- **ATTESTABLEAUDIT.** The provider and the auditor send the encrypted model and the encrypted audit code and data. The enclave runs the audit in a sandbox and publishes an attestation binding model, audit and result [[S-0009]].\n- **INFERENCE.** The enclave loads the earlier attestations, checks that the provider's model hash matches, and returns each encrypted response with an attestation linking model, prompt, output and audit result [[S-0009]].\n- **Verification.** Verifiers check platform configuration registers (PCRs) against known images and check signatures with the TEE vendor's key or attestation service. Including the base image in the measurement allows revocation when vulnerabilities are found [[S-0009]].\n- **Prototype.** The implementation is written in Rust with bindings to llama.cpp. A 4-core enclave on an m5.2xlarge instance is compared with CPU baselines on m5.xlarge and m5.2xlarge instances and an NVIDIA L40S GPU baseline. Enclave throughput was 1.84 tokens/s at $5.80 per 100K tokens, against 202 tokens/s and $0.12 on the GPU baseline. Zero-shot MMLU accuracy was 51.4% at 4-bit (57.4% excluding unparsable responses). The authors call this similar to the unquantized model on the GPU baseline, which scored 54.6% in the text and 58.9% in Table 2. Copying models into the enclave took at most 2 minutes [[S-0009]].", "category": "on-chip", "secondary_categories": [ "cryptographic-computational" ], "verifies": [ { "claim": "C-0005", "role": "primary", "note": "Users can check that the model answering them is the audited one." }, { "claim": "C-0006", "role": "supporting", "note": "Attests that declared safety benchmarks were run on that model and what they scored; it does not attest runtime safeguards." } ], "threat_model": "semi-trusted", "adversarial_evaluation": "analysis", "hardware_requirement": "existing-features", "prover_cooperation": "required", "confidentiality": "preserving", "depends_on": [ { "target": "M-0008", "note": "Built on AWS Nitro Enclaves attestation." } ], "readiness": { "assessment": true, "level": "R2", "scope": "showing users that the model answering them is the audited one", "rubric_version": "1.1", "rationale": "One developer paper reports end-to-end results on commercial hardware, but there is no public code and no independent reproduction, so confidence is low.\n\n- **R1** met: the protocol, goals and adversaries are published [[S-0009]].\n- **R2** met through reproducible published end-to-end results. The paper specifies the protocol and its algorithms, the enclave and baseline instances, the models and the benchmarks. It reports results for Llama-3.1-8B on AWS Nitro Enclaves, which are commercially available hardware, against a stated adversary [[S-0009]]. The reported evaluation covers the audit step. The paper links no code, which the rubric does not require for this route.\n- **R3** not met: no party other than the developers is documented relying on it for a verification decision, and the prototype is CPU-only research code rather than a production-grade system.\n- **R4** not met: as of September 2026 no independent security evaluation has been published.", "evidence": [ "S-0009" ], "next_level_gaps": [ "Reliance by a party other than the developers for a verification decision, or a production-grade, available implementation, for example on GPU confidential computing at realistic model scale.", "An independent public security evaluation (audit, red-team or peer-reviewed analysis)." ], "confidence": "low", "assessed_by": [ "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Relies on the TEE vendor and inherits TEE attacks", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "The design depends on trusting the TEE vendor, AWS in the prototype [[S-0009]]. The authors cite memory-aliasing, ciphertext side-channel and malicious-interrupt attacks on confidential VMs (BadRAM, CIPHERLEAKS, Heckler). Their answer is to revoke vulnerable base images once such attacks are discovered [[S-0009]].", "sources": [ "S-0009" ], "response": null }, { "assessment": true, "title": "Prompt-based model exfiltration is a residual gap", "kind": "open-question", "severity": "significant", "status": "open", "description": "The authors state that \"prompt-based model exfiltration during the user interaction step remains a residual gap\" [[S-0009]].", "sources": [ "S-0009" ], "response": null }, { "assessment": true, "title": "CPU-only enclaves force small, quantized models and high cost", "kind": "open-question", "severity": "minor", "status": "open", "description": "Memory limits required 4-bit quantization, and the quantized model scored 51.4% on zero-shot MMLU. CPU inference cost 21.7 times as much per token as GPU inference, and the enclave roughly doubled the CPU cost [[S-0009]]. The authors wrote that H100 confidential computing had no multi-GPU support [[S-0009]]. NVIDIA's white paper of August 2025 describes a protected-PCIe mode that passes all eight GPUs of a Hopper HGX node to one confidential VM, with NVLink traffic unencrypted [[S-1200]].", "sources": [ "S-0009", "S-1200" ], "response": null } ], "blockers": [ { "text": "The prototype needs porting to GPU confidential computing to handle larger models; the authors expect an overhead as small as 5 times there.", "theme": "performance-compatibility", "blocked_by": "M-0008", "sources": [ "S-0009" ] }, { "text": "As of September 2026 no code has been released for the prototype.", "theme": "adversarial-validation", "blocked_by": null, "sources": [ "S-0009" ] } ], "challenge_themes": [ "evidence-binding", "hardware-trust", "performance-compatibility" ], "organizations": [ "O-0142" ], "people": [], "sources": [ { "source": "S-0009", "supports": "all protocol, prototype, results and limitations statements", "locator": "§2 (background, GPU CC), §3 (goals, adversaries, Algorithms 1-3), §5 (Table 2), §7 (limitations)" }, { "source": "S-1200", "supports": "Hopper protected-PCIe multi-GPU mode with unencrypted NVLink", "locator": "p. 13" } ], "concepts": [ "K-0006", "K-0004", "K-0024", "K-0019", "K-0022" ], "kind": "research-prototype", "developer": [ "O-0142" ], "realises": [ "M-0008", "M-0012" ], "type": "implementation", "url": "https://trustbutveri.fyi/implementations/attestable-audits/", "source_file": "content/implementations/attestable-audits.md", "flags_all": [ "provider-reported" ], "body_markdown": "## What it is\n\nAttestable Audits is a protocol and prototype by four researchers in the University of Cambridge's Department of Computer Science and Technology [[S-0009]]. It targets a specific problem: benchmarks \"typically do not offer verifiable results and lack confidentiality for model IP and benchmark datasets\" [[S-0009]]. The design runs audits inside a TEE, so that a model provider and an auditor who do not trust each other can produce a result that users and regulators can verify [[S-0009]]. It is an implementation of [[M-0008]] and, through its inference step, of [[M-0012]].\n\n## How it works\n\nThe protocol has three steps [[S-0009]]:\n\n1. **Prepare.** The enclave attests its boot image and publishes a public key. The model provider sends its model, encrypted to that key. The enclave quantizes and hashes the model and posts an attestation to a transparency log.\n2. **Audit.** The provider sends the model and the auditor sends its audit code and data, each encrypted. The enclave runs the audit in a sandbox and publishes an attestation that binds the model hash, the audit and the result.\n3. **Inference.** The enclave loads the earlier attestations and checks that the served model's hash matches. It then answers each encrypted prompt with an attestation that links the model, prompt, output and audit result.\n\nVerifiers compare the enclave's measurement registers with known images and check the vendor's signature [[S-0009]]. The paper considers two adversaries: network attackers, and privileged or physical attackers who may take RAM snapshots, roll back VMs or use side channels [[S-0009]].\n\n## Evidence\nThe authors report the following results:\n\n- **Setup.** The prototype ran on CPU-only AWS Nitro Enclaves with a 4-bit Llama-3.1-8B-Instruct and three benchmarks: MMLU, XSum and ToxicChat. The reported evaluation covers the audit step [[S-0009]].\n- **Throughput.** The enclave produced 1.84 tokens per second, against 202 on an NVIDIA L40S GPU baseline [[S-0009]].\n- **Cost.** The enclave cost $5.80 per 100K tokens, against $2.61 on a plain CPU instance with the same number of cores and $0.12 on the GPU baseline. The authors put the cost of CPU inference at 21.7 times that of GPU inference. They attribute the roughly twofold enclave cost to needing a larger instance, or giving up cores [[S-0009]].\n- **Accuracy.** Zero-shot MMLU accuracy was 51.4% at 4-bit, which the authors describe as similar to the unquantized model on the GPU baseline [[S-0009]].\n- **Loading.** Copying a model into the enclave took at most 2 minutes [[S-0009]].\n- **Outlook.** The authors expect a production version on confidential-computing GPUs to have \"an overhead as small as 5×\" [[S-0009]].\n- **Code.** The paper links no code repository [[S-0009]].\n\n## Limitations\n- **Vendor trust.** The approach assumes the TEE vendor is trusted [[S-0009]].\n- **Attacks on confidential VMs.** The authors cite memory aliasing (BadRAM), ciphertext side channels (CIPHERLEAKS) and malicious interrupts (Heckler), and propose revoking affected images when such attacks are found [[S-0009]]. Physical attacks on the underlying TEEs, such as memory-bus interposition, are discussed under [[M-0008]].\n- **Exfiltration.** The authors name prompt-based model exfiltration during user interaction as a residual gap [[S-0009]].\n- **Model size.** Memory limits forced 4-bit quantization, which lowered MMLU accuracy [[S-0009]].\n- **Multi-GPU.** The authors wrote that H100 confidential computing had no multi-GPU support [[S-0009]]. NVIDIA's August 2025 white paper describes an eight-GPU protected-PCIe mode on Hopper [[S-1200]].", "body_text": "What it is Attestable Audits is a protocol and prototype by four researchers in the University of Cambridge's Department of Computer Science and Technology [S-0009]. It targets a specific problem: benchmarks \"typically do not offer verifiable results and lack confidentiality for model IP and benchmark datasets\" [S-0009]. The design runs audits inside a TEE, so that a model provider and an auditor who do not trust each other can produce a result that users and regulators can verify [S-0009]. It is an implementation of TEE remote attestation for AI workloads and, through its inference step, of Model identity attestation. How it works The protocol has three steps [S-0009]: 1. Prepare. The enclave attests its boot image and publishes a public key. The model provider sends its model, encrypted to that key. The enclave quantizes and hashes the model and posts an attestation to a transparency log. 2. Audit. The provider sends the model and the auditor sends its audit code and data, each encrypted. The enclave runs the audit in a sandbox and publishes an attestation that binds the model hash, the audit and the result. 3. Inference. The enclave loads the earlier attestations and checks that the served model's hash matches. It then answers each encrypted prompt with an attestation that links the model, prompt, output and audit result. Verifiers compare the enclave's measurement registers with known images and check the vendor's signature [S-0009]. The paper considers two adversaries: network attackers, and privileged or physical attackers who may take RAM snapshots, roll back VMs or use side channels [S-0009]. Evidence The authors report the following results: - Setup. The prototype ran on CPU-only AWS Nitro Enclaves with a 4-bit Llama-3.1-8B-Instruct and three benchmarks: MMLU, XSum and ToxicChat. The reported evaluation covers the audit step [S-0009]. - Throughput. The enclave produced 1.84 tokens per second, against 202 on an NVIDIA L40S GPU baseline [S-0009]. - Cost. The enclave cost $5.80 per 100K tokens, against $2.61 on a plain CPU instance with the same number of cores and $0.12 on the GPU baseline. The authors put the cost of CPU inference at 21.7 times that of GPU inference. They attribute the roughly twofold enclave cost to needing a larger instance, or giving up cores [S-0009]. - Accuracy. Zero-shot MMLU accuracy was 51.4% at 4-bit, which the authors describe as similar to the unquantized model on the GPU baseline [S-0009]. - Loading. Copying a model into the enclave took at most 2 minutes [S-0009]. - Outlook. The authors expect a production version on confidential-computing GPUs to have \"an overhead as small as 5×\" [S-0009]. - Code. The paper links no code repository [S-0009]. Limitations - Vendor trust. The approach assumes the TEE vendor is trusted [S-0009]. - Attacks on confidential VMs. The authors cite memory aliasing (BadRAM), ciphertext side channels (CIPHERLEAKS) and malicious interrupts (Heckler), and propose revoking affected images when such attacks are found [S-0009]. Physical attacks on the underlying TEEs, such as memory-bus interposition, are discussed under TEE remote attestation for AI workloads. - Exfiltration. The authors name prompt-based model exfiltration during user interaction as a residual gap [S-0009]. - Model size. Memory limits forced 4-bit quantization, which lowered MMLU accuracy [S-0009]. - Multi-GPU. The authors wrote that H100 confidential computing had no multi-GPU support [S-0009]. NVIDIA's August 2025 white paper describes an eight-GPU protected-PCIe mode on Hopper [S-1200].", "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" }, { "id": "O-0142", "title": "University of Cambridge", "url": "https://trustbutveri.fyi/organizations/university-of-cambridge/" } ] }, { "id": "I-0005", "slug": "attestable-zk-inference", "title": "Attestable zero-knowledge inference prover", "aliases": [ "Attestable ZK prover" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "one_liner": "Attestable's zero-knowledge prover, which the company reports proves large language model outputs came from committed weights at tens of tokens per second.", "summary": "Attestable, a company, reports a zero-knowledge prover for transformer inference that would let an AI developer prove that an output came from a committed model, input and random seed without revealing the weights. On one NVIDIA H100 GPU, Attestable reports proving a 31-billion-parameter model at 53 tokens per second for one 16,000-token sequence, and 77 tokens per second across four 4,000-token sequences. It reports proofs of a few megabytes that a CPU verifies in under a second. It also proposes using such proofs to support verifiable limits on AI development. All evidence comes from the company's blog, and no paper, protocol specification or code is public, so the results cannot be independently checked. The company lists its own limits: a 16,000-token context, 8-bit integer matrix multiplications, and proofs that cover only the computation they are about.", "category": "cryptographic-computational", "secondary_categories": [], "verifies": [ { "claim": "C-0005", "role": "primary", "note": "Attestable reports proving y = F(W, x, r) for committed weights W." }, { "claim": "C-0004", "role": "supporting", "note": "Proposed use: showing an accounted workload used an approved, unchanged model." }, { "claim": "C-0006", "role": "supporting", "note": "Proposed use: showing an agreed input classifier was applied." } ], "threat_model": "adversarial", "adversarial_evaluation": "none", "hardware_requirement": "none", "prover_cooperation": "required", "confidentiality": "preserving", "depends_on": [], "readiness": { "assessment": true, "level": "R1", "scope": "proving an output came from committed weights", "rubric_version": "1.1", "rationale": "Attestable states the claim, its security basis and its limits, but has published no code, protocol description or artifact.\n\n- **R1** met: Attestable publicly states the claim proven (an output y = F(W, x, r) for committed weights W, input x and seed r), its security basis (hash functions only, 100-bit security) and its limits [[S-1101]]. It also states the verification uses it proposes [[S-1102]] [[S-1103]].\n- **R2** not met. The rubric excludes results that are claimed but not public or not reproducible, and Attestable's figures are published without code, a protocol description or an artifact [[S-1101]].", "evidence": [ "S-1101", "S-1102", "S-1103" ], "next_level_gaps": [ "A public working implementation, or reproducible published end-to-end results, such as a paper with a protocol specification and benchmarks others can rerun.", "Any independent security analysis of the proof system." ], "confidence": "low", "assessed_by": [ "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Proves an 8-bit quantised variant of the model", "kind": "open-question", "severity": "significant", "status": "open", "description": "Attestable reports that matrix multiplications are dynamically quantised to 8-bit integers, while non-linear operations are proven in floating point. It reports that its IFEval result \"shows where the current quantization still needs improvement\" [[S-1101]]. The proven model is therefore a quantised variant, which a verifier must accept as the declared model.", "sources": [ "S-1101" ], "response": null }, { "assessment": true, "title": "A proof covers only the computation it is about", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Attestable states that \"a proof of some computation is not a proof of all computation\" and that a proof \"cannot discover a datacenter that was never declared\" [[S-1102]].", "sources": [ "S-1102" ], "response": null } ], "blockers": [ { "text": "No paper, protocol specification or code is public, so the reported results cannot be reproduced.", "theme": "adversarial-validation", "blocked_by": null, "sources": [ "S-1101" ] }, { "text": "Attestable reports a context window limited to 16K tokens.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-1101" ] }, { "text": "Covering computation that is not proven relies on proof-of-work accounting, which Attestable has only proposed.", "theme": "coverage-hidden-compute", "blocked_by": "M-0007", "sources": [ "S-1102" ] } ], "challenge_themes": [ "adversarial-validation", "coverage-hidden-compute", "performance-compatibility" ], "organizations": [ "O-0120" ], "people": [], "sources": [ { "source": "S-1101", "supports": "statement proven, security basis, performance figures, limitations (provider-reported)" }, { "source": "S-1102", "supports": "proposed verification uses and proof-of-work accounting; coverage argument (provider-reported)" }, { "source": "S-1103", "supports": "verification-firewall proposal with random sampling of outputs; stated threats (provider-reported)" } ], "concepts": [ "K-0010", "K-0024", "K-0020" ], "kind": "product", "developer": [ "O-0120" ], "realises": [ "M-0004" ], "homepage": "https://attestable.com/", "type": "implementation", "url": "https://trustbutveri.fyi/implementations/attestable-zk-inference/", "source_file": "content/implementations/attestable-zk-inference.md", "flags_all": [ "provider-reported" ], "body_markdown": "## What it is\n\nIn an August 2026 blog post, the company Attestable ([[O-0120]]) reports a prover for [[M-0004|zero-knowledge proofs of inference]] on \"production-scale transformer inference\" [[S-1101]]. The post links no paper, code or technical documentation [[S-1101]].\n\nAttestable reports that a proof \"can establish that a committed model produced a specific output from a specific input, without revealing the model's weights or other private information\" [[S-1101]].\n\n## How it works\n\n**The statement proven.** Attestable writes the proven statement as y = F(W, x, r), where:\n- F is the model;\n- W is the committed weights;\n- x is the input;\n- r is a random seed;\n- y is the output [[S-1101]].\n\nThe proof can also bind the model configuration, the decoding parameters and \"other policy-relevant values\" [[S-1101]]. A standalone CPU verifier checks it without the weights and without rerunning the model [[S-1101]].\n\n**Security basis.** Attestable reports that its results rest \"only on the security of cryptographic hash functions\" [[S-1101]]. It claims 100-bit security and post-quantum security [[S-1101]].\n\n**Arithmetic.** Matrix multiplications are dynamically quantised to 8-bit integers, and non-linear operations are proven in floating point [[S-1101]]. The post does not name the proof system [[S-1101]].\n\n**Proposed uses.** Attestable proposes two ways to deploy the prover:\n- **A verification firewall** at the external interfaces of an environment that holds model weights. It would select outputs for proof at random. The selection \"must remain unpredictable to the datacenter\" and happen only after each output is committed [[S-1103]]. Attestable describes its solution as \"already practical today\" [[S-1103]].\n- **Support for pacing agreements.** An auditor \"could verify that an accounted workload used an approved, unchanged model rather than a newly trained replacement\". Where an input classifier is required, a proof could show that the agreed classifier accepted the input [[S-1102]].\n\n## Evidence\nAll results come from Attestable's own blog. Attestable reports these figures on a single NVIDIA H100 GPU:\n- **Proofs.** Proof sizes range from 4.35 to 7.92 MiB, and CPU verification takes 157 to 648 milliseconds [[S-1101]].\n- **Throughput.** For a 31-billion-parameter Gemma model, it reports proving 53 tokens per second for one 16K-token sequence, and 77 tokens per second across four 4K-token sequences [[S-1101]].\n- **Accuracy.** It reports that its quantisation preserved performance on the GPQA Diamond reasoning benchmark better than a standard INT8 baseline [[S-1101]].\n- **Comparison.** It sets these figures against an ezkl result from the end of 2023: 16 minutes to prove a 1-million-parameter model on a CPU [[S-1101]].\n\nAs of September 2026 no independent reproduction of these results has been published.\n\n## Limitations\nAttestable lists its current limits:\n- a context window of up to 16K tokens;\n- matrix multiplications quantised to 8-bit integers [[S-1101]].\n\nIt reports that its IFEval result \"shows where the current quantization still needs improvement\" [[S-1101]].\n\nOn coverage, Attestable writes that \"a proof of some computation is not a proof of all computation\" [[S-1102]]. It proposes pairing inference proofs with [[M-0007|proof-of-work accounting]], which would need \"a credible estimate of the compute available\" to the actor [[S-1102]].\n\nFor the firewall proposal, Attestable lists threats that the proofs do not address:\n- physical attacks, such as probing memory buses;\n- leakage through timing, packet sizes, power and proof-generation latency [[S-1103]].\n\nAs of September 2026 the protocol is unpublished, and no attack on it or security analysis of it has been published.", "body_text": "What it is In an August 2026 blog post, the company Attestable (Attestable) reports a prover for zero-knowledge proofs of inference on \"production-scale transformer inference\" [S-1101]. The post links no paper, code or technical documentation [S-1101]. Attestable reports that a proof \"can establish that a committed model produced a specific output from a specific input, without revealing the model's weights or other private information\" [S-1101]. How it works The statement proven. Attestable writes the proven statement as y = F(W, x, r), where: - F is the model; - W is the committed weights; - x is the input; - r is a random seed; - y is the output [S-1101]. The proof can also bind the model configuration, the decoding parameters and \"other policy-relevant values\" [S-1101]. A standalone CPU verifier checks it without the weights and without rerunning the model [S-1101]. Security basis. Attestable reports that its results rest \"only on the security of cryptographic hash functions\" [S-1101]. It claims 100-bit security and post-quantum security [S-1101]. Arithmetic. Matrix multiplications are dynamically quantised to 8-bit integers, and non-linear operations are proven in floating point [S-1101]. The post does not name the proof system [S-1101]. Proposed uses. Attestable proposes two ways to deploy the prover: - A verification firewall at the external interfaces of an environment that holds model weights. It would select outputs for proof at random. The selection \"must remain unpredictable to the datacenter\" and happen only after each output is committed [S-1103]. Attestable describes its solution as \"already practical today\" [S-1103]. - Support for pacing agreements. An auditor \"could verify that an accounted workload used an approved, unchanged model rather than a newly trained replacement\". Where an input classifier is required, a proof could show that the agreed classifier accepted the input [S-1102]. Evidence All results come from Attestable's own blog. Attestable reports these figures on a single NVIDIA H100 GPU: - Proofs. Proof sizes range from 4.35 to 7.92 MiB, and CPU verification takes 157 to 648 milliseconds [S-1101]. - Throughput. For a 31-billion-parameter Gemma model, it reports proving 53 tokens per second for one 16K-token sequence, and 77 tokens per second across four 4K-token sequences [S-1101]. - Accuracy. It reports that its quantisation preserved performance on the GPQA Diamond reasoning benchmark better than a standard INT8 baseline [S-1101]. - Comparison. It sets these figures against an ezkl result from the end of 2023: 16 minutes to prove a 1-million-parameter model on a CPU [S-1101]. As of September 2026 no independent reproduction of these results has been published. Limitations Attestable lists its current limits: - a context window of up to 16K tokens; - matrix multiplications quantised to 8-bit integers [S-1101]. It reports that its IFEval result \"shows where the current quantization still needs improvement\" [S-1101]. On coverage, Attestable writes that \"a proof of some computation is not a proof of all computation\" [S-1102]. It proposes pairing inference proofs with proof-of-work accounting, which would need \"a credible estimate of the compute available\" to the actor [S-1102]. For the firewall proposal, Attestable lists threats that the proofs do not address: - physical attacks, such as probing memory buses; - leakage through timing, packet sizes, power and proof-generation latency [S-1103]. As of September 2026 the protocol is unpublished, and no attack on it or security analysis of it has been published.", "referenced_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "O-0120", "title": "Attestable", "url": "https://trustbutveri.fyi/organizations/attestable/" } ] }, { "id": "I-0016", "slug": "batch-invariant-inference-kernels", "title": "Batch-invariant inference kernels (Thinking Machines)", "aliases": [ "batch_invariant_ops", "VLLM_BATCH_INVARIANT", "SGLang deterministic inference" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "one_liner": "Open-source kernels from Thinking Machines Lab that make LLM outputs independent of batch size, adopted in vLLM and SGLang to give reproducible inference.", "summary": "Batch-invariant kernels make a language model give the same output for the same input whatever else the server is processing. Thinking Machines Lab identified changing batch sizes as the main reason LLM endpoints are nondeterministic, and published kernels whose reduction order does not depend on the batch. With them, 1,000 temperature-zero completions from a 235-billion-parameter model were identical. SGLang built its deterministic mode on the kernels, and vLLM's batch-invariant mode, still in beta, is based on the same work. For verification, identical outputs let a verifier that runs the same model, engine and hardware require an exact match instead of a tolerance. The kernels were built for reproducibility, not verification, and no verification system is documented using them. They cost throughput: 61.5% in the original test and 34% on average in SGLang.", "category": "cryptographic-computational", "secondary_categories": [], "verifies": [ { "claim": "C-0005", "role": "supporting", "note": "Makes exact-match recomputation of served outputs possible when the verifier runs the same model, engine and hardware (S-0016)." } ], "threat_model": "cooperative", "adversarial_evaluation": "none", "hardware_requirement": "none", "prover_cooperation": "required", "confidentiality": "revealing", "depends_on": [], "readiness": { "assessment": true, "level": "R2", "scope": "exact recomputation of served outputs by a verifier, with a cooperating provider", "rubric_version": "1.1", "rationale": "Public kernels made a 235-billion-parameter model's outputs identical across 1,000 runs and underpin the deterministic modes of two production engines, but no verification system is documented using them.\n\n- **R1** met: Thinking Machines sets out the design and the property it gives, outputs that do not depend on batch size [[S-1009]]. The condition for identical outputs, a fixed model, inference implementation and device, is stated in the verification literature [[S-0016]].\n- **R2** met: the code is public under an MIT licence, with a deterministic vLLM example [[S-1813]]. With the kernels, 1,000 temperature-zero completions from Qwen3-235B were identical, against 80 unique completions without them [[S-1009]].\n- **R3** not met for this use. The kernels have reached production engines: SGLang builds its deterministic mode on them [[S-1012]], and vLLM's developers state that its batch-invariant mode, labelled beta, is based on the same work [[S-1814]] [[S-1013]]. The kernels were built for reproducibility, such as on-policy reinforcement learning [[S-1009]]. [[I-0015|Gensyn's REE]] and Eigen Labs' EigenAI, which build verification services on exact replay, use their own kernels [[S-1809]] [[S-3020]]. No party is documented relying on these kernels for a verification decision.\n- **R4** not met: no independent security evaluation has been published.\n\nConfidence is low, because whether an engine's deterministic mode counts as production-grade for verification is a judgment call.", "evidence": [ "S-1009", "S-1813", "S-1012", "S-1013", "S-1814", "S-0016", "S-1809", "S-3020" ], "next_level_gaps": [ "A production-grade verification stack that uses these kernels for exact-match checks, or a party other than the developers relying on such checks for a verification decision." ], "confidence": "low", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [], "blockers": [ { "text": "Batch invariance costs throughput: on Qwen3-8B the improved deterministic build took 42 s against 26 s for vLLM's default, and SGLang reports an average slowdown of 34.35% on its FlashInfer and FlashAttention 3 backends.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-1009", "S-1012" ] }, { "text": "Outputs are identical only while the model, inference implementation and device stay fixed, so provider and verifier must run the same stack.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-0016" ] } ], "challenge_themes": [ "performance-compatibility", "adversarial-validation", "privacy-leakage" ], "organizations": [], "people": [], "sources": [ { "source": "S-1009", "supports": "main cause of nondeterminism; operations made batch-invariant; fixed split size for attention; vLLM integration; Qwen3-235B and Qwen3-8B results; on-policy RL motivation (provider-reported)", "locator": "whole post" }, { "source": "S-1813", "supports": "MIT-licensed library, replaced PyTorch operations, deterministic vLLM example", "locator": "README" }, { "source": "S-1012", "supports": "SGLang's integration of the kernels, its attention kernels, the 61.5% figure for the original post and SGLang's average slowdown", "locator": "whole post" }, { "source": "S-1013", "supports": "vLLM batch-invariant mode, beta status, supported hardware (NVIDIA compute capability 8.0+, Intel XPUs), tested model families", "locator": "whole page" }, { "source": "S-1814", "supports": "vLLM developers' statement that batch-invariance support is based on the Thinking Machines post; open work items" }, { "source": "S-0016", "supports": "identical results only with fixed model, implementation and device; statistical verification for heterogeneous stacks" }, { "source": "S-1809", "supports": "RepOps, Gensyn's own reproducible operators (rationale comparison)", "locator": "§3.2" }, { "source": "S-3020", "supports": "EigenAI's own deterministic kernels (rationale comparison; provider-reported)", "locator": "abstract; kernels section" } ], "concepts": [ "K-0008", "K-0009" ], "kind": "open-source-project", "developer": [ "O-0214" ], "realises": [ "M-0002" ], "homepage": "https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/", "repo": "https://github.com/thinking-machines-lab/batch_invariant_ops", "type": "implementation", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/", "source_file": "content/implementations/batch-invariant-inference-kernels.md", "flags_all": [ "provider-reported" ], "body_markdown": "## What it is\n\nThinking Machines Lab published batch-invariant kernels for LLM inference in September 2025 [[S-1009]]. Its post argues that \"the primary reason nearly all LLM inference endpoints are nondeterministic is that the load (and thus batch-size) nondeterministically varies\" [[S-1009]]. The companion library, batch_invariant_ops, is MIT-licensed [[S-1813]].\n\nThe kernels are one route to [[M-0002|deterministic and bit-exact inference]]. SGLang built its deterministic mode on them [[S-1012]], and vLLM's developers state that its batch-invariant mode is based on the post [[S-1814]].\n\n## How it works\n\nA kernel is batch-invariant if it computes each output element with the same reduction order whatever the batch size [[S-1009]]. The post identifies three operations that involve reductions: RMSNorm, matrix multiplication and attention [[S-1009]]. For attention it fixes the size of each split along the key-value dimension, rather than the number of splits [[S-1009]].\n\nThe library replaces PyTorch's matrix multiplication (torch.mm and torch.addmm), log-softmax and mean through torch.Library, so existing models can use the kernels with small code changes [[S-1813]]. The post runs the result in vLLM through its FlexAttention backend [[S-1009]]. vLLM now exposes its own mode with the environment variable VLLM_BATCH_INVARIANT=1, on NVIDIA GPUs of compute capability 8.0 or higher and on Intel XPUs [[S-1013]].\n\nFor verification, identical outputs hold when the model, inference implementation and device are fixed [[S-0016]]. A verifier that runs the same stack can then require an exact match, instead of the tolerance that [[M-0001|sampled recomputation]] schemes allow.\n\n## Evidence\n- For 1,000 temperature-zero completions of one prompt on Qwen3-235B, Thinking Machines reports 80 unique outputs with default kernels and one with batch-invariant kernels [[S-1009]].\n- The library's vLLM example gave 18 unique samples out of 1,000 completions without the upstream vLLM change, and one with it [[S-1813]].\n- SGLang integrated the mean, log-softmax and matrix-multiplication kernels and wrote batch-invariant attention kernels for several backends. It reports an average slowdown of 34.35% on its FlashInfer and FlashAttention 3 backends, against the 61.5% in the original post [[S-1012]].\n- vLLM documents its batch-invariant mode as beta and lists tested model families including DeepSeek V3, Qwen3, Llama 3.1 and GPT-OSS [[S-1013]].\n\n## Limitations\n- Outputs are identical only while the model, inference implementation and device stay fixed. For varied stacks and GPU types, DiFR's authors expect statistical verification to remain necessary [[S-0016]].\n- In Thinking Machines' Qwen3-8B test, vLLM's default took 26 s, the unoptimised deterministic build 55 s and the build with an improved attention kernel 42 s [[S-1009]].\n- vLLM's mode is in beta, and its tracking issue lists open work on performance, AMD hardware and speculative decoding [[S-1013]] [[S-1814]].\n- The post's motivation is reproducibility, including on-policy reinforcement learning, not verification against a cheating provider [[S-1009]].", "body_text": "What it is Thinking Machines Lab published batch-invariant kernels for LLM inference in September 2025 [S-1009]. Its post argues that \"the primary reason nearly all LLM inference endpoints are nondeterministic is that the load (and thus batch-size) nondeterministically varies\" [S-1009]. The companion library, batchinvariantops, is MIT-licensed [S-1813]. The kernels are one route to deterministic and bit-exact inference. SGLang built its deterministic mode on them [S-1012], and vLLM's developers state that its batch-invariant mode is based on the post [S-1814]. How it works A kernel is batch-invariant if it computes each output element with the same reduction order whatever the batch size [S-1009]. The post identifies three operations that involve reductions: RMSNorm, matrix multiplication and attention [S-1009]. For attention it fixes the size of each split along the key-value dimension, rather than the number of splits [S-1009]. The library replaces PyTorch's matrix multiplication (torch.mm and torch.addmm), log-softmax and mean through torch.Library, so existing models can use the kernels with small code changes [S-1813]. The post runs the result in vLLM through its FlexAttention backend [S-1009]. vLLM now exposes its own mode with the environment variable VLLMBATCHINVARIANT=1, on NVIDIA GPUs of compute capability 8.0 or higher and on Intel XPUs [S-1013]. For verification, identical outputs hold when the model, inference implementation and device are fixed [S-0016]. A verifier that runs the same stack can then require an exact match, instead of the tolerance that sampled recomputation schemes allow. Evidence - For 1,000 temperature-zero completions of one prompt on Qwen3-235B, Thinking Machines reports 80 unique outputs with default kernels and one with batch-invariant kernels [S-1009]. - The library's vLLM example gave 18 unique samples out of 1,000 completions without the upstream vLLM change, and one with it [S-1813]. - SGLang integrated the mean, log-softmax and matrix-multiplication kernels and wrote batch-invariant attention kernels for several backends. It reports an average slowdown of 34.35% on its FlashInfer and FlashAttention 3 backends, against the 61.5% in the original post [S-1012]. - vLLM documents its batch-invariant mode as beta and lists tested model families including DeepSeek V3, Qwen3, Llama 3.1 and GPT-OSS [S-1013]. Limitations - Outputs are identical only while the model, inference implementation and device stay fixed. For varied stacks and GPU types, DiFR's authors expect statistical verification to remain necessary [S-0016]. - In Thinking Machines' Qwen3-8B test, vLLM's default took 26 s, the unoptimised deterministic build 55 s and the build with an improved attention kernel 42 s [S-1009]. - vLLM's mode is in beta, and its tracking issue lists open work on performance, AMD hardware and speculative decoding [S-1013] [S-1814]. - The post's motivation is reproducibility, including on-policy reinforcement learning, not verification against a cheating provider [S-1009].", "referenced_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "O-0214", "title": "Thinking Machines Lab", "url": "https://trustbutveri.fyi/organizations/thinking-machines-lab/" } ] }, { "id": "I-0002", "slug": "difr", "title": "DiFR (Divergence From Reference)", "aliases": [ "Token-DiFR", "Activation-DiFR" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "one_liner": "DiFR checks that an inference provider ran its declared model by comparing output tokens or activations with a trusted re-run using the same random seed.", "summary": "DiFR (Divergence From Reference) is a pair of methods for checking that an inference provider ran the model and settings it declared. Both tolerate the small numerical differences that make re-runs disagree. In Token-DiFR, provider and verifier share the random seed for token sampling. The verifier re-runs the sequence and scores how far each claimed token departs from the reference model's choice. Activation-DiFR compares compressed fingerprints of internal activations instead. On models of 8 to 30 billion parameters on A100 and H200 GPUs, both detected 4-bit quantization almost perfectly (AUC above 0.999). Token-DiFR did so within 300 output tokens and Activation-DiFR within 2. The code is public, with a vLLM integration, and another team has built it into a prototype. The verifier needs the model weights and must know the provider's sampling procedure. Its statistical tolerance leaves a covert channel, which an independent study widened in a related exfiltration detector.", "technical": "- The specification φ fixes the architecture and weights, numerical precision, sampling hyperparameters and the seed of the pseudorandom number generator [[S-0016]].\n- Token-DiFR works with Gumbel-Max sampling. From the shared seed, both parties derive the same Gumbel noise vector g. The verifier computes post-Gumbel scores z_i = l_i + T·g_i from its reference logits l and temperature T, and takes the token it would have chosen. It scores each claimed token by the logit-margin difference between the two tokens, clipped at a maximum. Scores are pooled per batch. A batch is flagged if the statistic exceeds a threshold set on a calibration set generated under φ on trusted hardware [[S-0016]]. Verification is one prefill pass over prompt plus output [[S-0016]]. The companion exfiltration paper gives a variant for inverse-probability-transform samplers [[S-0015]].\n- Activation-DiFR has both parties generate the same random orthogonal projection P from a shared seed. P is k by D, with k much smaller than D. The provider sends f = P·a for activation vector a, possibly only for every J-th token. The verifier checks the L2 distance to its own projection [[S-0016]]. For detecting FP8 KV-cache quantization on H200 GPUs, the authors report an AUC of at least 0.9999 with under 0.5 bytes per token. For Qwen3-30B-A3B on A100 GPUs, an AUC of 0.999 took 7.25 bytes per token, against 32 for TOPLOC [[S-0016]].\n- DiFR works with unmodified vLLM, which exposes per-request seeds as a standard sampling parameter [[S-0016]].\n- At temperature zero, Token-DiFR needs no seed synchronization. The verifier checks that the provider chose the most likely token at each position. The authors warn that such greedy spot checks are open to selective cheating. A provider could serve the declared model at temperature zero and a degraded one otherwise [[S-0016]].", "category": "cryptographic-computational", "secondary_categories": [], "verifies": [ { "claim": "C-0005", "role": "primary", "note": "Checks that outputs are consistent with the declared model, precision and sampling configuration." }, { "claim": "C-0009", "role": "supporting", "note": "Used as the estimator in a weight-exfiltration detection scheme (S-0015)." } ], "threat_model": "adversarial", "adversarial_evaluation": "analysis", "hardware_requirement": "none", "prover_cooperation": "required", "confidentiality": "revealing", "depends_on": [], "readiness": { "assessment": true, "level": "R2", "scope": "checking that outputs match the declared model, precision and sampling settings", "rubric_version": "1.1", "rationale": "The code is public and the results reproduce on data-centre GPUs, but only its developers rely on it and no one has independently evaluated its consistency check.\n\n- **R1** met: the paper states the verification claim, the specification the provider must follow and the trust assumptions [[S-0016]]. The assumptions are a trusted reference, calibration on trusted hardware and synchronized seeds. A companion paper embeds the method in a formal security game [[S-0015]].\n- **R2** met: public MIT-licensed code with a vLLM integration exists [[S-1005]]. Results are reproducible on A100 and H200 GPUs with models of 8 to 30 billion parameters [[S-0016]]. A separate team built it into a prototype and tested models of up to 120 billion parameters [[S-1006]] [[S-1007]].\n- **R3** not met: Amodo's prototype is research code, not production software [[S-1007]], and no party is documented relying on DiFR for a verification decision.\n- **R4** not met: as of September 2026 no independent audit, red-team or peer-reviewed security analysis of DiFR's consistency check has been published. Amodo lists recomputation red-teaming as not started [[S-1008]]. Amodo's own tests are independent of the authors but used two fine-tuned models, which by Amodo's account cover only a fraction of possible attacks [[S-3001]]. The one independent attack targets a weight-exfiltration detector built on the same Gumbel-margin statistic [[S-1507]]. That attack bears on the supporting exfiltration use, assessed under [[M-0024]]. It does not bear on the primary use.", "evidence": [ "S-0016", "S-0015", "S-1005", "S-1006", "S-1007", "S-1008", "S-1507", "S-3001" ], "next_level_gaps": [ "Reliance by a party other than the developers on DiFR for a verification decision, or a production-grade release.", "An independent public security evaluation (audit, red-team or peer-reviewed analysis) against adaptive adversaries." ], "confidence": "medium", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Statistical tolerance leaves a covert channel", "kind": "demonstrated-attack", "severity": "significant", "status": "open", "description": "Statistical schemes can put an upper bound on an adversary's covert bandwidth, but cannot close it [[S-0020]]. In the companion exfiltration study, the detector cut exfiltratable information to under 0.5% under benign prompt traffic. It did not cut it to zero [[S-0015]]. Its authors called the channel's size under adversarial prompts an open empirical question [[S-0015]]. An independent study attacked that detector, which uses the same Gumbel-margin statistic. An adversary who controls the prompts roughly doubled the bits leaked per token. This cut the slowdown from over 200 times to 60–118 times [[S-1507]]. The attack widens the exfiltration bound. It does not target DiFR's check that outputs match the declared configuration.", "sources": [ "S-0020", "S-0015", "S-1507" ], "response": null }, { "assessment": true, "title": "Mixed hardware widens the honest baseline", "kind": "open-question", "severity": "minor", "status": "open", "description": "For Qwen3-30B-A3B, benign differences between A100 and H200 deployments broadened the honest score distribution. This made the smallest tested change, a temperature raised by 0.1, harder to separate.", "sources": [ "S-0016" ], "response": null }, { "assessment": true, "title": "Speculative decoding and multi-model sampling not evaluated", "kind": "open-question", "severity": "significant", "status": "open", "description": "The algorithms and experiments cover sampling from a single LLM. Speculative decoding was not evaluated. The authors sketch an extension to one speculative-decoding algorithm, without experiments. They note that other variants would need modified verification and extra metadata.", "sources": [ "S-0016" ], "response": null } ], "blockers": [ { "text": "The verifier needs the model weights, so outsiders cannot use the method to verify providers of closed-weights models.", "theme": "privacy-leakage", "blocked_by": null, "sources": [ "S-0016" ] }, { "text": "The verifier must know and match the provider's sampling procedure, and in one prototype a sampling mismatch in a newer vLLM version produced large spurious logit differences.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-0016", "S-1006" ] }, { "text": "No independent red-team of DiFR's consistency check has been published, Amodo rates recomputation red-teaming 'not started', and the one independent attack study targets an exfiltration detector built on the same statistic.", "theme": "adversarial-validation", "blocked_by": null, "sources": [ "S-1008", "S-1507" ] } ], "challenge_themes": [ "adversarial-validation", "protocol-soundness", "privacy-leakage", "performance-compatibility" ], "organizations": [ "O-0101" ], "people": [], "sources": [ { "source": "S-0016", "supports": "method, specification, experiments, results, comparison with TOPLOC and distributional methods, provider case study, deployment considerations, limitations, speculative-decoding sketch", "locator": "abstract; §2-3; §5; §5.1; §6.2; Table 1; §7.2-7.4; Appendix D; Appendix F" }, { "source": "S-0015", "supports": "companion security game and exfiltration results using Token-DiFR estimators; adversarial prompts an open question", "locator": "abstract; contributions; §4; §5.3" }, { "source": "S-1005", "supports": "public code, licence, vLLM and API modes", "locator": "README" }, { "source": "S-1006", "supports": "independent prototype, scale and vLLM sampling mismatch", "locator": "whole note" }, { "source": "S-1007", "supports": "prototype code, research code not production software; modified copy of the difr library", "locator": "README" }, { "source": "S-3001", "supports": "first Amodo prototype reusing the DiFR library; tests against two LoRA fine-tuning attacks on 8×H100; pass-rate gap; short answers; limits of the tests", "locator": "setup; results; limitations" }, { "source": "S-0020", "supports": "limits of statistical verification", "locator": "§1" }, { "source": "S-1008", "supports": "status of red-teaming", "locator": "status items" }, { "source": "S-1507", "supports": "independent attack on a weight-exfiltration detector built on the Token-DiFR Gumbel-margin statistic; scope limited to the exfiltration bound", "locator": "abstract; introduction" } ], "concepts": [ "K-0008", "K-0009", "K-0020" ], "kind": "research-prototype", "developer": [], "realises": [ "M-0001" ], "repo": "https://github.com/adamkarvonen/difr", "type": "implementation", "url": "https://trustbutveri.fyi/implementations/difr/", "source_file": "content/implementations/difr.md", "flags_all": [ "provider-reported" ], "body_markdown": "## What it is\n\nDiFR (Divergence From Reference) is a pair of methods for checking that an inference provider performed inference correctly. The checks work despite the benign numerical noise that makes re-runs of the same model differ [[S-0016]]. Authors at the ML Alignment and Theory Scholars (MATS) programme, FAR AI, Anthropic and Harvard University introduced it in a 2025 preprint [[S-0016]].\n\n- **Token-DiFR** compares the tokens a provider generated with the predictions of a trusted reference run that uses the same random seed [[S-0016]].\n- **Activation-DiFR** compares compressed fingerprints of internal activations [[S-0016]].\n\nA companion paper uses the same family of estimators to detect model weights hidden in inference outputs [[S-0015]]. The code is public under an MIT licence [[S-1005]].\n\n## How it works\n\nThe provider commits to a specification: model and weights, numerical precision, sampling settings and random seed [[S-0016]]. Token-DiFR relies on Gumbel-Max sampling, which picks each token by adding seeded random noise to the model's scores and taking the largest. With a shared seed, the verifier knows the random noise the provider used [[S-0016]]. It can therefore compute which token the reference model would have chosen at each position, and how far the claimed token falls short of it [[S-0016]].\n\nThe verifier pools these margins over many tokens into a score. It compares the score with a threshold calibrated on trusted hardware [[S-0016]]. The verifier feeds the prompt and output through the model once, which gives logits and activations for every position at once [[S-0016]]. Because the seed pins down sampling, over 98% of tokens match exactly between provider and verifier [[S-0016]].\n\nActivation-DiFR uses a shared random orthogonal projection to compress activations into short fingerprints. The verifier compares distances between its fingerprints and the provider's [[S-0016]]. This checks the forward pass but not the sampling step [[S-0016]].\n\n## Evidence\n- The authors tested Llama 3.1 8B-Instruct, Qwen3-8B and Qwen3-30B-A3B on 2,000 UltraChat prompts [[S-0016]]. The four inference configurations were H200 with four-way tensor parallelism, A100 with and without it, and H200 without it running Hugging Face [[S-0016]].\n- The faults tested were FP8 key-value cache quantization, 4-bit model quantization, a temperature of 1.1 instead of 1.0, an incorrect seed, and a simulated sampling bug that picks uniformly among the top-k tokens 1% of the time [[S-0016]].\n- The authors report that Token-DiFR detects 4-bit quantization with an AUC above 0.999 within 300 output tokens [[S-0016]]. Activation-DiFR does so within 2 tokens, while cutting communication by 25–75% relative to existing methods [[S-0016]].\n- The authors report that Activation-DiFR Pareto-dominates [[I-0001|TOPLOC]] in communication cost against detection accuracy when detecting FP8 KV-cache quantization [[S-0016]]. They also cite a report that a distributional test called RUT failed to reliably separate 8-bit quantized models. Token-DiFR detects such differences within a few thousand tokens [[S-0016]].\n- The companion paper's detector limited exfiltratable information to under 0.5% at a false-positive rate below 0.01% on a mixture-of-experts Qwen 30B model, under benign prompt traffic [[S-0015]]. An independent study found that choosing the prompts roughly doubles the leakage per token [[S-1507]].\n- In a small case study in November 2025, the authors spot-checked five public Llama 3.1 8B API deployments with greedy queries. Scores varied substantially across them. The authors read higher scores as evidence of a different effective specification, such as a different chat template, not as a direct measure of model quality [[S-0016]].\n- [[O-0101|Amodo Design]], a separate team, built the DiFR library into recomputation prototypes [[S-3001]] [[S-1007]]. Its first prototype, on eight H100 GPUs, judged each request on its own. The best threshold separated the pass rates of the honest model and two models fine-tuned to append text by about 25 percentage points, and most missed cases were answers shorter than five tokens [[S-3001]]. A later asynchronous version was tested on models of 1.5 to 120 billion parameters on H100 and H200 GPUs [[S-1006]].\n\n## Limitations\n- The methods need the model weights [[S-0016]]. Providers can check their own traffic and users can check open-weights models, but the methods cannot verify closed-weights providers [[S-0016]].\n- Deployment requires knowing the provider's sampling procedure [[S-0016]]. In Amodo's prototype, vLLM v0.25.1 used a different random-number implementation from the one DiFR expected. This produced large logit differences until Amodo forced the older model runner [[S-1006]].\n- For Qwen3-30B-A3B, mixing A100 and H200 reference runs made the smallest tested change harder to separate from honest runs [[S-0016]].\n- Only single-model sampling was studied. Speculative decoding was not evaluated [[S-0016]].\n- A statistical tolerance can bound covert bandwidth but cannot close it [[S-0020]].\n- Adversarial testing is limited. The authors show that simple adversarial temperature tuning defeats a cross-entropy baseline, while Token-DiFR still detects the change [[S-0016]]. An independent study reports that attacker-chosen prompts roughly double the leakage allowed by a weight-exfiltration detector built on the same Gumbel-margin statistic. This cut the detector's slowdown to 60–118 times [[S-1507]]. Amodo rates red-teaming of recomputation schemes as not started [[S-1008]].", "body_text": "What it is DiFR (Divergence From Reference) is a pair of methods for checking that an inference provider performed inference correctly. The checks work despite the benign numerical noise that makes re-runs of the same model differ [S-0016]. Authors at the ML Alignment and Theory Scholars (MATS) programme, FAR AI, Anthropic and Harvard University introduced it in a 2025 preprint [S-0016]. - Token-DiFR compares the tokens a provider generated with the predictions of a trusted reference run that uses the same random seed [S-0016]. - Activation-DiFR compares compressed fingerprints of internal activations [S-0016]. A companion paper uses the same family of estimators to detect model weights hidden in inference outputs [S-0015]. The code is public under an MIT licence [S-1005]. How it works The provider commits to a specification: model and weights, numerical precision, sampling settings and random seed [S-0016]. Token-DiFR relies on Gumbel-Max sampling, which picks each token by adding seeded random noise to the model's scores and taking the largest. With a shared seed, the verifier knows the random noise the provider used [S-0016]. It can therefore compute which token the reference model would have chosen at each position, and how far the claimed token falls short of it [S-0016]. The verifier pools these margins over many tokens into a score. It compares the score with a threshold calibrated on trusted hardware [S-0016]. The verifier feeds the prompt and output through the model once, which gives logits and activations for every position at once [S-0016]. Because the seed pins down sampling, over 98% of tokens match exactly between provider and verifier [S-0016]. Activation-DiFR uses a shared random orthogonal projection to compress activations into short fingerprints. The verifier compares distances between its fingerprints and the provider's [S-0016]. This checks the forward pass but not the sampling step [S-0016]. Evidence - The authors tested Llama 3.1 8B-Instruct, Qwen3-8B and Qwen3-30B-A3B on 2,000 UltraChat prompts [S-0016]. The four inference configurations were H200 with four-way tensor parallelism, A100 with and without it, and H200 without it running Hugging Face [S-0016]. - The faults tested were FP8 key-value cache quantization, 4-bit model quantization, a temperature of 1.1 instead of 1.0, an incorrect seed, and a simulated sampling bug that picks uniformly among the top-k tokens 1% of the time [S-0016]. - The authors report that Token-DiFR detects 4-bit quantization with an AUC above 0.999 within 300 output tokens [S-0016]. Activation-DiFR does so within 2 tokens, while cutting communication by 25–75% relative to existing methods [S-0016]. - The authors report that Activation-DiFR Pareto-dominates TOPLOC in communication cost against detection accuracy when detecting FP8 KV-cache quantization [S-0016]. They also cite a report that a distributional test called RUT failed to reliably separate 8-bit quantized models. Token-DiFR detects such differences within a few thousand tokens [S-0016]. - The companion paper's detector limited exfiltratable information to under 0.5% at a false-positive rate below 0.01% on a mixture-of-experts Qwen 30B model, under benign prompt traffic [S-0015]. An independent study found that choosing the prompts roughly doubles the leakage per token [S-1507]. - In a small case study in November 2025, the authors spot-checked five public Llama 3.1 8B API deployments with greedy queries. Scores varied substantially across them. The authors read higher scores as evidence of a different effective specification, such as a different chat template, not as a direct measure of model quality [S-0016]. - Amodo Design, a separate team, built the DiFR library into recomputation prototypes [S-3001] [S-1007]. Its first prototype, on eight H100 GPUs, judged each request on its own. The best threshold separated the pass rates of the honest model and two models fine-tuned to append text by about 25 percentage points, and most missed cases were answers shorter than five tokens [S-3001]. A later asynchronous version was tested on models of 1.5 to 120 billion parameters on H100 and H200 GPUs [S-1006]. Limitations - The methods need the model weights [S-0016]. Providers can check their own traffic and users can check open-weights models, but the methods cannot verify closed-weights providers [S-0016]. - Deployment requires knowing the provider's sampling procedure [S-0016]. In Amodo's prototype, vLLM v0.25.1 used a different random-number implementation from the one DiFR expected. This produced large logit differences until Amodo forced the older model runner [S-1006]. - For Qwen3-30B-A3B, mixing A100 and H200 reference runs made the smallest tested change harder to separate from honest runs [S-0016]. - Only single-model sampling was studied. Speculative decoding was not evaluated [S-0016]. - A statistical tolerance can bound covert bandwidth but cannot close it [S-0020]. - Adversarial testing is limited. The authors show that simple adversarial temperature tuning defeats a cross-entropy baseline, while Token-DiFR still detects the change [S-0016]. An independent study reports that attacker-chosen prompts roughly double the leakage allowed by a weight-exfiltration detector built on the same Gumbel-margin statistic. This cut the detector's slowdown to 60–118 times [S-1507]. Amodo rates red-teaming of recomputation schemes as not started [S-1008].", "referenced_by": [ { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ] }, { "id": "I-0014", "slug": "ezkl", "title": "EZKL", "aliases": [ "ezkl", "Easy Zero-Knowledge Inference" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "EZKL is a library from Zkonduit that turns neural networks into zero-knowledge circuits, so a prover can show an output came from a committed model.", "summary": "EZKL is a library and command-line tool, developed by the company Zkonduit, for proving neural-network inference in zero knowledge. It compiles a model exported in the ONNX format into a halo2 circuit, quantizing its values, so a prover can show that a model produced an output without revealing whichever of the model or the data is private. Its source code is public. A Trail of Bits audit in 2025 found proof-forgery bugs in its circuits and smart contracts, and all the high-severity ones were fixed. Trail of Bits reports that other projects use its verifier contracts in production. Its main limit for AI verification is scale: the largest language model in South et al.'s published results is a 250,000-parameter nanoGPT, which took 46 minutes to prove with a 219 GB proving key. Quantization also leaves a gap between the proven circuit and the full-precision model.", "category": "cryptographic-computational", "secondary_categories": [], "verifies": [ { "claim": "C-0005", "role": "primary", "note": "Proves an output follows from a committed model. South et al.'s results reach about a million parameters (S-0024)." } ], "threat_model": "adversarial", "adversarial_evaluation": "independent-red-team", "hardware_requirement": "none", "prover_cooperation": "required", "confidentiality": "partial", "depends_on": [], "readiness": { "assessment": true, "level": "R2", "scope": "proving an output came from a committed model, against a prover who cheats", "rubric_version": "1.1", "rationale": "Public, independently audited code proves small models against a cheating prover, but its published results stop far below the size of the language models that verification claims concern.\n\n- **R1** met: the claim is that a stated model produced an output from a stated input, with either kept private [[S-1806]]. South et al. set out its use for verifiable evaluations of models with private weights [[S-0024]].\n- **R2** met: the code is public [[S-1806]]. The stated adversary is a prover who tries to convince a verifier of an incorrect result, and an independent audit tested for exactly that [[S-0070]]. South et al. report end-to-end proofs for models of up to about a million parameters [[S-0024]].\n- **R3** not met for this use, which is checking that a served language model is the declared one. The library is public and versioned [[S-1806]], and Trail of Bits reports that other projects use its attestation and verifier contracts in production [[S-0070]]. No source says what those projects prove, and no party is documented relying on EZKL to verify a language model's outputs. The largest language model South et al. prove with it has 250,000 parameters [[S-0024]], far below the models that claims about served AI concern. This matches the assessment of [[M-0004]].\n- **R4** not met, because R3 is not. Trail of Bits' 2025 audit would otherwise count: at its fix review no high-severity finding remained unresolved [[S-0070]].", "evidence": [ "S-1806", "S-0024", "S-0070" ], "next_level_gaps": [ "A production-grade release that proves language models at the scale verification claims concern, or reliance by another party on such proofs for a verification decision." ], "confidence": "medium", "assessed_by": [ "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Circuit and contract bugs allowed forged proofs", "kind": "demonstrated-attack", "severity": "critical", "status": "mitigated", "description": "Trail of Bits found three high-severity soundness bugs in EZKL's circuits: an unsound shuffle argument for min, max and top-k, a decomposition that did not fix the sign of zero, and missing range checks for division and reciprocals. Each would let a malicious prover convince a verifier of incorrect calculations, for example that [1,1,1] is a valid permutation of [1,2,3]. It also found four ways to bypass the data attestation and KZG commitments in EZKL's smart contracts. All were resolved at the March 2025 fix review. Some fixes were first made in private repositories, to allow disclosure to projects using the contracts in production [[S-0070]].", "sources": [ "S-0070" ], "response": null }, { "assessment": true, "title": "Quantization can activate a backdoor dormant in the full-precision model", "kind": "demonstrated-attack", "severity": "significant", "status": "open", "description": "EZKL quantizes values to represent them in a finite field. Trail of Bits built a ResNet-18 whose backdoor is dormant at full precision and active after EZKL's quantization. Larger models and smaller quantization scales make the attack easier. Whether the backdoor persists through the witness and proof stages was left for further investigation. The fix was documentation of the risk [[S-0070]].", "sources": [ "S-0070" ], "response": null } ], "blockers": [ { "text": "Proving cost grows steeply with model size: a 250,000-parameter nanoGPT took 2,781 s to prove and needed a 219 GB proving key, which South et al. name as the main limit on model size.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-0024" ] } ], "challenge_themes": [ "performance-compatibility", "adversarial-validation", "evidence-binding" ], "organizations": [], "people": [], "sources": [ { "source": "S-1806", "supports": "what EZKL proves, ONNX input, halo2 backend, quantization note, maintainer (provider-reported)", "locator": "README" }, { "source": "S-0024", "supports": "verifiable evaluation attestations with EZKL; model sizes, proving and verification times, proving-key sizes; proving key as the main limit", "locator": "abstract; §6.1 Table 1" }, { "source": "S-0070", "supports": "independent audit: scope, soundness and contract findings, quantization backdoor, coverage limits, production use of the contracts, fix review", "locator": "Executive Summary; Project Coverage; TOB-EZKL-4 to 6, 13 to 17, 34; App. D" } ], "concepts": [ "K-0010", "K-0024", "K-0001", "K-0002" ], "kind": "product", "developer": [ "O-0213" ], "realises": [ "M-0004" ], "homepage": "https://docs.ezkl.xyz/", "repo": "https://github.com/zkonduit/ezkl", "type": "implementation", "url": "https://trustbutveri.fyi/implementations/ezkl/", "source_file": "content/implementations/ezkl.md", "flags_all": [], "body_markdown": "## What it is\n\nEZKL (\"Easy Zero-Knowledge Inference\") is a library and command-line tool for proving neural-network inference in zero knowledge, developed by Zkonduit [[S-1806]]. It supports statements such as \"I ran this publicly available neural network on some private data and it produced this output\", and the converse with a private model and public data [[S-1806]]. It implements [[M-0004|zero-knowledge proofs of inference]].\n\n## How it works\n\nEZKL takes a model exported in the ONNX format, for example from PyTorch or TensorFlow, and turns it into a circuit for the halo2 proof system [[S-1806]]. Operations are quantized on the way, so EZKL's outputs can differ slightly from the original model's [[S-1806]]. A verifier checks the resulting proof natively or through a smart contract on a blockchain [[S-0070]].\n\nSouth et al. use EZKL to bundle proofs over a dataset into an attestation that a model with fixed private weights reaches a stated score on public inputs [[S-0024]].\n\n## Evidence\n- South et al., whose authors include two from EZKL, report proofs for eight models, from a 30-parameter linear regression to a 1.07-million-parameter VAE decoder [[S-0024]]. A 250,000-parameter nanoGPT took 2,781 s to prove and 2.69 s to verify, with a 219 GB proving key [[S-0024]].\n- Trail of Bits reviewed EZKL and its verifier contracts for Zkonduit in January 2025, with 11 engineer-weeks of effort [[S-0070]]. It reported 34 findings, 8 of high severity, including circuit soundness bugs and ways to bypass data attestation. At its March 2025 fix review every high-severity finding was resolved [[S-0070]].\n- Trail of Bits reports that other projects use EZKL's attestation and verifier contracts in production [[S-0070]].\n\n## Limitations\n- South et al. write that proof time and resource requirements \"grow dramatically with large models\", and that the proving key's size limits model size [[S-0024]].\n- The proof covers the quantized circuit. Trail of Bits showed that quantization can activate a backdoor that is dormant in the full-precision model [[S-0070]].\n- The audit did not fully review several higher-level operations, such as einsum and scatter_nd, or all of the command-line tools [[S-0070]].\n- Limits common to proofs of inference, such as proofs that do not bind the computation spent, are covered in [[M-0004]].", "body_text": "What it is EZKL (\"Easy Zero-Knowledge Inference\") is a library and command-line tool for proving neural-network inference in zero knowledge, developed by Zkonduit [S-1806]. It supports statements such as \"I ran this publicly available neural network on some private data and it produced this output\", and the converse with a private model and public data [S-1806]. It implements zero-knowledge proofs of inference. How it works EZKL takes a model exported in the ONNX format, for example from PyTorch or TensorFlow, and turns it into a circuit for the halo2 proof system [S-1806]. Operations are quantized on the way, so EZKL's outputs can differ slightly from the original model's [S-1806]. A verifier checks the resulting proof natively or through a smart contract on a blockchain [S-0070]. South et al. use EZKL to bundle proofs over a dataset into an attestation that a model with fixed private weights reaches a stated score on public inputs [S-0024]. Evidence - South et al., whose authors include two from EZKL, report proofs for eight models, from a 30-parameter linear regression to a 1.07-million-parameter VAE decoder [S-0024]. A 250,000-parameter nanoGPT took 2,781 s to prove and 2.69 s to verify, with a 219 GB proving key [S-0024]. - Trail of Bits reviewed EZKL and its verifier contracts for Zkonduit in January 2025, with 11 engineer-weeks of effort [S-0070]. It reported 34 findings, 8 of high severity, including circuit soundness bugs and ways to bypass data attestation. At its March 2025 fix review every high-severity finding was resolved [S-0070]. - Trail of Bits reports that other projects use EZKL's attestation and verifier contracts in production [S-0070]. Limitations - South et al. write that proof time and resource requirements \"grow dramatically with large models\", and that the proving key's size limits model size [S-0024]. - The proof covers the quantized circuit. Trail of Bits showed that quantization can activate a backdoor that is dormant in the full-precision model [S-0070]. - The audit did not fully review several higher-level operations, such as einsum and scatternd, or all of the command-line tools [S-0070]. - Limits common to proofs of inference, such as proofs that do not bind the computation spent, are covered in Zero-knowledge proofs of inference.", "referenced_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "O-0213", "title": "Zkonduit", "url": "https://trustbutveri.fyi/organizations/zkonduit/" } ] }, { "id": "I-0015", "slug": "gensyn-verde-repops", "title": "Verde and RepOps (Gensyn)", "aliases": [ "Verde", "RepOps", "Reproducible Operators", "Gensyn Reproducible Execution Environment (REE)" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "one_liner": "Gensyn's system for checking delegated machine-learning jobs, which settles disagreements between providers by re-running a single operation with bitwise-reproducible operators.", "summary": "Verde is a protocol from the company Gensyn for checking machine-learning jobs, such as inference, fine-tuning or training, that a client hands to untrusted compute providers. Several providers run the same job. If their results differ, a bisection game narrows the dispute to a single operation, which a referee re-runs to decide who is right. The client gets the correct result if at least one provider is honest. This needs identical results on different hardware, which Gensyn reports its RepOps library provides by fixing the order of floating-point operations. Gensyn reports settling markets in its Delphi app with REE, a runtime built on RepOps whose receipts let anyone re-run the inference, and states that Verde runs in its Judge service. RepOps ships as closed binaries. The research paper reports that RepOps roughly doubled Llama-8B inference time. As of September 2026 no independent evaluation has been published.", "category": "cryptographic-computational", "secondary_categories": [], "verifies": [ { "claim": "C-0005", "role": "primary", "note": "A client learns that a delegated inference output came from the declared model and input, if at least one provider is honest (S-1809, S-1810)." }, { "claim": "C-0007", "role": "supporting", "note": "Also covers training and fine-tuning jobs delegated to several providers (S-1809)." } ], "threat_model": "adversarial", "adversarial_evaluation": "analysis", "hardware_requirement": "none", "prover_cooperation": "required", "confidentiality": "revealing", "depends_on": [], "readiness": { "assessment": true, "level": "R3", "scope": "showing a delegated output came from the declared model, if one provider is honest", "rubric_version": "1.1", "rationale": "Gensyn reports using its public runtime in production to settle markets whose answers anyone can re-run, but all evidence of that use comes from Gensyn, and no independent security evaluation of Verde or RepOps has been published.\n\n- **R1** met: the paper states the claim, that the client obtains the correct result if at least one provider is honest, together with the dispute protocol and its assumptions [[S-1809]].\n- **R2** met: Gensyn's Reproducible Execution Environment, which runs LLM inference with RepOps and writes receipts for re-execution, is public as binaries with an MIT-licensed SDK [[S-1812]]. The paper measures RepOps on NVIDIA T4, RTX 3090 and A100 GPUs, including Llama-8B inference and fine-tuning on an A100, against a stated adversary: dishonest compute providers [[S-1809]]. The paper's evaluation reports overheads. The claim of bitwise-identical results across hardware comes from Gensyn's posts [[S-1810]].\n- **R3** met on the provider's own account, as a production service that others can check. Gensyn reports that Delphi, its information-market app, is live on its mainnet, that hosted REE settlement is available for partner markets, and that markets settled by open models inside REE produce receipts anyone can re-run to verify the answer [[S-3022]]. The post, from May 2026, is the latest documented use, and it describes no dispute process. REE's current release is v0.6.0, which Gensyn does not label alpha or beta [[S-3023]]. Gensyn also states that Verde and RepOps are deployed in Judge, its AI evaluation service, without saying how Judge uses them [[S-1810]] [[S-1811]]. No party other than Gensyn is documented relying on Verde or REE for a verification decision.\n- **R4** not met: no independent audit, red-team or peer-reviewed security analysis has been published.\n\nConfidence is low: R3 rests on Gensyn's own posts, and REE's documentation warns that receipts from one release may not re-verify under the next [[S-3023]].", "evidence": [ "S-1809", "S-1810", "S-1811", "S-1812", "S-3022", "S-3023" ], "next_level_gaps": [ "An independent public security evaluation of the Verde dispute protocol and of RepOps reproducibility across hardware, which the paper asserts but does not test." ], "confidence": "low", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [], "blockers": [ { "text": "Reproducibility costs throughput: RepOps added 98% to Llama-8B inference time on an A100 in the paper, and Gensyn reports a threefold cut in REE's reproducible-mode overhead without absolute figures.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-1809", "S-1812" ] }, { "text": "The providers who re-run a job and the referee need the model and data, and the guarantee holds only if at least one provider is honest.", "theme": "privacy-leakage", "blocked_by": null, "sources": [ "S-1809" ] } ], "challenge_themes": [ "performance-compatibility", "adversarial-validation", "privacy-leakage", "protocol-soundness" ], "organizations": [], "people": [], "sources": [ { "source": "S-1809", "supports": "refereed delegation design, guarantee and its limit, bisection, single-operator re-execution, RepOps design, overheads, FP32 and single-GPU scope, author affiliations", "locator": "abstract; §1; §3.2; §4 Table 2; limitations" }, { "source": "S-1810", "supports": "deployment in Judge; bitwise reproducibility across hardware; tasks covered; what the guarantee does not cover (provider-reported)" }, { "source": "S-1811", "supports": "Judge launch with a reasoning task framed as a prediction market (provider-reported)" }, { "source": "S-1812", "supports": "public REE binaries and SDK, licences, receipts, release limited to reproducible LLM inference, pipeline parallelism up to 72B and threefold overhead cut in v0.2.0 (provider-reported)", "locator": "README; patch notes" }, { "source": "S-3022", "supports": "Delphi live on Gensyn's mainnet; REE settlement receipts that anyone can re-run; hosted REE settlement for partner markets (provider-reported)", "locator": "settlement section" }, { "source": "S-3023", "supports": "REE modes, current release v0.6.0, reproducible int8 attention and MoE kernels, receipts across releases (provider-reported)", "locator": "whole page" } ], "concepts": [ "K-0008", "K-0009" ], "kind": "product", "developer": [ "O-0212" ], "realises": [ "M-0002", "M-0006" ], "homepage": "https://www.gensyn.ai/research/verde-verification-system-in-production", "repo": "https://github.com/gensyn-ai/ree", "type": "implementation", "url": "https://trustbutveri.fyi/implementations/gensyn-verde-repops/", "source_file": "content/implementations/gensyn-verde-repops.md", "flags_all": [ "provider-reported" ], "body_markdown": "## What it is\n\nVerde is a protocol from Gensyn for checking machine-learning programs, such as inference, fine-tuning and training of language models, that a client delegates to untrusted compute providers [[S-1809]]. It adapts refereed delegation, a cryptographic technique in which a client sends the same job to several providers and a referee settles disagreements [[S-1809]]. RepOps (Reproducible Operators) is the library the paper introduces to make honest providers' results bitwise identical on different hardware [[S-1809]]. Most of the paper's authors work at Gensyn [[S-1809]].\n\nVerde relies on [[M-0002|bit-exact inference]], and for training jobs it resolves disputes over checkpoints, as in [[M-0006|training-transcript verification]]. Gensyn reports that Verde and RepOps are deployed in Judge, its service for verifiable AI evaluation [[S-1810]] [[S-1811]]. It also reports settling markets in Delphi, its information-market app, with REE [[S-3022]].\n\n## How it works\n\n1. The client gives the same job to two or more providers [[S-1809]], and arbitration starts only if their outputs differ [[S-1810]].\n2. A two-level bisection game finds the first step, and then the first operation in the computational graph, on which the providers disagree [[S-1809]] [[S-1810]].\n3. The referee re-runs that one operation to decide which output is correct. The authors state that this cuts the referee's communication and compute by two orders of magnitude compared with re-running the whole disputed step [[S-1809]].\n\nRepOps fixes the order of floating-point operations for common operators such as matrix multiplication, so that an honest provider's result does not depend on its GPU [[S-1809]] [[S-1810]]. Gensyn's Reproducible Execution Environment (REE) packages this for LLM inference. It writes a receipt holding the model, prompt, output and the metadata needed to reproduce the run [[S-1812]]. Its reproducible mode uses RepOps kernels. A separate deterministic mode uses PyTorch's deterministic algorithms and reproduces results only on the same hardware [[S-3023]].\n\n## Evidence\n- The paper reports RepOps overheads against PyTorch on NVIDIA T4, RTX 3090 and A100 GPUs, for DistilBERT and Llama models [[S-1809]]. For Llama-8B on an A100 with 80 GB, RepOps added 98% to inference time and 126% to LoRA fine-tuning time [[S-1809]].\n- Gensyn reports that RepOps gives bitwise-identical results across hardware, and that Verde and RepOps are deployed in Judge [[S-1810]]. Gensyn introduced Judge in August 2025 with a reasoning task framed as a prediction market [[S-1811]].\n- Gensyn reports that Delphi is live on its mainnet, that markets settled by open models inside REE produce a receipt that anyone can re-run, and that hosted REE settlement is available for partner markets [[S-3022]].\n- REE is public as an MIT-licensed SDK with proprietary binaries [[S-1812]]. Its release notes add pipeline parallelism for models of up to 72B parameters [[S-1812]], and its documentation gives version 0.6.0 as current, with reproducible int8 attention and mixture-of-experts kernels [[S-3023]].\n\n## Limitations\n- The guarantee needs at least one honest provider. The authors state that if all providers are dishonest, the referee accepts an incorrect output [[S-1809]].\n- The paper's RepOps supports 32-bit floating point only, and reproducibility only when each setup runs the program on a single GPU [[S-1809]]. Gensyn reports that REE has since added pipeline parallelism and int8 attention [[S-1812]] [[S-3023]].\n- Receipts are tied to an REE release. The documentation warns that some v0.5.0 receipts may not re-verify with v0.6.0 [[S-3023]].\n- Gensyn states that Verde shows the output was produced by the declared model and data, not that the model itself is correct [[S-1810]].\n- The RepOps kernels ship only as binaries under a proprietary licence [[S-1812]].\n- All evidence of production use comes from Gensyn [[S-1810]] [[S-1811]] [[S-3022]].", "body_text": "What it is Verde is a protocol from Gensyn for checking machine-learning programs, such as inference, fine-tuning and training of language models, that a client delegates to untrusted compute providers [S-1809]. It adapts refereed delegation, a cryptographic technique in which a client sends the same job to several providers and a referee settles disagreements [S-1809]. RepOps (Reproducible Operators) is the library the paper introduces to make honest providers' results bitwise identical on different hardware [S-1809]. Most of the paper's authors work at Gensyn [S-1809]. Verde relies on bit-exact inference, and for training jobs it resolves disputes over checkpoints, as in training-transcript verification. Gensyn reports that Verde and RepOps are deployed in Judge, its service for verifiable AI evaluation [S-1810] [S-1811]. It also reports settling markets in Delphi, its information-market app, with REE [S-3022]. How it works 1. The client gives the same job to two or more providers [S-1809], and arbitration starts only if their outputs differ [S-1810]. 2. A two-level bisection game finds the first step, and then the first operation in the computational graph, on which the providers disagree [S-1809] [S-1810]. 3. The referee re-runs that one operation to decide which output is correct. The authors state that this cuts the referee's communication and compute by two orders of magnitude compared with re-running the whole disputed step [S-1809]. RepOps fixes the order of floating-point operations for common operators such as matrix multiplication, so that an honest provider's result does not depend on its GPU [S-1809] [S-1810]. Gensyn's Reproducible Execution Environment (REE) packages this for LLM inference. It writes a receipt holding the model, prompt, output and the metadata needed to reproduce the run [S-1812]. Its reproducible mode uses RepOps kernels. A separate deterministic mode uses PyTorch's deterministic algorithms and reproduces results only on the same hardware [S-3023]. Evidence - The paper reports RepOps overheads against PyTorch on NVIDIA T4, RTX 3090 and A100 GPUs, for DistilBERT and Llama models [S-1809]. For Llama-8B on an A100 with 80 GB, RepOps added 98% to inference time and 126% to LoRA fine-tuning time [S-1809]. - Gensyn reports that RepOps gives bitwise-identical results across hardware, and that Verde and RepOps are deployed in Judge [S-1810]. Gensyn introduced Judge in August 2025 with a reasoning task framed as a prediction market [S-1811]. - Gensyn reports that Delphi is live on its mainnet, that markets settled by open models inside REE produce a receipt that anyone can re-run, and that hosted REE settlement is available for partner markets [S-3022]. - REE is public as an MIT-licensed SDK with proprietary binaries [S-1812]. Its release notes add pipeline parallelism for models of up to 72B parameters [S-1812], and its documentation gives version 0.6.0 as current, with reproducible int8 attention and mixture-of-experts kernels [S-3023]. Limitations - The guarantee needs at least one honest provider. The authors state that if all providers are dishonest, the referee accepts an incorrect output [S-1809]. - The paper's RepOps supports 32-bit floating point only, and reproducibility only when each setup runs the program on a single GPU [S-1809]. Gensyn reports that REE has since added pipeline parallelism and int8 attention [S-1812] [S-3023]. - Receipts are tied to an REE release. The documentation warns that some v0.5.0 receipts may not re-verify with v0.6.0 [S-3023]. - Gensyn states that Verde shows the output was produced by the declared model and data, not that the model itself is correct [S-1810]. - The RepOps kernels ship only as binaries under a proprietary licence [S-1812]. - All evidence of production use comes from Gensyn [S-1810] [S-1811] [S-3022].", "referenced_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" }, { "id": "I-0016", "title": "Batch-invariant inference kernels (Thinking Machines)", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/" }, { "id": "O-0212", "title": "Gensyn", "url": "https://trustbutveri.fyi/organizations/gensyn/" } ] }, { "id": "I-0012", "slug": "low-trust-compute-verification-system-overview", "title": "Low-trust AI compute verification system overview", "aliases": [ "Cankaya system overview", "Near-term, low-trust AI compute verification" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "one_liner": "A retrofittable reference design in which network taps commit to all facility traffic, and air-gapped, independently sourced checkers later re-run randomly challenged records.", "summary": "Naci Cankaya's system overview, published by the Machine Intelligence Research Institute's Technical Governance Team, proposes a privacy-preserving verification system that can be added to existing AI data centres, for an agreement between states that distrust each other. Network taps hash all traffic crossing the monitored boundary, and only these hashes are shared with the verifier. Later, the verifier challenges random records. The operator supplies the matching data, and at least three independently sourced computing clusters, each air-gapped inside the operator's data centre, re-run the workload and screen it, for example for training versus inference or banned uses. One-way optical links, cross-checks of every output and re-transmission by the operator limit what either side can leak. The document is a working draft, and key parts are open research questions. The main obstacles are bit-exact replay, physical security and side-channel defence. The author says the threat model is under-developed.", "technical": "- **Capture.** Evidence capture runs online at line rate. Taps parse only link-to-transport-layer structure, hash payloads over groups of packets without regard to object boundaries, and emit no timestamps, because trailing bits could encode information [[S-0018]].\n- **Replay metadata.** Exact replay needs the hardware SKU, the exact weights and quantization format, the parallelism topology, the software versions and custom kernels, and the batch size of each forward pass [[S-0018]].\n- **Instructor and executor.** An untrusted but fast executor re-runs the workload without seeing the claimed output, and a small, formally verifiable instructor compares its result with the prover's claim. The cited precedent is an FPGA sentry of 3,543 lines of hardware-description code [[S-0018]].\n- **Sampling statistics.** P(detect at least one) = 1 - (1 - p)^n. Checking 3,000 random samples detects a 0.1% flaw rate with 95.03% probability [[S-0018]].", "category": "isolation-architecture", "secondary_categories": [ "off-chip-devices", "cryptographic-computational" ], "verifies": [ { "claim": "C-0004", "role": "primary", "note": "Challenged records are screened for inference versus training." }, { "claim": "C-0005", "role": "supporting", "note": "Screening checks that the model is on an agreed whitelist." }, { "claim": "C-0006", "role": "supporting", "note": "Screening checks that outputs are free of blacklisted uses, including with inspector agents." }, { "claim": "C-0010", "role": "supporting", "note": "Memory challenges and resource accounting are proposed against hidden workloads." } ], "threat_model": "adversarial", "adversarial_evaluation": "analysis", "hardware_requirement": "retrofit-device", "prover_cooperation": "required", "confidentiality": "preserving", "depends_on": [ { "target": "M-0013", "note": "Taps are the default evidence-capture mechanism." }, { "target": "M-0002", "note": "Plan A evaluation relies on exact replay of declared computation." } ], "readiness": { "assessment": true, "level": "R1", "scope": "screening challenged records to show declared inference compute is not training", "rubric_version": "1.1", "rationale": "A detailed end-to-end design states the claim and threat model, but there is no integrated implementation and there are no results.\n\n- **R1** met: the overview describes the system end to end, the rules it would support (such as inference versus training, model whitelists and blacklisted uses), a worst-case threat model in which both prover and verifier are hostile nation-states, and its practical requirements [[S-0018]].\n- **R2** not met. The document is a working draft that sets out the design and open research questions, not results from an integrated system [[S-0018]]. Its companion preprint specifies the tap subsystem and states that empirical validation is still required [[S-1300]]. Several building blocks remain open, including passive optical splitting at 53–112 GBaud [[S-0018]].\n\nConfidence is medium: the design is detailed, but the author states that its threat model is under-developed [[S-0018]].", "evidence": [ "S-0018", "S-1300" ], "next_level_gaps": [ "A public working implementation or reproducible end-to-end results for the capture-then-challenge pipeline, at realistic line rates or against a stated adversary.", "Demonstrated bit-exact replay of production inference inside a secure auditing environment built from independently sourced components.", "A developed threat model and red-teaming of the side-channel, egress and inspector-agent components." ], "confidence": "medium", "assessed_by": [ "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Mismatches cannot be attributed to cheating or error", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "A failed hash or replay does not show whether it came from an evasion attempt, a random bit flip or an evaluation error. The author notes that if detected anomalies can plausibly be waved off as malfunctions, deterrence becomes less effective, so the parties need an agreed escalation procedure that ends in attribution.", "sources": [ "S-0018" ], "response": null }, { "assessment": true, "title": "Deliberate faults leak a bit each", "kind": "theoretical-argument", "severity": "minor", "status": "open", "description": "A malicious device can leak one bit by deliberately outputting a wrong result, which blocks a disclosure when the cross-comparison fails. The design therefore needs a pre-agreed budget of tolerated faults.", "sources": [ "S-0018" ], "response": null }, { "assessment": true, "title": "Inspector agents may be manipulable", "kind": "open-question", "severity": "significant", "status": "open", "description": "Automated compliance screening with LLM-based inspector agents must resist prompt-injection attacks. Adversarially trained systems might hide malicious actions with steganography, which makes backdoor detection an open problem.", "sources": [ "S-0018" ], "response": null } ], "blockers": [ { "text": "Empirical feasibility of passive optical splitting at 53–112 GBaud under realistic conditions is an open question.", "theme": "performance-compatibility", "blocked_by": "M-0013", "sources": [ "S-0018" ] }, { "text": "Exact replay needs complete hardware and software metadata, and the tolerable slowdown from emulation is an open question.", "theme": "performance-compatibility", "blocked_by": "M-0002", "sources": [ "S-0018" ] }, { "text": "Tamper-evident, rapidly mass-manufacturable and retrofittable enclosures for side-channel defence are an open research question, and physical security against covert communication in every monitored data centre is challenging.", "theme": "hardware-trust", "blocked_by": "M-0017", "sources": [ "S-0018" ] }, { "text": "A mass-manufacturable, good-enough side-channel defence, particularly power-line filtering, has not been constructed or red-teamed.", "theme": "coverage-hidden-compute", "blocked_by": "M-0022", "sources": [ "S-0018" ] }, { "text": "Distinguishing one server's DRAM contents from another's by challenge-response timing, and a general challenge-response protocol for diverse data types, are open.", "theme": "coverage-hidden-compute", "blocked_by": "M-0016", "sources": [ "S-0018" ] }, { "text": "The threat model is under-developed and needs input from cybersecurity and AI threat-modelling experts.", "theme": "adversarial-validation", "blocked_by": null, "sources": [ "S-0018" ] } ], "challenge_themes": [ "coverage-hidden-compute", "hardware-trust", "protocol-soundness", "performance-compatibility", "adversarial-validation", "privacy-leakage" ], "organizations": [ "O-0202" ], "people": [], "sources": [ { "source": "S-0018", "supports": "purpose; rules; threat model; requirements; execution trace; subsystems; engineering approaches; open problems; prior work; sampling statistics", "locator": "§1; §2a-2c; §3.1-3.2; §4.1-4.3; §5.1-5.3; Appendix A1" }, { "source": "S-1300", "supports": "companion secure gateway and tap design; demonstration cost; validation and red-teaming still required", "locator": "abstract; discussion of next steps" } ], "concepts": [ "K-0001", "K-0002", "K-0009", "K-0010", "K-0013", "K-0014", "K-0015", "K-0018", "K-0020", "K-0024" ], "kind": "proposed-architecture", "developer": [ "O-0202" ], "realises": [ "M-0013", "M-0001", "M-0002", "M-0004", "M-0015", "M-0016", "M-0022" ], "homepage": "https://intelligence.org/wp-content/uploads/2026/06/A-system-overview-for-near-term-low-trust-AI-compute-verification.pdf", "type": "implementation", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/", "source_file": "content/implementations/low-trust-compute-verification-system-overview.md", "flags_all": [ "provider-reported" ], "body_markdown": "## What it is\n\nNaci Cankaya's overview, from the Machine Intelligence Research Institute's Technical Governance Team, is a working draft [[S-0018]]. It describes the author's current best idea for a privacy-preserving, retrofittable AI compute verification system, meant to build confidence in an arms-control-style AI agreement [[S-0018]]. It offers example rules the system should support: monitorable chains of thought, cryptographic commitment of checkpoints, whitelisting of models before deployment, and a blacklist of illicit uses [[S-0018]].\n\nThe author argues that checking aggregate properties, such as total training FLOPs, is not enough, and sets collision-resistant evidence capture of individual workloads as a requirement [[S-0018]]. He assumes the worst case on both sides: the prover may covertly break the agreement's rules, and the verifier may try to exfiltrate the prover's secrets [[S-0018]].\n\n## How it works\n\nThe design separates capturing evidence from evaluating it [[S-0018]].\n\n### Evidence capture\n\n- **Network taps.** Taps on the monitored links hash traffic in groups of packets at line rate [[S-0018]]. See [[M-0013]].\n- **Shared hash stream.** The hashes leave over a split, sanitized fibre, so prover and verifier see identical copies. The prover recomputes the hashes from its own copy of the traffic, confirming that the tap sent nothing but legitimate commitments [[S-0018]].\n- **Prover-held data.** Beyond hashes, no information is shared with the verifier; the prover keeps the plaintext and a manifest [[S-0018]].\n- **Memory challenges and wiping.** These are an optional second form of evidence [[S-0018]]. See [[M-0016]] and [[M-0015]].\n\nA companion preprint specifies a secure gateway device for the taps that needs no processor trusted by both parties [[S-1300]].\n\n### Evidence evaluation\n\n1. The verifier issues a challenge over a one-way link enforced by a data diode [[S-0018]].\n2. The prover supplies the matching data, manifest entries and replay metadata [[S-0018]].\n3. Each party's trusted computing base checks the hashes, re-runs the computation under the declared metadata and screens it for compliance [[S-0018]]. See [[M-0002]].\n4. The trusted computing bases cross-check their verdicts, and nothing is released unless they agree; by default, disclosure is minimal [[S-0018]].\n\nA missing or mismatched record fails the challenge, while a failed replay or a disagreement triggers investigation [[S-0018]]. The evaluation hardware is at least three distinct trusted computing bases from different supply chains, trusted respectively by the prover, the verifier and a third party, each inside its own air-gap in the prover's data centre [[S-0018]]. Zero-knowledge proofs are a tentative \"plan B\" for evaluation, if they mature [[S-0018]]. See [[M-0004]].\n\n### Subsystems\n\nThe design avoids the need for mutually trusted silicon with these building blocks [[S-0018]]:\n\n- **Analog data movement control.** Fused fibre splitters duplicate traffic, optical isolators act as data diodes, and enclosures protect the equipment [[S-0018]].\n- **Controlled ingress.** A one-way path carries the verifier's inputs into the auditing environment [[S-0018]].\n- **Output cross-checks.** The prover clones the output of each verifier device with a fibre splitter, to confirm it sends nothing but the correctly computed result [[S-0018]].\n- **Sanitized egress.** The prover re-transmits verifier outputs at batched or randomized times, blocking optical and timing side channels [[S-0018]].\n- **Secure performance offshoring.** Untrusted fast hardware re-runs workloads, while small trusted devices compare the results [[S-0018]].\n\nSide-channel defence and resource accounting support the rest of the system [[S-0018]]. See [[M-0022]].\n\n## Evidence\n- **Status.** The overview sets out a design and open research questions [[S-0018]]. Its companion tap preprint states that empirical validation is still required [[S-1300]].\n- **Prior work it builds on.** Verde obtained bitwise-identical inference results across several NVIDIA GPUs by controlling the order of floating-point operations [[S-0018]]. The TrustGuard sentry, which re-executes instructions, was prototyped on an FPGA [[S-0018]].\n- **Taps.** Amodo Design is investigating passive optical splitting at 53–112 GBaud [[S-0018]]. The companion preprint expects a demonstration gateway to cost roughly as much to develop as a small team of engineers for a few months [[S-1300]].\n- **Cost target.** The author expects acceptance to depend on retrofit costs below 10% of the monitored hardware, ideally below 1% [[S-0018]].\n\n## Limitations\n- **Attribution.** A technical mismatch does not show whether it came from evasion, a random bit flip or faulty evaluation software [[S-0018]].\n- **Fault leakage.** A malicious device can leak one bit per deliberately wrong output, so the design needs a fault budget [[S-0018]].\n- **Inspector agents.** Screening agents must resist prompt injection [[S-0018]].\n- **Physical security.** Securing every monitored data centre against covert communication is challenging [[S-0018]].\n- **Zero-knowledge option.** Proofs work over integers, while accelerated inference accumulates floating-point rounding errors [[S-0018]].\n- **Threat model.** The author calls the threat model section under-developed [[S-0018]].", "body_text": "What it is Naci Cankaya's overview, from the Machine Intelligence Research Institute's Technical Governance Team, is a working draft [S-0018]. It describes the author's current best idea for a privacy-preserving, retrofittable AI compute verification system, meant to build confidence in an arms-control-style AI agreement [S-0018]. It offers example rules the system should support: monitorable chains of thought, cryptographic commitment of checkpoints, whitelisting of models before deployment, and a blacklist of illicit uses [S-0018]. The author argues that checking aggregate properties, such as total training FLOPs, is not enough, and sets collision-resistant evidence capture of individual workloads as a requirement [S-0018]. He assumes the worst case on both sides: the prover may covertly break the agreement's rules, and the verifier may try to exfiltrate the prover's secrets [S-0018]. How it works The design separates capturing evidence from evaluating it [S-0018]. Evidence capture - Network taps. Taps on the monitored links hash traffic in groups of packets at line rate [S-0018]. See Network taps and certifiers. - Shared hash stream. The hashes leave over a split, sanitized fibre, so prover and verifier see identical copies. The prover recomputes the hashes from its own copy of the traffic, confirming that the tap sent nothing but legitimate commitments [S-0018]. - Prover-held data. Beyond hashes, no information is shared with the verifier; the prover keeps the plaintext and a manifest [S-0018]. - Memory challenges and wiping. These are an optional second form of evidence [S-0018]. See Timed challenge-response and memory-occupation challenges and Memory wiping and proofs of secure erasure. A companion preprint specifies a secure gateway device for the taps that needs no processor trusted by both parties [S-1300]. Evidence evaluation 1. The verifier issues a challenge over a one-way link enforced by a data diode [S-0018]. 2. The prover supplies the matching data, manifest entries and replay metadata [S-0018]. 3. Each party's trusted computing base checks the hashes, re-runs the computation under the declared metadata and screens it for compliance [S-0018]. See Deterministic and bit-exact inference. 4. The trusted computing bases cross-check their verdicts, and nothing is released unless they agree; by default, disclosure is minimal [S-0018]. A missing or mismatched record fails the challenge, while a failed replay or a disagreement triggers investigation [S-0018]. The evaluation hardware is at least three distinct trusted computing bases from different supply chains, trusted respectively by the prover, the verifier and a third party, each inside its own air-gap in the prover's data centre [S-0018]. Zero-knowledge proofs are a tentative \"plan B\" for evaluation, if they mature [S-0018]. See Zero-knowledge proofs of inference. Subsystems The design avoids the need for mutually trusted silicon with these building blocks [S-0018]: - Analog data movement control. Fused fibre splitters duplicate traffic, optical isolators act as data diodes, and enclosures protect the equipment [S-0018]. - Controlled ingress. A one-way path carries the verifier's inputs into the auditing environment [S-0018]. - Output cross-checks. The prover clones the output of each verifier device with a fibre splitter, to confirm it sends nothing but the correctly computed result [S-0018]. - Sanitized egress. The prover re-transmits verifier outputs at batched or randomized times, blocking optical and timing side channels [S-0018]. - Secure performance offshoring. Untrusted fast hardware re-runs workloads, while small trusted devices compare the results [S-0018]. Side-channel defence and resource accounting support the rest of the system [S-0018]. See Side-channel suppression for isolated facilities. Evidence - Status. The overview sets out a design and open research questions [S-0018]. Its companion tap preprint states that empirical validation is still required [S-1300]. - Prior work it builds on. Verde obtained bitwise-identical inference results across several NVIDIA GPUs by controlling the order of floating-point operations [S-0018]. The TrustGuard sentry, which re-executes instructions, was prototyped on an FPGA [S-0018]. - Taps. Amodo Design is investigating passive optical splitting at 53–112 GBaud [S-0018]. The companion preprint expects a demonstration gateway to cost roughly as much to develop as a small team of engineers for a few months [S-1300]. - Cost target. The author expects acceptance to depend on retrofit costs below 10% of the monitored hardware, ideally below 1% [S-0018]. Limitations - Attribution. A technical mismatch does not show whether it came from evasion, a random bit flip or faulty evaluation software [S-0018]. - Fault leakage. A malicious device can leak one bit per deliberately wrong output, so the design needs a fault budget [S-0018]. - Inspector agents. Screening agents must resist prompt injection [S-0018]. - Physical security. Securing every monitored data centre against covert communication is challenging [S-0018]. - Zero-knowledge option. Proofs work over integers, while accelerated inference accumulates floating-point rounding errors [S-0018]. - Threat model. The author calls the threat model section under-developed [S-0018].", "referenced_by": [ { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" }, { "id": "O-0209", "title": "Hardware AI Governance Lab", "url": "https://trustbutveri.fyi/organizations/oxford-hardware-ai-governance-lab/" } ] }, { "id": "I-0009", "slug": "lucid-location-certificates", "title": "Lucid sovereignty (location) certificates", "aliases": [ "Sovereignty Certificates", "Ping-based location attestation", "sovcert" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "one_liner": "A draft specification, hosted by Lucid Computing, for short-lived certificates that bound where a workload runs by timing signed exchanges with fixed anchors.", "summary": "Sovereignty Certificates are a draft industry specification for proving where a computing workload runs. An agent inside a trusted execution environment exchanges signed messages with anchor servers at known locations and times the replies. Signals cannot travel faster than light, so each reply time caps the distance to that anchor. A verifier checks the hardware attestation and the timings, computes a feasible region and issues a certificate that expires after a few minutes. The specification is hosted by Lucid Computing and attributed to a working group. As of September 2026 it is still draft 0.1.0, dated October 2025, with no public implementation or measured results. It trusts the hardware root of trust and the TEE, and leaves sophisticated physical attacks as a residual risk. Known attacks on delay-based location, such as faster network paths and compromised anchors, also apply.", "technical": "The specification follows the IETF RATS architecture (RFC 9334) and the Entity Attestation Token format (RFC 9711) [[S-1404]]. Its roles are an Attester (a workload with a sidecar agent), a Verifier, an Anchor Fleet at fixed locations, an Endorser that publishes a signed directory of anchors with their public keys and coordinates, and a Relying Party [[S-1404]]. The Attester generates an ephemeral key pair each cycle. A hash binding that key and the verifier's nonce goes into a register included in the hardware root of trust's signed quote [[S-1404]]. The Attester may probe anchors directly or query a local trusted proxy [[S-1404]]. The anchors probed must be chosen to minimize geometric dilution of precision (GDOP) [[S-1404]].\n\nAnchor receipts carry a high-precision timestamp of the probe's arrival, the probe's nonce and the anchor's identifier, and are signed [[S-1404]]. The normative text has the Verifier derive round-trip times from the receipts' timestamps; in the Annex B example, the Attester sends a UDP probe to each of eight anchors and records the local arrival time of each reply [[S-1404]]. Anchors must be synchronized to a precise time source, preferably UTC [[S-1404]]. The Verifier must reject evidence whose distance bounds have no common intersection [[S-1404]].\n\nThe EAT profile defines private claims for the platform quote, the ephemeral public key, the location receipts, a location claim with latitude, longitude, radius in metres and confidence, and the policies passed [[S-1404]]. In the Annex B example the certificate expires after 10 minutes and is renewed about every five. If a certificate expires, the workload must fail closed [[S-1404]]. Long-lived anchor, verifier and endorser keys must sit in hardware security modules, and signatures should use ECDSA with P-256 or stronger [[S-1404]].", "category": "accounting-provenance", "secondary_categories": [ "on-chip" ], "verifies": [ { "claim": "C-0002", "role": "primary", "note": "Certifies a bounded region in which an attested workload's platform was running at a given time." } ], "threat_model": "semi-trusted", "adversarial_evaluation": "analysis", "hardware_requirement": "existing-features", "prover_cooperation": "required", "confidentiality": "preserving", "depends_on": [ { "target": "M-0008", "note": "Location evidence is bound to a hardware-rooted TEE attestation quote." } ], "readiness": { "assessment": true, "level": "R1", "scope": "certifying the region where an attested workload ran at a given time", "rubric_version": "1.1", "rationale": "A public draft states the claim and threat model, but nothing has been built or measured in public.\n\n- **R1** met: the draft states what the certificate is meant to prove, the protocol, the threat model and its trust assumptions [[S-1404]].\n- **R2** not met: as of September 2026 the repository holds only the specification, with no reference implementation, code or measured location results [[S-1404]]. The only location figures are an illustrative example in an annex [[S-1404]]. Lucid's homepage states that each of its clusters can prove where it is, but neither the homepage nor the developer documentation publishes how location is determined or any results [[S-1406]] [[S-1407]]. The working group gave January 2026 as the target for a final version, and the draft is still version 0.1.0 [[S-1405]] [[S-1404]].", "evidence": [ "S-1404", "S-1405", "S-1406", "S-1407" ], "next_level_gaps": [ "A public reference implementation, or reproducible results with a real anchor fleet and realistic network paths.", "Published location accuracy and false-rejection rates.", "An independent security review of the protocol and its trust assumptions." ], "confidence": "medium", "assessed_by": [ "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Physical attacks on the trusted hardware are out of scope", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "The specification places the hardware root of trust and the TEE in the trusted computing base. It assumes they resist software attacks, notes that the attacker may have physical access, and leaves sophisticated physical attacks, such as bus probing and side-channel analysis, as a residual risk [[S-1404]]. It says that future revisions may add requirements for physical tamper evidence [[S-1404]].", "sources": [ "S-1404" ], "response": null }, { "assessment": true, "title": "On-chip keys may be extractable", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Tee and Happel argue that ping-based location protocols backed by keys stored on the chip can be compromised if an adversary with physical access extracts those keys [[S-1403]]. In this specification, the evidence chain rests on the hardware root of trust's signed quote, whose signing key must be protected by the hardware [[S-1404]].", "sources": [ "S-1403", "S-1404" ], "response": null }, { "assessment": true, "title": "General delay and landmark attacks apply", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Attacks on delay-based location verification in general also apply. Brass and Aarne discuss adding delay, using faster paths such as dark fibre, and compromising landmarks [[S-1400]]. The specification counters anchor impersonation with a signed anchor directory. Against collusion it recommends anchors in diverse places run by several independent operators, and peer monitoring that temporarily removes anchors whose timings deviate [[S-1404]].", "sources": [ "S-1400", "S-1404" ], "response": null } ], "blockers": [ { "text": "The specification is an unfinished draft with no public implementation or evaluation.", "theme": "adversarial-validation", "blocked_by": null, "sources": [ "S-1404", "S-1405" ] }, { "text": "It needs a globally distributed, trusted anchor fleet and an endorser to run the anchor directory.", "theme": "access-governance", "blocked_by": null, "sources": [ "S-1404" ] } ], "challenge_themes": [ "hardware-trust", "adversarial-validation", "protocol-soundness", "access-governance" ], "organizations": [ "O-0180" ], "people": [], "sources": [ { "source": "S-1404", "supports": "status, version, scope, roles, protocol, binding, timing, claims, threat model, security considerations, worked example", "locator": "title block; §0-0.3, §3, §4.1-4.2, §5.1-5.6, §6.2-6.5, §7.1, §8.1-8.4; Annexes A-B" }, { "source": "S-1405", "supports": "working-group description; target date for a final version", "locator": "homepage" }, { "source": "S-1406", "supports": "Lucid's confidential-computing offering, its statement that clusters can prove their location, and link to Sovereignty Certificates", "locator": "homepage" }, { "source": "S-1407", "supports": "sovereignty-related auditor listed without method details", "locator": "reference/auditor-catalog.html; concepts/architecture.html" }, { "source": "S-1403", "supports": "key-extraction weakness of ping-based protocols", "locator": "Abstract" }, { "source": "S-1400", "supports": "general attacks on delay-based location verification; typical precision of delay-based methods", "locator": "Delay-based methods sections; brief survey of location verification methods" } ], "concepts": [ "K-0004", "K-0005", "K-0006", "K-0018" ], "kind": "standard", "developer": [ "O-0180" ], "realises": [ "M-0018" ], "homepage": "https://sovcert.org/", "repo": "https://github.com/Lucid-Computing/sovereignty-certificate-specification", "type": "implementation", "url": "https://trustbutveri.fyi/implementations/lucid-location-certificates/", "source_file": "content/implementations/lucid-location-certificates.md", "flags_all": [ "provider-reported" ], "body_markdown": "## What it is\n\nSovereignty Certificates are a draft specification for \"cryptographic credentials that provide verifiable, unforgeable proof of the physical location of a computing workload at a specific point in time\", according to the repository README [[S-1404]]. The document is version 0.1.0, dated 2025-10-21, marked \"Draft\" and \"Prepared by: Sovereignty Certificates Working Group\" [[S-1404]]. It describes itself as an industry-led specification, not an official standard from a body such as ISO or IEC [[S-1404]]. It is licensed CC BY 4.0 [[S-1404]].\n\nThe specification applies [[M-0018|delay-based location verification]] to computing devices such as CPUs and GPUs [[S-1404]]. Its repository sits in Lucid Computing's GitHub organization [[S-1404]]. The working group's website describes the group as an independent initiative [[S-1405]]. It lists participants by category, including chip makers, cloud providers, frontier AI labs and compute-governance organizations, without naming them [[S-1405]]. Lucid's homepage links to the effort and describes Lucid's own offering as attested, confidential-computing GPU clusters with data-residency controls [[S-1406]].\n\n## How it works\n\nThe specification calls its method ping-based location attestation [[S-1404]]. Signals cannot travel faster than light, so a measured round-trip time gives an upper bound on the device's distance from each anchor [[S-1404]]. The specification describes the following steps [[S-1404]]:\n\n1. The verifier sends the attester a fresh nonce and a signed directory of trusted anchors.\n2. From inside a trusted execution environment, the attester probes several anchors chosen to give good geometry.\n3. Each anchor returns a signed receipt with a timestamp and the probe's nonce, and the attester records when each reply arrives.\n4. The attester bundles the receipts with a hardware attestation quote that binds a per-cycle key and the verifier's nonce to the platform.\n5. The verifier checks signatures, freshness and platform integrity, then runs a deterministic multilateration to compute a feasible region.\n6. The verifier issues a short-lived certificate stating the location, radius and confidence, and which policies passed.\n\nWorkloads must fail closed when a certificate expires [[S-1404]]. The specification argues that tunnels and VPNs add latency, so they generally cannot make a device appear closer to an anchor than it is, and usually produce geometrically inconsistent measurements [[S-1404]]. It requires rejecting evidence whose distance bounds have an empty or impossible intersection [[S-1404]]. Anchors should monitor each other's round-trip times, and an anchor that deviates from the physical baseline should be temporarily removed from the directory [[S-1404]].\n\n## Evidence\n- **Specification.** The draft is version 0.1.0, dated 2025-10-21 [[S-1404]]. The working group's site listed January 2026 as the target for a final version [[S-1405]]. As of September 2026 the repository still holds draft 0.1.0 and no reference implementation [[S-1404]].\n- **Worked example.** Annex B illustrates a result with a 45 m radius and 0.92 confidence from seven anchor receipts. It is an example, not a measurement [[S-1404]]. For comparison, Brass and Aarne report that delay-based methods locate devices to within about 10 km to 1,000 km, depending on the algorithm [[S-1400]].\n- **Lucid's statements.** Lucid's homepage says that \"every cluster can prove where it is, whose silicon ran, what executed, and how data moved\" [[S-1406]]. Its developer documentation lists a \"Data Sovereignty & Localization\" auditor, described as \"Ensuring data remains within approved geographic jurisdictions\" [[S-1407]]. Neither explains how location is determined or reports results [[S-1406]] [[S-1407]].\n\nAs of September 2026 no test deployment or measured location results have been published [[S-1404]] [[S-1406]] [[S-1407]].\n\n## Limitations\n- **Trusted hardware.** The threat model lets the attacker control the network and hold root on the host, including where the host is an untrusted cloud provider. It trusts the hardware root of trust and the TEE, and leaves sophisticated physical attacks as a residual risk [[S-1404]].\n- **Key extraction.** Tee and Happel argue that keys stored on the chip, on which ping-based protocols rely, may be extractable by adversaries with physical access [[S-1403]].\n- **General delay attacks.** Adding delay, using faster network paths and compromising landmarks are known weaknesses of delay-based location verification [[S-1400]]. The specification relies on signed anchor directories, HSM-protected keys, diverse and independently operated anchors, and anchor peer monitoring against some of these [[S-1404]].", "body_text": "What it is Sovereignty Certificates are a draft specification for \"cryptographic credentials that provide verifiable, unforgeable proof of the physical location of a computing workload at a specific point in time\", according to the repository README [S-1404]. The document is version 0.1.0, dated 2025-10-21, marked \"Draft\" and \"Prepared by: Sovereignty Certificates Working Group\" [S-1404]. It describes itself as an industry-led specification, not an official standard from a body such as ISO or IEC [S-1404]. It is licensed CC BY 4.0 [S-1404]. The specification applies delay-based location verification to computing devices such as CPUs and GPUs [S-1404]. Its repository sits in Lucid Computing's GitHub organization [S-1404]. The working group's website describes the group as an independent initiative [S-1405]. It lists participants by category, including chip makers, cloud providers, frontier AI labs and compute-governance organizations, without naming them [S-1405]. Lucid's homepage links to the effort and describes Lucid's own offering as attested, confidential-computing GPU clusters with data-residency controls [S-1406]. How it works The specification calls its method ping-based location attestation [S-1404]. Signals cannot travel faster than light, so a measured round-trip time gives an upper bound on the device's distance from each anchor [S-1404]. The specification describes the following steps [S-1404]: 1. The verifier sends the attester a fresh nonce and a signed directory of trusted anchors. 2. From inside a trusted execution environment, the attester probes several anchors chosen to give good geometry. 3. Each anchor returns a signed receipt with a timestamp and the probe's nonce, and the attester records when each reply arrives. 4. The attester bundles the receipts with a hardware attestation quote that binds a per-cycle key and the verifier's nonce to the platform. 5. The verifier checks signatures, freshness and platform integrity, then runs a deterministic multilateration to compute a feasible region. 6. The verifier issues a short-lived certificate stating the location, radius and confidence, and which policies passed. Workloads must fail closed when a certificate expires [S-1404]. The specification argues that tunnels and VPNs add latency, so they generally cannot make a device appear closer to an anchor than it is, and usually produce geometrically inconsistent measurements [S-1404]. It requires rejecting evidence whose distance bounds have an empty or impossible intersection [S-1404]. Anchors should monitor each other's round-trip times, and an anchor that deviates from the physical baseline should be temporarily removed from the directory [S-1404]. Evidence - Specification. The draft is version 0.1.0, dated 2025-10-21 [S-1404]. The working group's site listed January 2026 as the target for a final version [S-1405]. As of September 2026 the repository still holds draft 0.1.0 and no reference implementation [S-1404]. - Worked example. Annex B illustrates a result with a 45 m radius and 0.92 confidence from seven anchor receipts. It is an example, not a measurement [S-1404]. For comparison, Brass and Aarne report that delay-based methods locate devices to within about 10 km to 1,000 km, depending on the algorithm [S-1400]. - Lucid's statements. Lucid's homepage says that \"every cluster can prove where it is, whose silicon ran, what executed, and how data moved\" [S-1406]. Its developer documentation lists a \"Data Sovereignty & Localization\" auditor, described as \"Ensuring data remains within approved geographic jurisdictions\" [S-1407]. Neither explains how location is determined or reports results [S-1406] [S-1407]. As of September 2026 no test deployment or measured location results have been published [S-1404] [S-1406] [S-1407]. Limitations - Trusted hardware. The threat model lets the attacker control the network and hold root on the host, including where the host is an untrusted cloud provider. It trusts the hardware root of trust and the TEE, and leaves sophisticated physical attacks as a residual risk [S-1404]. - Key extraction. Tee and Happel argue that keys stored on the chip, on which ping-based protocols rely, may be extractable by adversaries with physical access [S-1403]. - General delay attacks. Adding delay, using faster network paths and compromising landmarks are known weaknesses of delay-based location verification [S-1400]. The specification relies on signed anchor directories, HSM-protected keys, diverse and independently operated anchors, and anchor peer monitoring against some of these [S-1404].", "referenced_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "O-0180", "title": "Lucid Computing", "url": "https://trustbutveri.fyi/organizations/lucid-computing/" } ] }, { "id": "I-0004", "slug": "pearl-proof-of-useful-work", "title": "Pearl proof-of-useful-work blockchain", "aliases": [ "Pearl", "Pearl protocol", "PRL" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "one_liner": "A blockchain whose mining is designed to be a by-product of GPU matrix multiplications in AI workloads, with public node and miner code.", "summary": "Pearl is a blockchain developed by Pearl Research Labs whose mining is designed to be a by-product of the matrix multiplications that AI models perform on GPUs. Each block carries evidence that a certain amount of matrix-multiplication work was done. Its network is live and its code is public. Pearl reports that a mining variant of a 70-billion-parameter model served more tokens per second on four H200 GPUs than the original model, in a benchmark that does not isolate the cost of mining. An independent study found that the network's dominant mining software appears to multiply random matrices, which verification does not distinguish from AI work. Its R3 (in production) rating is for proving that matrix-multiplication work was done on a blockchain, not for verifying how AI hardware is used. Its security rests on new hardness assumptions, and Pearl lists known shortcuts that let miners do less work than nominal.", "category": "cryptographic-computational", "secondary_categories": [], "verifies": [ { "claim": "C-0004", "role": "primary", "note": "Proves matrix-multiplication work for consensus; not applied to bounding the spare capacity of declared hardware that could run training." }, { "claim": "C-0003", "role": "supporting", "note": "Proves work for consensus; keeping declared hardware provably busy, as an alternative to showing it idle, is not demonstrated." }, { "claim": "C-0010", "role": "supporting", "note": "On declared hardware only; cannot find undeclared facilities. Not applied to bounding spare capacity." } ], "threat_model": "adversarial", "adversarial_evaluation": "analysis", "hardware_requirement": "none", "prover_cooperation": "required", "confidentiality": "partial", "depends_on": [ { "target": "M-0002", "note": "The verifier recomputes a tile bit for bit, reproducing GPU arithmetic off the GPU with the Hawkeye technique." } ], "readiness": { "assessment": true, "level": "R3", "scope": "proving GPUs did matrix-multiplication work in blockchain mining, not bounding spare capacity", "rubric_version": "1.1", "rationale": "R3 for the narrow claim that GPUs performed a given amount of matrix-multiplication work. The network is live, but no independent security evaluation exists, and the capacity-bounding use in [[M-0007]] is not demonstrated.\n\n- **R1** met: the protocol, verifier and hardness assumption are specified [[S-1105]], building on a published construction [[S-1609]].\n- **R2** met: code for a full node, a GPU miner and a proof-of-work circuit and verifier is public [[S-1107]]. Pearl reports running it on H200 GPUs alongside LLM serving [[S-1106]], against miners who try to win more often than honest work allows [[S-1105]].\n- **R3** met for the narrow claim, as production software that is available and relied on by other parties. Pearl reports that the chain went live when the node code became public [[S-1106]] [[S-1107]]. The repository supports mainnet, and no release up to v1.2.1 is labelled alpha, beta or preview [[S-1107]]. An independent study counted 8,012 online workers among the top 15 miners of one mining pool in May 2026, ran experiments on mainnet, and had shares accepted by that pool, whose acceptance rests on the protocol's verification [[S-0071]].\n- **R4** not met: the README mentions no audit [[S-1107]], and that study measures how the network is used rather than auditing or attacking the protocol [[S-0071]]. As of September 2026 no independent security evaluation has been published.", "evidence": [ "S-1107", "S-1105", "S-1106", "S-1609", "S-0071" ], "next_level_gaps": [ "An independent public security evaluation of the protocol or its implementation.", "For verification use: a demonstration that proof rates can bound the spare capacity of declared hardware." ], "confidence": "low", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Known mining speedups reduce work per proof", "kind": "theoretical-argument", "severity": "minor", "status": "open", "description": "Pearl lists known speedups: crafted inputs, precision shortcuts, seed or commitment grinding, work reuse, and faster kernels or hardware. Its jackpot policy checks limit crafted inputs, and a policy check caps skippable summands at one-sixteenth of those in a tile [[S-1105]]. Pearl describes faster honest kernels or hardware as \"not an attack on the protocol\" [[S-1105]].", "sources": [ "S-1105" ], "response": null }, { "assessment": true, "title": "Security rests on a new, informal hardness assumption", "kind": "open-question", "severity": "significant", "status": "open", "description": "The FP8 scheme relies on \"Assumption 1 (Informal quantized-subspace hardness)\": quantised products of noised matrices are assumed not to be substantially easier than generic ones [[S-1105]]. The integer construction it extends lists PoUW from more standard assumptions as an open problem [[S-1609]].", "sources": [ "S-1105", "S-1609" ], "response": null }, { "assessment": true, "title": "Verification does not check that mined matrices come from AI workloads", "kind": "theoretical-argument", "severity": "minor", "status": "open", "description": "Miners choose their own matrices [[S-1609]]. Basu reports that Pearl's verification \"does not check whether the matrices originate from an AI model\", that random matrices pass it, and that Pearl's reference mining code generates uniformly random matrices, with vLLM inference as an option. String analysis suggests that the dominant third-party mining software contains no inference code. Basu also finds that a naive fixed-threshold check of matrix kurtosis is defeated, at negligible cost, by sampling clipped Gaussian matrices. Basu calls the gap \"a design property\" rather than a vulnerability [[S-0071]]. It does not affect the claim that work was performed, but it means the \"useful\" part of the work is not verified.", "sources": [ "S-1609", "S-0071" ], "response": null } ], "blockers": [ { "text": "Built for consensus rather than capacity bounding; verifying that declared hardware has no spare capacity would also need a credible compute estimate.", "theme": "capacity-bounds", "blocked_by": null, "sources": [ "S-1102" ] }, { "text": "Performance figures are provider-reported, and the benchmark reports no baseline of the certified model without mining.", "theme": "adversarial-validation", "blocked_by": null, "sources": [ "S-1106" ] }, { "text": "Bit-exact verification depends on reproducing GPU arithmetic deterministically.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-1105" ] } ], "challenge_themes": [ "capacity-bounds", "protocol-soundness", "adversarial-validation" ], "organizations": [ "O-0121" ], "people": [], "sources": [ { "source": "S-1105", "supports": "FP8 protocol, verifier, jackpot policy, known speedups, ZK verifier, assumption (provider-reported)", "locator": "abstract; §1; §2; §4.1; §5.1; §6; §7; App. A–B" }, { "source": "S-1106", "supports": "INT scheme, launch statement, benchmark table, zkSNARK size (provider-reported)", "locator": "§3; §4.2; §4.7; §7 Table 1" }, { "source": "S-3571", "supports": "provider-reported end-to-end serving overhead for the integer proof-of-useful-work kernel", "locator": "Measured in production" }, { "source": "S-1107", "supports": "public node, miner and ZK prover code; releases", "locator": "README; release v1.2.1 (commit e72ad36)" }, { "source": "S-1609", "supports": "underlying PoUW construction and its assumptions; miners choose the instance", "locator": "abstract; §2; §3" }, { "source": "S-0071", "supports": "independent measurement of Pearl's mainnet (integer scheme): worker count, mining software, random-matrix mining, verification scope, statistical check defeated", "locator": "abstract; measurement and verification sections; conclusion (v2)" }, { "source": "S-1010", "supports": "Hawkeye, the bit-exact GPU reproduction the FP8 verifier uses; Komargodski's affiliation with Pearl Research Labs", "locator": "author block; abstract" }, { "source": "S-1102", "supports": "a proof of some computation is not a proof of all computation; capacity bounding needs a compute estimate (Attestable's proposal)" } ], "concepts": [ "K-0011", "K-0024", "K-0008" ], "kind": "open-source-project", "developer": [ "O-0121" ], "realises": [ "M-0007" ], "homepage": "https://pearlresearch.ai/", "repo": "https://github.com/pearl-research-labs/pearl", "type": "implementation", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/", "source_file": "content/implementations/pearl-proof-of-useful-work.md", "flags_all": [ "provider-reported" ], "body_markdown": "## What it is\n\nPearl is a layer-1 blockchain developed by [[O-0121]]. It replaces hash-based mining with a [[M-0007|proof of useful work]] built on matrix multiplication [[S-1106]] [[S-1107]]. The chain launched with an integer scheme [[S-1106]].\n\nPearl's September 2026 specification describes a protocol \"in which the unit of mining work is FP8 matrix multiplication on GPUs\" [[S-1105]]. This extends the integer proof of useful work of Komargodski and Weinstein [[S-1105]] [[S-1609]]. Komargodski is at Pearl Research Labs [[S-1010]].\n\nPearl presents the scheme as \"2-for-1\": GPUs produce proof of work \"as a side-effect of AI training and inference workloads\" [[S-1106]]. Its public monorepo contains several components [[S-1107]]:\n- a reference full node;\n- a vLLM-based GPU miner;\n- a Plonky2/STARKy-based proof-of-work circuit and verifier.\n\n## How it works\n\n**Mining.** A miner multiplies matrices of its choosing, which Pearl describes as \"the operands of an AI training or inference workload\" [[S-1105]]. Each block involves four steps [[S-1105]]:\n1. Low-rank noise matrices are derived from public seeds.\n2. The noised product is computed with quantised FP8 arithmetic.\n3. Output tiles are hashed as lottery tickets.\n4. A tile that meets the target wins the block.\n\nThe miner then subtracts lower-order correction terms to recover the useful product \"at FP8-level accuracy\" [[S-1105]].\n\n**Checks on winning tiles.** A jackpot policy approves a winning tile only when several statistical checks pass. These checks are meant to exclude degenerate inputs [[S-1105]].\n\n**Verification.** The verifier \"checks one tile of the matmul, never the whole product\" [[S-1105]]. It recomputes that tile bit for bit, reproducing the GPU's rounding and accumulation order off the GPU with the technique of Badash et al. [[S-1105]] [[S-1010]] (see [[M-0002]]). A zero-knowledge verifier can keep the miner's operand data private while proving that the tile satisfies the protocol rules [[S-1105]]. The integer whitepaper describes hash-based SNARKs (Plonky2), compressed to \"below 60KB\" by recursion [[S-1106]]. An appendix extends verification to mixture-of-experts layers [[S-1105]].\n\n## Evidence\n- **A live network.** Pearl's integer whitepaper states that the chain became live when the node code was made public [[S-1106]]. The repository supports mainnet and test networks, and has published release v1.2.1 [[S-1107]].\n- **A serving benchmark.** Pearl reports a benchmark on four H200 GPUs comparing Llama 3.3 70B with its \"two-for-one\" Pearl-certified variant, which re-implements a layer with a new quantisation mechanism [[S-1106]]:\n - the variant reached 17,206.26 tokens per second with pipeline parallelism, and 18,291.66 with four-way data parallelism;\n - the original bf16 model reached at most 15,269.81 tokens per second, and ran out of memory with four-way data parallelism;\n - MMLU scores were close: 0.8180 to 0.8198 for the variant against 0.8193 to 0.8198 [[S-1106]].\n- **Kernel overhead.** In a separate benchmark against stock serving engines, Pearl reports 5.08% end-to-end overhead for Llama 70B with four-way data parallelism on four H200 GPUs, and 3.9% for DeepSeek V3.2 with eight-way data parallelism plus expert parallelism on eight H200 GPUs [[S-3571]]. These are developer-reported measurements of its integer proof-of-useful-work kernel.\n- **The underlying theory.** The construction behind Pearl proves a multiplicative overhead of 1 + o(1) over naive matrix multiplication [[S-1609]]. Pearl's FP8 specification states a \"1 + o(1) factor\" overhead, without a measured percentage [[S-1105]].\n- **Independent measurement.** Basu measured Pearl's mainnet, running the integer scheme, in May 2026, counting 8,012 online workers among the top 15 miners on the AlphaPool mining pool, which Basu estimates to hold about 21% of network hashrate. String analysis suggests that the dominant mining software, from a third party, contains no inference code and generates matrices from random seeds. Random matrices passed verification in the study's tests, and a mining pool accepted shares mined with them [[S-0071]].\n\n## Limitations\n**Known speedups.** Pearl documents known mining speedups and the checks against them [[S-1105]]. It treats faster honest kernels or hardware as outside the threat model [[S-1105]].\n\n**Hardness assumption.** Security rests on an informal hardness assumption specific to quantised, noised matrices [[S-1105]].\n\n**Usefulness is not verified.** Verification checks the multiplication, not where the matrices came from [[S-0071]].\n\n**Benchmark caveats.** The benchmark compares a Pearl-certified model with the original model, and reports no run of the same certified model without mining [[S-1106]].\n\n**Fit to verification.** Pearl's proofs show that work was performed, not that no other work ran. Attestable, which proposes proof-of-work accounting for AI agreements, notes that \"a proof of some computation is not a proof of all computation\", and that bounding spare capacity needs a credible estimate of the compute available [[S-1102]].", "body_text": "What it is Pearl is a layer-1 blockchain developed by Pearl Research Labs. It replaces hash-based mining with a proof of useful work built on matrix multiplication [S-1106] [S-1107]. The chain launched with an integer scheme [S-1106]. Pearl's September 2026 specification describes a protocol \"in which the unit of mining work is FP8 matrix multiplication on GPUs\" [S-1105]. This extends the integer proof of useful work of Komargodski and Weinstein [S-1105] [S-1609]. Komargodski is at Pearl Research Labs [S-1010]. Pearl presents the scheme as \"2-for-1\": GPUs produce proof of work \"as a side-effect of AI training and inference workloads\" [S-1106]. Its public monorepo contains several components [S-1107]: - a reference full node; - a vLLM-based GPU miner; - a Plonky2/STARKy-based proof-of-work circuit and verifier. How it works Mining. A miner multiplies matrices of its choosing, which Pearl describes as \"the operands of an AI training or inference workload\" [S-1105]. Each block involves four steps [S-1105]: 1. Low-rank noise matrices are derived from public seeds. 2. The noised product is computed with quantised FP8 arithmetic. 3. Output tiles are hashed as lottery tickets. 4. A tile that meets the target wins the block. The miner then subtracts lower-order correction terms to recover the useful product \"at FP8-level accuracy\" [S-1105]. Checks on winning tiles. A jackpot policy approves a winning tile only when several statistical checks pass. These checks are meant to exclude degenerate inputs [S-1105]. Verification. The verifier \"checks one tile of the matmul, never the whole product\" [S-1105]. It recomputes that tile bit for bit, reproducing the GPU's rounding and accumulation order off the GPU with the technique of Badash et al. [S-1105] [S-1010] (see Deterministic and bit-exact inference). A zero-knowledge verifier can keep the miner's operand data private while proving that the tile satisfies the protocol rules [S-1105]. The integer whitepaper describes hash-based SNARKs (Plonky2), compressed to \"below 60KB\" by recursion [S-1106]. An appendix extends verification to mixture-of-experts layers [S-1105]. Evidence - A live network. Pearl's integer whitepaper states that the chain became live when the node code was made public [S-1106]. The repository supports mainnet and test networks, and has published release v1.2.1 [S-1107]. - A serving benchmark. Pearl reports a benchmark on four H200 GPUs comparing Llama 3.3 70B with its \"two-for-one\" Pearl-certified variant, which re-implements a layer with a new quantisation mechanism [S-1106]: - the variant reached 17,206.26 tokens per second with pipeline parallelism, and 18,291.66 with four-way data parallelism; - the original bf16 model reached at most 15,269.81 tokens per second, and ran out of memory with four-way data parallelism; - MMLU scores were close: 0.8180 to 0.8198 for the variant against 0.8193 to 0.8198 [S-1106]. - Kernel overhead. In a separate benchmark against stock serving engines, Pearl reports 5.08% end-to-end overhead for Llama 70B with four-way data parallelism on four H200 GPUs, and 3.9% for DeepSeek V3.2 with eight-way data parallelism plus expert parallelism on eight H200 GPUs [S-3571]. These are developer-reported measurements of its integer proof-of-useful-work kernel. - The underlying theory. The construction behind Pearl proves a multiplicative overhead of 1 + o(1) over naive matrix multiplication [S-1609]. Pearl's FP8 specification states a \"1 + o(1) factor\" overhead, without a measured percentage [S-1105]. - Independent measurement. Basu measured Pearl's mainnet, running the integer scheme, in May 2026, counting 8,012 online workers among the top 15 miners on the AlphaPool mining pool, which Basu estimates to hold about 21% of network hashrate. String analysis suggests that the dominant mining software, from a third party, contains no inference code and generates matrices from random seeds. Random matrices passed verification in the study's tests, and a mining pool accepted shares mined with them [S-0071]. Limitations Known speedups. Pearl documents known mining speedups and the checks against them [S-1105]. It treats faster honest kernels or hardware as outside the threat model [S-1105]. Hardness assumption. Security rests on an informal hardness assumption specific to quantised, noised matrices [S-1105]. Usefulness is not verified. Verification checks the multiplication, not where the matrices came from [S-0071]. Benchmark caveats. The benchmark compares a Pearl-certified model with the original model, and reports no run of the same certified model without mining [S-1106]. Fit to verification. Pearl's proofs show that work was performed, not that no other work ran. Attestable, which proposes proof-of-work accounting for AI agreements, notes that \"a proof of some computation is not a proof of all computation\", and that bounding spare capacity needs a credible estimate of the compute available [S-1102].", "referenced_by": [ { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "K-0011", "title": "Proof of (useful) work", "url": "https://trustbutveri.fyi/concepts/proof-of-useful-work/" }, { "id": "O-0121", "title": "Pearl Research Labs", "url": "https://trustbutveri.fyi/organizations/pearl-research/" } ] }, { "id": "I-0017", "slug": "pysyft-double-blind-evaluations", "title": "PySyft double-blind evaluations", "aliases": [ "PySyft confidential evaluations" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "one_liner": "PySyft coordinates an attested enclave where a model owner and evaluator run tests without sharing weights or private prompts.", "summary": "OpenMined's PySyft coordinates evaluations in which a model owner keeps its weights from the evaluator and the evaluator keeps its prompts from the model owner. Both parties check an enclave's attestation, submit code and assets, and approve the code before it runs. In 2026, AVERI evaluated Gemini 2.5 Flash Lite on private MLCommons prompts in Google Cloud Confidential Space using PySyft v0.10.x on an NVIDIA H100 with Intel TDX. Singapore AISI ran a separate private-prompt evaluation. The participants report end-to-end operation, but not all model code could be inspected or allowlisted. The guest operating system's builds were not independently reproducible, and Google's services signed and verified the attestation. The participant report contains no independent security evaluation of this workflow.", "category": "cryptographic-computational", "secondary_categories": [ "on-chip" ], "verifies": [ { "claim": "C-0005", "role": "primary", "note": "Both parties approve the attested workload that runs a private evaluation against the model and prompts they submit (S-3320)." } ], "threat_model": "semi-trusted", "adversarial_evaluation": "analysis", "hardware_requirement": "existing-features", "prover_cooperation": "required", "confidentiality": "preserving", "depends_on": [ { "target": "M-0008", "note": "Both parties rely on remote attestation of the Intel TDX and NVIDIA H100 confidential-computing stack." } ], "readiness": { "assessment": true, "level": "R2", "scope": "evaluating a private model on private prompts, neither party seeing the other's inputs", "rubric_version": "1.1", "rationale": "A participant report describes an end-to-end evaluation with private assets on commercial GPU hardware; PySyft's double-blind workflow has not been shown as a generally available service.\n- **R1** met: the report states the mutual-confidentiality claim, the enclave trust assumptions and the submission and approval procedure [[S-3320]].\n- **R2** met: AVERI evaluated Gemini 2.5 Flash Lite with private MLCommons prompts on an H100 with Intel TDX and PySyft v0.10.x. The report gives the stack and workflow, and states that Singapore AISI ran a separate evaluation with private prompts [[S-3320]].\n- **R3** not met for this implementation: OpenMined documents a pilot with real private assets, but not an available production service or reliance on its result for a verification decision [[S-3320]] [[S-3563]].\n- **R4** not met: the participant report includes no independent public security evaluation of the workflow.\nConfidence is medium. The demonstration and its limits come from the participants' own report [[S-3320]].", "evidence": [ "S-3320", "S-3563" ], "next_level_gaps": [ "A generally available production workflow, or documented reliance by another party on its result for a verification decision.", "An independent public security evaluation that leaves no critical flaw open." ], "confidence": "medium", "assessed_by": [ "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [], "blockers": [ { "text": "The pilot could not inspect or allowlist all model code, and the guest operating system builds were not independently reproducible.", "theme": "hardware-trust", "blocked_by": "M-0008", "sources": [ "S-3320" ] }, { "text": "The pilot ran on one H100; the authors name many-node confidential GPU clusters as the next scale target.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-3320" ] } ], "challenge_themes": [ "privacy-leakage", "hardware-trust", "performance-compatibility" ], "organizations": [ "O-0215" ], "people": [], "sources": [ { "source": "S-3320", "supports": "procedure, trust boundary, participants, model, private prompts, hardware, results and limits", "locator": "§2.5; §3; §4" }, { "source": "S-3563", "supports": "OpenMined's description of PySyft and the two 2026 evaluations", "locator": "Executive Summary" } ], "concepts": [ "K-0004", "K-0006", "K-0018" ], "kind": "research-prototype", "developer": [ "O-0215" ], "realises": [ "M-0025" ], "homepage": "https://openmined.org/blog/double-blind-evaluation-frontier-ai-model/", "type": "implementation", "url": "https://trustbutveri.fyi/implementations/pysyft-double-blind-evaluations/", "source_file": "content/implementations/pysyft-double-blind-evaluations.md", "flags_all": [ "provider-reported" ], "body_markdown": "## What it is\n\nPySyft is OpenMined's software for running approved jobs across private assets [[S-3563]]. In its double-blind evaluation workflow, a model owner and an evaluator submit their assets to a confidential GPU enclave after checking its attestation [[S-3320]]. Neither party receives the other's weights or prompts [[S-3320]].\n\n## How it works\n\nEach party first checks a signed measurement of the enclave software [[S-3320]]. The model owner uploads weights and inference code; the evaluator uploads private prompts and evaluation code. Both review and approve the code before the enclave runs it. PySyft coordinates the submissions and approvals [[S-3320]].\n\n## Evidence\n\n- In the 2026 pilot, AVERI evaluated Gemini 2.5 Flash Lite on private MLCommons AILuminate prompts in Google Cloud Confidential Space. The instance used one NVIDIA H100 with Intel TDX and PySyft v0.10.x. AVERI staff decrypted and scored the outputs [[S-3320]].\n- The same report states that Singapore AISI ran a separate evaluation with a private prompt set focused on harmful content in Singapore's context [[S-3320]].\n\n## Limitations\n\n- The pilot's authors could not inspect or allowlist all model code. AVERI accepted that condition [[S-3320]].\n- The guest operating system builds were not independently reproducible, and Google's services signed and verified the attestation [[S-3320]].\n- The published evaluation used one H100. The authors name many-node confidential GPU clusters as a next step [[S-3320]].", "body_text": "What it is PySyft is OpenMined's software for running approved jobs across private assets [S-3563]. In its double-blind evaluation workflow, a model owner and an evaluator submit their assets to a confidential GPU enclave after checking its attestation [S-3320]. Neither party receives the other's weights or prompts [S-3320]. How it works Each party first checks a signed measurement of the enclave software [S-3320]. The model owner uploads weights and inference code; the evaluator uploads private prompts and evaluation code. Both review and approve the code before the enclave runs it. PySyft coordinates the submissions and approvals [S-3320]. Evidence - In the 2026 pilot, AVERI evaluated Gemini 2.5 Flash Lite on private MLCommons AILuminate prompts in Google Cloud Confidential Space. The instance used one NVIDIA H100 with Intel TDX and PySyft v0.10.x. AVERI staff decrypted and scored the outputs [S-3320]. - The same report states that Singapore AISI ran a separate evaluation with a private prompt set focused on harmful content in Singapore's context [S-3320]. Limitations - The pilot's authors could not inspect or allowlist all model code. AVERI accepted that condition [S-3320]. - The guest operating system builds were not independently reproducible, and Google's services signed and verified the attestation [S-3320]. - The published evaluation used one H100. The authors name many-node confidential GPU clusters as a next step [S-3320].", "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "O-0215", "title": "OpenMined", "url": "https://trustbutveri.fyi/organizations/openmined/" } ] }, { "id": "I-0010", "slug": "rand-secure-inference-data-centers", "title": "RAND secure inference data center (SIDC) design", "aliases": [ "Secure inference data center", "SIDC", "Highly Secure Inference Data Centers" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "one_liner": "A RAND design for a purpose-built facility that serves already-trained AI models while protecting weights and inference data against state-level attackers.", "summary": "RAND's secure inference data center (SIDC) is a design for a compact, purpose-built facility that serves already-trained AI models, for example to national-security users. It aims to keep model weights, inference algorithms, prompts and responses confidential and intact against a well-resourced, state-backed attacker. The facility is split into physically isolated realms linked only by one-way data diodes in a fixed layout, with formally verified protocols at every boundary, human screening of each prompt and response, and checks of loaded weights against reference measurements. RAND estimates $37–50 million for a proof-of-concept and $277–345 million for an enterprise-scale facility, built in as few as 14 months under emergency or national-priority conditions. It is a published design, and RAND recommends prototyping its key security features now. It describes no way for an outside party to check the facility's properties, and its guarantees rest on an uncompromised setup phase.", "technical": "- **Assurance chain.** The cross-realm solution that mediates every boundary crossing is justified step by step [[S-1510]]. A hazard analysis (STPA-Sec) justifies the component, and the requirements are allocated in a system model. A finite-state protocol specification fixes authorized behaviour, sequencing, timing and fail-secure transitions. Cryptographic protocol analysis with Tamarin and CryptoVerif covers end-point authentication, secrecy, replay resistance and key freshness under stated assumptions, and state-machine model checking shows that unsafe control states are unreachable. The specified behaviour can then be translated into synthesizable hardware logic, such as FPGA-based channel control, and circuit-level assertions can check selected temporal properties at the register-transfer level [[S-1510]].\n- **Hardware assumptions.** Cost estimates assume 150 GPUs at about 250 kW for the proof-of-concept and 375 GPUs at about 3 MW at enterprise scale; the design is agnostic to the accelerator type [[S-1510]].\n- **Staffing.** About 100 cleared staff for a proof-of-concept and 300 for a deployment-scale facility [[S-1510]].\n- **Schedule.** The 14-month estimate assumes emergency or national-priority conditions and a government-owned facility built inside an existing hangar or warehouse. Otherwise RAND expects two to two and a half years to deploy, plus about a year for accreditation [[S-1510]].", "category": "isolation-architecture", "secondary_categories": [ "off-chip-devices" ], "verifies": [ { "claim": "C-0009", "role": "primary", "note": "Designed to keep weights and inference data inside the facility; the report does not describe how an external party would verify this." }, { "claim": "C-0005", "role": "supporting", "note": "The compute sanctum checks resident weights against reference measurements before serving." }, { "claim": "C-0004", "role": "supporting", "note": "Scoped to serving already-trained models." } ], "threat_model": "semi-trusted", "adversarial_evaluation": "analysis", "hardware_requirement": "retrofit-device", "prover_cooperation": "required", "confidentiality": "partial", "depends_on": [ { "target": "M-0012", "note": "Integrity checks compare loaded weights with reference measurements from a trusted setup." } ], "readiness": { "assessment": true, "level": "R1", "scope": "the operator's own weight security, with no outside verification described", "rubric_version": "1.1", "rationale": "The design is detailed and costed but has no prototype. Its stated objectives are security properties for the operator, not claims an outside party could check.\n\n- **R1** met for the security design: the report states its protection objectives, including that each inference response is verifiable given valid weights, architecture and retrieved context. It also states its threat model and assumptions, such as a trusted setup and likely supply-chain compromise of commodity hardware [[S-1510]].\n- **R2** not met: no facility or prototype has been published, and RAND recommends prototyping key security features now [[S-1510]].\n\nConfidence is low because the stated verification use is only partly addressed. The report describes internal integrity checks and audit logging, but no way for a party outside the operator to verify the facility's properties, and it omits architectural blueprints and detailed implementations from the public version [[S-1510]].", "evidence": [ "S-1510" ], "next_level_gaps": [ "A public working prototype, or reproducible published results, for key features such as the diode-gated realm topology and cross-realm protocols.", "A published way for a party other than the operator to verify the facility's claims, for example weight confidentiality or which model is served." ], "confidence": "low", "assessed_by": [ "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Everything rests on the trusted setup", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Reference measurements for model weights and reference data are established in a trusted setup phase. The report states that the system cannot detect compromise that happened before ingestion if the trusted setup itself is compromised.", "sources": [ "S-1510" ], "response": null }, { "assessment": true, "title": "Security weakens over long operation", "kind": "theoretical-argument", "severity": "minor", "status": "open", "description": "The authors claim that the facility can withstand attacks at the OC5 level for a five-year operational period. They expect its ability to withstand long OC5 campaigns to become less robust the longer the facility remains in operation.", "sources": [ "S-1510" ], "response": null } ], "blockers": [ { "text": "No prototype exists; RAND recommends prototyping key security features and integration now.", "theme": "adversarial-validation", "blocked_by": null, "sources": [ "S-1510" ] }, { "text": "The report describes internal integrity checks, audit logging and accreditation, but no way for a party outside the operator to verify the facility's properties.", "theme": "access-governance", "blocked_by": null, "sources": [ "S-1510" ] }, { "text": "Human review of every prompt and response makes each request take three to five minutes, with the review steps as the rate-limiting factor.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-1510" ] }, { "text": "Detailed design information is withheld from the public report and is to be evaluated privately with stakeholders, which limits independent public scrutiny.", "theme": "access-governance", "blocked_by": null, "sources": [ "S-1510" ] } ], "challenge_themes": [ "hardware-trust", "evidence-binding", "access-governance", "adversarial-validation", "performance-compatibility" ], "organizations": [ "O-0200" ], "people": [], "sources": [ { "source": "S-1510", "supports": "title, authors and date; objectives; OC5 threat model; realm architecture; request lifecycle; hazards and energy monitoring; trusted setup; formal-methods chain; costs, schedule and staffing; limitations; recommendations", "locator": "Summary (p. v); ch. 1 (pp. 1-3); ch. 3 (pp. 9-15); ch. 4 (pp. 18-22); ch. 5 (p. 24); Appendix B (pp. 27-30)" }, { "source": "S-1706", "supports": "Intelligence Security Laboratories' stated aim, its use of STPA-Sec, and its reference to this report" }, { "source": "S-1707", "supports": "ISL's executive director, Gabriel Kulp, a co-author of the report" } ], "concepts": [ "K-0005", "K-0015", "K-0017", "K-0018", "K-0022" ], "kind": "proposed-architecture", "developer": [ "O-0200" ], "realises": [ "M-0014", "M-0022" ], "homepage": "https://www.rand.org/pubs/research_reports/RRA4827-1.html", "type": "implementation", "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/", "source_file": "content/implementations/rand-secure-inference-data-centers.md", "flags_all": [ "provider-reported" ], "body_markdown": "## What it is\n\nRAND's *Highly Secure Inference Data Centers* report, published in August 2026, sets out a strategy for secure inference data centers (SIDCs): purpose-built facilities designed to protect AI models from advanced nation-state adversaries [[S-1510]]. SIDCs are compact facilities that deploy models already trained elsewhere [[S-1510]]. The authors write for AI laboratories, cloud providers and government stakeholders who need to protect strategically important deployments [[S-1510]].\n\nThe target adversary is at RAND's OC5 level: an operation comparable to about 1,000 experts working years ahead of the public state of the art, over several years, with a budget of up to $1 billion and state-level infrastructure [[S-1510]]. The report states these protection objectives:\n\n- confidentiality and integrity of the copies of model weights, inference algorithms and reference data inside the facility [[S-1510]];\n- confidentiality and integrity of prompts and responses while they are inside [[S-1510]];\n- verifiability of each inference response, given valid weights, model architecture and the context retrieved from reference data [[S-1510]];\n- resilience, so that compromise of any single subsystem or person does not enable a violation of these objectives [[S-1510]].\n\nThe objectives are defences against outside attackers. The public report does not describe how a party other than the operator would check that they hold [[S-1510]].\n\n## How it works\n\nThe design rests on three invariants [[S-1510]]:\n\n- Realm partitioning divides the facility into security domains, called realms, that do not trust one another [[S-1510]].\n- A fixed diode-gated topology routes all communication between realms through data diodes in a fixed arrangement, which enforce one-way flow at the physical layer [[S-1510]]. See [[M-0014]].\n- Verified boundary protocols mediate every crossing between realms, and each protocol implementation is verified against its security specification [[S-1510]].\n\n### Realms\n\nA realm is a physically isolated domain with exclusive control over its compute, storage, devices and personnel access [[S-1510]]. The main realms are:\n\n- **Interface room (IR).** The only point of contact with users. Staff screen plaintext prompts and responses here [[S-1510]].\n- **Payload exchange room (PER).** It accepts only two inbound payloads: prompts, passed to the IR, and reference-data updates, passed to the information technology management realm (ITM) [[S-1510]].\n- **Information technology management (ITM).** It ingests and validates vetted reference data under two-person integrity controls [[S-1510]].\n- **Trusted inference compute sanctum (TICS).** It holds the weights, the inference algorithm, reference data storage and all compute hardware [[S-1510]].\n- **Trustworthy recovery vault (TRV).** It keeps read-only copies of critical state and the reference measurements, as the integrity anchor for the TICS [[S-1510]].\n- **Protection operations center (POC).** It receives hazard signals from every realm over one-way inbound channels and leads incident response [[S-1510]].\n\n### Serving a request\n\nIn the report's example, a national-security user submits a signed, encrypted prompt over a classified network [[S-1510]]:\n\n1. The PER receives the prompt and passes it to the IR [[S-1510]].\n2. IR staff screen the prompt for compliance and adversarial inputs, then pass it to the TICS [[S-1510]].\n3. The TICS checks that the resident weights and reference data match the reference measurements held in the TRV, and produces the response [[S-1510]]. See [[M-0012]].\n4. IR staff screen the response, which leaves through the PER [[S-1510]].\n\n### Monitoring\n\nAny deviation from expected control, data, matter, energy or personnel flows raises a hazard and triggers immediate containment [[S-1510]]. Power delivery, heat and electromagnetic emissions are measured continuously and kept within bounds [[S-1510]]. See [[M-0022]]. All realms keep audit logs of interactions within and between realms [[S-1510]].\n\n### Trusted setup\n\nA trusted setup phase establishes the root of trust [[S-1510]]:\n\n- All hardware is destructively sampled and verified before installation. Components that pass are sealed, logged and installed under two-person integrity controls [[S-1510]].\n- Cryptographic keys are generated inside the facility and bound to verified hardware [[S-1510]].\n- Weights and reference data are checked against reference measurements supplied by the model provider [[S-1510]].\n\nThe authors assume that supply-chain compromise of commodity compute hardware is likely [[S-1510]]. Their \"concept to circuit\" method traces each security requirement down to formally verified protocols and circuit-level designs [[S-1510]].\n\n## Evidence\n- **Design and cost study.** RAND estimates $37–50 million for a proof-of-concept facility and $277–345 million for an enterprise-scale version [[S-1510]]. It reports that an SIDC can be built today with proven, off-the-shelf compute hardware, and that no fundamental research breakthroughs are required [[S-1510]].\n- **Schedule.** Construction and deployment could take as few as 14 months under emergency or national-priority conditions, and two to two and a half years otherwise, plus time for accreditation [[S-1510]].\n- **No prototype yet.** RAND recommends prototyping key security features and integration now, and notes that FPGA-based channel control with formally verified protocols and optical diodes can be tested immediately [[S-1510]]. Its other recommendations are to begin procurement, engage a system integrator and select a site early [[S-1510]]. [[O-0210|Intelligence Security Laboratories]], a nonprofit led by one of the report's co-authors, states that it aims to develop and demonstrate the security that critical AI deployments need, applies the same STPA-Sec method, and points to the report for details [[S-1706]] [[S-1707]].\n\n## Limitations\n- **Trusted setup.** The facility cannot detect compromise that happened before ingestion if the trusted setup itself was compromised [[S-1510]].\n- **No external check.** The public report omits architectural blueprints, detailed technical implementations and deployment procedures, and says these must be evaluated privately with stakeholders [[S-1510]]. It describes internal checks, audit logging and accreditation, but no verification path for outside parties [[S-1510]].\n- **Scope.** The facility does not defend against undetected biases or latent model behaviours [[S-1510]].\n- **Latency.** Human review makes a full request–response cycle take three to five minutes; automating some human-mediated steps could bring the overhead under a minute [[S-1510]].\n- **Time horizon.** The security claims are framed over a five-year operational period, and the authors expect resistance to long OC5 campaigns to weaken the longer the facility operates [[S-1510]].", "body_text": "What it is RAND's Highly Secure Inference Data Centers report, published in August 2026, sets out a strategy for secure inference data centers (SIDCs): purpose-built facilities designed to protect AI models from advanced nation-state adversaries [S-1510]. SIDCs are compact facilities that deploy models already trained elsewhere [S-1510]. The authors write for AI laboratories, cloud providers and government stakeholders who need to protect strategically important deployments [S-1510]. The target adversary is at RAND's OC5 level: an operation comparable to about 1,000 experts working years ahead of the public state of the art, over several years, with a budget of up to $1 billion and state-level infrastructure [S-1510]. The report states these protection objectives: - confidentiality and integrity of the copies of model weights, inference algorithms and reference data inside the facility [S-1510]; - confidentiality and integrity of prompts and responses while they are inside [S-1510]; - verifiability of each inference response, given valid weights, model architecture and the context retrieved from reference data [S-1510]; - resilience, so that compromise of any single subsystem or person does not enable a violation of these objectives [S-1510]. The objectives are defences against outside attackers. The public report does not describe how a party other than the operator would check that they hold [S-1510]. How it works The design rests on three invariants [S-1510]: - Realm partitioning divides the facility into security domains, called realms, that do not trust one another [S-1510]. - A fixed diode-gated topology routes all communication between realms through data diodes in a fixed arrangement, which enforce one-way flow at the physical layer [S-1510]. See Bandwidth limits and compartmentalization. - Verified boundary protocols mediate every crossing between realms, and each protocol implementation is verified against its security specification [S-1510]. Realms A realm is a physically isolated domain with exclusive control over its compute, storage, devices and personnel access [S-1510]. The main realms are: - Interface room (IR). The only point of contact with users. Staff screen plaintext prompts and responses here [S-1510]. - Payload exchange room (PER). It accepts only two inbound payloads: prompts, passed to the IR, and reference-data updates, passed to the information technology management realm (ITM) [S-1510]. - Information technology management (ITM). It ingests and validates vetted reference data under two-person integrity controls [S-1510]. - Trusted inference compute sanctum (TICS). It holds the weights, the inference algorithm, reference data storage and all compute hardware [S-1510]. - Trustworthy recovery vault (TRV). It keeps read-only copies of critical state and the reference measurements, as the integrity anchor for the TICS [S-1510]. - Protection operations center (POC). It receives hazard signals from every realm over one-way inbound channels and leads incident response [S-1510]. Serving a request In the report's example, a national-security user submits a signed, encrypted prompt over a classified network [S-1510]: 1. The PER receives the prompt and passes it to the IR [S-1510]. 2. IR staff screen the prompt for compliance and adversarial inputs, then pass it to the TICS [S-1510]. 3. The TICS checks that the resident weights and reference data match the reference measurements held in the TRV, and produces the response [S-1510]. See Model identity attestation. 4. IR staff screen the response, which leaves through the PER [S-1510]. Monitoring Any deviation from expected control, data, matter, energy or personnel flows raises a hazard and triggers immediate containment [S-1510]. Power delivery, heat and electromagnetic emissions are measured continuously and kept within bounds [S-1510]. See Side-channel suppression for isolated facilities. All realms keep audit logs of interactions within and between realms [S-1510]. Trusted setup A trusted setup phase establishes the root of trust [S-1510]: - All hardware is destructively sampled and verified before installation. Components that pass are sealed, logged and installed under two-person integrity controls [S-1510]. - Cryptographic keys are generated inside the facility and bound to verified hardware [S-1510]. - Weights and reference data are checked against reference measurements supplied by the model provider [S-1510]. The authors assume that supply-chain compromise of commodity compute hardware is likely [S-1510]. Their \"concept to circuit\" method traces each security requirement down to formally verified protocols and circuit-level designs [S-1510]. Evidence - Design and cost study. RAND estimates $37–50 million for a proof-of-concept facility and $277–345 million for an enterprise-scale version [S-1510]. It reports that an SIDC can be built today with proven, off-the-shelf compute hardware, and that no fundamental research breakthroughs are required [S-1510]. - Schedule. Construction and deployment could take as few as 14 months under emergency or national-priority conditions, and two to two and a half years otherwise, plus time for accreditation [S-1510]. - No prototype yet. RAND recommends prototyping key security features and integration now, and notes that FPGA-based channel control with formally verified protocols and optical diodes can be tested immediately [S-1510]. Its other recommendations are to begin procurement, engage a system integrator and select a site early [S-1510]. Intelligence Security Laboratories, a nonprofit led by one of the report's co-authors, states that it aims to develop and demonstrate the security that critical AI deployments need, applies the same STPA-Sec method, and points to the report for details [S-1706] [S-1707]. Limitations - Trusted setup. The facility cannot detect compromise that happened before ingestion if the trusted setup itself was compromised [S-1510]. - No external check. The public report omits architectural blueprints, detailed technical implementations and deployment procedures, and says these must be evaluated privately with stakeholders [S-1510]. It describes internal checks, audit logging and accreditation, but no verification path for outside parties [S-1510]. - Scope. The facility does not defend against undetected biases or latent model behaviours [S-1510]. - Latency. Human review makes a full request–response cycle take three to five minutes; automating some human-mediated steps could bring the overhead under a minute [S-1510]. - Time horizon. The security claims are framed over a five-year operational period, and the authors expect resistance to long OC5 campaigns to weaken the longer the facility operates [S-1510].", "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "O-0210", "title": "Intelligence Security Laboratories", "url": "https://trustbutveri.fyi/organizations/intelligence-security-laboratories/" }, { "id": "O-0200", "title": "RAND", "url": "https://trustbutveri.fyi/organizations/rand/" } ] }, { "id": "I-0008", "slug": "sash-confidential-network-logger", "title": "SASH confidential network logger", "aliases": [ "Confidential Network Logger (CNL)", "SASH inference verification prototype" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "one_liner": "An open-source prototype that routes a facility's inference traffic through a logger and re-runs requests on a separate cluster to check it serves inference.", "summary": "The Singapore AI Safety Hub (SASH) describes confidential network loggers (CNLs) as systems that split and copy traffic leaving a data centre's cable. In SASH's planned next prototype, an FPGA certifies the traffic it sees, and a random sample of inputs and outputs goes to a separate recomputation cluster, which re-runs them on its own model copy and flags mismatches; the verifier would see only encrypted hashes. The aim is to show a facility serves inference, not training. The first prototype's code is public; it re-runs every request rather than a sample and computes no hashes. A hardware demonstration used DGX Spark GPUs and Raspberry Pis. As of September 2026 it has not been tested at realistic scale or against a stated adversary. SASH says the recomputation cluster's security cannot yet be monitored, that a Raspberry Pi logger ties the design to particular supply chains, and that side channels remain.", "technical": "- **Services.** The repository starts five services together: a Svelte dashboard (port 5173) and FastAPI services for the gateway (8010), network logger (8020), main cluster (8030) and recomputation cluster (8040) [[S-1319]].\n- **Gateway.** It wraps each inference request in an envelope encrypted with AES-GCM-SIV, using a 256-bit key derived by SHA-256 from a configured host key and a nonce derived from the request ID and direction, and posts it to the network logger [[S-1319]].\n- **Network logger.** It forwards the encrypted request to the main cluster, then posts the encrypted request and response to the recomputation cluster's /verify endpoint. It returns the response only if the result is \"verified\"; otherwise it returns an HTTP 400 \"Recomputation failed\" [[S-1319]].\n- **Recomputation cluster.** It decrypts both, re-runs inference and compares response text for exact equality. Inference calls an OpenAI-compatible endpoint with seed 0 [[S-1319]].\n- **Model and demo modes.** The Docker configuration serves google/gemma-3-270m-it on a CPU vLLM image, and a mock mode returns placeholder text. A demo switch replaces output characters with random hex characters to \"simulate covert training\" [[S-1319]].\n- **Difference from the blog post.** In this version the logger is an inline software proxy that forwards every request for verification and computes no hashes of the traffic, whereas the blog post describes random sampling and hashes for the next, FPGA-based prototype [[S-1319]] [[S-1320]].", "category": "off-chip-devices", "secondary_categories": [ "cryptographic-computational" ], "verifies": [ { "claim": "C-0004", "role": "primary", "note": "SASH describes the aim as distinguishing inference from training in data centres (S-1320)." }, { "claim": "C-0005", "role": "supporting", "note": "Recomputation uses another copy of the declared model (S-1320, S-1319)." } ], "threat_model": "semi-trusted", "adversarial_evaluation": "none", "hardware_requirement": "retrofit-device", "prover_cooperation": "required", "confidentiality": "partial", "depends_on": [ { "target": "M-0002", "note": "The recomputation check compares output text exactly." }, { "target": "M-0017", "note": "The logger and recomputation cluster must be protected against tampering." } ], "readiness": { "assessment": true, "level": "R1", "scope": "telling inference from training on a mutually inspected cluster", "rubric_version": "1.1", "rationale": "Public code exists, but it runs a toy model through a software proxy with no stated adversary.\n\n- **R1** met: SASH publicly describes the design (a traffic-splitting logger plus sampled recomputation on a mutually inspected cluster), the claim (inference rather than training) and its stated limitations [[S-1320]]. Public code exists [[S-1319]].\n- **R2** not met: the implementation does not run under conditions representative of the verification use in any key respect. The model is Gemma 3 270M on a CPU inference image, and the logger is a software proxy [[S-1319]]. The hardware demonstration used DGX Spark GPUs and Raspberry Pis [[S-1320]]. The only adversary is a demo switch that replaces outputs with random characters [[S-1319]]. As of September 2026 no evaluation by others has been published.\n\nConfidence is medium: the code can be read directly, but the hardware demonstration is known only from SASH's blog post.", "evidence": [ "S-1320", "S-1319" ], "next_level_gaps": [ "A hardware logger (the planned FPGA version) on a real data-centre link, with realistic model size or traffic volume.", "A stated adversary and threat model, with testing against it.", "Random sampling and hashed or certified traffic records, as described in the blog post, implemented in the public code." ], "confidence": "medium", "assessed_by": [ "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Recomputation cluster security is not monitored", "kind": "open-question", "severity": "significant", "status": "open", "description": "SASH states that the current prototype has no means to monitor the security of the recomputation cluster, which holds a copy of the model and decides whether an alarm is raised.", "sources": [ "S-1320" ], "response": null }, { "assessment": true, "title": "Side channels not yet addressed", "kind": "open-question", "severity": "significant", "status": "open", "description": "SASH lists eliminating a wider range of side-channel vulnerabilities as future work.", "sources": [ "S-1320" ], "response": null }, { "assessment": true, "title": "Logger hardware supply chain", "kind": "open-question", "severity": "minor", "status": "open", "description": "The prototype uses a Raspberry Pi 5 as the logger, which SASH says locks the design into Broadcom and Arm supply chains.", "sources": [ "S-1320" ], "response": null } ], "blockers": [ { "text": "The planned FPGA logger has not yet been built.", "theme": "hardware-trust", "blocked_by": null, "sources": [ "S-1320" ] }, { "text": "The design has not been scaled to production traffic volumes.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-1320" ] }, { "text": "Exact-match recomputation requires reproducible inference.", "theme": "evidence-binding", "blocked_by": "M-0002", "sources": [ "S-1319" ] } ], "challenge_themes": [ "hardware-trust", "performance-compatibility", "evidence-binding", "adversarial-validation" ], "organizations": [ "O-0160", "O-0206" ], "people": [], "sources": [ { "source": "S-1320", "supports": "project aims, CNL description, workflow, hardware demo, partners, limitations and future work", "locator": "whole post" }, { "source": "S-1319", "supports": "components, data flow, encryption, recomputation comparison, model, demo switch, licence", "locator": "README.md; run-dev.sh; components/gateway, network_logger, recomputation_cluster, lib/" }, { "source": "S-3200", "supports": "planned v2 of the logger based on a zero-knowledge proof system", "locator": "paragraph beginning 'SASH is contributing to this effort'" } ], "concepts": [ "K-0014", "K-0009", "K-0020", "K-0025" ], "kind": "research-prototype", "developer": [ "O-0160" ], "realises": [ "M-0013", "M-0001" ], "homepage": "https://www.aisafety.sg/blog/making-ai-verification-international", "repo": "https://github.com/sg-ai-safety-hub/inference-verification", "type": "implementation", "url": "https://trustbutveri.fyi/implementations/sash-confidential-network-logger/", "source_file": "content/implementations/sash-confidential-network-logger.md", "flags_all": [ "provider-reported" ], "body_markdown": "## What it is\n\nThe Singapore AI Safety Hub (SASH) describes an international collaboration to build inference-verification mechanisms that distinguish model inference from training in data centres [[S-1320]]. Its early partners include the Oxford Martin AI Governance Initiative and the Future of Life Institute [[S-1320]]. The researchers span Singapore, the US, the UK, Hungary, Canada and Germany [[S-1320]].\n\nThe core component is a Confidential Network Logger (CNL): \"specialised traffic analysis systems which split and copy network traffic coming out of a cable\" [[S-1320]]. SASH argues that, unlike on-chip mechanisms, CNLs \"can be installed on most datacenters\" [[S-1320]]. The first prototype is published as open-source code under the MIT licence, and its README calls it \"a prototype demo for verifying LLM inference\" [[S-1319]]. It implements [[M-0013|network logging]] combined with [[M-0001|recomputation]]. SASH's planned next prototype samples requests, while the published code re-runs all of them [[S-1320]] [[S-1319]].\n\n## How it works\n\nSASH describes the flow planned for its next prototype as follows [[S-1320]]:\n\n1. An FPGA placed between the host's cluster and its output gateway issues \"cryptographic certificates attesting to the encrypted traffic it has seen\" as traffic passes through [[S-1320]].\n2. A random sample is sent to a recomputation cluster \"which has been mutually inspected by both parties\" and houses another copy of the model [[S-1320]].\n3. The cluster recomputes the received inputs, and an alarm is triggered if the outputs do not match [[S-1320]].\n\nSASH says the verifier would see only \"encrypted hashes\" of inputs and outputs, so it can tell whether they match without reading them [[S-1320]]. SASH motivates the approach by the different traffic patterns of the two workloads [[S-1320]]. It describes inference as many small bursts of activity on relatively few GPUs, and training of large models as sustained, cyclical activity on many GPUs [[S-1320]].\n\nThe published code splits the system into a gateway, a network logger, a main cluster, a recomputation cluster and a dashboard [[S-1319]]:\n\n- The gateway encrypts each request and passes it to the logger [[S-1319]].\n- The logger forwards it to the main cluster, then sends the request and response to the recomputation cluster, and releases the response only if the check passes [[S-1319]].\n- The recomputation cluster re-runs the request and compares the response text for an exact match [[S-1319]].\n\nIn this version, every request is forwarded for verification and no hashes of the traffic are computed; the random sampling and hashes described in the blog post belong to the planned next prototype [[S-1319]] [[S-1320]]. Reproducible inference, which exact matching needs, is covered in [[M-0002]].\n\n## Evidence\n- **Hardware demonstration.** In SASH's demonstration, \"a pair of DGX Spark GPUs sit in for the audited datacenter and recomputation cluster\", and Raspberry Pis stand in for the logger and the data-centre frontend [[S-1320]].\n- **Public code.** The code's container setup serves the Gemma 3 270M instruction-tuned model on a CPU inference image, and a demo switch simulates covert training by replacing output with random characters [[S-1319]].\n- **Next prototype.** SASH says a further prototype with an FPGA is planned [[S-1320]]. In July 2026 it said it plans a v2 of the logger based on a zero-knowledge proof system [[S-3200]].\n\n## Limitations\nSASH lists the following [[S-1320]]:\n\n- \"There is no means to monitor the security of the recomputation cluster\" [[S-1320]].\n- Using a Raspberry Pi 5 as the logger locks the design into Broadcom and Arm supply chains [[S-1320]].\n- Future versions would add zero-knowledge proofs to the recomputation method, eliminate a wider range of side-channel vulnerabilities, and scale the design to production traffic volumes [[S-1320]].\n\nThe general limitations of network taps, such as covert capacity in model outputs and line-rate hashing, are covered in [[M-0013]].", "body_text": "What it is The Singapore AI Safety Hub (SASH) describes an international collaboration to build inference-verification mechanisms that distinguish model inference from training in data centres [S-1320]. Its early partners include the Oxford Martin AI Governance Initiative and the Future of Life Institute [S-1320]. The researchers span Singapore, the US, the UK, Hungary, Canada and Germany [S-1320]. The core component is a Confidential Network Logger (CNL): \"specialised traffic analysis systems which split and copy network traffic coming out of a cable\" [S-1320]. SASH argues that, unlike on-chip mechanisms, CNLs \"can be installed on most datacenters\" [S-1320]. The first prototype is published as open-source code under the MIT licence, and its README calls it \"a prototype demo for verifying LLM inference\" [S-1319]. It implements network logging combined with recomputation. SASH's planned next prototype samples requests, while the published code re-runs all of them [S-1320] [S-1319]. How it works SASH describes the flow planned for its next prototype as follows [S-1320]: 1. An FPGA placed between the host's cluster and its output gateway issues \"cryptographic certificates attesting to the encrypted traffic it has seen\" as traffic passes through [S-1320]. 2. A random sample is sent to a recomputation cluster \"which has been mutually inspected by both parties\" and houses another copy of the model [S-1320]. 3. The cluster recomputes the received inputs, and an alarm is triggered if the outputs do not match [S-1320]. SASH says the verifier would see only \"encrypted hashes\" of inputs and outputs, so it can tell whether they match without reading them [S-1320]. SASH motivates the approach by the different traffic patterns of the two workloads [S-1320]. It describes inference as many small bursts of activity on relatively few GPUs, and training of large models as sustained, cyclical activity on many GPUs [S-1320]. The published code splits the system into a gateway, a network logger, a main cluster, a recomputation cluster and a dashboard [S-1319]: - The gateway encrypts each request and passes it to the logger [S-1319]. - The logger forwards it to the main cluster, then sends the request and response to the recomputation cluster, and releases the response only if the check passes [S-1319]. - The recomputation cluster re-runs the request and compares the response text for an exact match [S-1319]. In this version, every request is forwarded for verification and no hashes of the traffic are computed; the random sampling and hashes described in the blog post belong to the planned next prototype [S-1319] [S-1320]. Reproducible inference, which exact matching needs, is covered in Deterministic and bit-exact inference. Evidence - Hardware demonstration. In SASH's demonstration, \"a pair of DGX Spark GPUs sit in for the audited datacenter and recomputation cluster\", and Raspberry Pis stand in for the logger and the data-centre frontend [S-1320]. - Public code. The code's container setup serves the Gemma 3 270M instruction-tuned model on a CPU inference image, and a demo switch simulates covert training by replacing output with random characters [S-1319]. - Next prototype. SASH says a further prototype with an FPGA is planned [S-1320]. In July 2026 it said it plans a v2 of the logger based on a zero-knowledge proof system [S-3200]. Limitations SASH lists the following [S-1320]: - \"There is no means to monitor the security of the recomputation cluster\" [S-1320]. - Using a Raspberry Pi 5 as the logger locks the design into Broadcom and Arm supply chains [S-1320]. - Future versions would add zero-knowledge proofs to the recomputation method, eliminate a wider range of side-channel vulnerabilities, and scale the design to production traffic volumes [S-1320]. The general limitations of network taps, such as covert capacity in model outputs and line-rate hashing, are covered in Network taps and certifiers.", "referenced_by": [ { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "O-0206", "title": "Future of Life Institute", "url": "https://trustbutveri.fyi/organizations/future-of-life-institute/" }, { "id": "O-0205", "title": "Oxford Martin AI Governance Initiative", "url": "https://trustbutveri.fyi/organizations/oxford-martin-ai-governance-initiative/" }, { "id": "O-0160", "title": "Singapore AI Safety Hub (SASH)", "url": "https://trustbutveri.fyi/organizations/singapore-ai-safety-hub/" } ] }, { "id": "I-0006", "slug": "tinfoil-model-identity", "title": "Tinfoil model identity (Modelwrap)", "aliases": [ "Modelwrap", "Tinfoil model integrity" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "one_liner": "Tinfoil's method for proving which model weights its enclave-hosted inference service runs, by binding a dm-verity hash of the weights into remote attestation.", "summary": "Modelwrap is an open-source tool that lets clients of Tinfoil's enclave-hosted inference service check which model weights it runs. Tinfoil reports running the service on AMD or Intel confidential virtual machines with NVIDIA GPUs in confidential-computing mode. Modelwrap packs the weights into a read-only disk image with a 32-byte Merkle-tree commitment. That hash is written into the enclave's measured boot configuration, and the kernel checks every block read against it. Clients check the hardware-signed attestation against measurements published through Sigstore, and anyone can rebuild the hash for a public model. As of September 2026 no independent security evaluation has been published. The chain inherits the limits of the underlying TEEs. Researchers with physical access have forged Intel TDX and AMD SEV-SNP attestations, and a software-only attack forged SEV-SNP attestations before AMD released firmware fixes. For private models, clients can confirm only that the same weights are served each time.", "technical": "- **Build.** Modelwrap downloads a pinned Hugging Face revision, normalizes the directory structure so the result is reproducible, and writes an EROFS image. It then computes a dm-verity root hash, a 32-byte commitment, with veritysetup [[S-0013]]. Its outputs are the image and a metadata file holding the root hash, offset and verity UUID. An encryption mode supports private models [[S-1209]].\n- **Binding.** The root hash goes on the kernel command line, which the enclave measurement includes. At runtime dm-verity checks each block read by the inference engine against the root and fails on any mismatch [[S-0013]].\n- **Boot chain.** The CPU measures the OVMF firmware first, then the kernel and initrd. A tinfoil-config.yml is checked against a hash on the kernel command line. The enclave checks each NVIDIA GPU's attestation with NVIDIA's local-gpu-verifier to confirm confidential-computing mode, and aborts the boot if the check fails. Model volumes are mounted read-only and checked against their Modelwrap commitments [[S-1207]].\n- **Client check.** The SDK verifies the attestation certificate chain to the CPU vendor's root and verifies a Sigstore bundle of expected measurements. It then confirms that the TLS public key matches the key in the attestation document [[S-1207]].\n- **Reported costs.** The hash tree adds about 0.8% to image size, and builds take 5 s for a 549 MB model and 13 min 25 s for a 554 GB model. Cold-cache loading takes about 80% longer [[S-0013]].", "category": "cryptographic-computational", "secondary_categories": [ "on-chip" ], "verifies": [ { "claim": "C-0005", "role": "primary", "note": "Clients check that the served weights match a committed root hash." } ], "threat_model": "semi-trusted", "adversarial_evaluation": "analysis", "hardware_requirement": "existing-features", "prover_cooperation": "required", "confidentiality": "preserving", "depends_on": [ { "target": "M-0008", "note": "Relies on AMD SEV-SNP or Intel TDX attestation and NVIDIA GPU confidential computing." } ], "readiness": { "assessment": true, "level": "R3", "scope": "showing clients that the served weights match a committed hash", "rubric_version": "1.1", "rationale": "Tinfoil reports running the chain in its production service, but almost all the evidence comes from Tinfoil, no independent evaluation exists, and the underlying TEEs have open critical flaws.\n\n- **R1** met: the design is published [[S-0013]].\n- **R2** met: a public working implementation under an MIT license [[S-1209]] is deployed on realistic hardware, AMD SEV-SNP or Intel TDX with NVIDIA H100, H200 or B200 (provider-reported) [[S-1206]]. Build results are reported for models of up to 554 GB [[S-0013]].\n- **R3** met on the provider's own account, as a production service that clients can use. Tinfoil reports serving each model from multiple enclaves in its production deployment, with client SDKs that verify the attestation before sending any data [[S-1208]]. The enclave checks model volumes against Modelwrap commitments at boot [[S-1207]].\n- **R4** not met. As of September 2026 no independent audit, red-team or peer-reviewed analysis of Modelwrap or Tinfoil's model-identity chain has been published. Independent research on the underlying TEEs used physical access to forge Intel TDX attestations and, by pairing them with relayed H100 attestations, passed a workload outside TEE protection [[S-1202]]. Other independent research forged AMD SEV-SNP attestations [[S-1210]] [[S-1212]]. Tinfoil's documentation acknowledges physical attacks [[S-1206]].", "evidence": [ "S-0013", "S-1206", "S-1207", "S-1208", "S-1209", "S-1202", "S-1210", "S-1212" ], "next_level_gaps": [ "An independent security evaluation of Modelwrap and the attestation chain that leaves no critical flaw open.", "A supported tool for audit-time verification from transparency records.", "A way for third parties to learn something about private models beyond consistency across requests." ], "confidence": "medium", "assessed_by": [ "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Inherits attacks on the underlying TEEs", "kind": "demonstrated-attack", "severity": "critical", "status": "open", "description": "Tinfoil's documentation states that an attacker with physical access \"can potentially compromise the enclave\". It notes that attestation forgery has been demonstrated for Intel TDX, and that researchers breached confidentiality on AMD SEV-SNP through key extraction; such attacks need \"significant resources and physical proximity\" [[S-1206]]. In that research, TEE.fail, the AMD key recovered was an ECDSA key used by OpenSSL inside the VM [[S-1202]]. Other independent attacks did forge SEV-SNP attestations. Battering RAM did so with a DDR4 memory interposer, and RMPocalypse did so from a malicious hypervisor without physical access [[S-1210]] [[S-1212]]. AMD reports firmware fixes for RMPocalypse [[S-1213]]. The TEE.fail authors report that NVIDIA does not bind the H100 to the identities of specific VMs. Pairing relayed H100 attestations with forged TDX attestations, they made a system outside TEE protection pass both checks [[S-1202]]. They also report that Intel and AMD treat interposer attacks as outside their threat models, and they advise users to run confidential workloads only on physically secure servers [[S-1202]]. Tinfoil reports that its enclave checks each GPU's attestation at boot, refuses to start if the check fails, and links the GPU attestation to the CPU attestation report [[S-1207]]. The GPU evidence is then only as trustworthy as the CPU attestation, which TEE.fail forged for Intel TDX [[S-1202]].", "sources": [ "S-1206", "S-1202", "S-1210", "S-1212", "S-1213", "S-1207" ], "response": null }, { "assessment": true, "title": "Side channels, I/O leakage and denial of service are outside enclave protection", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "Tinfoil's documentation lists timing, power and electromagnetic side channels, host observation of access patterns and I/O, denial of service, supply-chain compromise and rollback as limitations [[S-1206]].", "sources": [ "S-1206" ], "response": null }, { "assessment": true, "title": "Private models can be checked only for consistency", "kind": "open-question", "severity": "significant", "status": "open", "description": "For unpublished weights, the root hash appears in the attestation without the weights being exposed. Users can then confirm only that they get the same model each time [[S-0013]].", "sources": [ "S-0013" ], "response": null } ], "blockers": [ { "text": "The underlying TEE attestation does not resist attackers with physical access to the host.", "theme": "hardware-trust", "blocked_by": "M-0008", "sources": [ "S-1202", "S-1206" ] }, { "text": "No independent evaluation of the model-identity chain has been published.", "theme": "adversarial-validation", "blocked_by": null, "sources": [] } ], "challenge_themes": [ "evidence-binding", "hardware-trust", "adversarial-validation" ], "organizations": [ "O-0141" ], "people": [], "sources": [ { "source": "S-0013", "supports": "Modelwrap design, binding to kernel command line, runtime enforcement, private models, overheads (provider-reported)" }, { "source": "S-1206", "supports": "supported hardware, trust model, documented limitations (provider-reported)", "locator": "Supported hardware; Trust model; Limitations" }, { "source": "S-1207", "supports": "boot measurement chain, boot-time GPU attestation check linked to the CPU attestation report, Sigstore publication, client verification, closed-source components (provider-reported)" }, { "source": "S-1208", "supports": "connection-time vs audit-time verification; production deployment; no supported audit tool (provider-reported)", "locator": "In-band vs. out-of-band verification" }, { "source": "S-1209", "supports": "open-source code, MIT license, release v0.3.0, outputs and encryption mode" }, { "source": "S-1202", "supports": "independent demonstration of Intel TDX attestation forgery, SEV-SNP OpenSSL key recovery and H100 attestation relay", "locator": "§1.1, §8.3, §10.2" }, { "source": "S-1210", "supports": "SEV-SNP attestation breach with a DDR4 interposer (Battering RAM)", "locator": "Abstract; site FAQ" }, { "source": "S-1212", "supports": "software-only SEV-SNP attestation forgery by a malicious hypervisor (RMPocalypse)", "locator": "Abstract; site" }, { "source": "S-1213", "supports": "AMD firmware fixes for RMPocalypse (vendor-reported)", "locator": "Mitigation tables" } ], "concepts": [ "K-0024", "K-0004", "K-0006", "K-0005", "K-0019" ], "kind": "product", "developer": [ "O-0141" ], "realises": [ "M-0012", "M-0008" ], "homepage": "https://tinfoil.sh/", "repo": "https://github.com/tinfoilsh/modelwrap", "type": "implementation", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/", "source_file": "content/implementations/tinfoil-model-identity.md", "flags_all": [ "provider-reported", "critical-flaw-open" ], "body_markdown": "## What it is\n\nTinfoil reports that it operates an AI inference service inside secure enclaves. It runs on AMD EPYC processors with SEV-SNP or Intel Xeon processors with TDX, together with NVIDIA H100, H200 or B200 GPUs in confidential-computing mode [[S-1206]]. Its stated trust model is that clients trust the hardware manufacturer, but not the cloud provider's software stack, the hypervisor or the host OS [[S-1206]].\n\nModelwrap is Tinfoil's open-source tool, under an MIT license, that \"builds reproducible dm-verity EROFS images of Hugging Face models\" [[S-1209]]. It lets clients check which weights the service is running [[S-0013]]. The chain implements [[M-0012|model identity attestation]] on top of [[M-0008|TEE remote attestation]].\n\n## How it works\n\nTinfoil states the problem as follows: \"attestation measures launch state, not runtime state\", and weights are read from disk after the enclave boots [[S-0013]]. The system works in three phases [[S-0013]]:\n\n1. **Commit.** Modelwrap builds a normalized, read-only image of a pinned model revision and computes a Merkle-tree root hash.\n2. **Bind.** The root hash is passed to dm-verity on the kernel command line, which the enclave measurement covers, so the attestation vouches for the hash.\n3. **Enforce.** Whenever the inference server reads the weights, dm-verity checks each block against the attested root and fails on any mismatch.\n\nTinfoil contrasts this with signing weights, for example with Sigstore Model Transparency. It argues that a signature protects the weights only at the moment it is checked, while a malicious hypervisor could alter the disk afterwards [[S-0013]].\n\nThe rest of the boot chain is measured too: firmware, kernel, initrd and a configuration file. The enclave checks each GPU's attestation at boot and does not start if the check fails [[S-1207]]. Tinfoil's software development kit (SDK) then runs the client-side checks:\n\n- it verifies the attestation back to the CPU vendor's root certificate;\n- it compares the measurements with a Sigstore-published record of the expected values;\n- it binds the TLS connection to the key in the attestation [[S-1207]].\n\nFor a public model, anyone can rerun Modelwrap on the same Hugging Face commit and compare root hashes. For a private model, the customer builds the image and can encrypt it with dm-crypt, and the attestation shows only the root hash [[S-0013]]. Tinfoil lists billing and account management as closed-source components outside the security-critical path [[S-1207]].\n\n## Evidence\n- Tinfoil reports that the hash tree adds about 0.8% to storage and that builds take 5 s for a 549 MB model and 13 min 25 s for a 554 GB model. Cold-cache loading takes about 80% longer with verification, but inference runs at full speed once the weights are in GPU memory [[S-0013]].\n- Modelwrap is public under an MIT license, and v0.3.0 was the latest release in September 2026 [[S-1209]].\n- Tinfoil reports serving each model from multiple enclaves in its production deployment [[S-1208]]. Its attestation architecture checks model volumes against Modelwrap commitments at boot [[S-1207]].\n\n## Limitations\nTinfoil documents several limitations of enclaves [[S-1206]]:\n\n- an attacker with physical access \"can potentially compromise the enclave\", with demonstrated attestation forgery for Intel TDX and key extraction for AMD SEV-SNP;\n- timing, power and electromagnetic side channels;\n- host observation of access patterns and I/O;\n- denial of service, supply-chain compromise and rollback.\n\nOther limits:\n\n- Independent researchers report that NVIDIA does not bind the H100 to the identities of specific VMs. By pairing relayed H100 attestations with forged TDX attestations, they made a system running outside TEE protection pass both checks [[S-1202]]. Tinfoil reports that its GPU check runs inside the enclave at boot and is linked to the CPU attestation report [[S-1207]].\n- For private models, clients learn only that the same weights are served each time [[S-0013]].\n- For audit-time checks of its transparency records, Tinfoil \"does not provide a supported tool for querying these records\", and it notes that rebuilding binaries to check measurements independently is expensive [[S-1208]].", "body_text": "What it is Tinfoil reports that it operates an AI inference service inside secure enclaves. It runs on AMD EPYC processors with SEV-SNP or Intel Xeon processors with TDX, together with NVIDIA H100, H200 or B200 GPUs in confidential-computing mode [S-1206]. Its stated trust model is that clients trust the hardware manufacturer, but not the cloud provider's software stack, the hypervisor or the host OS [S-1206]. Modelwrap is Tinfoil's open-source tool, under an MIT license, that \"builds reproducible dm-verity EROFS images of Hugging Face models\" [S-1209]. It lets clients check which weights the service is running [S-0013]. The chain implements model identity attestation on top of TEE remote attestation. How it works Tinfoil states the problem as follows: \"attestation measures launch state, not runtime state\", and weights are read from disk after the enclave boots [S-0013]. The system works in three phases [S-0013]: 1. Commit. Modelwrap builds a normalized, read-only image of a pinned model revision and computes a Merkle-tree root hash. 2. Bind. The root hash is passed to dm-verity on the kernel command line, which the enclave measurement covers, so the attestation vouches for the hash. 3. Enforce. Whenever the inference server reads the weights, dm-verity checks each block against the attested root and fails on any mismatch. Tinfoil contrasts this with signing weights, for example with Sigstore Model Transparency. It argues that a signature protects the weights only at the moment it is checked, while a malicious hypervisor could alter the disk afterwards [S-0013]. The rest of the boot chain is measured too: firmware, kernel, initrd and a configuration file. The enclave checks each GPU's attestation at boot and does not start if the check fails [S-1207]. Tinfoil's software development kit (SDK) then runs the client-side checks: - it verifies the attestation back to the CPU vendor's root certificate; - it compares the measurements with a Sigstore-published record of the expected values; - it binds the TLS connection to the key in the attestation [S-1207]. For a public model, anyone can rerun Modelwrap on the same Hugging Face commit and compare root hashes. For a private model, the customer builds the image and can encrypt it with dm-crypt, and the attestation shows only the root hash [S-0013]. Tinfoil lists billing and account management as closed-source components outside the security-critical path [S-1207]. Evidence - Tinfoil reports that the hash tree adds about 0.8% to storage and that builds take 5 s for a 549 MB model and 13 min 25 s for a 554 GB model. Cold-cache loading takes about 80% longer with verification, but inference runs at full speed once the weights are in GPU memory [S-0013]. - Modelwrap is public under an MIT license, and v0.3.0 was the latest release in September 2026 [S-1209]. - Tinfoil reports serving each model from multiple enclaves in its production deployment [S-1208]. Its attestation architecture checks model volumes against Modelwrap commitments at boot [S-1207]. Limitations Tinfoil documents several limitations of enclaves [S-1206]: - an attacker with physical access \"can potentially compromise the enclave\", with demonstrated attestation forgery for Intel TDX and key extraction for AMD SEV-SNP; - timing, power and electromagnetic side channels; - host observation of access patterns and I/O; - denial of service, supply-chain compromise and rollback. Other limits: - Independent researchers report that NVIDIA does not bind the H100 to the identities of specific VMs. By pairing relayed H100 attestations with forged TDX attestations, they made a system running outside TEE protection pass both checks [S-1202]. Tinfoil reports that its GPU check runs inside the enclave at boot and is linked to the CPU attestation report [S-1207]. - For private models, clients learn only that the same weights are served each time [S-0013]. - For audit-time checks of its transparency records, Tinfoil \"does not provide a supported tool for querying these records\", and it notes that rebuilding binaries to check measurements independently is expensive [S-1208].", "referenced_by": [ { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "O-0140", "title": "NVIDIA", "url": "https://trustbutveri.fyi/organizations/nvidia/" }, { "id": "O-0141", "title": "Tinfoil", "url": "https://trustbutveri.fyi/organizations/tinfoil/" } ] }, { "id": "I-0001", "slug": "toploc", "title": "TOPLOC", "aliases": [ "TOPLOC v2" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "one_liner": "TOPLOC is a hashing scheme from Prime Intellect that lets a verifier check whether an inference provider ran the model, prompt and precision it claims.", "summary": "TOPLOC is a hashing scheme for checking that an inference provider ran the model, prompt and numerical precision it claims. During generation, the provider records a compact fingerprint of the model's largest last-layer activations. A verifier re-runs the sequence in one pass and checks that the fingerprints match within set tolerances. The tolerances absorb harmless differences between GPUs. The peer-reviewed paper reports catching every tested change to model, prompt or precision, with no false positives or negatives. Proofs take 258 bytes per 32 generated tokens. TOPLOC is open source. Prime Intellect, its developer, used it in 2025 to accept or reject work from untrusted computers in decentralized training and data-generation runs. As of September 2026 no independent security evaluation has been published. The authors list attacks it cannot yet catch, such as speculative decoding with a cheaper model. Subtle changes are also harder to detect than large ones.", "technical": "- The prover commits to its activations every 32 generated tokens. It takes the top-k values of the last hidden layer, with k = 128 in the main configuration. It encodes their indices and values as a polynomial over an integer field, with a modulus chosen to be injective on the index set [[S-1000]]. The result is k two-byte coefficients. For Llama 3.1-8B-Instruct that is 258 bytes per 32 tokens, against 262 KB for storing the embeddings directly [[S-1000]].\n- The verifier decodes the proof and recomputes the top-k values with a prefill pass. It counts exponent mismatches and computes the mean and median mantissa differences. Validation succeeds if all three are below their thresholds. For bf16 the thresholds are 38, 10 and 8 [[S-1000]].\n- The hardware tests used 1× A100, 1× RTX 4090 and 2× RTX 4090 GPUs, with FlashAttention 2, PyTorch SDPA and FlexAttention. The authors read the activations through a vLLM hook [[S-1000]].\n- Prime Intellect reports that validation is up to 100 times faster than the original inference [[S-1002]] [[S-1003]]. In SYNTHETIC-2 it reports a median verification cost, averaged across models, 25 times lower than re-running the inference [[S-3000]]. It reports that proof generation cut tokens-per-second throughput by about 1% in INTELLECT-2 [[S-1003]].\n- Prime Intellect reports that TOPLOC v2 adds reproducible Gumbel noise for categorical sampling, so that verifiers can check token sampling. Version 2 also extends the scheme to pipeline-parallel inference. A pass at the final pipeline stage accepts all stages, and a failure triggers a stage-by-stage replay that finds the first faulty node [[S-1004]] [[S-3000]].\n- The package is published on PyPI as `toploc`. The latest tag is v0.1.6 [[S-1001]].", "category": "cryptographic-computational", "secondary_categories": [], "verifies": [ { "claim": "C-0005", "role": "primary", "note": "Checks that the provider produced outputs with the claimed model weights, prompt and precision." } ], "threat_model": "adversarial", "adversarial_evaluation": "analysis", "hardware_requirement": "none", "prover_cooperation": "required", "confidentiality": "revealing", "depends_on": [], "readiness": { "assessment": true, "level": "R3", "scope": "checking that untrusted providers used the claimed model, prompt and precision", "rubric_version": "1.1", "rationale": "Prime Intellect, its developer, has used the public package in production to accept or reject work from untrusted computers, but no independent security audit or red-team has been published.\n\n- **R1** met: the peer-reviewed paper sets out the design, the claim and the threat [[S-1000]]. The claim is that the provider used the stated model, prompt and precision. The threat is undisclosed changes to any of them.\n- **R2** met: a public MIT-licensed implementation exists [[S-1001]]. The paper reports results on A100 and RTX 4090 GPUs across several models, attention implementations and one- and two-GPU tensor parallelism [[S-1000]].\n- **R3** met through the developer's own use: the package is public [[S-1001]], and Prime Intellect reports using it in production to accept or reject work from untrusted inference workers, in a 32-billion-parameter decentralized training run and a data-generation run on 1,253 GPUs [[S-1003]] [[S-3000]]. The latest documented use is the data-generation run, whose results Prime Intellect released in July 2025 [[S-3000]]. No other party is documented relying on TOPLOC for a verification decision.\n- **R4** not met: no independent audit, red-team or peer-reviewed security analysis has been published. DiFR's comparison measures detection accuracy against communication cost [[S-0016]]. It is not a security evaluation.\n\nConfidence is low: the production use is the developer's own, and none is documented after July 2025. The package has had no release since April 2025 [[S-1001]].", "evidence": [ "S-1000", "S-1001", "S-1003", "S-1004", "S-3000" ], "next_level_gaps": [ "An independent public security evaluation, such as an audit, red-team or peer-reviewed analysis, that tests adaptive attacks like the spoofing and speculative-decoding cases the TOPLOC authors list." ], "confidence": "low", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Speculative decoding goes undetected", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "The TOPLOC authors state that it cannot detect speculative decoding. In speculative decoding, a provider decodes with a cheaper model and uses the larger model only for prefill.", "sources": [ "S-1000" ], "response": null }, { "assessment": true, "title": "Last-layer activations could be spoofed", "kind": "open-question", "severity": "significant", "status": "open", "description": "The TOPLOC authors name spoofing of the last hidden layer's activations as a potential attack. A provider could do this by pruning intermediate layers or by using a smaller model.", "sources": [ "S-1000" ], "response": null }, { "assessment": true, "title": "Subtle modifications are harder to detect", "kind": "open-question", "severity": "significant", "status": "open", "description": "The TOPLOC authors state that large changes to the model or prompt are straightforward to detect, but subtle modifications are harder. In preliminary experiments, the margin separating fp8 from bf16 generation was small. The authors did not test whether TOPLOC distinguishes types of KV-cache compression.", "sources": [ "S-1000" ], "response": null }, { "assessment": true, "title": "Tolerance leaves covert bandwidth", "kind": "theoretical-argument", "severity": "significant", "status": "open", "description": "TOPLOC accepts approximate matches. A check of this kind can put an upper bound on the covert bandwidth available to an adversary, but it cannot close that bandwidth. The limit applies to all statistical verification schemes.", "sources": [ "S-0020" ], "response": null } ], "blockers": [ { "text": "No independent security evaluation has been published, and Amodo Design rates red-teaming of recomputation schemes as 'not started'.", "theme": "adversarial-validation", "blocked_by": null, "sources": [ "S-1008" ] }, { "text": "The verifier must run the model itself, which suits the paper's setting of providers serving open-weights models.", "theme": "privacy-leakage", "blocked_by": null, "sources": [ "S-1000" ] } ], "challenge_themes": [ "adversarial-validation", "protocol-soundness", "privacy-leakage" ], "organizations": [ "O-0100" ], "people": [], "sources": [ { "source": "S-1000", "supports": "design, commitment and validation algorithm, thresholds, experiments, limitations", "locator": "abstract; §3.1; §4; §5.1-5.7; §6.1-6.5" }, { "source": "S-1001", "supports": "public implementation, licence, release tag", "locator": "README; releases" }, { "source": "S-1002", "supports": "provider-reported validation speed and SGLang/vLLM integrations", "locator": "whole post" }, { "source": "S-1003", "supports": "provider-reported use in INTELLECT-2; checks; eviction of failing nodes; proof-generation overhead; validation speed", "locator": "§2.3; §2.4.2" }, { "source": "S-1004", "supports": "provider-reported TOPLOC v2 sampling verification, pipeline-parallel extension and pipeline replay in SYNTHETIC-2", "locator": "verification section" }, { "source": "S-3000", "supports": "provider-reported TOPLOC v2 use in SYNTHETIC-2: group-level accept/reject, stage-by-stage replay, sampling proofs, false-positive rate, verification cost, 1,253 GPUs", "locator": "verification section; GPU section" }, { "source": "S-0016", "supports": "independent comparison with Activation-DiFR on detecting FP8 KV-cache quantization", "locator": "§6.2; Table 1" }, { "source": "S-0017", "supports": "independent description of the TOPLOC scheme", "locator": "TOPLOC section" }, { "source": "S-0020", "supports": "limits of statistical verification", "locator": "§1" }, { "source": "S-1008", "supports": "status as an initial recomputation scheme under testing; recomputation red-teaming rated 'not started'", "locator": "Recomputation algorithms and Recomputation red-teaming items" } ], "concepts": [ "K-0008", "K-0009", "K-0024" ], "kind": "open-source-project", "developer": [ "O-0100" ], "realises": [ "M-0001" ], "homepage": "https://www.primeintellect.ai/blog/toploc", "repo": "https://github.com/PrimeIntellect-ai/toploc", "type": "implementation", "url": "https://trustbutveri.fyi/implementations/toploc/", "source_file": "content/implementations/toploc.md", "flags_all": [ "provider-reported" ], "body_markdown": "## What it is\n\nTOPLOC is a method for checking that an inference provider used the model configuration it claims [[S-1000]]. It hashes intermediate activations with a compact locality-sensitive hashing scheme, a kind of hash that gives similar outputs for similar inputs [[S-1000]]. Researchers at Prime Intellect and Together AI published it at ICML 2025 [[S-1000]].\n\nThe paper targets providers that run open-weights models on their own hardware and serve them through APIs [[S-1000]]. It considers four undisclosed changes such a provider might make [[S-1000]]:\n\n- lower precision;\n- KV-cache compression;\n- altered weights, whether distilled, merged or pruned;\n- an altered system prompt.\n\nThe code is MIT-licensed and installable as a Python package [[S-1001]]. Prime Intellect reports integrations with vLLM and a fork of SGLang [[S-1002]].\n\n## How it works\n\nThe provider records the 128 largest-magnitude values of the last hidden layer, and their indices, as it generates [[S-1000]] [[S-0017]]. It encodes them compactly as a polynomial and stores the result as a proof [[S-1000]] [[S-0017]]. The verifier re-runs the full sequence in a single forward pass. It recomputes the same top values and compares them with the proof [[S-1000]] [[S-0017]].\n\nThe verifier accepts if the number of exponent mismatches and the mean and median mantissa differences are all below set thresholds [[S-1000]]. The thresholds absorb reorderings of the computation caused by different GPU types, tensor-parallel layouts and attention kernels [[S-1000]].\n\nPrime Intellect's INTELLECT-2 run combined TOPLOC with other checks. Each inference worker generated a TOPLOC proof for every sequence [[S-1003]]. Validators also checked termination, the logit distribution and deterministic data sampling [[S-1003]]. Prime Intellect reports that a later version, TOPLOC v2, adds reproducible Gumbel noise so that verifiers can also check token sampling [[S-1004]].\n\n## Evidence\n- The TOPLOC paper reports detecting unauthorized changes to models, prompts or precision with 100% accuracy, and no false positives or negatives in its evaluations [[S-1000]]. The tests used Llama 3.1-8B-Instruct, INTELLECT-1-Instruct and Gemma-2-9B on UltraChat prompts. A model-differentiation test also included Llama 3.1-70B-Instruct [[S-1000]]. Proofs take 258 bytes per 32 new tokens [[S-1000]].\n- Validation held across A100 and RTX 4090 GPUs, one- and two-GPU tensor parallelism, and three attention implementations [[S-1000]].\n- Prime Intellect reports using TOPLOC to verify rollouts from untrusted inference workers when training a 32-billion-parameter model [[S-1003]]. Nodes whose files fail validation are \"slashed and evicted\" [[S-1003]]. Prime Intellect reports that TOPLOC v2 checked the inference workers in SYNTHETIC-2, a distributed data-generation run on 1,253 GPUs, with a false-positive rate of 0.000925% (37 slashes) over 4 million samples [[S-3000]].\n- The authors of [[I-0002|DiFR]], a related scheme, report that Activation-DiFR Pareto-dominates TOPLOC (does at least as well on both communication cost and detection accuracy) when detecting FP8 KV-cache quantization in their tests [[S-0016]].\n- Amodo Design's status page lists TOPLOC among initial recomputation schemes now being tested on relevant hardware and models [[S-1008]].\n\n## Limitations\nThe TOPLOC paper lists five limitations [[S-1000]]:\n\n- The margin separating fp8 from bf16 generation is small, and the authors did not test KV-cache compression.\n- The method cannot detect speculative decoding in which a cheaper model does the decoding.\n- Inference consumers could mine for \"unstable\" prompts that tend to fail validation.\n- An attacker could spoof last-layer activations by pruning intermediate layers or using a smaller model.\n- Subtle modifications are harder to detect than large ones.\n\nThe original method checks activations, not token sampling [[S-1000]]. Prime Intellect reports that TOPLOC v2 adds sampling checks, described so far only in its blog posts [[S-1004]] [[S-3000]].\n\nTOPLOC accepts approximate matches, so it shares the general limit of statistical schemes. It can bound an adversary's covert freedom but cannot eliminate it [[S-0020]].\n\nNo independent red-team or audit has been published. All deployment evidence comes from the developer [[S-1003]] [[S-1004]] [[S-3000]].", "body_text": "What it is TOPLOC is a method for checking that an inference provider used the model configuration it claims [S-1000]. It hashes intermediate activations with a compact locality-sensitive hashing scheme, a kind of hash that gives similar outputs for similar inputs [S-1000]. Researchers at Prime Intellect and Together AI published it at ICML 2025 [S-1000]. The paper targets providers that run open-weights models on their own hardware and serve them through APIs [S-1000]. It considers four undisclosed changes such a provider might make [S-1000]: - lower precision; - KV-cache compression; - altered weights, whether distilled, merged or pruned; - an altered system prompt. The code is MIT-licensed and installable as a Python package [S-1001]. Prime Intellect reports integrations with vLLM and a fork of SGLang [S-1002]. How it works The provider records the 128 largest-magnitude values of the last hidden layer, and their indices, as it generates [S-1000] [S-0017]. It encodes them compactly as a polynomial and stores the result as a proof [S-1000] [S-0017]. The verifier re-runs the full sequence in a single forward pass. It recomputes the same top values and compares them with the proof [S-1000] [S-0017]. The verifier accepts if the number of exponent mismatches and the mean and median mantissa differences are all below set thresholds [S-1000]. The thresholds absorb reorderings of the computation caused by different GPU types, tensor-parallel layouts and attention kernels [S-1000]. Prime Intellect's INTELLECT-2 run combined TOPLOC with other checks. Each inference worker generated a TOPLOC proof for every sequence [S-1003]. Validators also checked termination, the logit distribution and deterministic data sampling [S-1003]. Prime Intellect reports that a later version, TOPLOC v2, adds reproducible Gumbel noise so that verifiers can also check token sampling [S-1004]. Evidence - The TOPLOC paper reports detecting unauthorized changes to models, prompts or precision with 100% accuracy, and no false positives or negatives in its evaluations [S-1000]. The tests used Llama 3.1-8B-Instruct, INTELLECT-1-Instruct and Gemma-2-9B on UltraChat prompts. A model-differentiation test also included Llama 3.1-70B-Instruct [S-1000]. Proofs take 258 bytes per 32 new tokens [S-1000]. - Validation held across A100 and RTX 4090 GPUs, one- and two-GPU tensor parallelism, and three attention implementations [S-1000]. - Prime Intellect reports using TOPLOC to verify rollouts from untrusted inference workers when training a 32-billion-parameter model [S-1003]. Nodes whose files fail validation are \"slashed and evicted\" [S-1003]. Prime Intellect reports that TOPLOC v2 checked the inference workers in SYNTHETIC-2, a distributed data-generation run on 1,253 GPUs, with a false-positive rate of 0.000925% (37 slashes) over 4 million samples [S-3000]. - The authors of DiFR, a related scheme, report that Activation-DiFR Pareto-dominates TOPLOC (does at least as well on both communication cost and detection accuracy) when detecting FP8 KV-cache quantization in their tests [S-0016]. - Amodo Design's status page lists TOPLOC among initial recomputation schemes now being tested on relevant hardware and models [S-1008]. Limitations The TOPLOC paper lists five limitations [S-1000]: - The margin separating fp8 from bf16 generation is small, and the authors did not test KV-cache compression. - The method cannot detect speculative decoding in which a cheaper model does the decoding. - Inference consumers could mine for \"unstable\" prompts that tend to fail validation. - An attacker could spoof last-layer activations by pruning intermediate layers or using a smaller model. - Subtle modifications are harder to detect than large ones. The original method checks activations, not token sampling [S-1000]. Prime Intellect reports that TOPLOC v2 adds sampling checks, described so far only in its blog posts [S-1004] [S-3000]. TOPLOC accepts approximate matches, so it shares the general limit of statistical schemes. It can bound an adversary's covert freedom but cannot eliminate it [S-0020]. No independent red-team or audit has been published. All deployment evidence comes from the developer [S-1003] [S-1004] [S-3000].", "referenced_by": [ { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" }, { "id": "O-0100", "title": "Prime Intellect", "url": "https://trustbutveri.fyi/organizations/prime-intellect/" } ] }, { "id": "I-0003", "slug": "zkllm", "title": "zkLLM", "aliases": [ "zkLLM-CCS2024" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "one_liner": "zkLLM is a GPU-accelerated zero-knowledge proof system that proves a large language model's output came from committed weights without revealing those weights.", "summary": "zkLLM is an academic zero-knowledge proof system for large language models, published at ACM CCS 2024. It lets a model's owner prove that its committed model computed an output, without revealing the weights. The verifier supplies the prompt and sees the output. On one NVIDIA A100 GPU, zkLLM proved one 2,048-token forward pass of LLaMa-2-13B in about 13 minutes. The 188 kB proof verifies in about four seconds. The code is public and passed the conference's artifact evaluation. The main obstacles to real use are cost and engineering maturity. Its authors state the code is unaudited, not ready for industrial use and no longer maintained. For verification, the main weakness is that it proves a fixed-point approximation of a model whose architecture must be public. Its zero-knowledge guarantee also assumes a verifier that follows the protocol.", "technical": "The design has four parts [[S-0023]]:\n\n- tlookup is a parallel lookup argument for non-arithmetic tensor operations. The authors state it adds no asymptotic overhead in memory or running time (§4).\n- zkAttn proves softmax attention by splitting the exponentiation into K segments, each checked with tlookup (§5).\n- The commitments use Hyrax, a Pedersen variant, over BLS12-381 under discrete-log hardness (§3).\n- Tensors are scaled by 2^16 and rounded into the field. The resulting total L1 error on the output is about 10^-2 (§7–8).\n\nThe paper's security analysis is in §7.2 [[S-0023]]:\n\n- Theorems 7.2 and 7.3 give tlookup a completeness error of O(N/|F|). They show that a cheating probabilistic polynomial-time prover succeeds only with negligible probability. The rest of the protocol applies the sumcheck protocol and proofs of opening for committed tensors.\n- Theorem 7.4 covers zero knowledge. It states that a simulator with only oracle access to the output produces a view indistinguishable from the real one. The theorem assumes zero-knowledge variants of sumcheck and Pedersen commitments. The threat model assumes a semi-honest verifier (§3.6).\n\nTable 1 reports these costs on an A100 40 GB GPU at sequence length 2,048 [[S-0023]]:\n\n- **OPT-13B** took 1,270 s to commit and 713 s to prove. The proof was 160 kB, verified in 3.71 s and used 22.9 GB of memory.\n- **LLaMa-2-13B** took 986 s to commit and 803 s to prove. The proof was 188 kB, verified in 3.95 s and used 23.1 GB of memory.\n\nThe public code covers LLaMa-2 7B and 13B, runs prover and verifier side by side, and is interactive [[S-1108]].", "category": "cryptographic-computational", "secondary_categories": [], "verifies": [ { "claim": "C-0005", "role": "primary", "note": "Proves an output follows from committed weights and a public architecture." } ], "threat_model": "adversarial", "adversarial_evaluation": "analysis", "hardware_requirement": "none", "prover_cooperation": "required", "confidentiality": "partial", "depends_on": [], "readiness": { "assessment": true, "level": "R2", "scope": "proving an output came from committed weights, against a prover who cheats", "rubric_version": "1.1", "rationale": "Public, artifact-evaluated code proves 13-billion-parameter models in peer-reviewed tests, but its authors say it is unaudited and not ready for production.\n\n- **R1** met: the paper states the claim, the threat model and the security theorems [[S-0023]].\n- **R2** met: the code is public, tagged and archived on Zenodo. It received the CCS 2024 badges \"Artifacts Available\" and \"Artifacts Evaluated--Functional\" [[S-1108]]. The published end-to-end results use 13-billion-parameter models on a data-centre GPU [[S-0023]]. The stated adversary is a cheating polynomial-time prover [[S-0023]].\n- **R3** not met: the README says the code \"is NOT ready for industrial applications\" and is no longer maintained [[S-1108]]. As of September 2026 no third party is known to rely on it.\n- **R4** not met: the README says the code \"has NOT undergone security auditing\" [[S-1108]], and as of September 2026 no independent security evaluation of it has been published.", "evidence": [ "S-0023", "S-1108" ], "next_level_gaps": [ "A production-grade implementation, with prover and verifier separated and non-interactive proofs, or reliance by a third party for a verification decision.", "An independent public security evaluation (audit, red-team or third-party peer-reviewed analysis)." ], "confidence": "medium", "assessed_by": [ "claude-review", "codex-review" ], "assessed_on": "2026-09-25", "status": "current", "dispute": null }, "flaws": [ { "assessment": true, "title": "Reference code is interactive and runs prover and verifier together", "kind": "open-question", "severity": "significant", "status": "open", "description": "The README states that prover and verifier work \"are implemented side-by-side\", and that intermediate values written to files are for the prover's reference only. It says an industrial deployment would need to separate the two and apply Fiat–Shamir to make proofs non-interactive [[S-1108]]. The released code gives a verifier no standalone check.", "sources": [ "S-1108" ], "response": null }, { "assessment": true, "title": "Proves a fixed-point approximation of a publicly known architecture", "kind": "theoretical-argument", "severity": "minor", "status": "open", "description": "The prover's model must have a \"publicly known structure\". Tensors are discretised by scaling and rounding [[S-0023]]. The authors report perplexity changes of 0.008 to 0.09 on C4 [[S-0023]]. The proof covers the quantised computation.", "sources": [ "S-0023" ], "response": null } ], "blockers": [ { "text": "Proving takes about 12 to 13 minutes of A100 time per 2,048-token forward pass at 13B parameters, plus a one-time weight commitment of 16 to 21 minutes.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-0023" ] }, { "text": "The repository was archived on 10 July 2025 and the author states there is no plan for upgrades or maintenance.", "theme": "performance-compatibility", "blocked_by": null, "sources": [ "S-1108" ] }, { "text": "No security audit of the code has been carried out.", "theme": "adversarial-validation", "blocked_by": null, "sources": [ "S-1108" ] } ], "challenge_themes": [ "performance-compatibility", "adversarial-validation", "privacy-leakage" ], "organizations": [ "O-0122" ], "people": [], "sources": [ { "source": "S-0023", "supports": "design, threat model, security theorems, overheads, accuracy, comparison with zkML", "locator": "abstract; §3.6; §4; §5; §7; §8 Table 1 and Fig. 4; §9" }, { "source": "S-1108", "supports": "public code, artifact badges, supported models, README caveats, archive status", "locator": "README; Zenodo record 13621754" }, { "source": "S-0018", "supports": "use of zkLLM's figures in a verification system design", "locator": "§5.2.4" }, { "source": "S-1112", "supports": "independent Hollow-LLM analysis: valid proofs in zkLLM-style deployments do not bind computational effort; demonstrated with zkGPT", "locator": "Abstract; threat model; §V" } ], "concepts": [ "K-0010", "K-0024", "K-0001", "K-0002" ], "kind": "research-prototype", "developer": [ "O-0122" ], "realises": [ "M-0004" ], "repo": "https://github.com/jvhs0706/zkllm-ccs2024", "type": "implementation", "url": "https://trustbutveri.fyi/implementations/zkllm/", "source_file": "content/implementations/zkllm.md", "flags_all": [ "provider-reported" ], "body_markdown": "## What it is\n\nzkLLM is a system for [[M-0004|zero-knowledge proofs of inference]] on large language models. Researchers at the University of Waterloo published it at ACM CCS 2024 [[S-0023]] [[S-1108]]. Its authors call it \"the inaugural specialized zero-knowledge proof tailored for LLMs\" [[S-0023]].\n\nThe setting has two parties [[S-0023]]:\n- **The prover**, such as an AI company, owns a model with a publicly known structure and treats the weights as intellectual property.\n- **The verifier**, such as a regulator, submits a prompt and asks for proof that the output returned is correct for that prompt and the confidential model.\n\nThe CUDA implementation is public under an MIT license [[S-1108]]. A tagged release, archived on Zenodo, received artifact-evaluation badges [[S-1108]].\n\n## How it works\n\nThe weights are committed once, using Hyrax commitments over the BLS12-381 curve [[S-0023]]. For each prompt, the prover shows that the model's tensor operations were computed correctly, using sumcheck-based arguments [[S-0023]].\n\nTwo new components handle the hard parts of a transformer:\n- **tlookup** is a parallel lookup argument for non-arithmetic tensor operations. The authors state it adds no asymptotic overhead in memory or running time [[S-0023]].\n- **zkAttn** is a proof for softmax attention. It splits the exponential into segments, each checked by lookups [[S-0023]].\n\nAll values are scaled by 2^16 and rounded into a finite field [[S-0023]].\n\nThe public code works through a LLaMa-2 model one layer at a time:\n1. Generate public parameters.\n2. Commit the weights.\n3. Run the proofs for normalisation, attention and feed-forward components [[S-1108]].\n\nIt supports LLaMa-2 models of 7 and 13 billion parameters [[S-1108]].\n\n## Evidence\n- **Setup.** The paper reports results on one NVIDIA A100 GPU with 40 GB of memory, 12 CPU cores and 124.5 GB of system memory [[S-0023]]. The models were OPT (125M to 13B) and LLaMa-2 (7B and 13B), run on 2,048-token samples from C4 [[S-0023]].\n- **LLaMa-2-13B.** Proving took 803 seconds and produced a 188 kB proof that verified in 3.95 seconds, using 23.1 GB of memory; the one-time weight commitment took 986 seconds [[S-0023]].\n- **Accuracy.** Perplexity changed little: for LLaMa-2-13B it moved from 6.520 to 6.528 [[S-0023]].\n- **Comparison.** The authors compare zkLLM with an earlier system, zkML, on the same hardware [[S-0023]]. zkML ran out of memory beyond the size of GPT-2 (1.5 billion parameters), so its times for larger models are the authors' estimates [[S-0023]].\n- **Artifact evaluation.** The artifact received the CCS 2024 badges \"Artifacts Available\" and \"Artifacts Evaluated--Functional\" [[S-1108]].\n- **Use by others.** A verification system design for AI agreements uses zkLLM's figure of 803 seconds per 2,048-token forward pass on an A100 to judge ZKP overheads [[S-0018]].\n\n## Limitations\n**Code maturity.** The README states the code \"has NOT undergone security auditing and is NOT ready for industrial applications\" [[S-1108]]. It also names these gaps:\n- prover and verifier run side by side;\n- intermediate files are not meant as verifier inputs;\n- an industrial deployment would need to separate the two parties and apply Fiat–Shamir [[S-1108]].\n\n**Maintenance.** The repository was archived in July 2025 [[S-1108]]. The author states the project is no longer actively maintained [[S-1108]].\n\n**Assumptions.** The paper assumes a publicly known model structure [[S-0023]]. Its zero-knowledge guarantee is stated for a semi-honest verifier, which \"accurately reports the outcome of the proof verification\" but tries to learn the hidden parameters [[S-0023]].\n\n**Scope.** Proofs cover inference only [[S-0023]]. The authors write that extending zero-knowledge proofs to training LLMs \"may pose insurmountable challenges\" [[S-0023]].\n\n**Attacks.** As of September 2026 no attack on the soundness of zkLLM's proofs has been published. Its security rests on the paper's soundness and zero-knowledge theorems [[S-0023]]. An independent analysis, whose setting follows deployments such as zkLLM, shows that valid proofs do not bind the computation spent, so a much smaller model can pass as the declared one [[S-1112]]. Its authors demonstrated this with another system, zkGPT, on a small transformer [[S-1112]]. See [[M-0004]].", "body_text": "What it is zkLLM is a system for zero-knowledge proofs of inference on large language models. Researchers at the University of Waterloo published it at ACM CCS 2024 [S-0023] [S-1108]. Its authors call it \"the inaugural specialized zero-knowledge proof tailored for LLMs\" [S-0023]. The setting has two parties [S-0023]: - The prover, such as an AI company, owns a model with a publicly known structure and treats the weights as intellectual property. - The verifier, such as a regulator, submits a prompt and asks for proof that the output returned is correct for that prompt and the confidential model. The CUDA implementation is public under an MIT license [S-1108]. A tagged release, archived on Zenodo, received artifact-evaluation badges [S-1108]. How it works The weights are committed once, using Hyrax commitments over the BLS12-381 curve [S-0023]. For each prompt, the prover shows that the model's tensor operations were computed correctly, using sumcheck-based arguments [S-0023]. Two new components handle the hard parts of a transformer: - tlookup is a parallel lookup argument for non-arithmetic tensor operations. The authors state it adds no asymptotic overhead in memory or running time [S-0023]. - zkAttn is a proof for softmax attention. It splits the exponential into segments, each checked by lookups [S-0023]. All values are scaled by 2^16 and rounded into a finite field [S-0023]. The public code works through a LLaMa-2 model one layer at a time: 1. Generate public parameters. 2. Commit the weights. 3. Run the proofs for normalisation, attention and feed-forward components [S-1108]. It supports LLaMa-2 models of 7 and 13 billion parameters [S-1108]. Evidence - Setup. The paper reports results on one NVIDIA A100 GPU with 40 GB of memory, 12 CPU cores and 124.5 GB of system memory [S-0023]. The models were OPT (125M to 13B) and LLaMa-2 (7B and 13B), run on 2,048-token samples from C4 [S-0023]. - LLaMa-2-13B. Proving took 803 seconds and produced a 188 kB proof that verified in 3.95 seconds, using 23.1 GB of memory; the one-time weight commitment took 986 seconds [S-0023]. - Accuracy. Perplexity changed little: for LLaMa-2-13B it moved from 6.520 to 6.528 [S-0023]. - Comparison. The authors compare zkLLM with an earlier system, zkML, on the same hardware [S-0023]. zkML ran out of memory beyond the size of GPT-2 (1.5 billion parameters), so its times for larger models are the authors' estimates [S-0023]. - Artifact evaluation. The artifact received the CCS 2024 badges \"Artifacts Available\" and \"Artifacts Evaluated--Functional\" [S-1108]. - Use by others. A verification system design for AI agreements uses zkLLM's figure of 803 seconds per 2,048-token forward pass on an A100 to judge ZKP overheads [S-0018]. Limitations Code maturity. The README states the code \"has NOT undergone security auditing and is NOT ready for industrial applications\" [S-1108]. It also names these gaps: - prover and verifier run side by side; - intermediate files are not meant as verifier inputs; - an industrial deployment would need to separate the two parties and apply Fiat–Shamir [S-1108]. Maintenance. The repository was archived in July 2025 [S-1108]. The author states the project is no longer actively maintained [S-1108]. Assumptions. The paper assumes a publicly known model structure [S-0023]. Its zero-knowledge guarantee is stated for a semi-honest verifier, which \"accurately reports the outcome of the proof verification\" but tries to learn the hidden parameters [S-0023]. Scope. Proofs cover inference only [S-0023]. The authors write that extending zero-knowledge proofs to training LLMs \"may pose insurmountable challenges\" [S-0023]. Attacks. As of September 2026 no attack on the soundness of zkLLM's proofs has been published. Its security rests on the paper's soundness and zero-knowledge theorems [S-0023]. An independent analysis, whose setting follows deployments such as zkLLM, shows that valid proofs do not bind the computation spent, so a much smaller model can pass as the declared one [S-1112]. Its authors demonstrated this with another system, zkGPT, on a small transformer [S-1112]. See Zero-knowledge proofs of inference.", "referenced_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "O-0122", "title": "University of Waterloo", "url": "https://trustbutveri.fyi/organizations/university-of-waterloo/" } ] }, { "id": "C-0008", "slug": "bandwidth-is-bounded", "title": "Communication between compute groups is bounded", "aliases": [ "bandwidth limits", "interconnect limits" ], "status": "published", "last_reviewed": "2026-09-23", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Data flowing between specified groups of chips, or out of a facility, stays below a declared rate, so the groups cannot jointly run large workloads.", "summary": "The claim is that data flowing between specified groups of chips, or out of a facility, stays below a declared rate. A proposed pod design aims to allow inference tokens while restricting training gradients if the served model and routing fit within each pod. A verified bound on communication between compute groups could therefore restrict training while allowing inference. A bound on a facility's external links also limits how much weights or data could leave it. The claim is negative: no channel exceeds the bound, including channels outside the monitored network. Proposed designs cap the external bandwidth of small pods of chips or restrict chip networking in hardware, and use network taps to observe the links. The hard parts are finding every path, including physical side channels; choosing bounds that stay meaningful as distributed training methods improve; and monitoring high-speed accelerator fabrics that are difficult to tap.", "claim_class": "negative", "editors_synthesis": { "assessment": true, "markdown": "A software bandwidth monitor has been demonstrated (R2). It runs on the operator's nodes, and its authors say the measurements are trivially spoofable [[S-3220]]. No bandwidth cap that a verifier can check has been demonstrated publicly.\n\n[[M-0014|Bandwidth limits and compartmentalization]] (R2 for software monitoring) proposes to cap or remove links between accelerator groups. Its pod-cap design aims to allow inference tokens while restricting training gradients, provided the served model and routing fit within each pod [[S-0005]] [[S-3565]]. It needs no access to a facility's code [[S-0005]]. Hardware versions include RAND's fixed-set design [[S-0057]] and [[M-0009|guarantee processors]] (R1), and the [[I-0011|AI 2040 stack]] (R1) removes back-end networking [[S-0067]]. [[M-0022|Side-channel suppression]] (R1) addresses paths outside the network [[S-0038]]. [[M-0017|Tamper evidence]] (R2) would protect the enforcing devices, and [[M-0024|bounding unexplained information]] (R2) limits what the permitted bandwidth can carry.\n\nLucid Computing's pod-cap design is \"not yet implemented or red-teamed\" [[S-1301]], and Amodo's rate limiting on 400G links was for weight security with a cooperating operator [[S-1313]].\n\nCopper scale-up links and encrypted interconnects are hard to observe [[S-0031]], and the designs aim to cut side channels to low rates, not to eliminate them [[S-0018]] [[S-0038]]. Sastry and colleagues note that decentralised training, if it becomes more viable, might undermine the detectability of training runs [[S-0053]], and training methods that need less communication could likewise erode a bound that separates training from inference today.", "text": "A software bandwidth monitor has been demonstrated (R2). It runs on the operator's nodes, and its authors say the measurements are trivially spoofable [S-3220]. No bandwidth cap that a verifier can check has been demonstrated publicly. Bandwidth limits and compartmentalization (R2 for software monitoring) proposes to cap or remove links between accelerator groups. Its pod-cap design aims to allow inference tokens while restricting training gradients, provided the served model and routing fit within each pod [S-0005] [S-3565]. It needs no access to a facility's code [S-0005]. Hardware versions include RAND's fixed-set design [S-0057] and guarantee processors (R1), and the AI 2040 stack (R1) removes back-end networking [S-0067]. Side-channel suppression (R1) addresses paths outside the network [S-0038]. Tamper evidence (R2) would protect the enforcing devices, and bounding unexplained information (R2) limits what the permitted bandwidth can carry. Lucid Computing's pod-cap design is \"not yet implemented or red-teamed\" [S-1301], and Amodo's rate limiting on 400G links was for weight security with a cooperating operator [S-1313]. Copper scale-up links and encrypted interconnects are hard to observe [S-0031], and the designs aim to cut side channels to low rates, not to eliminate them [S-0018] [S-0038]. Sastry and colleagues note that decentralised training, if it becomes more viable, might undermine the detectability of training runs [S-0053], and training methods that need less communication could likewise erode a bound that separates training from inference today." }, "sources": [ { "source": "S-0005", "supports": "pods with high internal and very low external bandwidth; enough for inference tokens not training gradients; implementable with modest changes, physical access and monitoring, without code access", "locator": "Interconnect bandwidth limits" }, { "source": "S-0053", "supports": "compute caps via physical limits on chip-to-chip networking; decentralised training could undermine detectability", "locator": "enforcement; §3.B.1 Detectability" }, { "source": "S-1314", "supports": "DiLoCo on 8 workers matched fully synchronous training while communicating 500 times less", "locator": "abstract" }, { "source": "S-0057", "supports": "fixed-set HEM restricting networking of small GPU sets", "locator": "p. viii" }, { "source": "S-0018", "supports": "front-end vs back-end bandwidth; inference bandwidth assumption; covert side-channel target in kilobits per second; egress explainable by ingress", "locator": "inference vs training; open problems" }, { "source": "S-0031", "supports": "front-end vs back-end tapping costs and feasibility; copper links and scale-up domains; encrypted interconnect", "locator": "frontend vs backend; open problems" }, { "source": "S-0038", "supports": "physical side channels can bypass network monitoring; defences; tolerable low rate", "locator": "channels of concern; defences" }, { "source": "S-0029", "supports": "retroactively confirming data-centre topology is beyond scope", "locator": "open problems" }, { "source": "S-0002", "supports": "network taps intercepting inter-chip data", "locator": "§4.2" }, { "source": "S-0067", "supports": "removing back-end networking to create isolated inference units", "locator": "verification mechanisms" }, { "source": "S-1301", "supports": "pod-level traffic cap design; not yet implemented or red-teamed", "locator": "summary; status" }, { "source": "S-1313", "supports": "DPU-enforced rate limiting on 400G links for weight security, with the limits set by a trusted operator's controller", "locator": "whole note" }, { "source": "S-3220", "supports": "software bandwidth-monitor prototype on AI GPUs; authors say operator-controlled measurements are trivially spoofable", "locator": "Abstract; Pros and Cons analysis" }, { "source": "S-1508", "supports": "egress limits cap what can be stolen", "locator": "§5.1" }, { "source": "S-3565", "supports": "expert-parallel MoE inference can require cross-device all-to-all communication", "locator": "abstract" } ], "concepts": [ "K-0021", "K-0017", "K-0014", "K-0013" ], "order": 8, "type": "claim", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/", "source_file": "content/claims/bandwidth-is-bounded.md", "flags_all": [], "body_markdown": "## Why it matters\nDistributed training depends on fast communication between chips, so limiting communication limits training [[S-0005]].\n\n- Scher and Thiergart describe pods of chips with high-bandwidth interconnect inside each pod but very low external bandwidth: enough for inference tokens, not for training gradients [[S-0005]]. They suggest this could be implemented with modest changes to existing technology, physical access and monitoring, without access to code [[S-0005]].\n- That token-only description applies when the served model and its expert routing fit within a pod. Expert-parallel mixture-of-experts inference can require substantial all-to-all traffic between devices [[S-3565]]. A pod boundary must therefore be sized and tested for the declared serving workload.\n- Compute-governance analyses list physical limits on chip-to-chip networking as a way to enforce compute caps [[S-0053]]. RAND's \"fixed set\" design restricts networking so that small, fixed sets of GPUs cannot be aggregated into large clusters [[S-0057]].\n- A low-trust system overview asks how far a facility can be required to make all egress traffic explainable by its ingress [[S-0018]]. It assumes that the information bandwidth of inference is not more than two orders of magnitude above expected API traffic at full utilisation [[S-0018]]. Rinberg and colleagues note that egress limits cap theft directly: if only 10 GB leaves a data centre, an adversary cannot steal more than 10 GB [[S-1508]]. This links the claim to [[C-0009]].\n\n## Why it is hard\nThe claim requires knowing every path.\n\n- **Taps see only the links they are on.** RAND's network-tap layer observes data exchanged between chips [[S-0002]]. The cost and feasibility of tapping differ by link: Cankaya describes front-end links to the outside world as the most viable to monitor, while the back-end compute fabric has far higher bandwidth, is latency-critical, and may require sampling rather than full capture [[S-0031]].\n- **Some links are hard to observe at all.** As scale-up domains grow, more traffic runs over copper links that passive optical taps cannot see, and encrypted interconnects complicate inspection [[S-0031]]. Shavit notes that methods for verifiers to confirm, after the fact, the topology in which chips were configured in a data centre were beyond the scope of his work [[S-0029]].\n- **Physical side channels.** Data could bypass network monitoring altogether. A MIRI Technical Governance Team post surveys electromagnetic, acoustic, conducted and fibre-optic channels, and proposes shielding, active jamming with sensing, inspection and filtering [[S-0038]]. The system overview's target is to suppress covert side-channel bandwidth to kilobits per second, not to zero [[S-0018]].\n- **Choosing the bound.** A threshold that separates inference from training today may not do so later. On the widely used C4 dataset, DiLoCo on 8 workers matched fully synchronous training while communicating 500 times less [[S-1314]]. Sastry and colleagues note that decentralised training across many data centres, or with lower-quality compute, could undermine compute detectability if it becomes more viable [[S-0053]].", "body_text": "Why it matters Distributed training depends on fast communication between chips, so limiting communication limits training [S-0005]. - Scher and Thiergart describe pods of chips with high-bandwidth interconnect inside each pod but very low external bandwidth: enough for inference tokens, not for training gradients [S-0005]. They suggest this could be implemented with modest changes to existing technology, physical access and monitoring, without access to code [S-0005]. - That token-only description applies when the served model and its expert routing fit within a pod. Expert-parallel mixture-of-experts inference can require substantial all-to-all traffic between devices [S-3565]. A pod boundary must therefore be sized and tested for the declared serving workload. - Compute-governance analyses list physical limits on chip-to-chip networking as a way to enforce compute caps [S-0053]. RAND's \"fixed set\" design restricts networking so that small, fixed sets of GPUs cannot be aggregated into large clusters [S-0057]. - A low-trust system overview asks how far a facility can be required to make all egress traffic explainable by its ingress [S-0018]. It assumes that the information bandwidth of inference is not more than two orders of magnitude above expected API traffic at full utilisation [S-0018]. Rinberg and colleagues note that egress limits cap theft directly: if only 10 GB leaves a data centre, an adversary cannot steal more than 10 GB [S-1508]. This links the claim to Model weights have not left the facility. Why it is hard The claim requires knowing every path. - Taps see only the links they are on. RAND's network-tap layer observes data exchanged between chips [S-0002]. The cost and feasibility of tapping differ by link: Cankaya describes front-end links to the outside world as the most viable to monitor, while the back-end compute fabric has far higher bandwidth, is latency-critical, and may require sampling rather than full capture [S-0031]. - Some links are hard to observe at all. As scale-up domains grow, more traffic runs over copper links that passive optical taps cannot see, and encrypted interconnects complicate inspection [S-0031]. Shavit notes that methods for verifiers to confirm, after the fact, the topology in which chips were configured in a data centre were beyond the scope of his work [S-0029]. - Physical side channels. Data could bypass network monitoring altogether. A MIRI Technical Governance Team post surveys electromagnetic, acoustic, conducted and fibre-optic channels, and proposes shielding, active jamming with sensing, inspection and filtering [S-0038]. The system overview's target is to suppress covert side-channel bandwidth to kilobits per second, not to zero [S-0018]. - Choosing the bound. A threshold that separates inference from training today may not do so later. On the widely used C4 dataset, DiLoCo on 8 workers matched fully synchronous training while communicating 500 times less [S-1314]. Sastry and colleagues note that decentralised training across many data centres, or with lower-quality compute, could undermine compute detectability if it becomes more viable [S-0053].", "addressed_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/", "role": "primary", "note": "Caps or removes communication between declared groups of accelerators (S-0067, S-1301)." }, { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/", "role": "supporting", "note": "Bounds effective unexplained output bandwidth rather than raw link bandwidth." }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/", "role": "supporting", "note": "Interlocks on NVLink or NICs, and RAND's fixed-set pods, would bound communication (S-1204, S-0057)." }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/", "role": "primary", "note": "Bounds the capacity of physical covert channels out of an enclosure, so that monitored links carry all significant traffic." }, { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/", "role": "supporting", "note": "Protects network devices that enforce or monitor bandwidth boundaries (S-0018)." }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/", "role": "supporting", "note": "Removing back-end networking limits communication between inference units." } ], "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "O-0211", "title": "Centre for the Governance of AI", "url": "https://trustbutveri.fyi/organizations/centre-for-the-governance-of-ai/" } ] }, { "id": "C-0002", "slug": "chips-are-where-declared", "title": "Chips are where they are declared to be", "aliases": [ "chip location", "location verification" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Specific AI chips are physically located at the sites a party has declared, throughout the declared period.", "summary": "The claim is that specific AI chips are at the sites a party has declared, throughout the declared period. Such claims underpin export controls and chip-tracking proposals. Checking them would make a registry of declared sites enforceable and diversion to undeclared facilities detectable. It concerns specific devices and can be tested positively, which makes it more tractable than proving that no chips exist elsewhere. The main technical approach has a chip answer timed challenges from trusted servers, so that the delay bounds its distance from them. A rudimentary prototype on NVIDIA H100 chips has been reported with one published result and no systematic measurements. NVIDIA is reported to be developing a similar scheme that uses its own servers. Known weaknesses are extraction of the chip's private key, which would let another device answer on its behalf, modification of the chip hardware, and limited reach into chips already in circulation.", "claim_class": "positive", "editors_synthesis": { "assessment": true, "markdown": "Chip location concerns known devices and can be checked positively, so it is one of the more tractable claims, but every publicly described scheme is still proposed (R1).\n\n[[M-0018|Chip location verification]] (R1) times a chip's signed replies to trusted servers, so that signal delay bounds its distance from them [[S-0001]] [[S-0005]]. [[I-0009|Lucid's sovereignty certificates]] (R1) are a draft specification of this approach [[S-1404]], and [[M-0019|chip registries]] (R1) supply the declared locations to test. [[M-0009|Guarantee processors]] (R1) could automate checks of approximate chip location, and their designers want them to be retrofittable to existing chip and server designs [[S-0035]].\n\nAn IAPS issue brief reports a single result from a rudimentary prototype on NVIDIA H100 chips, a landmark in Singapore bounding a chip in Singapore to within 300 miles [[S-1401]], and no systematic measurements, error rates or code have been published. NVIDIA has said that it is developing delay-based location verification using its own servers [[S-1402]], but it has published no design or results, and its announced fleet-management software is opt-in [[S-1413]].\n\nThe chip's private key must not be extractable, or another device can answer for it [[S-0005]]. Wasil and colleagues see location tracking as limited to newly produced chips [[S-0062]], while Brass and Aarne expect that the H100's trusted execution environment could implement it [[S-1400]].", "text": "Chip location concerns known devices and can be checked positively, so it is one of the more tractable claims, but every publicly described scheme is still proposed (R1). Chip location verification (R1) times a chip's signed replies to trusted servers, so that signal delay bounds its distance from them [S-0001] [S-0005]. Lucid's sovereignty certificates (R1) are a draft specification of this approach [S-1404], and chip registries (R1) supply the declared locations to test. Guarantee processors (R1) could automate checks of approximate chip location, and their designers want them to be retrofittable to existing chip and server designs [S-0035]. An IAPS issue brief reports a single result from a rudimentary prototype on NVIDIA H100 chips, a landmark in Singapore bounding a chip in Singapore to within 300 miles [S-1401], and no systematic measurements, error rates or code have been published. NVIDIA has said that it is developing delay-based location verification using its own servers [S-1402], but it has published no design or results, and its announced fleet-management software is opt-in [S-1413]. The chip's private key must not be extractable, or another device can answer for it [S-0005]. Wasil and colleagues see location tracking as limited to newly produced chips [S-0062], while Brass and Aarne expect that the H100's trusted execution environment could implement it [S-1400]." }, "sources": [ { "source": "S-0001", "supports": "export-controlled chips straightforward to smuggle; location and owner unknowable after export; data-processing laws; verifiable latencies to trusted servers; co-location", "locator": "§5.2.1" }, { "source": "S-0005", "supports": "location as a verification goal; time-based ping location attestation; private-key extraction enables spoofing; locate chips early and keep them monitored", "locator": "Verifying the location of AI compute; on-chip mechanisms" }, { "source": "S-0062", "supports": "chip location tracking via unique identifiers; evasion by hardware modification or location spoofing; limited to new chips; needs manufacturing agreements", "locator": "Hardware-dependent methods; Table 1; Figure 4" }, { "source": "S-0063", "supports": "declaration of chip locations; monitored facilities; inspectors with ongoing physical access; chip use verification", "locator": "§4; Articles V and VII (as summarised)" }, { "source": "S-0029", "supports": "chip owner directory with serial numbers; physical inspection of sampled chips", "locator": "§3; §5" }, { "source": "S-1400", "supports": "H100 trusted execution environment could likely implement location attestation", "locator": "Proposed Solution Requirements" }, { "source": "S-1401", "supports": "rudimentary location-verification prototype on NVIDIA H100 chips (builder not named); single Singapore result within 300 miles; summarises Brass and Aarne's 2024 report", "locator": "issue brief, pp. 1-2" }, { "source": "S-1402", "supports": "NVIDIA confirmed developing delay-based location verification with NVIDIA-run servers (citing Reuters, December 2025)", "locator": "§1.6" }, { "source": "S-1413", "supports": "NVIDIA's opt-in, customer-installed fleet-management service with read-only telemetry; NVIDIA's statement that its GPUs lack hardware tracking, kill switches and backdoors (provider self-description)", "locator": "blog post" }, { "source": "S-1404", "supports": "draft specification for location (sovereignty) certificates", "locator": "specification v0.1.0" }, { "source": "S-0035", "supports": "flexHEG could enable automated verification of approximate chip location; designs should be retrofittable on existing chip and server designs", "locator": "How FlexHEGs Could Address Risks (Malicious Use); Recommended Areas of Technical Research" }, { "source": "S-3382", "supports": "authors' model-based range for AI chip smuggling into China in 2024; recommendation for software-based location verification", "locator": "four lines of argument; recommendations 2 and 3" }, { "source": "S-3383", "supports": "December 2025 prosecution alleging GPU relabelling and misclassified exports", "locator": "criminal complaint summary" } ], "concepts": [ "K-0004", "K-0019", "K-0005", "K-0015" ], "order": 2, "type": "claim", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/", "source_file": "content/claims/chips-are-where-declared.md", "flags_all": [], "body_markdown": "## Why it matters\n- High-end data-centre AI chips are subject to US export controls, but the Open Problems survey describes them as at present straightforward to smuggle [[S-0001]]. It names as a key technical problem that a chip's location or owner cannot currently be known after export [[S-0001]]. Verified location could also help cloud users check that their data is processed in line with local data-processing laws [[S-0001]].\n- Grunewald and Fist estimate that 10,000 to several hundred thousand AI chips were smuggled into China in 2024. About 140,000 is the median of their modelled range, not a confirmed count [[S-3382]]. In a December 2025 US prosecution, the Justice Department alleged that workers replaced NVIDIA labels on GPUs with a fictitious brand before export or attempted export [[S-3383]].\n- Wasil and colleagues list chip location tracking, using unique identifiers and tracking mechanisms built into chips, among the hardware-dependent methods for verifying agreements [[S-0062]]. A draft international agreement requires parties to declare chip locations and to keep large concentrations of chips in monitored facilities where inspectors have ongoing physical access [[S-0063]].\n- Scher and Thiergart treat locating AI compute as one of their main verification goals [[S-0005]]. They favour tracking chips over trying to detect secret data centres, and propose locating chips at an initial point in time and then keeping them monitored [[S-0005]]. The Open Problems survey also calls for methods to verify that a large number of chips are co-located in a single data centre [[S-0001]].\n\n## Why it is hard\nThe main technical proposal is delay-based. A chip exchanges timed messages with a network of trusted servers, and the measured latencies constrain where it can be [[S-0001]]. Scher and Thiergart describe AI chips using time-based pings to servers around the world to locate themselves [[S-0005]]. An IAPS issue brief from May 2025, which summarises a 2024 report by Brass and Aarne, states that a rudimentary version has been prototyped on NVIDIA H100 chips [[S-1401]]. It shows one result: a landmark in Singapore verifying that a chip in Singapore is within 300 miles of Singapore [[S-1401]].\n\nAvellar and Grunewald report, citing Reuters reporting from December 2025, that NVIDIA has confirmed it is developing location verification that estimates a chip's location from communication delays with NVIDIA-run servers [[S-1402]]. NVIDIA's own announcement from that month describes an opt-in fleet-management service that customers install and that reports read-only telemetry [[S-1413]]. The announcement states that NVIDIA GPUs do not have hardware tracking technology, kill switches or backdoors [[S-1413]].\n\n- Scher and Thiergart identify the main security issue as ensuring that a chip's private key cannot be extracted [[S-0005]]. Extraction would let other chips pretend to be the chip in question, so its location could be spoofed [[S-0005]]. Wasil and colleagues list modifying AI chip hardware and spoofing location as evasion techniques, and note that sophisticated actors may try to disable tracking [[S-0062]].\n- Wasil and colleagues note that chip location tracking is limited to newly produced chips and requires agreements on manufacturing standards [[S-0062]]. Brass and Aarne, by contrast, expect that the H100's trusted execution environment could be used to implement location attestation [[S-1400]]. For chips without such features, location must be established by other means, such as physical inspection against a directory of chip serial numbers and owners [[S-0029]].\n- A verified location says where a chip is, not what it is computing or who controls it. Proposals therefore pair location with verification of chip use [[S-0063]], which on this site falls under [[C-0003]] and [[C-0004]].", "body_text": "Why it matters - High-end data-centre AI chips are subject to US export controls, but the Open Problems survey describes them as at present straightforward to smuggle [S-0001]. It names as a key technical problem that a chip's location or owner cannot currently be known after export [S-0001]. Verified location could also help cloud users check that their data is processed in line with local data-processing laws [S-0001]. - Grunewald and Fist estimate that 10,000 to several hundred thousand AI chips were smuggled into China in 2024. About 140,000 is the median of their modelled range, not a confirmed count [S-3382]. In a December 2025 US prosecution, the Justice Department alleged that workers replaced NVIDIA labels on GPUs with a fictitious brand before export or attempted export [S-3383]. - Wasil and colleagues list chip location tracking, using unique identifiers and tracking mechanisms built into chips, among the hardware-dependent methods for verifying agreements [S-0062]. A draft international agreement requires parties to declare chip locations and to keep large concentrations of chips in monitored facilities where inspectors have ongoing physical access [S-0063]. - Scher and Thiergart treat locating AI compute as one of their main verification goals [S-0005]. They favour tracking chips over trying to detect secret data centres, and propose locating chips at an initial point in time and then keeping them monitored [S-0005]. The Open Problems survey also calls for methods to verify that a large number of chips are co-located in a single data centre [S-0001]. Why it is hard The main technical proposal is delay-based. A chip exchanges timed messages with a network of trusted servers, and the measured latencies constrain where it can be [S-0001]. Scher and Thiergart describe AI chips using time-based pings to servers around the world to locate themselves [S-0005]. An IAPS issue brief from May 2025, which summarises a 2024 report by Brass and Aarne, states that a rudimentary version has been prototyped on NVIDIA H100 chips [S-1401]. It shows one result: a landmark in Singapore verifying that a chip in Singapore is within 300 miles of Singapore [S-1401]. Avellar and Grunewald report, citing Reuters reporting from December 2025, that NVIDIA has confirmed it is developing location verification that estimates a chip's location from communication delays with NVIDIA-run servers [S-1402]. NVIDIA's own announcement from that month describes an opt-in fleet-management service that customers install and that reports read-only telemetry [S-1413]. The announcement states that NVIDIA GPUs do not have hardware tracking technology, kill switches or backdoors [S-1413]. - Scher and Thiergart identify the main security issue as ensuring that a chip's private key cannot be extracted [S-0005]. Extraction would let other chips pretend to be the chip in question, so its location could be spoofed [S-0005]. Wasil and colleagues list modifying AI chip hardware and spoofing location as evasion techniques, and note that sophisticated actors may try to disable tracking [S-0062]. - Wasil and colleagues note that chip location tracking is limited to newly produced chips and requires agreements on manufacturing standards [S-0062]. Brass and Aarne, by contrast, expect that the H100's trusted execution environment could be used to implement location attestation [S-1400]. For chips without such features, location must be established by other means, such as physical inspection against a directory of chip serial numbers and owners [S-0029]. - A verified location says where a chip is, not what it is computing or who controls it. Proposals therefore pair location with verification of chip use [S-0063], which on this site falls under Declared hardware is idle or shut down and This compute runs inference, not training.", "addressed_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/", "role": "primary", "note": "Bounds how far a responding chip can be from trusted landmark servers at the time of the check." }, { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/", "role": "supporting", "note": "Records declared locations, which inspections or location checks can test." }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/", "role": "supporting", "note": "Automated verification of approximate chip location (S-0035); see M-0018." }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/", "role": "supporting", "note": "Speed-of-light bounds on signed challenge round trips underlie delay-based location checks; see M-0018." }, { "id": "I-0009", "title": "Lucid sovereignty (location) certificates", "url": "https://trustbutveri.fyi/implementations/lucid-location-certificates/", "role": "primary", "note": "Certifies a bounded region in which an attested workload's platform was running at a given time." } ], "referenced_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "I-0009", "title": "Lucid sovereignty (location) certificates", "url": "https://trustbutveri.fyi/implementations/lucid-location-certificates/" }, { "id": "C-0001", "title": "Compute stock is at most a declared amount", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" } ] }, { "id": "C-0001", "slug": "compute-stock-is-bounded", "title": "Compute stock is at most a declared amount", "aliases": [ "bounded compute stock", "chip inventory" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "A party holds no more AI-relevant compute, counted in chips or equivalent capacity, than the total it has declared.", "summary": "The claim is that a party holds no more AI-relevant compute, counted in chips or equivalent capacity, than the total it has declared. Many proposed AI agreements start from this kind of declaration, which other parties then check. A bounded stock caps how much training or inference a party could run in secret, and anchors other checks, such as monitoring how chips are used. The claim is negative, which makes it hard to verify. Millions of AI-relevant chips already exist without central tracking, and a party could acquire or build chips outside any registry. Proposals combine monitoring of the chip supply chain from fabrication onward, registries of chips and their owners, inspections, and intelligence methods such as customs and financial data. Supply-chain tracking can reach newly produced chips; the existing stock is the main gap.", "claim_class": "negative", "editors_synthesis": { "assessment": true, "markdown": "No mechanism can yet bound a party's chip stock, and every approach is proposed (R1). Chip tracking would reach new production far better than chips already in circulation, partly because the chip supply chain is concentrated [[S-0053]] [[S-0029]].\n\n[[M-0019|Chip registries and manufacturing records]] (R1) would follow each chip from the fab to its owner, so that inspectors can check a sample against the declared records [[S-0029]]. [[M-0020|Remote detection of data centres]] (R1) estimates the power capacity of large facilities from equipment visible outside, and public estimates exist for known large facilities [[S-1411]]. [[M-0011|Performance throttling and licensing]] (R1) would cap the work that declared chips can do [[S-0057]]. Beyond these estimates, only designs and policy analyses are public, and no chip registry has been built for verification.\n\nMillions of AI-relevant chips already exist with no central tracking [[S-0005]], and domestic chip manufacture and older chips are listed as evasion routes [[S-0062]]. Draft agreements therefore pair technical measures with intelligence, inspections and whistleblowers [[S-0063]].", "text": "No mechanism can yet bound a party's chip stock, and every approach is proposed (R1). Chip tracking would reach new production far better than chips already in circulation, partly because the chip supply chain is concentrated [S-0053] [S-0029]. Chip registries and manufacturing records (R1) would follow each chip from the fab to its owner, so that inspectors can check a sample against the declared records [S-0029]. Remote detection of data centres (R1) estimates the power capacity of large facilities from equipment visible outside, and public estimates exist for known large facilities [S-1411]. Performance throttling and licensing (R1) would cap the work that declared chips can do [S-0057]. Beyond these estimates, only designs and policy analyses are public, and no chip registry has been built for verification. Millions of AI-relevant chips already exist with no central tracking [S-0005], and domestic chip manufacture and older chips are listed as evasion routes [S-0062]. Draft agreements therefore pair technical measures with intelligence, inspections and whistleblowers [S-0063]." }, "sources": [ { "source": "S-0029", "supports": "supply-chain monitoring to prevent amassing untracked chips; chip owner directory, sampled inspection and chain of custody; many existing chips lack features and may not be locatable", "locator": "abstract; §3; §5" }, { "source": "S-0053", "supports": "compute is detectable, excludable and quantifiable with a concentrated supply chain; international chip registry listed; algorithmic progress and decentralised training", "locator": "abstract; §§ on properties of compute and visibility mechanisms; limitations" }, { "source": "S-0063", "supports": "chip consolidation into monitored facilities; >16 H100-equivalents only in monitored facilities; methods for locating chips; production monitoring", "locator": "§4; Articles V–VI (as summarised)" }, { "source": "S-0005", "supports": "millions of AI-relevant chips without central tracking; locate chips early then keep them monitored", "locator": "Verifying the location of AI compute" }, { "source": "S-0062", "supports": "customs data, financial intelligence and fab inspections; limits and evasions (domestic manufacture, older chips); chip location tracking limited to new chips", "locator": "Verification methods; Table 1; Figures 2–4" }, { "source": "S-0002", "supports": "large-scale defined as computing power of thousands of high-end AI chips under a single entity", "locator": "§2.2" }, { "source": "S-1411", "supports": "power capacity of known large data centres inferred from visible cooling equipment", "locator": "methodology" }, { "source": "S-0057", "supports": "offline licensing: a renewable licence grants a compute budget, after which the chip refuses or slows the relevant operations", "locator": "p. viii" } ], "concepts": [ "K-0016", "K-0023", "K-0003", "K-0020" ], "order": 1, "type": "claim", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/", "source_file": "content/claims/compute-stock-is-bounded.md", "flags_all": [], "body_markdown": "## Why it matters\nAn accurate chip count underpins other checks on compute. Proposals rely on it in several ways:\n\n- **Shavit's monitoring framework** has three stages: on-chip logging, proofs about training runs, and monitoring of the chip supply chain [[S-0029]]. The third stage exists so that no actor can avoid discovery by amassing a large quantity of untracked chips [[S-0029]]. Without it, a prover could covertly acquire chips and train on them without ever notifying the verifier, bypassing checks on the chips it did declare [[S-0029]].\n- **Sastry and colleagues** describe AI-relevant compute as detectable, excludable and quantifiable, and produced through an extremely concentrated supply chain [[S-0053]]. They list an international AI chip registry among possible mechanisms for regulatory visibility [[S-0053]].\n- **A draft international agreement** would prohibit concentrations of more than 16 H100-equivalents outside monitored facilities [[S-0063]]. It would consolidate existing chips into those facilities and track new production [[S-0063]]. Chips would be located through supply-chain tracking, mandatory reporting, intelligence gathering, open-source intelligence, power monitoring, challenge inspections and whistleblowers [[S-0063]].\n- **Scher and Thiergart** argue for locating AI chips at an initial point in time and then keeping them monitored, rather than relying on detecting secret data centres later [[S-0005]].\n\n## Why it is hard\nThe claim is negative: it asserts that no chips exist beyond the declared total.\n\n- **The existing stock.** Shavit noted in 2023 that hundreds of thousands of ML chips had already been sold, many lacking the security features his framework needs and possibly not locatable by governments [[S-0029]]. Scher and Thiergart write that millions of AI-relevant chips already exist with no central tracking, which could make an initial inventory difficult [[S-0005]].\n- **Tracking must start at the fab.** Shavit's design monitors the small number of fabrication facilities that make leading-edge chips [[S-0029]]. It records each chip's burned-in serial number in a directory of chip owners, kept up to date when chips are resold or damaged [[S-0029]]. Responsibility for any missing chip precursors lies with the most recent holder [[S-0029]].\n- **Supporting methods have gaps.** Wasil and colleagues note that customs data is less useful against countries that can manufacture components domestically, and that financial intelligence is limited because many hardware purchases have legitimate uses [[S-0062]]. Inspections of chip fabrication plants are resource-intensive and put intellectual property at risk [[S-0062]]. The same authors list local chip manufacture and the use of older chips as evasion routes, and note that chip location tracking would apply only to new chips [[S-0062]].\n- **Chips are not a fixed unit of capacity.** RAND's framework counts a cluster as large-scale if it has the computing power of thousands of high-end AI chips controlled by a single entity [[S-0002]]. Sastry and colleagues caution that algorithmic progress can reduce the compute needed for a given capability, and that decentralised training could undermine the detectability of compute [[S-0053]].\n\nThe claim is closely tied to [[C-0010]], which asks whether any compute lies outside the declared stock, and to [[C-0002]], which asks whether declared chips are where they are said to be.", "body_text": "Why it matters An accurate chip count underpins other checks on compute. Proposals rely on it in several ways: - Shavit's monitoring framework has three stages: on-chip logging, proofs about training runs, and monitoring of the chip supply chain [S-0029]. The third stage exists so that no actor can avoid discovery by amassing a large quantity of untracked chips [S-0029]. Without it, a prover could covertly acquire chips and train on them without ever notifying the verifier, bypassing checks on the chips it did declare [S-0029]. - Sastry and colleagues describe AI-relevant compute as detectable, excludable and quantifiable, and produced through an extremely concentrated supply chain [S-0053]. They list an international AI chip registry among possible mechanisms for regulatory visibility [S-0053]. - A draft international agreement would prohibit concentrations of more than 16 H100-equivalents outside monitored facilities [S-0063]. It would consolidate existing chips into those facilities and track new production [S-0063]. Chips would be located through supply-chain tracking, mandatory reporting, intelligence gathering, open-source intelligence, power monitoring, challenge inspections and whistleblowers [S-0063]. - Scher and Thiergart argue for locating AI chips at an initial point in time and then keeping them monitored, rather than relying on detecting secret data centres later [S-0005]. Why it is hard The claim is negative: it asserts that no chips exist beyond the declared total. - The existing stock. Shavit noted in 2023 that hundreds of thousands of ML chips had already been sold, many lacking the security features his framework needs and possibly not locatable by governments [S-0029]. Scher and Thiergart write that millions of AI-relevant chips already exist with no central tracking, which could make an initial inventory difficult [S-0005]. - Tracking must start at the fab. Shavit's design monitors the small number of fabrication facilities that make leading-edge chips [S-0029]. It records each chip's burned-in serial number in a directory of chip owners, kept up to date when chips are resold or damaged [S-0029]. Responsibility for any missing chip precursors lies with the most recent holder [S-0029]. - Supporting methods have gaps. Wasil and colleagues note that customs data is less useful against countries that can manufacture components domestically, and that financial intelligence is limited because many hardware purchases have legitimate uses [S-0062]. Inspections of chip fabrication plants are resource-intensive and put intellectual property at risk [S-0062]. The same authors list local chip manufacture and the use of older chips as evasion routes, and note that chip location tracking would apply only to new chips [S-0062]. - Chips are not a fixed unit of capacity. RAND's framework counts a cluster as large-scale if it has the computing power of thousands of high-end AI chips controlled by a single entity [S-0002]. Sastry and colleagues caution that algorithmic progress can reduce the compute needed for a given capability, and that decentralised training could undermine the detectability of compute [S-0053]. The claim is closely tied to There is no undeclared relevant compute, which asks whether any compute lies outside the declared stock, and to Chips are where they are declared to be, which asks whether declared chips are where they are said to be.", "addressed_by": [ { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/", "role": "primary", "note": "Gives a baseline of which chips were made and who declared owning them." }, { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/", "role": "supporting", "note": "A verified performance cap bounds the effective capacity of declared hardware; needs attestation that the cap is active." }, { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/", "role": "supporting", "note": "Estimates the power capacity, and so roughly the compute, of observed facilities." } ], "referenced_by": [ { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" }, { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "O-0211", "title": "Centre for the Governance of AI", "url": "https://trustbutveri.fyi/organizations/centre-for-the-governance-of-ai/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" } ] }, { "id": "C-0003", "slug": "declared-hardware-is-idle", "title": "Declared hardware is idle or shut down", "aliases": [ "chip non-use", "powered-down hardware" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Specified AI chips or facilities are not performing computation, or are powered off, throughout a declared period.", "summary": "The claim is that specified AI chips or facilities are not computing, or are powered off, throughout a declared period. Some agreement designs would use this to pause certain activities, or to hold a reserve of compute verified not to be in use. Verifying idleness would let a party show it is not using hardware it still owns. It is a negative claim, but a comparatively simple one: chips need power to compute, so a facility's power draw, knowledge of on-site generation and possibly thermal imaging could show whether hardware is running. One analysis expects this to be verifiable with less invasive methods than those needed to check what running chips compute. The difficulties are binding and coverage: showing that the monitored facility holds the declared chips, and obtaining reliable power data that cannot be masked. On-chip telemetry and hardware licensing could add chip-level evidence or enforcement.", "claim_class": "negative", "editors_synthesis": { "assessment": true, "markdown": "Idleness is one of the more approachable negative claims, because computing needs power and leaves physical traces. On-chip telemetry and timed challenges are demonstrated (R2) as indicators of GPU activity, not as proofs that all declared hardware is idle.\n\nPower draw, knowledge of on-site generation and possibly thermal imaging could show that a facility's chips are unpowered [[S-0005]], but energy monitoring is unproven in practice and open to masking [[S-0062]]. [[M-0010|On-chip telemetry]] (R2) and [[M-0016|timed challenges]] (R2) could show whether a declared chip is busy, and GPU timing and memory measurements correlate with compute activity even when host and device are untrusted [[S-0033]]. [[M-0011|Licensing and throttling]] (R1) would make chips refuse or slow work once a licensed budget is spent [[S-0057]]. [[M-0007|Proofs of useful work]] (R1) instead keep declared hardware provably busy with agreed work [[S-1102]].\n\nA dark facility shows only that the hardware inside it is idle, so the claim depends on knowing where the declared chips are ([[C-0002]]). Where chips must stay powered for permitted work, the claim becomes a bound on use ([[C-0004]], [[C-0007]]).", "text": "Idleness is one of the more approachable negative claims, because computing needs power and leaves physical traces. On-chip telemetry and timed challenges are demonstrated (R2) as indicators of GPU activity, not as proofs that all declared hardware is idle. Power draw, knowledge of on-site generation and possibly thermal imaging could show that a facility's chips are unpowered [S-0005], but energy monitoring is unproven in practice and open to masking [S-0062]. On-chip telemetry (R2) and timed challenges (R2) could show whether a declared chip is busy, and GPU timing and memory measurements correlate with compute activity even when host and device are untrusted [S-0033]. Licensing and throttling (R1) would make chips refuse or slow work once a licensed budget is spent [S-0057]. Proofs of useful work (R1) instead keep declared hardware provably busy with agreed work [S-1102]. A dark facility shows only that the hardware inside it is idle, so the claim depends on knowing where the declared chips are (Chips are where they are declared to be). Where chips must stay powered for permitted work, the claim becomes a bound on use (This compute runs inference, not training, A training run stayed within declared limits)." }, "sources": [ { "source": "S-0005", "supports": "chips without power cannot run a large training run; power draw, on-site generation and thermal imaging as less invasive verification", "locator": "Verifying that known compute is not being used for a large training run" }, { "source": "S-0062", "supports": "energy monitoring to detect facilities and approximate FLOPs; unproven, can be masked, data hard to obtain; evasions", "locator": "Energy monitoring; Table 1; Figure 2" }, { "source": "S-0067", "supports": "compute bank verified not to be in use during an agreement", "locator": "phase 3 (improving robustness)" }, { "source": "S-0002", "supports": "off-chip analog sensors; estimating utilisation and physical signature such as power as an R&D problem", "locator": "§4; Table 2, Appendix A.6" }, { "source": "S-0057", "supports": "offline licensing tying chip features to a renewable licence with a compute budget", "locator": "p. viii" }, { "source": "S-0037", "supports": "telemetry classifier accuracy overall and on adversarially disguised workloads", "locator": "abstract; §5.2" }, { "source": "S-0033", "supports": "timing and memory observables that correlate with GPU compute activity when host and device are untrusted", "locator": "abstract" }, { "source": "S-1102", "supports": "approved work plus protocol-defined work fills a required work budget (provider proposal)", "locator": "blog post" } ], "concepts": [ "K-0003", "K-0007", "K-0013", "K-0025" ], "order": 3, "type": "claim", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/", "source_file": "content/claims/declared-hardware-is-idle.md", "flags_all": [], "body_markdown": "## Why it matters\nIdleness lets a party show that hardware it still owns is not in use. Proposals use it in three ways:\n\n- Scher and Thiergart observe that a data-centre operator can claim its chips are not being used in a large training run if the chips are not receiving the power they need to operate [[S-0005]]. They add that this can likely be verified with less invasive methods, such as the data centre's power draw, knowledge of on-site backup power generation, or possibly thermal imaging [[S-0005]].\n- The AI 2040 verification plan suggests a small compute bank, verified not to be in use during an agreement, as a way to reduce the incentive to withdraw from it [[S-0067]].\n- Energy monitoring is one of the national technical means Wasil and colleagues consider for detecting unauthorised facilities, by analysing power consumption and converting energy estimates into an approximate count of operations [[S-0062]].\n\n## Why it is hard\nThe claim is negative: it asserts that no computation takes place.\n\n- **Power data describes a facility, not a chip.** A verifier must know about every power source, including on-site generation [[S-0005]]. Wasil and colleagues note that energy monitoring is unproven in practice, that energy use may be disguised as other high-energy activity, and that detailed consumption data is hard to obtain [[S-0062]]. They list masking a data centre's energy use and siting a data centre at a power plant as evasion techniques [[S-0062]].\n- **Binding.** A dark building shows that the chips inside it are idle, but not that the declared chips are the ones inside. The claim therefore depends on verifying chip location and custody, covered under [[C-0002]].\n- **Chip-level evidence is immature.** RAND's framework includes off-chip analog sensors, and lists estimating a workload's utilisation and associated physical signature, such as power, as a research problem [[S-0002]]. Monfared and colleagues report timing and memory measurements on GPUs that correlate with compute activity even when host and device are untrusted [[S-0033]]. One telemetry classifier detects training with 98.2% accuracy across its own corpus, falling to 43–87% on the most challenging disguised workloads held out from its training [[S-0037]].\n- **Hardware enforcement.** RAND's offline-licensing design grants a chip a compute budget through a renewable licence; once the budget is spent, the chip would refuse the relevant operations or perform them at a much lower rate [[S-0057]].\n\nWhere chips must stay powered for permitted tasks, the claim becomes a bound on use rather than idleness, and the questions of [[C-0004]] and [[C-0007]] apply.", "body_text": "Why it matters Idleness lets a party show that hardware it still owns is not in use. Proposals use it in three ways: - Scher and Thiergart observe that a data-centre operator can claim its chips are not being used in a large training run if the chips are not receiving the power they need to operate [S-0005]. They add that this can likely be verified with less invasive methods, such as the data centre's power draw, knowledge of on-site backup power generation, or possibly thermal imaging [S-0005]. - The AI 2040 verification plan suggests a small compute bank, verified not to be in use during an agreement, as a way to reduce the incentive to withdraw from it [S-0067]. - Energy monitoring is one of the national technical means Wasil and colleagues consider for detecting unauthorised facilities, by analysing power consumption and converting energy estimates into an approximate count of operations [S-0062]. Why it is hard The claim is negative: it asserts that no computation takes place. - Power data describes a facility, not a chip. A verifier must know about every power source, including on-site generation [S-0005]. Wasil and colleagues note that energy monitoring is unproven in practice, that energy use may be disguised as other high-energy activity, and that detailed consumption data is hard to obtain [S-0062]. They list masking a data centre's energy use and siting a data centre at a power plant as evasion techniques [S-0062]. - Binding. A dark building shows that the chips inside it are idle, but not that the declared chips are the ones inside. The claim therefore depends on verifying chip location and custody, covered under Chips are where they are declared to be. - Chip-level evidence is immature. RAND's framework includes off-chip analog sensors, and lists estimating a workload's utilisation and associated physical signature, such as power, as a research problem [S-0002]. Monfared and colleagues report timing and memory measurements on GPUs that correlate with compute activity even when host and device are untrusted [S-0033]. One telemetry classifier detects training with 98.2% accuracy across its own corpus, falling to 43–87% on the most challenging disguised workloads held out from its training [S-0037]. - Hardware enforcement. RAND's offline-licensing design grants a chip a compute budget through a renewable licence; once the budget is spent, the chip would refuse the relevant operations or perform them at a much lower rate [S-0057]. Where chips must stay powered for permitted tasks, the claim becomes a bound on use rather than idleness, and the questions of This compute runs inference, not training and A training run stayed within declared limits apply.", "addressed_by": [ { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/", "role": "supporting", "note": "Unlicensed hardware falls back to reduced capacity or shuts down (S-0006)." }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/", "role": "primary", "note": "Utilization and residency signals indicate whether declared-idle hardware is busy (S-0033)." }, { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/", "role": "supporting", "note": "Keeping declared hardware provably busy with agreed work, as an alternative to showing it idle." }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/", "role": "primary", "note": "Compute and memory probes can reveal whether a GPU is engaged in other work (S-0033)." }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/", "role": "supporting", "note": "Proves work for consensus; keeping declared hardware provably busy, as an alternative to showing it idle, is not demonstrated." } ], "referenced_by": [ { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" } ] }, { "id": "C-0005", "slug": "declared-model-is-served", "title": "The declared model is the one being served", "aliases": [ "model identity", "model substitution" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Outputs delivered to users or auditors come from the specific model, weights and configuration the provider declared, not from a substitute.", "summary": "The claim is that outputs delivered to users or auditors come from the specific model, weights and configuration the provider declared. Evaluations, audits and agreements often apply to one specific model. If a provider could evaluate one model and serve another, such as a cheaper, quantized or modified version, those checks would say little about what users receive. It is a positive claim that can be tested directly, but three problems make it hard. Numerical nondeterminism means honest recomputation does not match exactly. The verifier usually cannot see the weights, which are commercially or strategically sensitive. And the evidence must come from the actual serving system rather than a separate test instance. Approaches include statistical or exact recomputation of sampled outputs, hardware attestation of the loaded weights, and zero-knowledge proofs. They trade off cost, trust in hardware vendors and confidentiality.", "claim_class": "positive", "editors_synthesis": { "assessment": true, "markdown": "The served model can be checked directly on its outputs, and several mechanisms for it are demonstrated (R2) or in production (R3). None is deployment-ready (R4): independent security evaluations are either missing or, for trusted execution environments, found a critical flaw.\n\nFor example, users of a hosted model may want to know that the model answering them is the one an auditor evaluated.\n\n- [[M-0012|Model identity attestation]] (R3) extends [[M-0008|TEE remote attestation]] (R3) from the software an enclave booted to the weights it loads. [[M-0025|Confidential multi-party verification]] (R2) runs the audit in an enclave and binds its result to the model's hash [[S-0009]].\n- [[I-0007|Attestable Audits]] (R2) joins the two steps. At inference, the enclave checks the served model's hash against the audited one and returns each response with an attestation that links model, prompt, output and audit result [[S-0009]]. Its reported evaluation covers only the audit step, on CPU-only enclaves with a 4-bit, 8-billion-parameter model [[S-0009]]. [[I-0006|Tinfoil's Modelwrap]] (R3) covers serving alone, in what Tinfoil reports is a production service [[S-1208]].\n- The evidence is narrower than what users want. Attested enclaves report that the weights that answered have the same hash as the weights that were audited. For a public model, Modelwrap lets anyone rebuild the hash from the published weights. For a private model, users see only the hash [[S-0013]].\n- The chain trusts the TEE vendors' attestation keys. With physical access, researchers forged Intel TDX attestations and paired them with relayed H100 attestations, so that a system outside TEE protection passed both checks [[S-1202]]. [[M-0001|Sampled recomputation]] (R3), as in [[I-0002|DiFR]], avoids hardware trust but needs a trusted copy of the weights and evidence tied to the production serving path [[S-0018]]. [[M-0004|Zero-knowledge proofs of inference]] (R2) keep the weights private without trusting hardware, but proving remains expensive [[S-0018]].", "text": "The served model can be checked directly on its outputs, and several mechanisms for it are demonstrated (R2) or in production (R3). None is deployment-ready (R4): independent security evaluations are either missing or, for trusted execution environments, found a critical flaw. For example, users of a hosted model may want to know that the model answering them is the one an auditor evaluated. - Model identity attestation (R3) extends TEE remote attestation (R3) from the software an enclave booted to the weights it loads. Confidential multi-party verification (R2) runs the audit in an enclave and binds its result to the model's hash [S-0009]. - Attestable Audits (R2) joins the two steps. At inference, the enclave checks the served model's hash against the audited one and returns each response with an attestation that links model, prompt, output and audit result [S-0009]. Its reported evaluation covers only the audit step, on CPU-only enclaves with a 4-bit, 8-billion-parameter model [S-0009]. Tinfoil's Modelwrap (R3) covers serving alone, in what Tinfoil reports is a production service [S-1208]. - The evidence is narrower than what users want. Attested enclaves report that the weights that answered have the same hash as the weights that were audited. For a public model, Modelwrap lets anyone rebuild the hash from the published weights. For a private model, users see only the hash [S-0013]. - The chain trusts the TEE vendors' attestation keys. With physical access, researchers forged Intel TDX attestations and paired them with relayed H100 attestations, so that a system outside TEE protection passed both checks [S-1202]. Sampled recomputation (R3), as in DiFR, avoids hardware trust but needs a trusted copy of the weights and evidence tied to the production serving path [S-0018]. Zero-knowledge proofs of inference (R2) keep the weights private without trusting hardware, but proving remains expensive [S-0018]." }, "sources": [ { "source": "S-0002", "supports": "Subgoal 1.A (declared uses declared accurately, including inference) and 1.B (required properties; deployed models evaluated at intervals)", "locator": "§3.2" }, { "source": "S-0016", "supports": "need to verify inference; nondeterminism; Token-DiFR detects 4-bit quantization with AUC > 0.999 within 300 tokens", "locator": "abstract" }, { "source": "S-1009", "supports": "batch-size dependence as a cause of inference nondeterminism", "locator": "batch invariance section" }, { "source": "S-0020", "supports": "bit-exact reproduction across GPU variants given recomputation data", "locator": "abstract" }, { "source": "S-0004", "supports": "model fingerprint attestation; device-model mating with an encrypted model", "locator": "Appendix K (p. 157); Appendix L.4 (p. 159)" }, { "source": "S-0018", "supports": "whitelisted models for blacklisted uses; attributing forward passes to hardware and time; committed weights in auditing environments; ZKP cost", "locator": "verification goals; architecture; open problems" }, { "source": "S-0023", "supports": "zkLLM proves one 2,048-token forward pass of a 13B-parameter model in under 15 minutes with proofs under 200 kB, keeping parameters private", "locator": "abstract; §8 Table 1" }, { "source": "S-0012", "supports": "property attestation on Intel TDX + NVIDIA H100 with under 11% overhead for common operations", "locator": "abstract" }, { "source": "S-0014", "supports": "attestation-key holder can produce valid reports; side-channel and physical attacks; measurement coverage", "locator": "Limitations" }, { "source": "S-0009", "supports": "attestation linking model, audit result, prompt and response in a TEE; reported evaluation covers the audit step on CPU-only enclaves with a 4-bit 8B model", "locator": "abstract; inference protocol; §5" }, { "source": "S-0013", "supports": "public models' hashes can be rebuilt; private models expose only the hash (provider-reported)" }, { "source": "S-1208", "supports": "Modelwrap chain in Tinfoil's production service (provider-reported)", "locator": "In-band vs. out-of-band verification" }, { "source": "S-1202", "supports": "physical memory-bus interposition extracts a per-CPU Intel attestation key and forges TDX attestations", "locator": "abstract; §1.1" }, { "source": "S-1210", "supports": "Battering RAM forges SEV-SNP attestation with a DDR4 interposer" }, { "source": "S-1212", "supports": "RMPocalypse forges SEV-SNP attestation from a malicious hypervisor" }, { "source": "S-3540", "supports": "model equality testing: median 77.4% power with about 10 samples per prompt; 11 of 31 Llama API endpoints in summer 2024 served distributions different from the reference weights", "locator": "abstract" }, { "source": "S-3541", "supports": "output tests query-intensive and fail against subtle substitutions; log-probability tests defeated by inference nondeterminism", "locator": "abstract" } ], "concepts": [ "K-0009", "K-0008", "K-0004", "K-0006", "K-0010", "K-0019" ], "order": 5, "type": "claim", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/", "source_file": "content/claims/declared-model-is-served.md", "flags_all": [], "body_markdown": "## Why it matters\nChecks on a model's capabilities or safety apply to the model that was checked. Several parties depend on that link:\n\n- RAND's framework asks verifiers to confirm that declared inference is declared accurately, so that the prover actually did the claimed deployment [[S-0002]]. It then asks them to confirm that deployed models have the required properties, for example through evaluations at regular intervals [[S-0002]]. Both steps assume the evaluated model is the served model. A low-trust system overview identifies running approved models for prohibited uses as the most difficult violation to defend against, and aims to deploy only approved models [[S-0018]]. The Oxford Martin report includes appendices on model fingerprint attestation and on \"device-model mating\" with an encrypted model [[S-0004]].\n- The DiFR authors note that providers and their customers increasingly need to verify that inference is performed correctly, without errors or tampering [[S-0016]]. Their Token-DiFR method detected 4-bit quantization of a model with AUC above 0.999 within 300 output tokens [[S-0016]]. In an audit of commercial inference APIs for four Llama models in summer 2024, Gao, Liang and Guestrin found that 11 of 31 endpoints served a different output distribution from the published reference weights [[S-3540]].\n\n## Why it is hard\n- Statistical tests on an API's outputs need no cooperation from the provider, but they have limits [[S-3540]]. Gao, Liang and Guestrin's test reached a median statistical power of 77.4% against a range of distortions, using an average of 10 samples per prompt [[S-3540]]. Cai and colleagues find that tests on text outputs are query-intensive and fail against subtle substitutions, and that tests on log probabilities are defeated by inference nondeterminism in production [[S-3541]].\n- Re-running the same inference often gives slightly different results because of benign numerical variation, which makes it hard to tell legitimate variation from substitution [[S-0016]]. In LLM serving a major cause is that kernels are not invariant to batch size, which varies with server load [[S-1009]]. Statistical tests tolerate this noise [[S-0016]]. Alternatively, Cankaya reports that inference can be reproduced bit-exactly across several NVIDIA GPU variants in software, given enough recorded information about the original run [[S-0020]].\n- Recomputation requires the weights, which a provider or state may not reveal [[S-0018]]. One system design keeps weights cryptographically committed and uses them only inside air-gapped auditing environments [[S-0018]]. Zero-knowledge proofs avoid revealing weights at all. zkLLM reports proving one 2,048-token forward pass of a 13-billion-parameter model in under 15 minutes, with proofs under 200 kB [[S-0023]], but the low-trust overview describes zero-knowledge proof computation as still expensive [[S-0018]]. Trusted execution environments can attest which software and data were loaded [[S-0014]]. PAL\\*M reports property attestation on confidential computing hardware (Intel TDX with NVIDIA H100) at under 11% overhead for common operations [[S-0012]].\n- Evidence about a test instance says nothing about production unless it is tied to the serving hardware and time. The low-trust design aims to identify each forward pass uniquely and attribute it to the hardware and time it was processed [[S-0018]]. TEE-based binding relies on the vendor's keys: whoever holds the hardware's attestation key can produce valid reports, and every component that influences inference must be covered by launch measurement [[S-0014]]. A published memory-bus interposition attack, which needs physical access, extracted a per-CPU Intel attestation key and forged Intel TDX attestations [[S-1202]]. Other attacks forged AMD SEV-SNP attestations, one through a DDR4 interposer and one from software alone on platforms without AMD's fix [[S-1210]] [[S-1212]].", "body_text": "Why it matters Checks on a model's capabilities or safety apply to the model that was checked. Several parties depend on that link: - RAND's framework asks verifiers to confirm that declared inference is declared accurately, so that the prover actually did the claimed deployment [S-0002]. It then asks them to confirm that deployed models have the required properties, for example through evaluations at regular intervals [S-0002]. Both steps assume the evaluated model is the served model. A low-trust system overview identifies running approved models for prohibited uses as the most difficult violation to defend against, and aims to deploy only approved models [S-0018]. The Oxford Martin report includes appendices on model fingerprint attestation and on \"device-model mating\" with an encrypted model [S-0004]. - The DiFR authors note that providers and their customers increasingly need to verify that inference is performed correctly, without errors or tampering [S-0016]. Their Token-DiFR method detected 4-bit quantization of a model with AUC above 0.999 within 300 output tokens [S-0016]. In an audit of commercial inference APIs for four Llama models in summer 2024, Gao, Liang and Guestrin found that 11 of 31 endpoints served a different output distribution from the published reference weights [S-3540]. Why it is hard - Statistical tests on an API's outputs need no cooperation from the provider, but they have limits [S-3540]. Gao, Liang and Guestrin's test reached a median statistical power of 77.4% against a range of distortions, using an average of 10 samples per prompt [S-3540]. Cai and colleagues find that tests on text outputs are query-intensive and fail against subtle substitutions, and that tests on log probabilities are defeated by inference nondeterminism in production [S-3541]. - Re-running the same inference often gives slightly different results because of benign numerical variation, which makes it hard to tell legitimate variation from substitution [S-0016]. In LLM serving a major cause is that kernels are not invariant to batch size, which varies with server load [S-1009]. Statistical tests tolerate this noise [S-0016]. Alternatively, Cankaya reports that inference can be reproduced bit-exactly across several NVIDIA GPU variants in software, given enough recorded information about the original run [S-0020]. - Recomputation requires the weights, which a provider or state may not reveal [S-0018]. One system design keeps weights cryptographically committed and uses them only inside air-gapped auditing environments [S-0018]. Zero-knowledge proofs avoid revealing weights at all. zkLLM reports proving one 2,048-token forward pass of a 13-billion-parameter model in under 15 minutes, with proofs under 200 kB [S-0023], but the low-trust overview describes zero-knowledge proof computation as still expensive [S-0018]. Trusted execution environments can attest which software and data were loaded [S-0014]. PAL*M reports property attestation on confidential computing hardware (Intel TDX with NVIDIA H100) at under 11% overhead for common operations [S-0012]. - Evidence about a test instance says nothing about production unless it is tied to the serving hardware and time. The low-trust design aims to identify each forward pass uniquely and attribute it to the hardware and time it was processed [S-0018]. TEE-based binding relies on the vendor's keys: whoever holds the hardware's attestation key can produce valid reports, and every component that influences inference must be covered by launch measurement [S-0014]. A published memory-bus interposition attack, which needs physical access, extracted a per-CPU Intel attestation key and forged Intel TDX attestations [S-1202]. Other attacks forged AMD SEV-SNP attestations, one through a DDR4 interposer and one from software alone on platforms without AMD's fix [S-1210] [S-1212].", "addressed_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/", "role": "primary", "note": "Binds audit or capability-evaluation results to the model that is served, without revealing weights (S-0009, S-0011)." }, { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/", "role": "primary", "note": "Enables exact-match recomputation checks that the declared model, weights and software setup produced the outputs." }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/", "role": "supporting", "note": "Deployment only to approved flexHEG devices, and verification of evaluation scores (S-0035)." }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/", "role": "primary", "note": "Core purpose: responses come from the declared weights." }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/", "role": "supporting", "note": "Replaying challenged records with the declared model checks which model produced outputs (S-1300, S-0067)." }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/", "role": "supporting", "note": "Property and audit attestations bind responses to a measured model (S-0012, S-0009)." }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/", "role": "primary", "note": "Checks that sampled recorded outputs are consistent with the declared model, precision and sampling settings." }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/", "role": "primary", "note": "Attests the software stack that produced responses, and the model too when paired with a weight commitment (see M-0012)." }, { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/", "role": "primary", "note": "Binds each proven output to committed weights and a public architecture." }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/", "role": "supporting", "note": "Recomputation checks sampled packets against the declared model." }, { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/", "role": "supporting", "note": "Attests the software release that served a request. Apple reports that model assets share the code's integrity protection (S-1800)." }, { "id": "I-0007", "title": "Attestable Audits", "url": "https://trustbutveri.fyi/implementations/attestable-audits/", "role": "primary", "note": "Users can check that the model answering them is the audited one." }, { "id": "I-0005", "title": "Attestable zero-knowledge inference prover", "url": "https://trustbutveri.fyi/implementations/attestable-zk-inference/", "role": "primary", "note": "Attestable reports proving y = F(W, x, r) for committed weights W." }, { "id": "I-0016", "title": "Batch-invariant inference kernels (Thinking Machines)", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/", "role": "supporting", "note": "Makes exact-match recomputation of served outputs possible when the verifier runs the same model, engine and hardware (S-0016)." }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/", "role": "primary", "note": "Checks that outputs are consistent with the declared model, precision and sampling configuration." }, { "id": "I-0014", "title": "EZKL", "url": "https://trustbutveri.fyi/implementations/ezkl/", "role": "primary", "note": "Proves an output follows from a committed model. South et al.'s results reach about a million parameters (S-0024)." }, { "id": "I-0015", "title": "Verde and RepOps (Gensyn)", "url": "https://trustbutveri.fyi/implementations/gensyn-verde-repops/", "role": "primary", "note": "A client learns that a delegated inference output came from the declared model and input, if at least one provider is honest (S-1809, S-1810)." }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/", "role": "supporting", "note": "Screening checks that the model is on an agreed whitelist." }, { "id": "I-0017", "title": "PySyft double-blind evaluations", "url": "https://trustbutveri.fyi/implementations/pysyft-double-blind-evaluations/", "role": "primary", "note": "Both parties approve the attested workload that runs a private evaluation against the model and prompts they submit (S-3320)." }, { "id": "I-0010", "title": "RAND secure inference data center (SIDC) design", "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/", "role": "supporting", "note": "The compute sanctum checks resident weights against reference measurements before serving." }, { "id": "I-0008", "title": "SASH confidential network logger", "url": "https://trustbutveri.fyi/implementations/sash-confidential-network-logger/", "role": "supporting", "note": "Recomputation uses another copy of the declared model (S-1320, S-1319)." }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/", "role": "primary", "note": "Clients check that the served weights match a committed root hash." }, { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/", "role": "primary", "note": "Checks that the provider produced outputs with the claimed model weights, prompt and precision." }, { "id": "I-0003", "title": "zkLLM", "url": "https://trustbutveri.fyi/implementations/zkllm/", "role": "primary", "note": "Proves an output follows from committed weights and a public architecture." } ], "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/" }, { "id": "I-0007", "title": "Attestable Audits", "url": "https://trustbutveri.fyi/implementations/attestable-audits/" }, { "id": "I-0005", "title": "Attestable zero-knowledge inference prover", "url": "https://trustbutveri.fyi/implementations/attestable-zk-inference/" }, { "id": "I-0016", "title": "Batch-invariant inference kernels (Thinking Machines)", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" }, { "id": "I-0014", "title": "EZKL", "url": "https://trustbutveri.fyi/implementations/ezkl/" }, { "id": "I-0015", "title": "Verde and RepOps (Gensyn)", "url": "https://trustbutveri.fyi/implementations/gensyn-verde-repops/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "I-0017", "title": "PySyft double-blind evaluations", "url": "https://trustbutveri.fyi/implementations/pysyft-double-blind-evaluations/" }, { "id": "I-0010", "title": "RAND secure inference data center (SIDC) design", "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/" }, { "id": "I-0008", "title": "SASH confidential network logger", "url": "https://trustbutveri.fyi/implementations/sash-confidential-network-logger/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" }, { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "I-0003", "title": "zkLLM", "url": "https://trustbutveri.fyi/implementations/zkllm/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" }, { "id": "K-0003", "title": "Positive and negative claims", "url": "https://trustbutveri.fyi/concepts/positive-and-negative-claims/" } ] }, { "id": "C-0004", "slug": "inference-not-training", "title": "This compute runs inference, not training", "aliases": [ "inference-only verification", "training vs inference" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "A declared cluster is used only to run existing models to produce outputs, and not to train new or more capable models.", "summary": "The claim is that a declared cluster is used only to run existing models to produce outputs, not to train new or more capable models. Several agreement proposals would let existing AI models keep serving users while restricting further training. The claim combines a positive part (the declared inference is happening) with a negative part (nothing else, especially training, is). It matters because it could let most AI compute stay in productive use under a training restriction. It is hard because the same chips can do both, workloads can be disguised, and the negative part requires accounting for all of a facility's activity. Proposed approaches include capturing and recomputing the traffic that enters and leaves a facility, limiting bandwidth between groups of chips so that large training cannot be coordinated, and classifying workloads from telemetry. Each rests on open assumptions about numerical nondeterminism, hidden capacity and side channels.", "claim_class": "mixed", "editors_synthesis": { "assessment": true, "markdown": "Declared inference can be checked with components that are demonstrated (R2) or in production (R3), but the absence of training cannot yet be verified. That half rests on full-stack designs ([[I-0011]], [[I-0012]], [[I-0008]]) that are proposed (R1), and a team building the components reports nothing past a proof-of-principle prototype [[S-1512]].\n\nFor example, a verifier may want to know that a cluster declared for inference is not training a new model.\n\n- [[M-0013|Network taps]] (R1) record the cluster's front-end traffic with its users, and [[M-0001|sampled recomputation]] (R3) re-runs sampled requests on the declared model. [[I-0002|DiFR]] tolerates numerical noise in these re-runs on open-weight models of 8 to 30 billion parameters [[S-0016]].\n- [[I-0008|SASH's network logger]] (R1) is a public prototype of both. It passes every request through a logger and re-runs it on a separate cluster, with a 270-million-parameter model and no stated adversary [[S-1319]].\n- This covers the positive half at most. It shows that sampled outputs match the declared model. A tap on the cluster's external links does not stop covert workloads. It aims only to stop their results leaving over those links [[S-1300]].\n- The negative half needs the rest of the cluster accounted for. Training traffic runs on back-end fabric that is harder to tap [[S-0018]]. [[M-0014|Bandwidth limits]] (R2 for software monitoring) between groups of chips target that fabric, since distributed training must exchange gradients [[S-0005]]. Attestable proposes [[M-0007|proofs of useful work]] (R1) to keep declared hardware busy with approved or protocol-defined work, leaving little spare capacity [[S-1102]].\n- Known routes remain. Reinforcement-learning rollouts are inference, so declared servers could generate them while hidden compute updates the model [[S-3566]]. In one scenario, JoshC estimates that more than 95% of computation must be accounted for to constrain that strategy [[S-3566]]. [[M-0021|Workload classification]] (R2) is a lighter alternative. It detects training with 98.2% accuracy on its own corpus, but 43–87% on the most challenging disguised workloads held out from its training, and current GPUs lack the protections its telemetry needs to be trusted [[S-0037]] [[S-0033]].", "text": "Declared inference can be checked with components that are demonstrated (R2) or in production (R3), but the absence of training cannot yet be verified. That half rests on full-stack designs (AI 2040 inference-only verification stack, Low-trust AI compute verification system overview, SASH confidential network logger) that are proposed (R1), and a team building the components reports nothing past a proof-of-principle prototype [S-1512]. For example, a verifier may want to know that a cluster declared for inference is not training a new model. - Network taps (R1) record the cluster's front-end traffic with its users, and sampled recomputation (R3) re-runs sampled requests on the declared model. DiFR tolerates numerical noise in these re-runs on open-weight models of 8 to 30 billion parameters [S-0016]. - SASH's network logger (R1) is a public prototype of both. It passes every request through a logger and re-runs it on a separate cluster, with a 270-million-parameter model and no stated adversary [S-1319]. - This covers the positive half at most. It shows that sampled outputs match the declared model. A tap on the cluster's external links does not stop covert workloads. It aims only to stop their results leaving over those links [S-1300]. - The negative half needs the rest of the cluster accounted for. Training traffic runs on back-end fabric that is harder to tap [S-0018]. Bandwidth limits (R2 for software monitoring) between groups of chips target that fabric, since distributed training must exchange gradients [S-0005]. Attestable proposes proofs of useful work (R1) to keep declared hardware busy with approved or protocol-defined work, leaving little spare capacity [S-1102]. - Known routes remain. Reinforcement-learning rollouts are inference, so declared servers could generate them while hidden compute updates the model [S-3566]. In one scenario, JoshC estimates that more than 95% of computation must be accounted for to constrain that strategy [S-3566]. Workload classification (R2) is a lighter alternative. It detects training with 98.2% accuracy on its own corpus, but 43–87% on the most challenging disguised workloads held out from its training, and current GPUs lack the protections its telemetry needs to be trusted [S-0037] [S-0033]." }, "sources": [ { "source": "S-0063", "supports": "restricting the scale of training; chip use verification distinguishing inference on existing systems from training", "locator": "abstract; Article VII (as summarised)" }, { "source": "S-0067", "supports": "data centres converted to inference-only operation; network taps, recomputation and reproducible packets", "locator": "phases; verification mechanisms" }, { "source": "S-0053", "supports": "majority of AI compute used for inference; decentralised training could undermine detectability", "locator": "training vs inference; limitations" }, { "source": "S-0002", "supports": "declared inference (1.A.2) as a distinct subgoal; deterministic replication of inference as an R&D problem", "locator": "§3.2; Appendix A.9" }, { "source": "S-0018", "supports": "distinguishing inference from training; token-level front-end evidence; back-end harder to tap; egress explainable by ingress as open question; memory wiping; side channels", "locator": "verification goals; inference vs training; open problems" }, { "source": "S-0005", "supports": "inference-specialised chips repurposable for training; pods with limited external bandwidth", "locator": "Verifying that known compute is not being used for a large training run" }, { "source": "S-0029", "supports": "no straightforward way to tell whether a chip is running training or another job", "locator": "open problems" }, { "source": "S-0037", "supports": "telemetry classifier accuracy overall and on adversarially disguised workloads; required telemetry protections", "locator": "abstract; §5.2; deployment requirements" }, { "source": "S-0001", "supports": "workload classification; adversarial customers may obfuscate by adding noise", "locator": "§3.2.2 / §5.2.2 open problems" }, { "source": "S-0031", "supports": "front-end tapping most viable; back-end requires sampling", "locator": "frontend vs backend" }, { "source": "S-0016", "supports": "recomputation despite benign numerical noise on 8–30B open-weight models", "locator": "abstract; §5" }, { "source": "S-0020", "supports": "bit-exact inference verification across GPU variants", "locator": "abstract" }, { "source": "S-1512", "supports": "no verification component past a proof-of-principle prototype", "locator": "introduction" }, { "source": "S-0033", "supports": "current GPUs expose limited trusted telemetry", "locator": "abstract" }, { "source": "S-1102", "supports": "spare capacity could run an unauthorised training job; approved inference plus protocol-defined work fills a required work budget (provider proposal)", "locator": "blog post" }, { "source": "S-3566", "supports": "public analysis of covert reinforcement-learning rollouts on declared inference servers and updates on hidden compute; share of computation that must be accounted for (scenario estimate)", "locator": "§2.5; §3.4; §4" }, { "source": "S-1319", "supports": "SASH prototype re-runs every request through a logger on a separate cluster, with Gemma 3 270M and a demo switch as the only adversary", "locator": "README.md; implementation" }, { "source": "S-1300", "supports": "a tap on external links does not prevent covert workloads, only the exfiltration of their results" } ], "concepts": [ "K-0025", "K-0009", "K-0008", "K-0014", "K-0021", "K-0017", "K-0013" ], "order": 4, "type": "claim", "url": "https://trustbutveri.fyi/claims/inference-not-training/", "source_file": "content/claims/inference-not-training.md", "flags_all": [], "body_markdown": "## Why it matters\nSeveral proposals to restrict frontier AI development target training while preserving the use of existing models [[S-0063]] [[S-0067]]. Sastry and colleagues note that most AI compute is now used for inference [[S-0053]].\n\n- **A draft international agreement** restricts the scale of AI training. Its chip-use verification is meant to distinguish inference on existing systems from the training of new models [[S-0063]].\n- **The AI 2040 verification plan** would convert data centres to inference-only operation, preventing training runs while models continue to serve users [[S-0067]].\n- **RAND's framework** treats accurate declaration of AI inference as its own verification subgoal, separate from declared training [[S-0002]]. It lists deterministic replication of neural-network inference as a research problem [[S-0002]].\n- **A low-trust system overview** names distinguishing inference from training, and deploying only approved models, among its core objectives [[S-0018]].\n\n## Why it is hard\n- **The same hardware can do both.** Scher and Thiergart expect that some current inference-specialised chips could be repurposed for training without significant effort [[S-0005]]. Shavit notes that there is no straightforward way to determine whether an ML chip is running a training job or an unrelated one [[S-0029]].\n- **Classifiers invite evasion.** The Open Problems survey notes that adversarial customers may obfuscate their activities, for example by adding noise to how they use computational resources [[S-0001]]. One adversarial study of GPU-telemetry classifiers reports 98.2% accuracy at identifying training across its corpus, but 43–87% on the most challenging disguised workloads held out from its training [[S-0037]].\n- **Training traffic is harder to observe.** Inference produces token-level input and output data on the front-end links between a data centre and its users [[S-0018]]. Training traffic runs over back-end fabric that has much higher bandwidth, is latency-sensitive, and is harder to tap [[S-0018]]. Cankaya describes front-end tapping as the most viable option, and notes that back-end tapping may require sampling rather than full capture [[S-0031]]. The system overview leaves open how far a facility can be required to make all egress traffic explainable by its ingress [[S-0018]].\n- **Recomputation must cope with numerical noise.** Checks on sampled outputs handle it either statistically [[S-0016]] or by exact reproduction [[S-0020]].\n- **Bandwidth limits may erode.** Limits between pods of chips could prevent the gradient exchange that distributed training needs [[S-0005]]. Sastry and colleagues note that more viable decentralised training could undermine the detectability of training [[S-0053]].\n- **Hidden capacity and channels.** The negative part of the claim requires that no capacity or channel is hidden. The system overview addresses this with memory wiping and side-channel suppression, and lists open problems for both [[S-0018]].\n- **Inference can be part of training.** JoshC analyzes a covert reinforcement-learning strategy that generates rollouts on declared inference servers while updating the model on hidden compute [[S-3566]]. Observing inference outputs alone would not rule it out.", "body_text": "Why it matters Several proposals to restrict frontier AI development target training while preserving the use of existing models [S-0063] [S-0067]. Sastry and colleagues note that most AI compute is now used for inference [S-0053]. - A draft international agreement restricts the scale of AI training. Its chip-use verification is meant to distinguish inference on existing systems from the training of new models [S-0063]. - The AI 2040 verification plan would convert data centres to inference-only operation, preventing training runs while models continue to serve users [S-0067]. - RAND's framework treats accurate declaration of AI inference as its own verification subgoal, separate from declared training [S-0002]. It lists deterministic replication of neural-network inference as a research problem [S-0002]. - A low-trust system overview names distinguishing inference from training, and deploying only approved models, among its core objectives [S-0018]. Why it is hard - The same hardware can do both. Scher and Thiergart expect that some current inference-specialised chips could be repurposed for training without significant effort [S-0005]. Shavit notes that there is no straightforward way to determine whether an ML chip is running a training job or an unrelated one [S-0029]. - Classifiers invite evasion. The Open Problems survey notes that adversarial customers may obfuscate their activities, for example by adding noise to how they use computational resources [S-0001]. One adversarial study of GPU-telemetry classifiers reports 98.2% accuracy at identifying training across its corpus, but 43–87% on the most challenging disguised workloads held out from its training [S-0037]. - Training traffic is harder to observe. Inference produces token-level input and output data on the front-end links between a data centre and its users [S-0018]. Training traffic runs over back-end fabric that has much higher bandwidth, is latency-sensitive, and is harder to tap [S-0018]. Cankaya describes front-end tapping as the most viable option, and notes that back-end tapping may require sampling rather than full capture [S-0031]. The system overview leaves open how far a facility can be required to make all egress traffic explainable by its ingress [S-0018]. - Recomputation must cope with numerical noise. Checks on sampled outputs handle it either statistically [S-0016] or by exact reproduction [S-0020]. - Bandwidth limits may erode. Limits between pods of chips could prevent the gradient exchange that distributed training needs [S-0005]. Sastry and colleagues note that more viable decentralised training could undermine the detectability of training [S-0053]. - Hidden capacity and channels. The negative part of the claim requires that no capacity or channel is hidden. The system overview addresses this with memory wiping and side-channel suppression, and lists open problems for both [S-0018]. - Inference can be part of training. JoshC analyzes a covert reinforcement-learning strategy that generates rollouts on declared inference servers while updating the model on hidden compute [S-3566]. Observing inference outputs alone would not rule it out.", "addressed_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/", "role": "supporting", "note": "Intended to leave inference workable while making large training impractical (S-0067, S-1301)." }, { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/", "role": "supporting", "note": "In the compute-agreement framing, outputs must be predictable by policy-compliant computation, which limits what undeclared workloads can export (S-0019)." }, { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/", "role": "supporting", "note": "Bit-exact recomputation of declared inference removes the tolerance an operator could hide other work in (S-0020)." }, { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/", "role": "supporting", "note": "Periodic wipes are proposed so that only verified inference outputs persist (S-0067); Amodo frames wipes as verifying completeness of declared workloads (S-1302)." }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/", "role": "primary", "note": "Supplies the committed record of cluster I/O that recomputation checks against declared inference (S-0067, S-1300)." }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/", "role": "primary", "note": "Classifying training versus other workloads from counters (S-0037) or timing (S-0033)." }, { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/", "role": "primary", "note": "Bounds the spare capacity of declared hardware that could run unauthorised training; Attestable pairs work accounting with ZK inference proofs." }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/", "role": "primary", "note": "Proposed as the correctness check for an inference-only retrofit (S-0067)." }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/", "role": "supporting", "note": "Proposed as the correctness check in inference-only retrofit plans. Completeness needs other mechanisms (S-0067, S-0017)." }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/", "role": "supporting", "note": "Inference-only designs count on suppressing unmonitored physical channels so that all significant traffic passes the taps (S-0018)." }, { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/", "role": "supporting", "note": "Protects the integrity of taps, gateways and recomputation hardware used for inference-only verification (S-0018, S-0067)." }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/", "role": "supporting", "note": "Attests a declared inference deployment, but not that the same chips ran no other workloads (S-0014)." }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/", "role": "supporting", "note": "Listed as an alternative inference-verification direction that may not need a hardware retrofit (S-0067); bounds spare memory (S-0018)." }, { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/", "role": "primary", "note": "Classifies observed activity as training, inference or non-ML work." }, { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/", "role": "supporting", "note": "Attestable proposes using proofs to show accounted workloads used an approved, unchanged model." }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/", "role": "primary", "note": "The stack's stated purpose: retrofitted data centres run only inference." }, { "id": "I-0005", "title": "Attestable zero-knowledge inference prover", "url": "https://trustbutveri.fyi/implementations/attestable-zk-inference/", "role": "supporting", "note": "Proposed use: showing an accounted workload used an approved, unchanged model." }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/", "role": "primary", "note": "Challenged records are screened for inference versus training." }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/", "role": "primary", "note": "Proves matrix-multiplication work for consensus; not applied to bounding the spare capacity of declared hardware that could run training." }, { "id": "I-0010", "title": "RAND secure inference data center (SIDC) design", "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/", "role": "supporting", "note": "Scoped to serving already-trained models." }, { "id": "I-0008", "title": "SASH confidential network logger", "url": "https://trustbutveri.fyi/implementations/sash-confidential-network-logger/", "role": "primary", "note": "SASH describes the aim as distinguishing inference from training in data centres (S-1320)." } ], "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" }, { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "I-0005", "title": "Attestable zero-knowledge inference prover", "url": "https://trustbutveri.fyi/implementations/attestable-zk-inference/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" }, { "id": "I-0010", "title": "RAND secure inference data center (SIDC) design", "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/" }, { "id": "I-0008", "title": "SASH confidential network logger", "url": "https://trustbutveri.fyi/implementations/sash-confidential-network-logger/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "K-0003", "title": "Positive and negative claims", "url": "https://trustbutveri.fyi/concepts/positive-and-negative-claims/" }, { "id": "O-0211", "title": "Centre for the Governance of AI", "url": "https://trustbutveri.fyi/organizations/centre-for-the-governance-of-ai/" }, { "id": "O-0205", "title": "Oxford Martin AI Governance Initiative", "url": "https://trustbutveri.fyi/organizations/oxford-martin-ai-governance-initiative/" } ] }, { "id": "C-0010", "slug": "no-undeclared-compute", "title": "There is no undeclared relevant compute", "aliases": [ "no hidden compute", "completeness of declarations" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "A party controls no AI-relevant computing hardware or facilities, above an agreed threshold, beyond those it has declared.", "summary": "The claim is that a party controls no AI-relevant computing hardware or facilities, above an agreed threshold, beyond those it has declared. Every other check on declared hardware can be sidestepped if a party runs prohibited work on hardware it never declared. Verifying that no such compute exists is therefore central to many proposed AI agreements. It is also among the hardest claims to establish, because it asserts an absence across a whole country or organisation, and demonstrating non-existence is generally harder than demonstrating existence. No single technique establishes it. Proposals combine tracking chips from manufacture, so that the declared stock is complete; searching for undeclared facilities with satellite imagery, energy data, customs and financial intelligence; and human sources such as whistleblowers and inspections. Each has documented evasions, and the achievable assurance depends on how much compute a meaningful violation would require.", "claim_class": "negative", "editors_synthesis": { "assessment": true, "markdown": "No single verification method is foolproof for this broadest of negative claims [[S-0062]], so draft agreements pair technical measures with intelligence, challenge inspections and whistleblowers [[S-0063]]. The only mechanism mapped primarily to this claim, [[M-0020|remote detection of data centres]], is proposed (R1), as are the chip registries and location checks that support it. RAND splits the claim into undeclared use of declared clusters, which reduces to claims such as [[C-0003]] and [[C-0004]], and undeclared clusters [[S-0002]].\n\nOne strategy, which Scher and Thiergart favour, makes the declared stock complete from the start through [[M-0019|chip registries]] and [[M-0018|location verification]] [[S-0005]]. The other searches for what was missed, through remote detection of data centres and other national technical means [[S-0062]]. Satellite imagery, permits and utility filings already track the construction of known large facilities, but automated data-centre detection remains primarily conceptual [[S-1409]]. [[M-0007|Proofs of useful work]] would leave declared hardware little spare capacity, but cannot find a facility that was never declared [[S-1102]].\n\nChips sold before tracking began may not be locatable [[S-0029]], facilities can be hidden underground or camouflaged [[S-0062]], and it is unclear how small undeclared compute can be and still matter [[S-0002]] [[S-0053]].", "text": "No single verification method is foolproof for this broadest of negative claims [S-0062], so draft agreements pair technical measures with intelligence, challenge inspections and whistleblowers [S-0063]. The only mechanism mapped primarily to this claim, remote detection of data centres, is proposed (R1), as are the chip registries and location checks that support it. RAND splits the claim into undeclared use of declared clusters, which reduces to claims such as Declared hardware is idle or shut down and This compute runs inference, not training, and undeclared clusters [S-0002]. One strategy, which Scher and Thiergart favour, makes the declared stock complete from the start through chip registries and location verification [S-0005]. The other searches for what was missed, through remote detection of data centres and other national technical means [S-0062]. Satellite imagery, permits and utility filings already track the construction of known large facilities, but automated data-centre detection remains primarily conceptual [S-1409]. Proofs of useful work would leave declared hardware little spare capacity, but cannot find a facility that was never declared [S-1102]. Chips sold before tracking began may not be locatable [S-0029], facilities can be hidden underground or camouflaged [S-0062], and it is unclear how small undeclared compute can be and still matter [S-0002] [S-0053]." }, "sources": [ { "source": "S-0002", "supports": "Subgoal 2 (2.A, 2.B, 2.B.1, 2.B.2); focus on large-scale clusters; unclear whether dangerous deployment requires scale; personnel and intelligence layers", "locator": "§2.2; §3.2, Figure 4; §4" }, { "source": "S-0004", "supports": "existence easier to demonstrate than non-existence", "locator": "p. 31" }, { "source": "S-0029", "supports": "sampling fails if prover amasses untracked chips; existing chips possibly not locatable", "locator": "abstract; §5" }, { "source": "S-0005", "supports": "covert data centres may be hard to detect; tracking chips favoured; intelligence and whistleblowers as complements", "locator": "Verifying the location of AI compute (analysis)" }, { "source": "S-0062", "supports": "unauthorised data centres as a violation type; no single method foolproof; national technical means and their limitations and evasions", "locator": "What to verify; Table 1; Figures 2–4" }, { "source": "S-0063", "supports": "locating chips through supply-chain tracking, reporting, intelligence, OSINT, power monitoring, challenge inspections and whistleblowers", "locator": "§4; Articles V and X (as summarised)" }, { "source": "S-0053", "supports": "detectability of large facilities; low-compute narrow models; decentralised training; underground data centres raise cost", "locator": "properties of compute; limitations" }, { "source": "S-0014", "supports": "difficulty of verifying completeness of workload declarations and bounding unknown compute", "locator": "Limitations" }, { "source": "S-1409", "supports": "satellite imagery, permits and utility filings track known facilities; automated data-centre detection primarily conceptual", "locator": "Methodology; Opportunities for Further Research" }, { "source": "S-1102", "supports": "work-budget proposal; a proof cannot discover a datacenter that was never declared (provider proposal)", "locator": "blog post" }, { "source": "S-0060", "supports": "10^24, 10^25 and 10^26 FLOP thresholds evadable with $1.6M, $31M and $3.8B of hardware in sub-registration clusters; assumptions (DiLoCo-family training over 100 Mbps links, 16 H100-equivalents per node, about 740 days)", "locator": "§3.1; §4" } ], "concepts": [ "K-0016", "K-0003", "K-0020", "K-0023" ], "order": 10, "type": "claim", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/", "source_file": "content/claims/no-undeclared-compute.md", "flags_all": [], "body_markdown": "## Why it matters\nChecks on declared hardware say nothing about hardware that was never declared.\n\n- **RAND's framework** gives this its own goal: verifying that there are no undeclared uses of large-scale AI compute [[S-0002]]. It splits the goal into no undeclared use of declared clusters, and no undeclared clusters, whether inside known data centres or standalone [[S-0002]].\n- **Shavit's framework** depends on the same property. Without chip-ownership verification, a prover might covertly acquire a large quantity of chips and train on them without ever notifying the verifier [[S-0029]].\n- **Wasil and colleagues** list unauthorised data centres as one of two main violation types [[S-0062]].\n- **A draft international agreement** relies on a combination of supply-chain tracking, mandatory reporting, state intelligence, open-source intelligence, power monitoring, challenge inspections and whistleblowers to locate chips [[S-0063]].\n\n## Why it is hard\nThe claim asserts absence. The Oxford Martin report observes that demonstrating the existence of an object or process is often straightforward compared with demonstrating its non-existence [[S-0004]].\n\n- **Hidden facilities.** Scher and Thiergart judge that detecting covert data centres may be difficult, because AI compute may be hidden among other compute or in secret facilities [[S-0005]]. They see whistleblowers and intelligence as possible means of detection [[S-0005]]. Sastry and colleagues note that large training facilities are visible because of their size and power demands, and that hiding them underground would likely increase cost significantly [[S-0053]].\n- **Limits of each detection method.** Wasil and colleagues note that data centres could be concealed underground or camouflaged from satellite imagery, and that energy use can be disguised as other high-energy activity [[S-0062]]. Customs data is less useful against domestic chip production, and financial intelligence must separate illicit purchases from many legitimate ones [[S-0062]]. Whistleblowers may be deterred by fear of retaliation [[S-0062]].\n- **The existing stock.** Shavit notes that many chips already sold lack the security features his framework needs and may not be locatable by governments [[S-0029]]. A 2026 analysis of TEE-based monitoring notes the difficulty of verifying that workload declarations are complete and of forming tight bounds on unknown compute [[S-0014]].\n- **The threshold.** RAND's framework focuses on clusters with the computing power of thousands of high-end chips, while noting it is not clear that frontier AI deployment must happen at scale to be dangerous [[S-0002]]. Sastry and colleagues caution that low-compute narrow models can have dangerous capabilities, and that more viable decentralised training could undermine the detectability of compute [[S-0053]]. Rahman estimates that the 10^24, 10^25 and 10^26 FLOP thresholds could be evaded with $1.6 million, $31 million and $3.8 billion of hardware respectively, arranged in clusters smaller than any proposed registration requirement [[S-0060]]. The estimate assumes distributed training over consumer-grade internet, nodes of at most 16 H100-equivalents and a run of up to about two years [[S-0060]].", "body_text": "Why it matters Checks on declared hardware say nothing about hardware that was never declared. - RAND's framework gives this its own goal: verifying that there are no undeclared uses of large-scale AI compute [S-0002]. It splits the goal into no undeclared use of declared clusters, and no undeclared clusters, whether inside known data centres or standalone [S-0002]. - Shavit's framework depends on the same property. Without chip-ownership verification, a prover might covertly acquire a large quantity of chips and train on them without ever notifying the verifier [S-0029]. - Wasil and colleagues list unauthorised data centres as one of two main violation types [S-0062]. - A draft international agreement relies on a combination of supply-chain tracking, mandatory reporting, state intelligence, open-source intelligence, power monitoring, challenge inspections and whistleblowers to locate chips [S-0063]. Why it is hard The claim asserts absence. The Oxford Martin report observes that demonstrating the existence of an object or process is often straightforward compared with demonstrating its non-existence [S-0004]. - Hidden facilities. Scher and Thiergart judge that detecting covert data centres may be difficult, because AI compute may be hidden among other compute or in secret facilities [S-0005]. They see whistleblowers and intelligence as possible means of detection [S-0005]. Sastry and colleagues note that large training facilities are visible because of their size and power demands, and that hiding them underground would likely increase cost significantly [S-0053]. - Limits of each detection method. Wasil and colleagues note that data centres could be concealed underground or camouflaged from satellite imagery, and that energy use can be disguised as other high-energy activity [S-0062]. Customs data is less useful against domestic chip production, and financial intelligence must separate illicit purchases from many legitimate ones [S-0062]. Whistleblowers may be deterred by fear of retaliation [S-0062]. - The existing stock. Shavit notes that many chips already sold lack the security features his framework needs and may not be locatable by governments [S-0029]. A 2026 analysis of TEE-based monitoring notes the difficulty of verifying that workload declarations are complete and of forming tight bounds on unknown compute [S-0014]. - The threshold. RAND's framework focuses on clusters with the computing power of thousands of high-end chips, while noting it is not clear that frontier AI deployment must happen at scale to be dangerous [S-0002]. Sastry and colleagues caution that low-compute narrow models can have dangerous capabilities, and that more viable decentralised training could undermine the detectability of compute [S-0053]. Rahman estimates that the 10^24, 10^25 and 10^26 FLOP thresholds could be evaded with $1.6 million, $31 million and $3.8 billion of hardware respectively, arranged in clusters smaller than any proposed registration requirement [S-0060]. The estimate assumes distributed training over consumer-grade internet, nodes of at most 16 H100-equivalents and a run of up to about two years [S-0060].", "addressed_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/", "role": "supporting", "note": "Can flag enrolled chips that stop responding or answer from outside declared regions; says nothing about chips outside the scheme." }, { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/", "role": "supporting", "note": "Supports checks that recorded chips have not been assembled into undeclared clusters." }, { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/", "role": "supporting", "note": "Unreported batch elements alter the numerics, so covert computation inside batches becomes detectable (S-0020)." }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/", "role": "supporting", "note": "Contention from undeclared co-running workloads shifts challenge timing (S-0033); coverage of other chips needs other mechanisms." }, { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/", "role": "supporting", "note": "On declared hardware only: bounds capacity left for unmonitored work; cannot find undeclared facilities." }, { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/", "role": "primary", "note": "Searches for large facilities that have not been declared." }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/", "role": "supporting", "note": "Memory challenges and resource accounting are proposed against hidden workloads." }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/", "role": "supporting", "note": "On declared hardware only; cannot find undeclared facilities. Not applied to bounding spare capacity." } ], "referenced_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" }, { "id": "C-0001", "title": "Compute stock is at most a declared amount", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0003", "title": "Positive and negative claims", "url": "https://trustbutveri.fyi/concepts/positive-and-negative-claims/" } ] }, { "id": "C-0006", "slug": "safeguards-were-applied", "title": "Declared safeguards were applied during inference", "aliases": [ "safeguard verification", "deployment mitigations" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Specified safety measures, such as input filters, output checks or monitoring, actually ran on the requests a deployed model served.", "summary": "The claim is that specified safety measures, such as input filters, output checks or monitoring for misuse, actually ran on the requests a deployed model served. Rules for deploying AI systems often require such safeguards. A developer's statement that safeguards exist does not show that they ran on every request, or that the version checked is the one in production. Verifying application matters to regulators and to agreements that allow deployment on condition of mitigations. It is a positive claim, but it inherits the difficulties of verifying the served model and adds more. Safeguards are often separate software components whose configuration can change, and the verifier usually cannot see users' prompts or the provider's systems. Proposals use trusted execution environments to attest the serving stack, audits run inside such environments, and inspections. Whether a safeguard is effective is a separate question from whether it was applied.", "claim_class": "positive", "editors_synthesis": { "assessment": true, "markdown": "Safeguard application has been shown only in small research prototypes, which can show that a declared safeguard ran, not that it works.\n\n[[M-0023|Safeguard attestation]] (R2) runs the safeguard inside a trusted execution environment, whose hardware signs a measurement of its code with a commitment to each input and response [[S-1500]]. It builds on [[M-0008|TEE remote attestation]] (R3) and on knowing which model is served ([[C-0005]]). The proof-of-guardrail prototype demonstrates it on CPU enclaves in AWS and calls its guardrail model through an external API [[S-1500]]. In the authors' tests it detected modified guardrail code, attestations and responses [[S-1500]]. [[M-0025|Confidential multi-party verification]] (R2) can limit monitoring to a plan both parties sign [[S-1503]], and Auditor-in-a-Box demonstrates such plan-scoped monitoring, though its authors state that the demo's user data and plan execution are not actually secure [[S-1504]].\n\nEvery component that influences inference must be covered by the launch measurement [[S-0014]], and the prototype attests only the responses for which it offers attestation, so coverage of all traffic is not shown [[S-1500]]. Where hardware trust is unavailable, the sources fall back on inspections, audits and personnel-based layers [[S-0062]] [[S-0003]] [[S-0002]].", "text": "Safeguard application has been shown only in small research prototypes, which can show that a declared safeguard ran, not that it works. Safeguard attestation (R2) runs the safeguard inside a trusted execution environment, whose hardware signs a measurement of its code with a commitment to each input and response [S-1500]. It builds on TEE remote attestation (R3) and on knowing which model is served (The declared model is the one being served). The proof-of-guardrail prototype demonstrates it on CPU enclaves in AWS and calls its guardrail model through an external API [S-1500]. In the authors' tests it detected modified guardrail code, attestations and responses [S-1500]. Confidential multi-party verification (R2) can limit monitoring to a plan both parties sign [S-1503], and Auditor-in-a-Box demonstrates such plan-scoped monitoring, though its authors state that the demo's user data and plan execution are not actually secure [S-1504]. Every component that influences inference must be covered by the launch measurement [S-0014], and the prototype attests only the responses for which it offers attestation, so coverage of all traffic is not shown [S-1500]. Where hardware trust is unavailable, the sources fall back on inspections, audits and personnel-based layers [S-0062] [S-0003] [S-0002]." }, "sources": [ { "source": "S-0002", "supports": "deployment mitigations specified by inputs and outputs (filters, oversight checks); difficulty of choosing technical rules; whistleblower and interview layers", "locator": "Table 3; §4" }, { "source": "S-0062", "supports": "AI developer inspections to check authorised code and implementation of evaluations and safeguards; software can be quickly modified or hidden", "locator": "Access-dependent methods; Table 1" }, { "source": "S-0003", "supports": "third-party verification of developers' safety and security claims with deep, secure access; AI Assurance Levels", "locator": "abstract" }, { "source": "S-0014", "supports": "TEEs for policy-adherent inference; launch measurement must cover all components; attestation-key holder can forge reports", "locator": "main argument; Limitations" }, { "source": "S-0009", "supports": "TEE-based verifiable benchmarks keeping model IP and datasets confidential; prototype with Llama-3.1", "locator": "abstract" }, { "source": "S-0012", "supports": "property attestation across training and inference on TDX + H100; formal model in Tamarin", "locator": "abstract" }, { "source": "S-0018", "supports": "output cross-checks; inspector agents vulnerable to prompt injection", "locator": "architecture; open problems" }, { "source": "S-1500", "supports": "proof-of-guardrail protocol, prototype on AWS Nitro Enclaves with an external guardrail API, tamper tests, opt-in attestation, jailbreak risk", "locator": "abstract; §3; §4.1; Table 1; Appendix A" }, { "source": "S-1503", "supports": "monitoring scoped to a jointly signed plan and executed in an attested TEE", "locator": "abstract" }, { "source": "S-1504", "supports": "Auditor-in-a-Box demo of plan-scoped monitoring; authors state user data and plan execution are not actually secure and the demo has not been stress-tested by a counterparty", "locator": "whole post" } ], "concepts": [ "K-0006", "K-0004", "K-0019", "K-0005" ], "order": 6, "type": "claim", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/", "source_file": "content/claims/safeguards-were-applied.md", "flags_all": [], "body_markdown": "## Why it matters\nRules on deployment often take the form of required mitigations.\n\n- RAND gives the example of deployment mitigations specified in terms of input and output data, such as filtering out some kinds of inputs or running oversight checks on outputs [[S-0002]]. Proposals for frontier AI auditing call for rigorous third-party verification of developers' safety and security claims, grounded in deep, secure access to non-public information [[S-0003]].\n- Wasil and colleagues' verification methods for international AI agreements include inspections of AI developers to check that they run authorised code and properly implement model evaluations and safeguards [[S-0062]].\n- A 2026 analysis of privacy-preserving monitoring argues that trusted execution environments could let an auditor check that deployed software follows an agreed policy, with access control enforced so that neither the service provider nor the auditor needs to pair users' identities with their data [[S-0014]].\n\n## Why it is hard\n- **Knowing what software is serving.** With TEEs, hardware attestation can show that specified software is running unaltered, but only for what the launch measurement covers [[S-0014]]. The same analysis notes that every component influencing inference behaviour must be covered, and that incomplete specifications could let an adversary undermine deployment integrity [[S-0014]]. It also notes that whoever holds the hardware's attestation key can produce valid reports, and that side-channel and physical attacks remain concerns [[S-0014]].\n- **Confidentiality.** [[I-0007|Attestable Audits]] runs benchmarks inside TEEs so that results are verifiable while model weights and test data stay confidential, with a prototype tested on Llama-3.1 [[S-0009]]. PAL\\*M defines properties across training and inference and attests them using confidential virtual machines with GPU support [[S-0012]]. Its authors model the protocol formally with the Tamarin prover [[S-0012]].\n- **Software changes quickly.** Wasil and colleagues note that code can be rapidly modified or hidden, which limits what periodic inspections of developers can show [[S-0062]]. Automated inspection has its own weaknesses: a low-trust system overview notes that inspector agents may be vulnerable to prompt injection and that their classification precision is uncertain [[S-0018]].\n- **Application is not effectiveness.** Proof-of-guardrail's authors note that an attested guardrail can still err or be jailbroken [[S-1500]]. RAND notes that deciding which technical rules actually achieve policy objectives remains difficult [[S-0002]]. Personnel-based layers such as whistleblower programmes and interviews are proposed for cases where technical evidence is unavailable [[S-0002]].", "body_text": "Why it matters Rules on deployment often take the form of required mitigations. - RAND gives the example of deployment mitigations specified in terms of input and output data, such as filtering out some kinds of inputs or running oversight checks on outputs [S-0002]. Proposals for frontier AI auditing call for rigorous third-party verification of developers' safety and security claims, grounded in deep, secure access to non-public information [S-0003]. - Wasil and colleagues' verification methods for international AI agreements include inspections of AI developers to check that they run authorised code and properly implement model evaluations and safeguards [S-0062]. - A 2026 analysis of privacy-preserving monitoring argues that trusted execution environments could let an auditor check that deployed software follows an agreed policy, with access control enforced so that neither the service provider nor the auditor needs to pair users' identities with their data [S-0014]. Why it is hard - Knowing what software is serving. With TEEs, hardware attestation can show that specified software is running unaltered, but only for what the launch measurement covers [S-0014]. The same analysis notes that every component influencing inference behaviour must be covered, and that incomplete specifications could let an adversary undermine deployment integrity [S-0014]. It also notes that whoever holds the hardware's attestation key can produce valid reports, and that side-channel and physical attacks remain concerns [S-0014]. - Confidentiality. Attestable Audits runs benchmarks inside TEEs so that results are verifiable while model weights and test data stay confidential, with a prototype tested on Llama-3.1 [S-0009]. PAL*M defines properties across training and inference and attests them using confidential virtual machines with GPU support [S-0012]. Its authors model the protocol formally with the Tamarin prover [S-0012]. - Software changes quickly. Wasil and colleagues note that code can be rapidly modified or hidden, which limits what periodic inspections of developers can show [S-0062]. Automated inspection has its own weaknesses: a low-trust system overview notes that inspector agents may be vulnerable to prompt injection and that their classification precision is uncertain [S-0018]. - Application is not effectiveness. Proof-of-guardrail's authors note that an attested guardrail can still err or be jailbroken [S-1500]. RAND notes that deciding which technical rules actually achieve policy objectives remains difficult [S-0002]. Personnel-based layers such as whistleblower programmes and interviews are proposed for cases where technical evidence is unavailable [S-0002].", "addressed_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/", "role": "supporting", "note": "Plan-scoped monitoring runs an agreed classifier over private usage records (S-1503)." }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/", "role": "supporting", "note": "Could require deployment-time safeguards on approved devices (S-0035)." }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/", "role": "supporting", "note": "Links an attested evaluation to the model later served (S-0009)." }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/", "role": "primary", "note": "Attests that a measured safeguard program (guardrail, filter, monitor) mediated the attested responses; coverage of all traffic is not established." }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/", "role": "supporting", "note": "Can attest that measured policy software, such as filters and logging, wrapped the model (S-0014). The evaluation variant is I-0007." }, { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/", "role": "supporting", "note": "Attestable proposes that a proof could show an agreed input classifier was applied; South et al. prove evaluation results." }, { "id": "I-0007", "title": "Attestable Audits", "url": "https://trustbutveri.fyi/implementations/attestable-audits/", "role": "supporting", "note": "Attests that declared safety benchmarks were run on that model and what they scored; it does not attest runtime safeguards." }, { "id": "I-0005", "title": "Attestable zero-knowledge inference prover", "url": "https://trustbutveri.fyi/implementations/attestable-zk-inference/", "role": "supporting", "note": "Proposed use: showing an agreed input classifier was applied." }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/", "role": "supporting", "note": "Screening checks that outputs are free of blacklisted uses, including with inspector agents." } ], "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0007", "title": "Attestable Audits", "url": "https://trustbutveri.fyi/implementations/attestable-audits/" }, { "id": "I-0005", "title": "Attestable zero-knowledge inference prover", "url": "https://trustbutveri.fyi/implementations/attestable-zk-inference/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" } ] }, { "id": "C-0007", "slug": "training-within-declared-limits", "title": "A training run stayed within declared limits", "aliases": [ "training compute limits", "proof of training" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "A declared training run used no more compute than permitted and had its declared properties, such as data, hyperparameters and resulting weights.", "summary": "The claim is that a declared training run used no more compute than permitted and had its declared properties, such as its data, hyperparameters and resulting weights. Many proposed rules on frontier AI limit training, often through a threshold on total training compute measured in floating-point operations, sometimes combined with limits on data or methods. Verifying that a run stayed within them would let a regulator or treaty partner rely on something other than self-reporting. It is hard because training is long, distributed and expensive to repeat. Exact replay is complicated by numerical noise, the data and weights are sensitive, and work could be split across declared and undeclared hardware. Proposals include on-chip logging of weight snapshots with sampled recomputation of training segments, proof-of-learning protocols, zero-knowledge proofs and telemetry-based accounting. Published attacks show that the original proof-of-learning protocol can be spoofed.", "claim_class": "mixed", "editors_synthesis": { "assessment": true, "markdown": "Proof-based checks of training remain far below frontier-scale runs. Proof-of-learning and zero-knowledge proofs of training are both demonstrated (R2); hardware enforcement remains proposed (R1).\n\n[[M-0006|Proof-of-learning and transcript verification]] (R2) re-runs sampled training segments between logged weight snapshots [[S-0029]], and [[M-0005|zero-knowledge proofs of training]] (R2) prove that training followed a committed specification without revealing data or weights [[S-1110]]. [[M-0009|Guarantee processors such as flexHEG]] and [[M-0011|licensing]] (both R1) could enforce compute limits in hardware [[S-0035]] [[S-0057]].\n\nChoi and colleagues report proof-of-training-data experiments on language models of up to 1 billion parameters [[S-0030]], and Kaizen proves training iterations of a 10-million-parameter image model at about 15 minutes of proving per iteration [[S-1110]]. VeriLoRA proves individual low-rank fine-tuning steps on language models of up to 13 billion parameters [[S-3080]].\n\nPublished attacks spoof the original proof-of-learning protocol, and the attack's authors argue that a provably robust version needs a better understanding of deep-learning optimisation [[S-0027]]. Governance analyses have judged zero-knowledge proofs impractical at frontier scale, and one 2026 proposal argues this is a limit of current approaches, not a fundamental one [[S-0025]]. A compute limit bounds a run only if all the chips used are known, so this claim depends on [[C-0001]] and [[C-0010]].", "text": "Proof-based checks of training remain far below frontier-scale runs. Proof-of-learning and zero-knowledge proofs of training are both demonstrated (R2); hardware enforcement remains proposed (R1). Proof-of-learning and transcript verification (R2) re-runs sampled training segments between logged weight snapshots [S-0029], and zero-knowledge proofs of training (R2) prove that training followed a committed specification without revealing data or weights [S-1110]. Guarantee processors such as flexHEG and licensing (both R1) could enforce compute limits in hardware [S-0035] [S-0057]. Choi and colleagues report proof-of-training-data experiments on language models of up to 1 billion parameters [S-0030], and Kaizen proves training iterations of a 10-million-parameter image model at about 15 minutes of proving per iteration [S-1110]. VeriLoRA proves individual low-rank fine-tuning steps on language models of up to 13 billion parameters [S-3080]. Published attacks spoof the original proof-of-learning protocol, and the attack's authors argue that a provably robust version needs a better understanding of deep-learning optimisation [S-0027]. Governance analyses have judged zero-knowledge proofs impractical at frontier scale, and one 2026 proposal argues this is a limit of current approaches, not a fundamental one [S-0025]. A compute limit bounds a run only if all the chips used are known, so this claim depends on Compute stock is at most a declared amount and There is no undeclared relevant compute." }, "sources": [ { "source": "S-0029", "supports": "goal and example rules; weight snapshots; sampled segment recomputation; reasons full re-run is infeasible; PoTT harder than PoL; open problems (online RL, thresholds)", "locator": "abstract; §2–§3; open problems" }, { "source": "S-0053", "supports": "EO 14110 threshold; compute as a high-level proxy; thresholds must change with progress", "locator": "thresholds; limitations" }, { "source": "S-0069", "supports": "revocation of EO 14110 on 20 January 2025", "locator": "Sec. 2(ggg)" }, { "source": "S-0063", "supports": "training above 10^24 FLOP prohibited; runs above 10^22 FLOP approved and monitored", "locator": "§4" }, { "source": "S-0062", "supports": "unauthorised training above a FLOP threshold as a violation type", "locator": "What to verify" }, { "source": "S-0002", "supports": "Subgoal 1.A.1 verifying declared training", "locator": "§3.2" }, { "source": "S-0025", "supports": "enforcement rests on self-reporting; governance analyses judge ZKPs impractical at frontier scale, which the authors argue is paradigm-bound; proposed architecture; ~36-month estimate", "locator": "abstract" }, { "source": "S-0027", "supports": "reproducible PoL spoofing at a fraction of prior cost; provably robust PoL requires advances in understanding deep-learning optimisation", "locator": "abstract" }, { "source": "S-0018", "supports": "monitoring training needs larger perimeters or compute-fabric taps; back-end traffic harder to capture", "locator": "§5.1.1; inference vs training" }, { "source": "S-0035", "supports": "flexHEG compute limits for training", "locator": "abstract; Executive Summary" }, { "source": "S-0057", "supports": "offline licensing with a compute budget", "locator": "p. viii" }, { "source": "S-0030", "supports": "training-data verification experiments on GPT-2 and Pythia models up to 1B parameters", "locator": "§4" }, { "source": "S-1110", "supports": "Kaizen zero-knowledge proofs of training; VGG-11 (10M parameters) at about 15 minutes per iteration", "locator": "abstract; evaluation" }, { "source": "S-3080", "supports": "VeriLoRA proofs of individual LoRA fine-tuning steps on language models up to 13B parameters", "locator": "§VI-B–VI-D" }, { "source": "S-3542", "supports": "Art. 51(2) presumption of high-impact capabilities above 10^25 FLOP; Art. 52(1) notification within two weeks; Art. 113(b) Chapter V applies from 2 August 2025", "locator": "Arts 51, 52, 113" }, { "source": "S-3543", "supports": "10^25 FLOP presumption and two-week notification; entry into application on 2 August 2025", "locator": "§2.3.1–2.3.2; landing page" }, { "source": "S-3544", "supports": "frontier model defined by more than 10^26 integer or floating-point operations including fine-tuning; transparency duties", "locator": "§22757.11(i); §22757.12" }, { "source": "S-0059", "supports": "compute structuring: splitting or modifying workloads to avoid regulation", "locator": "abstract" } ], "concepts": [ "K-0023", "K-0009", "K-0020", "K-0010", "K-0024", "K-0008" ], "order": 7, "type": "claim", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/", "source_file": "content/claims/training-within-declared-limits.md", "flags_all": [], "body_markdown": "## Why it matters\nLimits on training are a common rule in compute-governance proposals [[S-0062]].\n\n- **Shavit's framework** aims to give governments high confidence that no actor uses large quantities of specialised ML chips for a training run that violates agreed rules [[S-0029]]. Its example rules include total training compute, properties of the training data, hyperparameters such as the share of reinforcement-learning steps, and combinations of these with benchmark performance [[S-0029]].\n- **US Executive Order 14110** required reporting for models trained with more than 10^26 operations [[S-0053]]. It was revoked in January 2025 [[S-0069]].\n- **The EU AI Act and California's SB 53** both set training-compute thresholds. The AI Act presumes that a general-purpose AI model trained with more than 10^25 FLOP has high-impact capabilities, one of the conditions for classing it as a model with systemic risk [[S-3542]]. Its provider must notify the European Commission within two weeks of meeting the threshold or learning that it will, an obligation that has applied since 2 August 2025 [[S-3542]] [[S-3543]]. SB 53, enacted in September 2025, sets transparency duties for developers of frontier models, defined as models trained with more than 10^26 integer or floating-point operations, including later fine-tuning and other material modifications [[S-3544]].\n- **A draft international agreement** prohibits training runs above 10^24 FLOP, and requires runs above 10^22 FLOP to be approved and monitored [[S-0063]].\n- **Wasil and colleagues** treat training above an agreed FLOP threshold as one of two main violation types to be detected [[S-0062]]. RAND lists verifying declared training as its own subgoal [[S-0002]].\n\nWithout technical verification, such limits rest on self-reporting, a gap the authors of a 2026 zero-knowledge proposal identify in current frontier AI governance frameworks [[S-0025]].\n\n## Why it is hard\n- **Training cannot simply be re-run.** Shavit gives three reasons: hardware noise makes training irreproducible, the compute cost is very large, and the training data may be confidential [[S-0029]]. His framework instead has chips save weight snapshots at random times, and the verifier re-runs short segments between reported checkpoints, accepting results within a tolerance [[S-0029]].\n- **Transcripts can be spoofed.** Shavit notes that proving a transcript is the original one is harder than proof-of-learning, because the adversary knows the true transcript and has more compute time to construct a spoof [[S-0029]]. Fang and colleagues introduced spoofing strategies against proof-of-learning that reproduce across verification configurations at a fraction of the cost of earlier attacks [[S-0027]]. They conclude that a provably robust proof-of-learning mechanism cannot be built without further understanding of optimisation in deep learning [[S-0027]].\n- **Coverage.** A limit on one run means little if other hardware contributes undeclared work, so Shavit pairs run verification with supply-chain monitoring [[S-0029]]. A developer could also split a run into workloads that each stay below a threshold, which Seferis and Fist call compute structuring [[S-0059]]. Training traffic runs over high-bandwidth back-end fabric that is harder to capture than inference traffic [[S-0018]].\n- **Open design problems.** Shavit lists extending pre-commitment to online reinforcement learning, where training data cannot be known in advance, and choosing principled thresholds when scaling laws do not reliably predict downstream capabilities [[S-0029]].\n- **Cryptographic cost.** A 2026 proposal notes that governance analyses judge zero-knowledge proofs currently impractical at frontier scale, and argues that this is a limit of current approaches rather than a fundamental one [[S-0025]]. It sets out an architecture combining a pre-committed training specification, network observations and Merkle commitments, and estimates a proof of concept within about 36 months [[S-0025]].\n- **Compute is a proxy.** Training compute is only a high-level proxy for capability, and thresholds may need to change as algorithms improve [[S-0053]].", "body_text": "Why it matters Limits on training are a common rule in compute-governance proposals [S-0062]. - Shavit's framework aims to give governments high confidence that no actor uses large quantities of specialised ML chips for a training run that violates agreed rules [S-0029]. Its example rules include total training compute, properties of the training data, hyperparameters such as the share of reinforcement-learning steps, and combinations of these with benchmark performance [S-0029]. - US Executive Order 14110 required reporting for models trained with more than 10^26 operations [S-0053]. It was revoked in January 2025 [S-0069]. - The EU AI Act and California's SB 53 both set training-compute thresholds. The AI Act presumes that a general-purpose AI model trained with more than 10^25 FLOP has high-impact capabilities, one of the conditions for classing it as a model with systemic risk [S-3542]. Its provider must notify the European Commission within two weeks of meeting the threshold or learning that it will, an obligation that has applied since 2 August 2025 [S-3542] [S-3543]. SB 53, enacted in September 2025, sets transparency duties for developers of frontier models, defined as models trained with more than 10^26 integer or floating-point operations, including later fine-tuning and other material modifications [S-3544]. - A draft international agreement prohibits training runs above 10^24 FLOP, and requires runs above 10^22 FLOP to be approved and monitored [S-0063]. - Wasil and colleagues treat training above an agreed FLOP threshold as one of two main violation types to be detected [S-0062]. RAND lists verifying declared training as its own subgoal [S-0002]. Without technical verification, such limits rest on self-reporting, a gap the authors of a 2026 zero-knowledge proposal identify in current frontier AI governance frameworks [S-0025]. Why it is hard - Training cannot simply be re-run. Shavit gives three reasons: hardware noise makes training irreproducible, the compute cost is very large, and the training data may be confidential [S-0029]. His framework instead has chips save weight snapshots at random times, and the verifier re-runs short segments between reported checkpoints, accepting results within a tolerance [S-0029]. - Transcripts can be spoofed. Shavit notes that proving a transcript is the original one is harder than proof-of-learning, because the adversary knows the true transcript and has more compute time to construct a spoof [S-0029]. Fang and colleagues introduced spoofing strategies against proof-of-learning that reproduce across verification configurations at a fraction of the cost of earlier attacks [S-0027]. They conclude that a provably robust proof-of-learning mechanism cannot be built without further understanding of optimisation in deep learning [S-0027]. - Coverage. A limit on one run means little if other hardware contributes undeclared work, so Shavit pairs run verification with supply-chain monitoring [S-0029]. A developer could also split a run into workloads that each stay below a threshold, which Seferis and Fist call compute structuring [S-0059]. Training traffic runs over high-bandwidth back-end fabric that is harder to capture than inference traffic [S-0018]. - Open design problems. Shavit lists extending pre-commitment to online reinforcement learning, where training data cannot be known in advance, and choosing principled thresholds when scaling laws do not reliably predict downstream capabilities [S-0029]. - Cryptographic cost. A 2026 proposal notes that governance analyses judge zero-knowledge proofs currently impractical at frontier scale, and argues that this is a limit of current approaches rather than a fundamental one [S-0025]. It sets out an architecture combining a pre-committed training specification, network observations and Merkle commitments, and estimates a proof of concept within about 36 months [S-0025]. - Compute is a proxy. Training compute is only a high-level proxy for capability, and thresholds may need to change as algorithms improve [S-0053].", "addressed_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/", "role": "supporting", "note": "Bounds the size of model that can be trained efficiently across pods (S-1301)." }, { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/", "role": "supporting", "note": "Zero-knowledge audits can prove properties of committed training data and weights (S-0022)." }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/", "role": "primary", "note": "Verifiable claims about total training compute, and enforcement of compute thresholds (S-0035)." }, { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/", "role": "supporting", "note": "Licenses that authorize a fixed amount of work would bound compute per license period (S-0057, S-0006)." }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/", "role": "supporting", "note": "Counters for FLOP, memory and interconnect traffic are proposed as meters for compute accounting (S-0057, S-0006)." }, { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/", "role": "primary", "note": "Transcript checks for rules on training compute, data and hyperparameters (Shavit; Choi et al.)." }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/", "role": "supporting", "note": "Proposed for later R&D verification by treating training steps as packets (S-0067, S-0017)." }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/", "role": "supporting", "note": "PAL*M attests single-node training and fine-tuning operations (S-0012). Distributed training is left open." }, { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/", "role": "supporting", "note": "Can flag training on hardware declared for other uses; does not measure training size by itself." }, { "id": "M-0005", "title": "Zero-knowledge proofs of training constraints", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-training-constraints/", "role": "primary", "note": "Proves training followed a committed specification and data; the frontier design adds compute-threshold attestations." }, { "id": "I-0015", "title": "Verde and RepOps (Gensyn)", "url": "https://trustbutveri.fyi/implementations/gensyn-verde-repops/", "role": "supporting", "note": "Also covers training and fine-tuning jobs delegated to several providers (S-1809)." } ], "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" }, { "id": "M-0005", "title": "Zero-knowledge proofs of training constraints", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-training-constraints/" }, { "id": "I-0015", "title": "Verde and RepOps (Gensyn)", "url": "https://trustbutveri.fyi/implementations/gensyn-verde-repops/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" } ] }, { "id": "C-0009", "slug": "weights-have-not-left", "title": "Model weights have not left the facility", "aliases": [ "no weight exfiltration", "Model weights or data have not left the facility" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "No copy of specified model weights has left a designated facility through networks, physical media or other channels.", "summary": "The claim is that no copy of specified model weights has left a designated facility through networks, physical media or other channels. Agreements and security commitments may require this, for example against theft by well-resourced attackers. Verification itself creates a related concern, since monitoring equipment could leak the prover's secrets. It is a negative claim over every channel: network links, physical media, people and physical side channels. Approaches include checking that outgoing traffic is explained by legitimate computation so that hidden data stands out, capping the bandwidth of external links, suppressing side channels, and physical security with inspection. At best they bound how much could leave. Checking inference outputs against a reference cut the information an attacker could hide in responses to under 0.5% in one test; in an independent study, an attacker who controlled the prompts roughly doubled the leak. Physical and insider channels remain harder to bound.", "claim_class": "negative", "editors_synthesis": { "assessment": true, "markdown": "Only the network channel has a demonstrated (R2) bound, and only for inference outputs. Physical side channels rely on proposed (R1) designs, and insiders and physical media are left to facility security [[S-1610]] and personnel measures such as whistleblowing [[S-0002]].\n\nFor example, a verifier may want to know that a facility serving a model through an API has not let the weights out inside its responses.\n\n- [[M-0013|Network taps]] (R1) record what the facility sends, and [[M-0001|sampled recomputation]] (R3) re-runs sampled requests on a trusted copy of the model with the same random seed [[S-0015]]. [[M-0024|Bounding unexplained information]] (R2) then scores how plausible each returned token is under honest sampling, which bounds how fast hidden bits can pass unnoticed [[S-0015]].\n- Rinberg and colleagues published code for this check. On a 30-billion-parameter model under benign prompts, it cut the information an attacker could hide to under 0.5%, at a false-positive rate below 0.01% [[S-0015]].\n- This establishes a rate limit on one channel, not that no copy left. In an independent study, an attacker who controlled the prompts roughly doubled the leak per token [[S-1507]]. One estimate still leaves about 40 Mbit/s of covert output for a 200,000-GPU inference cluster at full load, assuming about 0.1 bits per token survive replay checks [[S-1300]]. [[M-0002|Deterministic inference]] (R3) would remove the tolerance that hidden data exploits [[S-0020]].\n- The bound holds only if checked outputs are the only way out. Caps on external bandwidth ([[C-0008]]) limit the total that can leave [[S-1508]]. [[M-0022|Side-channel suppression]] (R1) is designed to hold physical leakage below kilobits per second, not to zero [[S-0018]]. [[I-0010|RAND's secure inference data centre]] (R1) is designed to protect weights against state-backed attackers but does not describe how an outside party would verify it [[S-1510]].", "text": "Only the network channel has a demonstrated (R2) bound, and only for inference outputs. Physical side channels rely on proposed (R1) designs, and insiders and physical media are left to facility security [S-1610] and personnel measures such as whistleblowing [S-0002]. For example, a verifier may want to know that a facility serving a model through an API has not let the weights out inside its responses. - Network taps (R1) record what the facility sends, and sampled recomputation (R3) re-runs sampled requests on a trusted copy of the model with the same random seed [S-0015]. Bounding unexplained information (R2) then scores how plausible each returned token is under honest sampling, which bounds how fast hidden bits can pass unnoticed [S-0015]. - Rinberg and colleagues published code for this check. On a 30-billion-parameter model under benign prompts, it cut the information an attacker could hide to under 0.5%, at a false-positive rate below 0.01% [S-0015]. - This establishes a rate limit on one channel, not that no copy left. In an independent study, an attacker who controlled the prompts roughly doubled the leak per token [S-1507]. One estimate still leaves about 40 Mbit/s of covert output for a 200,000-GPU inference cluster at full load, assuming about 0.1 bits per token survive replay checks [S-1300]. Deterministic inference (R3) would remove the tolerance that hidden data exploits [S-0020]. - The bound holds only if checked outputs are the only way out. Caps on external bandwidth (Communication between compute groups is bounded) limit the total that can leave [S-1508]. Side-channel suppression (R1) is designed to hold physical leakage below kilobits per second, not to zero [S-0018]. RAND's secure inference data centre (R1) is designed to protect weights against state-backed attackers but does not describe how an outside party would verify it [S-1510]." }, "sources": [ { "source": "S-1610", "supports": "importance of protecting frontier weights; 38 attack vectors; five security levels; adversaries up to nation-states; comprehensive defences", "locator": "summary" }, { "source": "S-0015", "supports": "steganographic exfiltration via inference server responses; security game; fixed-seed re-run and token-plausibility check; code release; <0.5% exfiltratable information at <0.01% FPR on MoE-Qwen-30B under benign prompts; >200x adversary slowdown", "locator": "§1 Contributions; §5–7" }, { "source": "S-1507", "supports": "prompt-controlling adversary roughly doubles bits leaked per token; slowdown falls to 60–118x", "locator": "abstract; results" }, { "source": "S-0005", "supports": "strong security to keep weights in a data centre plus close monitoring", "locator": "data-centre security discussion" }, { "source": "S-0018", "supports": "verifier aims to exfiltrate prover secrets; only commitments leave the facility; egress explainable by ingress; memory wiping; side-channel target", "locator": "threat model; architecture; open problems" }, { "source": "S-0038", "supports": "physical side channels can bypass network monitoring; defences", "locator": "channels of concern; defences" }, { "source": "S-0002", "supports": "confidentiality as protecting models, data and code from theft; whistleblower and interview layers", "locator": "§1; §4" }, { "source": "S-0031", "supports": "active taps scrubbing headers against covert channels on front-end links", "locator": "frontend vs backend" }, { "source": "S-1508", "supports": "egress limits cap what can be stolen", "locator": "§5.1" }, { "source": "S-0020", "supports": "approximate output matching leaves degrees of freedom, including steganography, that covert adversaries can exploit", "locator": "abstract" }, { "source": "S-1510", "supports": "secure inference data centre design against state-backed attackers; no external verification path described", "locator": "Summary; ch. 1" }, { "source": "S-1300", "supports": "residual covert egress of about 40 Mbit/s for a 200k-GPU inference cluster at about 0.1 bits per token after replay checks", "locator": "§5.2" } ], "concepts": [ "K-0022", "K-0013", "K-0021", "K-0014", "K-0012" ], "order": 9, "type": "claim", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/", "source_file": "content/claims/weights-have-not-left.md", "flags_all": [], "body_markdown": "## Why it matters\nModel weights are a primary target for theft [[S-1610]].\n\n- RAND researchers write that protecting frontier models from theft and misuse will become more important as they grow more capable [[S-1610]]. They identify 38 distinct attack vectors, and define five security levels for defending against actors ranging from opportunistic criminals to well-resourced nation-states [[S-1610]].\n- For verifying international agreements, Scher and Thiergart describe one approach as using strong security to prevent model weights from leaving a data centre, and then monitoring that data centre closely [[S-0005]].\n- RAND's framework treats confidentiality, meaning protection of models, data and code from unauthorised access and especially theft, as a requirement on verification [[S-0002]]. A low-trust system overview assumes the verifier may try to exfiltrate the prover's secrets [[S-0018]]. It therefore sends only cryptographic commitments out of the facility, and checks challenged data only in an auditing environment inside the prover's facility [[S-0018]].\n\n## Why it is hard\nThe claim must hold for every channel. The RAND weight-security study's 38 attack vectors span far more than network transfer [[S-1610]].\n\n- **Hidden data in ordinary outputs.** An attacker who controls an inference server could hide weights inside ordinary model responses using steganography [[S-0015]]. Rinberg and colleagues formalise this as a security game and verify outputs against a reference [[S-0015]]. On a 30-billion-parameter mixture-of-experts model, their detector reduced exfiltratable information to under 0.5% at a false-positive rate below 0.01%, which they equate to a slowdown of more than 200 times for an adversary [[S-0015]]. An independent study found that an attacker who chooses the prompts roughly doubles the bits leaked per token, cutting the slowdown to 60–118 times [[S-1507]].\n- **Capping egress.** Rinberg and colleagues note that egress limits cap theft directly: if only 10 GB leaves a data centre, an adversary cannot steal more than 10 GB [[S-1508]]. Bounding a facility's links in this way is the subject of [[C-0008]].\n- **Explaining all egress.** Making all egress traffic explainable by its ingress is an open design question, including which engineering compromises it would require [[S-0018]]. Active network taps on front-end links could scrub headers to reduce covert channels [[S-0031]].\n- **Physical side channels.** These offer routes around network monitoring altogether. Proposed defences include shielding, jamming, inspection and filtering [[S-0038]], with a design target of kilobits per second rather than zero [[S-0018]]. Residual copies in memory are addressed by memory wiping [[S-0018]].\n- **Insiders and physical media.** These are outside the reach of network-level mechanisms. RAND's verification framework includes whistleblower programmes and personnel interviews among its layers [[S-0002]], and the weight-security study treats defence as requiring comprehensive measures rather than isolated ones [[S-1610]].", "body_text": "Why it matters Model weights are a primary target for theft [S-1610]. - RAND researchers write that protecting frontier models from theft and misuse will become more important as they grow more capable [S-1610]. They identify 38 distinct attack vectors, and define five security levels for defending against actors ranging from opportunistic criminals to well-resourced nation-states [S-1610]. - For verifying international agreements, Scher and Thiergart describe one approach as using strong security to prevent model weights from leaving a data centre, and then monitoring that data centre closely [S-0005]. - RAND's framework treats confidentiality, meaning protection of models, data and code from unauthorised access and especially theft, as a requirement on verification [S-0002]. A low-trust system overview assumes the verifier may try to exfiltrate the prover's secrets [S-0018]. It therefore sends only cryptographic commitments out of the facility, and checks challenged data only in an auditing environment inside the prover's facility [S-0018]. Why it is hard The claim must hold for every channel. The RAND weight-security study's 38 attack vectors span far more than network transfer [S-1610]. - Hidden data in ordinary outputs. An attacker who controls an inference server could hide weights inside ordinary model responses using steganography [S-0015]. Rinberg and colleagues formalise this as a security game and verify outputs against a reference [S-0015]. On a 30-billion-parameter mixture-of-experts model, their detector reduced exfiltratable information to under 0.5% at a false-positive rate below 0.01%, which they equate to a slowdown of more than 200 times for an adversary [S-0015]. An independent study found that an attacker who chooses the prompts roughly doubles the bits leaked per token, cutting the slowdown to 60–118 times [S-1507]. - Capping egress. Rinberg and colleagues note that egress limits cap theft directly: if only 10 GB leaves a data centre, an adversary cannot steal more than 10 GB [S-1508]. Bounding a facility's links in this way is the subject of Communication between compute groups is bounded. - Explaining all egress. Making all egress traffic explainable by its ingress is an open design question, including which engineering compromises it would require [S-0018]. Active network taps on front-end links could scrub headers to reduce covert channels [S-0031]. - Physical side channels. These offer routes around network monitoring altogether. Proposed defences include shielding, jamming, inspection and filtering [S-0038], with a design target of kilobits per second rather than zero [S-0018]. Residual copies in memory are addressed by memory wiping [S-0018]. - Insiders and physical media. These are outside the reach of network-level mechanisms. RAND's verification framework includes whistleblower programmes and personnel interviews among its layers [S-0002], and the weight-security study treats defence as requiring comprehensive measures rather than isolated ones [S-1610].", "addressed_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/", "role": "supporting", "note": "A cap on outgoing bandwidth bounds how much weight data can leave a facility in a given time (S-1508)." }, { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/", "role": "primary", "note": "Bounds how much weight or other undeclared information can leave in checked outputs; does not close other channels." }, { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/", "role": "supporting", "note": "Removes the tolerance margin that steganographic exfiltration could use (S-0020)." }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/", "role": "supporting", "note": "The recomputation variant limits steganographic weight exfiltration through outputs (S-0015)." }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/", "role": "supporting", "note": "Aims to make covert exfiltration of results through tapped links infeasible (S-1300)." }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/", "role": "supporting", "note": "Bounds how much information can be hidden steganographically in checked outputs. It is not a stand-alone defence against weight exfiltration (S-0015)." }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/", "role": "supporting", "note": "Supports arguments that weights cannot leave by unmonitored physical routes." }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/", "role": "supporting", "note": "Used as the estimator in a weight-exfiltration detection scheme (S-0015)." }, { "id": "I-0010", "title": "RAND secure inference data center (SIDC) design", "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/", "role": "primary", "note": "Designed to keep weights and inference data inside the facility; the report does not describe how an external party would verify this." } ], "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" }, { "id": "I-0010", "title": "RAND secure inference data center (SIDC) design", "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "O-0210", "title": "Intelligence Security Laboratories", "url": "https://trustbutveri.fyi/organizations/intelligence-security-laboratories/" }, { "id": "O-0200", "title": "RAND", "url": "https://trustbutveri.fyi/organizations/rand/" } ] }, { "id": "K-0017", "slug": "compartmentalization", "title": "Compartmentalization", "aliases": [ "pods", "compute compartments" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Dividing a facility's accelerators into groups with restricted communication between them, so that combining groups for large training runs becomes slow or impractical.", "sources": [ { "source": "S-0005", "supports": "pods of well-connected chips; between pods, inference needs only tokens while other forms of parallelism transfer activations or gradients; efficient inference on dozens to low hundreds of chips versus thousands for large training", "locator": "Interconnect bandwidth limits" }, { "source": "S-3565", "supports": "expert-parallel MoE inference involves all-to-all cross-device communication", "locator": "abstract" }, { "source": "S-0057", "supports": "fixed-set HEM restricting networking of small, fixed sets of GPUs", "locator": "p. viii" }, { "source": "S-0053", "supports": "compute caps enforced via physical limits on chip-to-chip networking", "locator": "enforcement mechanisms" }, { "source": "S-0018", "supports": "air-gapped evaluation environments; optical splitters and data diodes as inspectable components enforcing one-way data movement", "locator": "system architecture" }, { "source": "S-0002", "supports": "network taps observing data exchanged between chips", "locator": "§4.2" }, { "source": "S-0038", "supports": "physical channels could bypass network monitoring", "locator": "side channels of concern" } ], "related": [ "K-0021", "K-0025", "K-0013" ], "type": "concept", "url": "https://trustbutveri.fyi/concepts/compartmentalization/", "source_file": "content/concepts/compartmentalization.md", "flags_all": [], "body_markdown": "Compartmentalization divides a facility's accelerators into groups and restricts communication between the groups, so that groups are hard to combine into a larger workload than is allowed [[S-0005]] [[S-0057]].\n\nScher and Thiergart describe pods of well-connected chips. Between pods, inference needs to pass only tokens, whereas other forms of parallelism transfer activations or gradients [[S-0005]]. That holds only while each inference replica, including any split of the model or its experts across devices, stays within one pod. Mixture-of-experts inference that spreads experts across devices uses all-to-all communication between them [[S-3565]]. Efficient inference fits within dozens to low hundreds of closely connected accelerators, while large-scale training links thousands [[S-0005]]. RAND's \"fixed set\" design likewise restricts networking so that small, fixed sets of GPUs cannot be combined into large clusters [[S-0057]], and Sastry and colleagues list physical limits on chip-to-chip networking as a way to enforce compute caps [[S-0053]]. These ideas underlie [[M-0014|bandwidth limits and compartmentalization]].\n\nCompartments can also separate trust domains. One low-trust design air-gaps its evaluation environments and uses optical splitters and data diodes, simple components that can be inspected for tampering, to enforce one-way data movement [[S-0018]]. The boundaries can be checked by observing traffic between accelerators with [[K-0014|network taps]] [[S-0002]], while physical channels that could bypass monitored links are the target of [[M-0022|side-channel suppression]] [[S-0038]].", "body_text": "Compartmentalization divides a facility's accelerators into groups and restricts communication between the groups, so that groups are hard to combine into a larger workload than is allowed [S-0005] [S-0057]. Scher and Thiergart describe pods of well-connected chips. Between pods, inference needs to pass only tokens, whereas other forms of parallelism transfer activations or gradients [S-0005]. That holds only while each inference replica, including any split of the model or its experts across devices, stays within one pod. Mixture-of-experts inference that spreads experts across devices uses all-to-all communication between them [S-3565]. Efficient inference fits within dozens to low hundreds of closely connected accelerators, while large-scale training links thousands [S-0005]. RAND's \"fixed set\" design likewise restricts networking so that small, fixed sets of GPUs cannot be combined into large clusters [S-0057], and Sastry and colleagues list physical limits on chip-to-chip networking as a way to enforce compute caps [S-0053]. These ideas underlie bandwidth limits and compartmentalization. Compartments can also separate trust domains. One low-trust design air-gaps its evaluation environments and uses optical splitters and data diodes, simple components that can be inspected for tampering, to enforce one-way data movement [S-0018]. The boundaries can be checked by observing traffic between accelerators with network taps [S-0002], while physical channels that could bypass monitored links are the target of side-channel suppression [S-0038].", "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "I-0010", "title": "RAND secure inference data center (SIDC) design", "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "K-0007", "title": "Hardware-enabled mechanism (HEM)", "url": "https://trustbutveri.fyi/concepts/hardware-enabled-mechanism/" }, { "id": "K-0025", "title": "Inference and training workloads", "url": "https://trustbutveri.fyi/concepts/inference-and-training-workloads/" }, { "id": "K-0021", "title": "Interconnect bandwidth", "url": "https://trustbutveri.fyi/concepts/interconnect-bandwidth/" }, { "id": "K-0013", "title": "Side channel", "url": "https://trustbutveri.fyi/concepts/side-channel/" } ] }, { "id": "K-0024", "slug": "cryptographic-commitment", "title": "Cryptographic commitment", "aliases": [ "commitment scheme", "hash commitment" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "A way to fix a value now without revealing it, so that it can later be opened and shown not to have changed.", "sources": [ { "source": "S-1606", "supports": "ideal commitment: value kept hidden, later retrieved with assurance it did not change", "locator": "§2.1.1, p. 19" }, { "source": "S-1607", "supports": "hash-tree commitment to N items with N−1 hash invocations; opening one item with L log N bits; binding property reduces to collision resistance", "locator": "§5 (hash trees)" }, { "source": "S-1600", "supports": "collision resistance of approved hash functions", "locator": "term: hash_function (FIPS 203/204/205)" }, { "source": "S-0004", "supports": "networking hardware and enclosures making cryptographic commitments about all traffic", "locator": "p. 13" }, { "source": "S-0018", "supports": "taps send out hashes of captured traffic while the prover keeps the plaintext", "locator": "§3.2.1" }, { "source": "S-0023", "supports": "one-time commitment to model weights, then per-query proofs against it", "locator": "§3; §8, Table 1" }, { "source": "S-0025", "supports": "pre-committed training specification and Merkle commitments of intermediate computation", "locator": "abstract" }, { "source": "S-0017", "supports": "prover commits a hash of sampled weights before it learns whether a step will be audited", "locator": "pre-training scheme" } ], "related": [ "K-0010", "K-0019", "K-0014", "K-0020" ], "type": "concept", "url": "https://trustbutveri.fyi/concepts/cryptographic-commitment/", "source_file": "content/concepts/cryptographic-commitment.md", "flags_all": [], "body_markdown": "A cryptographic commitment lets a party fix a value now while keeping it hidden, and later reveal it with assurance that it has not changed [[S-1606]].\n\nA commitment must keep the value hidden until it is opened [[S-1606]], and it must be binding: hard to open in more than one way [[S-1607]]. A Merkle hash tree commits to many items with one root hash, and any single item can later be opened with a proof whose size grows only logarithmically with the number of items [[S-1607]]. Its binding property reduces to the collision resistance of the hash function, meaning that finding two inputs with the same output is computationally infeasible [[S-1607]] [[S-1600]]. In AI verification, commitments fix evidence before anyone checks it:\n- **Traffic.** The Oxford Martin report describes networking hardware and enclosures that make commitments about all traffic [[S-0004]], and in one design taps send out hashes of captured traffic while the prover keeps the plaintext [[S-0018]], as in [[M-0013|network taps and certifiers]].\n- **Weights.** In [[M-0004|zero-knowledge proofs of inference]] the prover commits to its weights once and proves each output against that commitment [[S-0023]].\n- **Training.** A proposed zero-knowledge scheme combines a pre-committed training specification with Merkle commitments to intermediate computation [[S-0025]], as in [[M-0005|zero-knowledge proofs of training constraints]].\n- **Audits.** In one scheme the prover commits to sampled weights at each training step before it learns whether that step will be audited, so that it cannot fabricate consistent records afterwards [[S-0017]].", "body_text": "A cryptographic commitment lets a party fix a value now while keeping it hidden, and later reveal it with assurance that it has not changed [S-1606]. A commitment must keep the value hidden until it is opened [S-1606], and it must be binding: hard to open in more than one way [S-1607]. A Merkle hash tree commits to many items with one root hash, and any single item can later be opened with a proof whose size grows only logarithmically with the number of items [S-1607]. Its binding property reduces to the collision resistance of the hash function, meaning that finding two inputs with the same output is computationally infeasible [S-1607] [S-1600]. In AI verification, commitments fix evidence before anyone checks it: - Traffic. The Oxford Martin report describes networking hardware and enclosures that make commitments about all traffic [S-0004], and in one design taps send out hashes of captured traffic while the prover keeps the plaintext [S-0018], as in network taps and certifiers. - Weights. In zero-knowledge proofs of inference the prover commits to its weights once and proves each output against that commitment [S-0023]. - Training. A proposed zero-knowledge scheme combines a pre-committed training specification with Merkle commitments to intermediate computation [S-0025], as in zero-knowledge proofs of training constraints. - Audits. In one scheme the prover commits to sampled weights at each training step before it learns whether that step will be audited, so that it cannot fabricate consistent records afterwards [S-0017].", "referenced_by": [ { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "M-0005", "title": "Zero-knowledge proofs of training constraints", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-training-constraints/" }, { "id": "I-0007", "title": "Attestable Audits", "url": "https://trustbutveri.fyi/implementations/attestable-audits/" }, { "id": "I-0005", "title": "Attestable zero-knowledge inference prover", "url": "https://trustbutveri.fyi/implementations/attestable-zk-inference/" }, { "id": "I-0014", "title": "EZKL", "url": "https://trustbutveri.fyi/implementations/ezkl/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" }, { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "I-0003", "title": "zkLLM", "url": "https://trustbutveri.fyi/implementations/zkllm/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0019", "title": "Evidence binding", "url": "https://trustbutveri.fyi/concepts/evidence-binding/" }, { "id": "K-0014", "title": "Network tap", "url": "https://trustbutveri.fyi/concepts/network-tap/" }, { "id": "K-0012", "title": "Proof of space", "url": "https://trustbutveri.fyi/concepts/proof-of-space/" }, { "id": "K-0009", "title": "Recomputation", "url": "https://trustbutveri.fyi/concepts/recomputation/" }, { "id": "K-0020", "title": "Sampling and assurance", "url": "https://trustbutveri.fyi/concepts/sampling-and-assurance/" }, { "id": "K-0010", "title": "Zero-knowledge proof", "url": "https://trustbutveri.fyi/concepts/zero-knowledge-proof/" } ] }, { "id": "K-0019", "slug": "evidence-binding", "title": "Evidence binding", "aliases": [ "binding", "attribution of evidence" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Tying verification evidence to the specific device, workload, data and time it describes, so it cannot be substituted, replayed or misattributed.", "sources": [ { "source": "S-1603", "supports": "evidence must be securely associated with its target environment so a verifier cannot be tricked into accepting claims from a different environment; evidence generated with the attester's key material; signed nonces for freshness", "locator": "§3.1; §8.1; §10.2" }, { "source": "S-0018", "supports": "evidence can identify each forward pass uniquely and attribute it to the hardware and time it was processed on", "locator": "verification goals" }, { "source": "S-0005", "supports": "if a chip's location-attestation private key were extracted, its location could be spoofed", "locator": "On-chip mechanisms for location verification" }, { "source": "S-0031", "supports": "taps hash and timestamp captured traffic", "locator": "tap functions" }, { "source": "S-0029", "supports": "chip firmware hashes weight snapshots taken at random times; the hash is stored in firmware-only memory or signed with the chip's private key", "locator": "§4" }, { "source": "S-0012", "supports": "confidential VMs with GPUs and incremental multiset hashing to track dataset integrity for property attestation", "locator": "abstract" }, { "source": "S-0017", "supports": "prover commits a hash of sampled weights before it learns whether a step will be audited", "locator": "pre-training scheme" } ], "related": [ "K-0004", "K-0024", "K-0005" ], "type": "concept", "url": "https://trustbutveri.fyi/concepts/evidence-binding/", "source_file": "content/concepts/evidence-binding.md", "flags_all": [], "body_markdown": "Evidence binding ties a piece of verification evidence to the device, workload, data and time it describes, so that it cannot be replayed, substituted or attributed to something else [[S-1603]] [[S-0018]].\n\nThe IETF remote-attestation architecture states the requirement for devices: evidence must be securely associated with the environment it describes, so that a verifier cannot be tricked into accepting claims that originate elsewhere [[S-1603]]. Binding has several dimensions:\n- **Device.** Evidence is signed with key material held by the attesting device, as in [[M-0008|TEE remote attestation]] [[S-1603]]. Scher and Thiergart note that if the private key a chip uses to attest its location were extracted, the chip's location could be spoofed [[S-0005]].\n- **Time.** A nonce from the appraising party, signed into the evidence, shows that it is fresh rather than replayed [[S-1603]], and [[M-0013|network taps]] hash and timestamp the traffic they capture [[S-0031]].\n- **Workload.** Shavit's design has chip firmware hash weight snapshots taken at random times, then store the hash where only the firmware can write it or sign it with the chip's private key [[S-0029]]. One low-trust system aims to identify each forward pass uniquely and attribute it to the hardware and time that processed it [[S-0018]].\n- **Model and data.** PAL\\*M tracks dataset integrity with incremental multiset hashing inside confidential virtual machines, so that attested properties refer to the model and data actually used [[S-0012]].\n\nBinding also constrains when evidence is fixed: for sampled checks, the prover must [[K-0024|commit]] to its records before it learns which ones will be audited [[S-0017]].", "body_text": "Evidence binding ties a piece of verification evidence to the device, workload, data and time it describes, so that it cannot be replayed, substituted or attributed to something else [S-1603] [S-0018]. The IETF remote-attestation architecture states the requirement for devices: evidence must be securely associated with the environment it describes, so that a verifier cannot be tricked into accepting claims that originate elsewhere [S-1603]. Binding has several dimensions: - Device. Evidence is signed with key material held by the attesting device, as in TEE remote attestation [S-1603]. Scher and Thiergart note that if the private key a chip uses to attest its location were extracted, the chip's location could be spoofed [S-0005]. - Time. A nonce from the appraising party, signed into the evidence, shows that it is fresh rather than replayed [S-1603], and network taps hash and timestamp the traffic they capture [S-0031]. - Workload. Shavit's design has chip firmware hash weight snapshots taken at random times, then store the hash where only the firmware can write it or sign it with the chip's private key [S-0029]. One low-trust system aims to identify each forward pass uniquely and attribute it to the hardware and time that processed it [S-0018]. - Model and data. PAL*M tracks dataset integrity with incremental multiset hashing inside confidential virtual machines, so that attested properties refer to the model and data actually used [S-0012]. Binding also constrains when evidence is fixed: for sampled checks, the prover must commit to its records before it learns which ones will be audited [S-0017].", "referenced_by": [ { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/" }, { "id": "I-0007", "title": "Attestable Audits", "url": "https://trustbutveri.fyi/implementations/attestable-audits/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" }, { "id": "K-0024", "title": "Cryptographic commitment", "url": "https://trustbutveri.fyi/concepts/cryptographic-commitment/" }, { "id": "K-0014", "title": "Network tap", "url": "https://trustbutveri.fyi/concepts/network-tap/" }, { "id": "K-0004", "title": "Remote attestation", "url": "https://trustbutveri.fyi/concepts/remote-attestation/" } ] }, { "id": "K-0023", "slug": "flop-accounting", "title": "FLOP accounting", "aliases": [ "training compute accounting", "compute thresholds" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Estimating or verifying how many floating-point operations a training run or other workload used, often to compare against a threshold in a rule.", "sources": [ { "source": "S-0029", "supports": "total training compute as a rule and an indicator of capabilities; a threshold of H FLOPs converted to chip-days using each chip's FLOPs per day at full, perfectly parallel use", "locator": "§2.1; §3.2, Table 1" }, { "source": "S-0053", "supports": "EO 14110 reporting threshold of 10^26 operations; compute as a good high-level proxy for risk of general-purpose frontier systems; thresholds need changing with algorithmic and hardware progress", "locator": "thresholds; limitations" }, { "source": "S-0063", "supports": "Strict Threshold 10^24 FLOP and Monitored Threshold 10^22 FLOP", "locator": "§4" }, { "source": "S-3542", "supports": "EU AI Act 10^25 FLOP presumption, notification, application from 2 August 2025", "locator": "Arts 51(2), 52(1), 113(b)" }, { "source": "S-3543", "supports": "10^25 FLOP presumption and notification; entry into application on 2 August 2025", "locator": "§2.3.1–2.3.2; landing page" }, { "source": "S-3544", "supports": "frontier model defined by more than 10^26 integer or floating-point operations", "locator": "§22757.11(i)" }, { "source": "S-0062", "supports": "facility energy estimates could be converted into an approximation of FLOPs", "locator": "Energy monitoring" }, { "source": "S-0002", "supports": "estimating MFU and physical signature (e.g. power) as a research problem", "locator": "Table 2, Appendix A.6" }, { "source": "S-0057", "supports": "offline licensing with a renewable licence carrying a compute budget", "locator": "p. viii" }, { "source": "S-0018", "supports": "coarse metrics like total FLOPs insufficient; per-workload evidence sought", "locator": "verification goals" }, { "source": "S-0069", "supports": "revocation of EO 14110 on 20 January 2025", "locator": "Sec. 2(ggg)" } ], "related": [ "K-0025", "K-0016" ], "type": "concept", "url": "https://trustbutveri.fyi/concepts/flop-accounting/", "source_file": "content/concepts/flop-accounting.md", "flags_all": [], "body_markdown": "FLOP accounting estimates or verifies the total number of floating-point operations (FLOP) that a training run or other workload performs, usually to compare it with a threshold set by a rule [[S-0029]] [[S-0053]].\n\nShavit lists total training compute among the rules a verifier might enforce, noting that it has proven to be an indicator of model capabilities [[S-0029]]. US Executive Order 14110 required reporting for models trained with more than 10^26 operations [[S-0053]] until its revocation in January 2025 [[S-0069]], and a proposed international agreement sets a prohibited threshold of 10^24 FLOP and a monitored threshold of 10^22 FLOP [[S-0063]]. The EU AI Act presumes that a general-purpose AI model trained with more than 10^25 FLOP has high-impact capabilities, with notification of the European Commission required since 2 August 2025 [[S-3542]] [[S-3543]], and California's SB 53 defines frontier models by more than 10^26 operations of training compute [[S-3544]]. Proposed ways to count or cap FLOP include:\n- **Hardware time.** Shavit converts a FLOP threshold into accelerator-days by assuming that every accelerator runs at its full rate with perfect parallelization, a conservative assumption that gives the fewest accelerator-days a run of that size could occupy [[S-0029]].\n- **Energy.** Wasil and colleagues suggest that a data centre's measured energy use could be converted into an approximate FLOP count [[S-0062]].\n- **Telemetry.** RAND lists estimating a workload's model FLOPs utilization (MFU) and physical signature, such as power, as a research problem [[S-0002]], one that bears on [[M-0010|on-chip telemetry]].\n- **On-chip budgets.** Offline licensing ties chip use to a renewable licence carrying a compute budget [[S-0057]], as in [[M-0011|hardware performance throttling and licensing]].\n\nSastry and colleagues call compute a good high-level proxy for the risk of general-purpose frontier systems, though not necessarily of some narrow ones, and expect thresholds to need changing as algorithms and hardware improve [[S-0053]]. One system overview argues that coarse totals such as FLOP counts are not enough and seeks evidence about individual workloads [[S-0018]].", "body_text": "FLOP accounting estimates or verifies the total number of floating-point operations (FLOP) that a training run or other workload performs, usually to compare it with a threshold set by a rule [S-0029] [S-0053]. Shavit lists total training compute among the rules a verifier might enforce, noting that it has proven to be an indicator of model capabilities [S-0029]. US Executive Order 14110 required reporting for models trained with more than 10^26 operations [S-0053] until its revocation in January 2025 [S-0069], and a proposed international agreement sets a prohibited threshold of 10^24 FLOP and a monitored threshold of 10^22 FLOP [S-0063]. The EU AI Act presumes that a general-purpose AI model trained with more than 10^25 FLOP has high-impact capabilities, with notification of the European Commission required since 2 August 2025 [S-3542] [S-3543], and California's SB 53 defines frontier models by more than 10^26 operations of training compute [S-3544]. Proposed ways to count or cap FLOP include: - Hardware time. Shavit converts a FLOP threshold into accelerator-days by assuming that every accelerator runs at its full rate with perfect parallelization, a conservative assumption that gives the fewest accelerator-days a run of that size could occupy [S-0029]. - Energy. Wasil and colleagues suggest that a data centre's measured energy use could be converted into an approximate FLOP count [S-0062]. - Telemetry. RAND lists estimating a workload's model FLOPs utilization (MFU) and physical signature, such as power, as a research problem [S-0002], one that bears on on-chip telemetry. - On-chip budgets. Offline licensing ties chip use to a renewable licence carrying a compute budget [S-0057], as in hardware performance throttling and licensing. Sastry and colleagues call compute a good high-level proxy for the risk of general-purpose frontier systems, though not necessarily of some narrow ones, and expect thresholds to need changing as algorithms and hardware improve [S-0053]. One system overview argues that coarse totals such as FLOP counts are not enough and seeks evidence about individual workloads [S-0018].", "referenced_by": [ { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" }, { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "M-0005", "title": "Zero-knowledge proofs of training constraints", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-training-constraints/" }, { "id": "C-0001", "title": "Compute stock is at most a declared amount", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0025", "title": "Inference and training workloads", "url": "https://trustbutveri.fyi/concepts/inference-and-training-workloads/" }, { "id": "K-0016", "title": "Undeclared compute", "url": "https://trustbutveri.fyi/concepts/undeclared-compute/" } ] }, { "id": "K-0007", "slug": "hardware-enabled-mechanism", "title": "Hardware-enabled mechanism (HEM)", "aliases": [ "HEM", "hardware-enabled governance mechanism", "on-chip mechanism" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "A governance or verification function built into AI chips or closely attached hardware, such as usage reporting, location attestation or enforced limits.", "sources": [ { "source": "S-0057", "supports": "introduces HEMs to help achieve US AI governance goals including export controls; threats, attack vectors and protections; offline licensing and fixed-set designs; security under an adversary's physical possession is open", "locator": "abstract; pp. viii–x" }, { "source": "S-0006", "supports": "HEMs enabling verifiable reporting of compute quantity, cluster configuration or location, and policy enforcement", "locator": "abstract" }, { "source": "S-0056", "supports": "chips sold by leading firms already have many of the needed features", "locator": "summary" }, { "source": "S-0035", "supports": "flexHEG: auditable guarantee processor monitoring accelerator usage, plus a secure enclosure providing physical tamper protection", "locator": "abstract" } ], "related": [ "K-0005", "K-0004", "K-0015", "K-0017" ], "type": "concept", "url": "https://trustbutveri.fyi/concepts/hardware-enabled-mechanism/", "source_file": "content/concepts/hardware-enabled-mechanism.md", "flags_all": [], "body_markdown": "A hardware-enabled mechanism (HEM) is a governance or verification function built into AI chips or hardware attached to them, such as reporting how much compute was used and where, or enforcing limits on use [[S-0057]] [[S-0006]].\n\nA 2024 RAND report introduced the concept to help achieve US AI governance goals such as export controls, and analysed the threats, attack vectors and protective measures that apply to such mechanisms [[S-0057]]. CNAS notes that chips sold by several leading firms already have many of the security features HEMs would need [[S-0056]]. Proposed designs include:\n- **Offline licensing.** Use of certain chip features is tied to a renewable licence carrying a compute budget, as in [[M-0011|hardware performance throttling and licensing]] [[S-0057]].\n- **Fixed sets.** Networking is restricted so that small, fixed groups of GPUs cannot be combined into large clusters, a form of [[K-0017|compartmentalization]] [[S-0057]].\n- **Guarantee processors.** An auditable processor monitors accelerator usage inside a secure enclosure that provides physical tamper protection, as in [[M-0009|flexHEG]] [[S-0035]].\n- **Verifiable reporting.** HEMs could report properties of training, such as the quantity of compute used and the cluster's configuration or location [[S-0006]], the aim of [[M-0010|on-chip telemetry]] and [[M-0018|chip location verification]].\n\nA central open question is whether HEMs can stay secure when an adversary has the chips in its physical possession [[S-0057]].", "body_text": "A hardware-enabled mechanism (HEM) is a governance or verification function built into AI chips or hardware attached to them, such as reporting how much compute was used and where, or enforcing limits on use [S-0057] [S-0006]. A 2024 RAND report introduced the concept to help achieve US AI governance goals such as export controls, and analysed the threats, attack vectors and protective measures that apply to such mechanisms [S-0057]. CNAS notes that chips sold by several leading firms already have many of the security features HEMs would need [S-0056]. Proposed designs include: - Offline licensing. Use of certain chip features is tied to a renewable licence carrying a compute budget, as in hardware performance throttling and licensing [S-0057]. - Fixed sets. Networking is restricted so that small, fixed groups of GPUs cannot be combined into large clusters, a form of compartmentalization [S-0057]. - Guarantee processors. An auditable processor monitors accelerator usage inside a secure enclosure that provides physical tamper protection, as in flexHEG [S-0035]. - Verifiable reporting. HEMs could report properties of training, such as the quantity of compute used and the cluster's configuration or location [S-0006], the aim of on-chip telemetry and chip location verification. A central open question is whether HEMs can stay secure when an adversary has the chips in its physical possession [S-0057].", "referenced_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" }, { "id": "K-0005", "title": "Root of trust", "url": "https://trustbutveri.fyi/concepts/root-of-trust/" }, { "id": "K-0015", "title": "Tamper evidence and tamper resistance", "url": "https://trustbutveri.fyi/concepts/tamper-evidence/" } ] }, { "id": "K-0025", "slug": "inference-and-training-workloads", "title": "Inference and training workloads", "aliases": [ "training vs inference" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Training updates a model's weights from data. Inference uses fixed weights to produce outputs. Their different resource use underpins several verification methods.", "sources": [ { "source": "S-0029", "supports": "training steps update weights from data batches; no straightforward way to tell whether an ML chip is running training or an unrelated job", "locator": "§5.1; §4" }, { "source": "S-0018", "supports": "inference yields token-level input-output traffic on front-end links; training uses the back-end fabric", "locator": "inference vs training" }, { "source": "S-0002", "supports": "declared training (1.A.1) and inference (1.A.2) as distinct declared uses", "locator": "§3.2" }, { "source": "S-0053", "supports": "majority of AI compute used for inference; single training run needs far more compute than a single inference; copies of a model run on little compute", "locator": "training vs inference discussion" }, { "source": "S-0005", "supports": "large-scale training links thousands of chips and exchanges gradients; efficient inference on dozens to low hundreds of chips passes only tokens between pods; utilization and power often differ", "locator": "Interconnect bandwidth limits; workload classification with high-level chip measures" }, { "source": "S-3565", "supports": "expert-parallel MoE inference involves all-to-all cross-device communication", "locator": "abstract" }, { "source": "S-3566", "supports": "covert reinforcement-learning rollouts on declared inference servers with updates on hidden compute", "locator": "§3.4; §4" }, { "source": "S-0037", "supports": "NVML-telemetry classifier: 98.2% binary accuracy at identifying training across its corpus (9 GPU models); 43–87% against the white-box disguised workloads held out from training, after hardening", "locator": "abstract" } ], "related": [ "K-0021", "K-0023", "K-0017" ], "type": "concept", "url": "https://trustbutveri.fyi/concepts/inference-and-training-workloads/", "source_file": "content/concepts/inference-and-training-workloads.md", "flags_all": [], "body_markdown": "Training is the workload that updates a model's weights step by step from batches of data, and inference is the workload that runs a model with fixed weights on inputs to produce outputs such as tokens [[S-0029]] [[S-0018]].\n\nRAND's verification framework treats declared training and declared inference as distinct uses of compute, each to be verified [[S-0002]]. Sastry and colleagues note that most AI compute is used for inference, although a single training run needs far more compute than a single inference, and individual copies of a model can run on relatively little compute [[S-0053]]. Verification designs use the differences in resource use:\n- Large-scale training links thousands of accelerators and exchanges gradients or activations between groups of them, while inference between pods passes only tokens [[S-0005]], provided each inference replica, including any split of the model across devices, stays within one pod [[S-3565]]. [[M-0014|Bandwidth limits]] rely on this gap.\n- Training and inference often differ in accelerator utilization and power draw [[S-0005]], which [[M-0021|workload classification from telemetry and side channels]] uses.\n- One classifier using GPU telemetry reports 98.2% binary accuracy at identifying training across its corpus of nine GPU models, falling to 43–87% on the most challenging disguised workloads held out from its training [[S-0037]], as in [[M-0010|on-chip telemetry]].\n\nReinforcement learning blurs the line, because its rollouts are inference. JoshC analyzes a covert strategy that generates rollouts on declared inference servers while updating the model on hidden compute [[S-3566]]. Shavit notes that there is no straightforward way to tell whether an accelerator is running training or an unrelated workload [[S-0029]].", "body_text": "Training is the workload that updates a model's weights step by step from batches of data, and inference is the workload that runs a model with fixed weights on inputs to produce outputs such as tokens [S-0029] [S-0018]. RAND's verification framework treats declared training and declared inference as distinct uses of compute, each to be verified [S-0002]. Sastry and colleagues note that most AI compute is used for inference, although a single training run needs far more compute than a single inference, and individual copies of a model can run on relatively little compute [S-0053]. Verification designs use the differences in resource use: - Large-scale training links thousands of accelerators and exchanges gradients or activations between groups of them, while inference between pods passes only tokens [S-0005], provided each inference replica, including any split of the model across devices, stays within one pod [S-3565]. Bandwidth limits rely on this gap. - Training and inference often differ in accelerator utilization and power draw [S-0005], which workload classification from telemetry and side channels uses. - One classifier using GPU telemetry reports 98.2% binary accuracy at identifying training across its corpus of nine GPU models, falling to 43–87% on the most challenging disguised workloads held out from its training [S-0037], as in on-chip telemetry. Reinforcement learning blurs the line, because its rollouts are inference. JoshC analyzes a covert strategy that generates rollouts on declared inference servers while updating the model on hidden compute [S-3566]. Shavit notes that there is no straightforward way to tell whether an accelerator is running training or an unrelated workload [S-0029].", "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "I-0008", "title": "SASH confidential network logger", "url": "https://trustbutveri.fyi/implementations/sash-confidential-network-logger/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "K-0017", "title": "Compartmentalization", "url": "https://trustbutveri.fyi/concepts/compartmentalization/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" }, { "id": "K-0021", "title": "Interconnect bandwidth", "url": "https://trustbutveri.fyi/concepts/interconnect-bandwidth/" } ] }, { "id": "K-0021", "slug": "interconnect-bandwidth", "title": "Interconnect bandwidth", "aliases": [ "chip-to-chip bandwidth", "network bandwidth" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "The data rate of links between accelerators or groups of them; large-scale training needs far more of it than inference, so limiting it constrains workloads.", "sources": [ { "source": "S-0053", "supports": "communication bandwidth as a chip specification alongside operations per second and memory; EO cluster definition using network connections over 100 Gbit/s; decentralized training risk", "locator": "§ on quantifiability and detectability; limitations" }, { "source": "S-0005", "supports": "large-scale training links thousands of chips with high-bandwidth interconnect, efficient inference dozens to low hundreds; between pods inference needs tokens while training transfers gradients or activations; this gap is the target of bandwidth limits", "locator": "Interconnect bandwidth limits" }, { "source": "S-3565", "supports": "expert-parallel MoE inference involves all-to-all cross-device communication", "locator": "abstract" }, { "source": "S-0018", "supports": "front-end token-level traffic vs high-bandwidth, latency-sensitive back-end fabric that is harder to tap", "locator": "inference vs training" }, { "source": "S-0069", "supports": "revocation of Executive Order 14110 in January 2025", "locator": "§2(ggg)" } ], "related": [ "K-0017", "K-0025", "K-0014" ], "type": "concept", "url": "https://trustbutveri.fyi/concepts/interconnect-bandwidth/", "source_file": "content/concepts/interconnect-bandwidth.md", "flags_all": [], "body_markdown": "Interconnect bandwidth is the rate at which accelerators, servers or clusters can exchange data over the links between them; it is one of the measurable specifications of AI accelerators, alongside operations per second and memory capacity [[S-0053]].\n\nLarge-scale training links thousands of accelerators with high-bandwidth interconnect, while efficient inference can run on dozens to low hundreds of closely connected accelerators [[S-0005]]. Between such pods, inference needs to pass only tokens, whereas training exchanges gradients or activations; Scher and Thiergart identify this gap as the target of bandwidth limits, as in [[M-0014|bandwidth limits and compartmentalization]] [[S-0005]]. The gap holds only while each inference replica, including any split of the model or its experts across devices, stays within one pod. Mixture-of-experts inference that spreads experts across devices uses all-to-all communication between them [[S-3565]]. Inside a data centre, front-end links carry token-level inputs and outputs, while the back-end fabric between accelerators carries tensors and collective operations at much higher bandwidth, is latency-sensitive, and is harder to [[K-0014|tap]] [[S-0018]]. US Executive Order 14110 defined reportable computing clusters partly by network connections faster than 100 Gbit/s [[S-0053]]. It was revoked in January 2025 [[S-0069]]. Sastry et al. note that the detectability of compute could be undermined if decentralized training, spread across many data centres or using lower-quality compute, becomes more viable [[S-0053]].", "body_text": "Interconnect bandwidth is the rate at which accelerators, servers or clusters can exchange data over the links between them; it is one of the measurable specifications of AI accelerators, alongside operations per second and memory capacity [S-0053]. Large-scale training links thousands of accelerators with high-bandwidth interconnect, while efficient inference can run on dozens to low hundreds of closely connected accelerators [S-0005]. Between such pods, inference needs to pass only tokens, whereas training exchanges gradients or activations; Scher and Thiergart identify this gap as the target of bandwidth limits, as in bandwidth limits and compartmentalization [S-0005]. The gap holds only while each inference replica, including any split of the model or its experts across devices, stays within one pod. Mixture-of-experts inference that spreads experts across devices uses all-to-all communication between them [S-3565]. Inside a data centre, front-end links carry token-level inputs and outputs, while the back-end fabric between accelerators carries tensors and collective operations at much higher bandwidth, is latency-sensitive, and is harder to tap [S-0018]. US Executive Order 14110 defined reportable computing clusters partly by network connections faster than 100 Gbit/s [S-0053]. It was revoked in January 2025 [S-0069]. Sastry et al. note that the detectability of compute could be undermined if decentralized training, spread across many data centres or using lower-quality compute, becomes more viable [S-0053].", "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0017", "title": "Compartmentalization", "url": "https://trustbutveri.fyi/concepts/compartmentalization/" }, { "id": "K-0025", "title": "Inference and training workloads", "url": "https://trustbutveri.fyi/concepts/inference-and-training-workloads/" }, { "id": "K-0014", "title": "Network tap", "url": "https://trustbutveri.fyi/concepts/network-tap/" }, { "id": "K-0022", "title": "Weight exfiltration", "url": "https://trustbutveri.fyi/concepts/weight-exfiltration/" } ] }, { "id": "K-0014", "slug": "network-tap", "title": "Network tap", "aliases": [ "TAP", "traffic tap" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "A device that copies the traffic on a network link for inspection without disrupting it; proposed for checking what AI data centres compute.", "sources": [ { "source": "S-0031", "supports": "definition; passive vs active taps; taps commit to traffic by hashing; front-end links most viable; back-end fabric bandwidth, latency and sampling", "locator": "sections on definitions, north-south and east-west links" }, { "source": "S-0002", "supports": "mutually vetted devices intercept inter-chip data and check against declared uses", "locator": "§4.2" }, { "source": "S-0018", "supports": "fixed-function taps that send hashes of packet groups, not plaintext", "locator": "evidence capture" } ], "related": [ "K-0021", "K-0024", "K-0019" ], "type": "concept", "url": "https://trustbutveri.fyi/concepts/network-tap/", "source_file": "content/concepts/network-tap.md", "flags_all": [], "body_markdown": "A network tap is a device that copies the traffic on a network link for inspection without disrupting normal operation [[S-0031]].\n\nPassive taps observe without interfering, for example by optically splitting the light in a fibre; active taps interact with the traffic, for instance by re-timing it or modifying headers to reduce covert channels [[S-0031]]. RAND's verification framework includes mutually vetted devices that intercept data exchanged between accelerators and check it for discrepancies with declared uses [[S-0002]]. In proposed designs such as [[M-0013|network taps and certifiers]], taps send out hashes of the traffic as [[K-0024|cryptographic commitments]] instead of plaintext, so that the prover's declarations can later be checked against them [[S-0031]] [[S-0018]]. Cankaya describes taps on the front-end links between a data centre and the outside world as the most viable, because those links carry little bandwidth [[S-0031]]. The back-end fabric between accelerators has far higher [[K-0021|bandwidth]] and strict latency requirements, and may call for random sampling rather than full capture [[S-0031]].", "body_text": "A network tap is a device that copies the traffic on a network link for inspection without disrupting normal operation [S-0031]. Passive taps observe without interfering, for example by optically splitting the light in a fibre; active taps interact with the traffic, for instance by re-timing it or modifying headers to reduce covert channels [S-0031]. RAND's verification framework includes mutually vetted devices that intercept data exchanged between accelerators and check it for discrepancies with declared uses [S-0002]. In proposed designs such as network taps and certifiers, taps send out hashes of the traffic as cryptographic commitments instead of plaintext, so that the prover's declarations can later be checked against them [S-0031] [S-0018]. Cankaya describes taps on the front-end links between a data centre and the outside world as the most viable, because those links carry little bandwidth [S-0031]. The back-end fabric between accelerators has far higher bandwidth and strict latency requirements, and may call for random sampling rather than full capture [S-0031].", "referenced_by": [ { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "M-0005", "title": "Zero-knowledge proofs of training constraints", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-training-constraints/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "I-0008", "title": "SASH confidential network logger", "url": "https://trustbutveri.fyi/implementations/sash-confidential-network-logger/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0017", "title": "Compartmentalization", "url": "https://trustbutveri.fyi/concepts/compartmentalization/" }, { "id": "K-0024", "title": "Cryptographic commitment", "url": "https://trustbutveri.fyi/concepts/cryptographic-commitment/" }, { "id": "K-0021", "title": "Interconnect bandwidth", "url": "https://trustbutveri.fyi/concepts/interconnect-bandwidth/" }, { "id": "K-0022", "title": "Weight exfiltration", "url": "https://trustbutveri.fyi/concepts/weight-exfiltration/" } ] }, { "id": "K-0008", "slug": "numerical-nondeterminism", "title": "Numerical nondeterminism", "aliases": [ "floating-point nondeterminism", "non-reproducibility" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Differences between runs, or between machines, in the results of the same AI computation, because floating-point rounding depends on the order of operations.", "sources": [ { "source": "S-1010", "supports": "GPU non-determinism arises from unspecified details including rounding strategy, subnormal numbers and accumulation order, since floating-point arithmetic is not associative; results differ between GPU architectures", "locator": "abstract; §1" }, { "source": "S-0016", "supports": "re-running the same inference often gives different results due to benign numerical noise; comparison against a trusted reference conditioned on the same sampling seed", "locator": "abstract" }, { "source": "S-1009", "supports": "floating-point non-associativity; LLM forward pass run-to-run deterministic; lack of batch invariance with load-dependent batch size as the main cause of nondeterminism in LLM inference endpoints; batch-invariant kernels", "locator": "sections on non-associativity, the concurrency hypothesis and batch invariance" }, { "source": "S-0018", "supports": "metadata needed for bit-exact replay (hardware SKU, quantization, parallelism, kernels, batch size)", "locator": "§5.2.2" }, { "source": "S-0029", "supports": "accept a recomputed checkpoint within a small distance of the reported one", "locator": "§5.1" }, { "source": "S-0020", "supports": "software emulation predicting every bit of transformer forward passes across NVIDIA GPU architectures, validated on dense blocks on A100, L40, L40S and H100", "locator": "abstract; evaluation" } ], "related": [ "K-0009" ], "type": "concept", "url": "https://trustbutveri.fyi/concepts/numerical-nondeterminism/", "source_file": "content/concepts/numerical-nondeterminism.md", "flags_all": [], "body_markdown": "Numerical nondeterminism is variation in the results of the same computation on the same inputs, across repeated runs or across hardware and software setups, that comes from floating-point arithmetic rather than from intended randomness such as sampling [[S-1010]] [[S-0016]].\n\nFloating-point addition is not associative, so a rounded sum depends on the order in which its terms are accumulated [[S-1009]] [[S-1010]]. GPUs leave that order, their rounding strategy and their handling of subnormal numbers unspecified, and the same matrix multiplication can give different results on different GPU architectures [[S-1010]]. On a single machine, a Thinking Machines post finds the LLM forward pass run-to-run deterministic for a fixed batch, and traces the variation users see to kernels whose results change with batch size, which depends on server load [[S-1009]]. Exact replay can therefore need a record of the original hardware model, quantization, parallelism layout, kernels and batch size [[S-0018]]. For a verifier, this makes legitimate variation hard to tell from real problems [[S-0016]]. Verification designs either tolerate it or remove it:\n- [[M-0001|Sampled inference recomputation]] tolerates it by comparing outputs with a trusted reference that uses the same sampling seed [[S-0016]], and [[M-0006|training-transcript verification]] accepts a recomputed checkpoint within a small distance of the reported one [[S-0029]].\n- [[M-0002|Deterministic and bit-exact inference]] removes it with batch-invariant kernels [[S-1009]] or software emulation that predicts, bit for bit, the outputs of dense transformer blocks on four NVIDIA GPU models [[S-0020]].", "body_text": "Numerical nondeterminism is variation in the results of the same computation on the same inputs, across repeated runs or across hardware and software setups, that comes from floating-point arithmetic rather than from intended randomness such as sampling [S-1010] [S-0016]. Floating-point addition is not associative, so a rounded sum depends on the order in which its terms are accumulated [S-1009] [S-1010]. GPUs leave that order, their rounding strategy and their handling of subnormal numbers unspecified, and the same matrix multiplication can give different results on different GPU architectures [S-1010]. On a single machine, a Thinking Machines post finds the LLM forward pass run-to-run deterministic for a fixed batch, and traces the variation users see to kernels whose results change with batch size, which depends on server load [S-1009]. Exact replay can therefore need a record of the original hardware model, quantization, parallelism layout, kernels and batch size [S-0018]. For a verifier, this makes legitimate variation hard to tell from real problems [S-0016]. Verification designs either tolerate it or remove it: - Sampled inference recomputation tolerates it by comparing outputs with a trusted reference that uses the same sampling seed [S-0016], and training-transcript verification accepts a recomputed checkpoint within a small distance of the reported one [S-0029]. - Deterministic and bit-exact inference removes it with batch-invariant kernels [S-1009] or software emulation that predicts, bit for bit, the outputs of dense transformer blocks on four NVIDIA GPU models [S-0020].", "referenced_by": [ { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "M-0005", "title": "Zero-knowledge proofs of training constraints", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-training-constraints/" }, { "id": "I-0016", "title": "Batch-invariant inference kernels (Thinking Machines)", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" }, { "id": "I-0015", "title": "Verde and RepOps (Gensyn)", "url": "https://trustbutveri.fyi/implementations/gensyn-verde-repops/" }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" }, { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0009", "title": "Recomputation", "url": "https://trustbutveri.fyi/concepts/recomputation/" } ] }, { "id": "K-0003", "slug": "positive-and-negative-claims", "title": "Positive and negative claims", "aliases": [ "negative claim", "positive claim", "mixed claim" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "A positive claim asserts that something is present or happened; a negative claim, that an activity or resource is absent; a mixed claim, both.", "sources": [ { "source": "S-0004", "supports": "demonstrating the existence of an object or process is often straightforward compared with demonstrating its non-existence", "locator": "p. 31" }, { "source": "S-0002", "supports": "split between verifying that declared uses are accurate (Subgoal 1) and verifying no undeclared uses or clusters (Subgoal 2)", "locator": "§3.2, Figure 4" }, { "source": "S-0018", "supports": "memory wiping with incompressible noise to leave no residual capacity for hidden workloads", "locator": "system architecture (memory wiping)" }, { "source": "S-0005", "supports": "bandwidth limits target the gap between inference tokens and the activations or gradients of other parallelism between pods; covert data centres may be difficult to detect", "locator": "Interconnect bandwidth limits; Verifying the location of AI compute" }, { "source": "S-0029", "supports": "random chip sampling detects at least one chip from a violating run with a chosen probability", "locator": "§3.2, Equation 1" } ], "related": [ "K-0016", "K-0020", "C-0005", "C-0004" ], "type": "concept", "url": "https://trustbutveri.fyi/concepts/positive-and-negative-claims/", "source_file": "content/concepts/positive-and-negative-claims.md", "flags_all": [], "body_markdown": "A positive claim asserts that something is present or happened, and a negative claim asserts that an activity or resource is absent [[S-0004]]; a mixed claim bundles both.\n\nExamples are [[C-0005|the declared model being the one served]] (positive), [[C-0010|there being no undeclared compute]] (negative) and [[C-0004|compute running inference and not training]] (mixed). The Oxford Martin report observes that demonstrating the existence of an object or process is often straightforward compared with demonstrating its non-existence [[S-0004]]. RAND's framework mirrors the split: one subgoal verifies that declared uses of compute are accurate, and another verifies that there are no undeclared uses and no undeclared clusters [[S-0002]]. Proposed designs support negative claims indirectly:\n- **Leaving no spare capacity.** Filling or wiping memory removes residual capacity for hidden workloads, as in [[M-0015|memory wiping and proofs of secure erasure]] [[S-0018]].\n- **Limiting communication.** Capping the bandwidth between pods at what inference tokens need, below what the activations or gradients of training need, is meant to keep pods from joining a larger training run, as in [[M-0014|bandwidth limits]] [[S-0005]].\n- **Sampling.** Randomly inspecting accelerators makes it likely that at least one accelerator used in a violating run is found [[S-0029]].\n\nAll three act on declared hardware; Scher and Thiergart judge that detecting data centres that were never declared may be difficult [[S-0005]].", "body_text": "A positive claim asserts that something is present or happened, and a negative claim asserts that an activity or resource is absent [S-0004]; a mixed claim bundles both. Examples are the declared model being the one served (positive), there being no undeclared compute (negative) and compute running inference and not training (mixed). The Oxford Martin report observes that demonstrating the existence of an object or process is often straightforward compared with demonstrating its non-existence [S-0004]. RAND's framework mirrors the split: one subgoal verifies that declared uses of compute are accurate, and another verifies that there are no undeclared uses and no undeclared clusters [S-0002]. Proposed designs support negative claims indirectly: - Leaving no spare capacity. Filling or wiping memory removes residual capacity for hidden workloads, as in memory wiping and proofs of secure erasure [S-0018]. - Limiting communication. Capping the bandwidth between pods at what inference tokens need, below what the activations or gradients of training need, is meant to keep pods from joining a larger training run, as in bandwidth limits [S-0005]. - Sampling. Randomly inspecting accelerators makes it likely that at least one accelerator used in a violating run is found [S-0029]. All three act on declared hardware; Scher and Thiergart judge that detecting data centres that were never declared may be difficult [S-0005].", "referenced_by": [ { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/" }, { "id": "C-0001", "title": "Compute stock is at most a declared amount", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "K-0001", "title": "Prover", "url": "https://trustbutveri.fyi/concepts/prover/" }, { "id": "K-0020", "title": "Sampling and assurance", "url": "https://trustbutveri.fyi/concepts/sampling-and-assurance/" }, { "id": "K-0016", "title": "Undeclared compute", "url": "https://trustbutveri.fyi/concepts/undeclared-compute/" } ] }, { "id": "K-0012", "slug": "proof-of-space", "title": "Proof of space", "aliases": [ "proof of storage capacity" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "A protocol in which a prover shows that it is dedicating a given amount of storage or memory, rather than computation, to a task.", "sources": [ { "source": "S-1607", "supports": "definition of proofs of space as the storage analogue of proofs of work; initialization and execution phases; construction with pebbling-hard graphs and Merkle hash trees in the random oracle model", "locator": "abstract; §2" }, { "source": "S-0032", "supports": "proofs of secure erasure: the verifier fills the prover's memory with random data and queries random blocks; isolation assumption relaxed to slow communication with an external conspirator", "locator": "abstract; §2" }, { "source": "S-0018", "supports": "optional memory challenges using response latency; memory wiping with incompressible noise to remove residual capacity for hidden workloads", "locator": "system architecture; §5.1.2" } ], "related": [ "K-0011", "K-0024", "K-0016" ], "type": "concept", "url": "https://trustbutveri.fyi/concepts/proof-of-space/", "source_file": "content/concepts/proof-of-space.md", "flags_all": [], "body_markdown": "A proof of space is a protocol in which a [[K-0001|prover]] convinces a verifier that it is dedicating a given amount of storage, rather than computation as in a [[K-0011|proof of work]] [[S-1607]].\n\nIn an initialization phase the prover stores data of the required size. Later the verifier sends random challenges that the prover can answer quickly only if it still holds the data, because the data are expensive to regenerate [[S-1607]]. Dziembowski and colleagues built secure schemes in the random-oracle model from graphs with high \"pebbling complexity\" and Merkle hash trees [[S-1607]].\n\nA related primitive, the proof of secure erasure, uses memory-filling challenges to show that a device's memory has been overwritten [[S-0032]]. Most earlier protocols required the prover to be isolated during the protocol. Bursuc and colleagues relax this to slow communication with an outside accomplice [[S-0032]]. One low-trust system design applies these ideas with optional memory challenges that use response latency to check whether data is present, and with memory wiping to remove residual capacity for hidden workloads, as in [[M-0016|timed memory-occupation challenges]] and [[M-0015|memory wiping and proofs of secure erasure]] [[S-0018]].", "body_text": "A proof of space is a protocol in which a prover convinces a verifier that it is dedicating a given amount of storage, rather than computation as in a proof of work [S-1607]. In an initialization phase the prover stores data of the required size. Later the verifier sends random challenges that the prover can answer quickly only if it still holds the data, because the data are expensive to regenerate [S-1607]. Dziembowski and colleagues built secure schemes in the random-oracle model from graphs with high \"pebbling complexity\" and Merkle hash trees [S-1607]. A related primitive, the proof of secure erasure, uses memory-filling challenges to show that a device's memory has been overwritten [S-0032]. Most earlier protocols required the prover to be isolated during the protocol. Bursuc and colleagues relax this to slow communication with an outside accomplice [S-0032]. One low-trust system design applies these ideas with optional memory challenges that use response latency to check whether data is present, and with memory wiping to remove residual capacity for hidden workloads, as in timed memory-occupation challenges and memory wiping and proofs of secure erasure [S-0018].", "referenced_by": [ { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0011", "title": "Proof of (useful) work", "url": "https://trustbutveri.fyi/concepts/proof-of-useful-work/" } ] }, { "id": "K-0011", "slug": "proof-of-useful-work", "title": "Proof of (useful) work", "aliases": [ "proof of work", "PoW", "proof of useful work", "PoUW" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Evidence that a party spent a given amount of computation; in useful variants, the same work can also solve a problem someone wants solved.", "sources": [ { "source": "S-1607", "supports": "proofs of work proposed by Dwork and Naor: requester dedicates non-trivial computational work to each request; original uses against spam and denial of service; Bitcoin double spending", "locator": "abstract" }, { "source": "S-0005", "supports": "mining hashes message variants until one meets a target, which a verifier can check quickly", "locator": "'Proof-of-Work methods for crypto mining', in 'Verifying That Known Compute is Not Being Used for a Large Training Run'" }, { "source": "S-1608", "supports": "PoWs based on Orthogonal Vectors, 3SUM and All-Pairs Shortest Path whose work is useful; evaluation cannot be amortized across instances; energy waste motivation", "locator": "abstract" }, { "source": "S-1609", "supports": "PoUW for arbitrary matrix multiplication with 1+o(1) multiplicative overhead; GPUs could do AI work and mining at once", "locator": "abstract" }, { "source": "S-1105", "supports": "Pearl adapts the matrix-multiplication PoUW to FP8 on GPUs (provider-reported)", "locator": "abstract; §2" }, { "source": "S-0071", "supports": "independent measurement of Pearl's mainnet: random matrices pass verification", "locator": "abstract; measurement and verification sections" }, { "source": "S-1102", "supports": "proposal to use proof-of-work accounting to bound compute for unmonitored activity; needs a credible estimate of available compute; cannot discover an undeclared data centre (provider's own proposal)", "locator": "whole post" } ], "related": [ "K-0012", "K-0016" ], "type": "concept", "url": "https://trustbutveri.fyi/concepts/proof-of-useful-work/", "source_file": "content/concepts/proof-of-useful-work.md", "flags_all": [], "body_markdown": "A proof of work is evidence that a [[K-0001|prover]] spent a non-trivial amount of computation, which a verifier can check quickly; a proof of useful work lets that computation also solve a problem someone wants solved [[S-1607]] [[S-0005]] [[S-1608]].\n\nDwork and Naor proposed proofs of work to protect shared resources, for example against spam and denial of service, and Bitcoin later used them to prevent double spending [[S-1607]]. Conventional proofs of work waste the computation, so Ball and colleagues built ones whose work solves problems such as Orthogonal Vectors and 3SUM, and showed that the work cannot be amortized across instances [[S-1608]]. Komargodski and Weinstein give a proof of useful work for arbitrary matrix multiplication with 1 + o(1) multiplicative overhead, so that GPUs could do AI work and blockchain mining at once [[S-1609]]. [[I-0004|Pearl]]'s specification adapts the construction to FP8 matrix multiplication [[S-1105]]. The proof does not show where the matrices came from. On Pearl's network, an independent study found that random matrices passed verification [[S-0071]].\n\nAttestable proposes using proof-of-work accounting to bound the compute left for unmonitored activity, as in [[M-0007|proofs of useful work for capacity accounting]] [[S-1102]]. A proof shows that work was done, not that no other work was done: Attestable notes that its scheme needs a credible estimate of the compute available and cannot discover a data centre that was never declared [[S-1102]].", "body_text": "A proof of work is evidence that a prover spent a non-trivial amount of computation, which a verifier can check quickly; a proof of useful work lets that computation also solve a problem someone wants solved [S-1607] [S-0005] [S-1608]. Dwork and Naor proposed proofs of work to protect shared resources, for example against spam and denial of service, and Bitcoin later used them to prevent double spending [S-1607]. Conventional proofs of work waste the computation, so Ball and colleagues built ones whose work solves problems such as Orthogonal Vectors and 3SUM, and showed that the work cannot be amortized across instances [S-1608]. Komargodski and Weinstein give a proof of useful work for arbitrary matrix multiplication with 1 + o(1) multiplicative overhead, so that GPUs could do AI work and blockchain mining at once [S-1609]. Pearl's specification adapts the construction to FP8 matrix multiplication [S-1105]. The proof does not show where the matrices came from. On Pearl's network, an independent study found that random matrices passed verification [S-0071]. Attestable proposes using proof-of-work accounting to bound the compute left for unmonitored activity, as in proofs of useful work for capacity accounting [S-1102]. A proof shows that work was done, not that no other work was done: Attestable notes that its scheme needs a credible estimate of the compute available and cannot discover a data centre that was never declared [S-1102].", "referenced_by": [ { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" }, { "id": "K-0012", "title": "Proof of space", "url": "https://trustbutveri.fyi/concepts/proof-of-space/" } ] }, { "id": "K-0001", "slug": "prover", "title": "Prover", "aliases": [], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "The party that makes a claim about its own AI hardware, models or workloads and supplies the evidence a verifier checks.", "sources": [ { "source": "S-1606", "supports": "a prover tries to convince a verifier that a statement is true", "locator": "p. 1" }, { "source": "S-0004", "supports": "Prover defined as a state claiming to demonstrate its compliance with an agreement", "locator": "fn. 20" }, { "source": "S-0029", "supports": "Prover as an ML developer or state, modelled as a covert adversary; unlimited physical access can undermine attestation and signed-firmware enforcement; verifier re-runs training segments from reported checkpoints and data batches", "locator": "§2; §3.1; §5.1" }, { "source": "S-0002", "supports": "Prover may be a private institution or a government constraining companies in its territory", "locator": "§3.1" }, { "source": "S-1603", "supports": "Attester role: a device whose evidence is appraised to infer how far it is trustworthy", "locator": "§4 Terminology" }, { "source": "S-0023", "supports": "proof that an output came from running the model, without revealing its parameters", "locator": "abstract" } ], "related": [ "K-0002", "K-0004", "K-0018", "K-0003" ], "type": "concept", "url": "https://trustbutveri.fyi/concepts/prover/", "source_file": "content/concepts/prover.md", "flags_all": [], "body_markdown": "A prover is the party that asserts a claim, such as compliance with an agreement, and tries to convince a [[K-0002|verifier]] that it is true [[S-1606]] [[S-0004]].\n\nIn AI verification the prover may be a developer, another private institution or a state; under an international agreement it may be a government that constrains companies within its territory [[S-0029]] [[S-0002]]. The prover usually owns and physically controls the hardware being checked, and Shavit notes that unlimited physical access could undermine a chip's attestation and signed-firmware protections [[S-0029]]. He models the prover as a covert adversary that breaks the rules only if it expects to avoid detection [[S-0029]]. In [[K-0004|remote attestation]] the corresponding role is the attester, a device whose evidence is appraised to judge how far it can be trusted [[S-1603]].\n\nWhat the prover supplies depends on the mechanism:\n- In [[M-0006|proof-of-learning]], it reports checkpoints and data batches, so that the verifier can re-run segments of training [[S-0029]].\n- In [[M-0004|zero-knowledge proofs of inference]], it generates a proof that an output came from running its model, without revealing the weights [[S-0023]].\n- In [[M-0008|TEE remote attestation]], its hardware produces signed evidence about the software it runs [[S-1603]].", "body_text": "A prover is the party that asserts a claim, such as compliance with an agreement, and tries to convince a verifier that it is true [S-1606] [S-0004]. In AI verification the prover may be a developer, another private institution or a state; under an international agreement it may be a government that constrains companies within its territory [S-0029] [S-0002]. The prover usually owns and physically controls the hardware being checked, and Shavit notes that unlimited physical access could undermine a chip's attestation and signed-firmware protections [S-0029]. He models the prover as a covert adversary that breaks the rules only if it expects to avoid detection [S-0029]. In remote attestation the corresponding role is the attester, a device whose evidence is appraised to judge how far it can be trusted [S-1603]. What the prover supplies depends on the mechanism: - In proof-of-learning, it reports checkpoints and data batches, so that the verifier can re-run segments of training [S-0029]. - In zero-knowledge proofs of inference, it generates a proof that an output came from running its model, without revealing the weights [S-0023]. - In TEE remote attestation, its hardware produces signed evidence about the software it runs [S-1603].", "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0014", "title": "EZKL", "url": "https://trustbutveri.fyi/implementations/ezkl/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "I-0003", "title": "zkLLM", "url": "https://trustbutveri.fyi/implementations/zkllm/" }, { "id": "K-0012", "title": "Proof of space", "url": "https://trustbutveri.fyi/concepts/proof-of-space/" }, { "id": "K-0011", "title": "Proof of (useful) work", "url": "https://trustbutveri.fyi/concepts/proof-of-useful-work/" }, { "id": "K-0018", "title": "Threat model", "url": "https://trustbutveri.fyi/concepts/threat-model/" }, { "id": "K-0016", "title": "Undeclared compute", "url": "https://trustbutveri.fyi/concepts/undeclared-compute/" }, { "id": "K-0002", "title": "Verifier", "url": "https://trustbutveri.fyi/concepts/verifier/" }, { "id": "K-0010", "title": "Zero-knowledge proof", "url": "https://trustbutveri.fyi/concepts/zero-knowledge-proof/" }, { "id": "O-0205", "title": "Oxford Martin AI Governance Initiative", "url": "https://trustbutveri.fyi/organizations/oxford-martin-ai-governance-initiative/" } ] }, { "id": "K-0009", "slug": "recomputation", "title": "Recomputation", "aliases": [ "re-execution", "replay" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Checking a claimed computation by re-running all of it, or a random sample, on hardware the verifier trusts and comparing the results.", "sources": [ { "source": "S-0029", "supports": "full re-running of large training is infeasible; verifier re-runs segments from a reported checkpoint with the reported data batches and accepts if close to the next checkpoint; jointly trusted air-gapped cluster for confidentiality", "locator": "§5.1; §5.2" }, { "source": "S-0016", "supports": "trusted reference implementation recomputes predictions for generated tokens; nondeterminism makes legitimate variation hard to tell from real problems", "locator": "abstract" }, { "source": "S-0015", "supports": "inference verification against a reference to limit steganographic weight exfiltration", "locator": "abstract" }, { "source": "S-0017", "supports": "prover commits to sampled weights before learning whether a step will be audited; correctness vs completeness; schemes address correctness only", "locator": "pre-training scheme; introduction" }, { "source": "S-0018", "supports": "replay of the prover's data in air-gapped auditing environments, checked against committed hashes", "locator": "§1; §5.1; §5.2.1" } ], "related": [ "K-0008", "K-0020", "K-0024" ], "type": "concept", "url": "https://trustbutveri.fyi/concepts/recomputation/", "source_file": "content/concepts/recomputation.md", "flags_all": [], "body_markdown": "Recomputation checks a claimed computation by re-running it, or a random sample of it, from the same inputs on hardware the [[K-0002|verifier]] trusts, and comparing the results with those reported [[S-0029]] [[S-0016]].\n\nRe-running a large training run in full would be very costly, so in Shavit's framework the verifier re-runs selected segments, starting from a reported checkpoint and applying the reported data batches, and accepts if the result is close to the next reported checkpoint; this is the basis of [[M-0006|proof-of-learning and training-transcript verification]] [[S-0029]]. For inference, a trusted reference implementation recomputes what the model should have predicted at each generated token [[S-0016]], as in [[M-0001|sampled inference recomputation]]; the same check can limit how much of a model's weights can be hidden in its responses ([[M-0024]]) [[S-0015]]. Amodo Design distinguishes correctness, meaning that the workloads run match those declared, from completeness, meaning that every workload is reported, and notes that its recomputation schemes address only correctness [[S-0017]]. Recomputation also needs:\n- **Access to inputs and weights.** This raises confidentiality problems, which Shavit addresses with a jointly trusted, air-gapped cluster [[S-0029]]; one low-trust design replays the prover's data in air-gapped auditing environments and checks it against hashes committed earlier [[S-0018]].\n- **Commitment before sampling.** The prover must fix its records, for example by [[K-0024|committing]] to a hash of sampled weights, before it learns which step will be audited [[S-0017]].\n- **A way to handle [[K-0008|numerical nondeterminism]].** Benign noise otherwise makes legitimate variation hard to tell from real problems [[S-0016]].", "body_text": "Recomputation checks a claimed computation by re-running it, or a random sample of it, from the same inputs on hardware the verifier trusts, and comparing the results with those reported [S-0029] [S-0016]. Re-running a large training run in full would be very costly, so in Shavit's framework the verifier re-runs selected segments, starting from a reported checkpoint and applying the reported data batches, and accepts if the result is close to the next reported checkpoint; this is the basis of proof-of-learning and training-transcript verification [S-0029]. For inference, a trusted reference implementation recomputes what the model should have predicted at each generated token [S-0016], as in sampled inference recomputation; the same check can limit how much of a model's weights can be hidden in its responses (Bounding unexplained information in outputs) [S-0015]. Amodo Design distinguishes correctness, meaning that the workloads run match those declared, from completeness, meaning that every workload is reported, and notes that its recomputation schemes address only correctness [S-0017]. Recomputation also needs: - Access to inputs and weights. This raises confidentiality problems, which Shavit addresses with a jointly trusted, air-gapped cluster [S-0029]; one low-trust design replays the prover's data in air-gapped auditing environments and checks it against hashes committed earlier [S-0018]. - Commitment before sampling. The prover must fix its records, for example by committing to a hash of sampled weights, before it learns which step will be audited [S-0017]. - A way to handle numerical nondeterminism. Benign noise otherwise makes legitimate variation hard to tell from real problems [S-0016].", "referenced_by": [ { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "I-0016", "title": "Batch-invariant inference kernels (Thinking Machines)", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" }, { "id": "I-0015", "title": "Verde and RepOps (Gensyn)", "url": "https://trustbutveri.fyi/implementations/gensyn-verde-repops/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "I-0008", "title": "SASH confidential network logger", "url": "https://trustbutveri.fyi/implementations/sash-confidential-network-logger/" }, { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0008", "title": "Numerical nondeterminism", "url": "https://trustbutveri.fyi/concepts/numerical-nondeterminism/" }, { "id": "K-0020", "title": "Sampling and assurance", "url": "https://trustbutveri.fyi/concepts/sampling-and-assurance/" } ] }, { "id": "K-0004", "slug": "remote-attestation", "title": "Remote attestation", "aliases": [ "attestation" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "A process in which a device sends signed evidence about its state, such as software measurements, so a remote party can judge its trustworthiness.", "sources": [ { "source": "S-1603", "supports": "purpose of attestation; Attester, Evidence, Verifier, Relying Party, Attestation Result; evidence generated with key material; nonces for freshness", "locator": "§1; §3.1; §4; §10.2" }, { "source": "S-1600", "supports": "NIST definition of attestation as signing measurements securely stored in hardware, which the requester validates", "locator": "term: attestation (NIST SP 1800-19B)" }, { "source": "S-1602", "supports": "Root of Trust for Reporting provides authenticity and non-repudiation when attesting to platform characteristics", "locator": "p. 9" }, { "source": "S-1604", "supports": "attestability as an optional TEE attribute that confidential computing requires; attestation as validation of a hardware-signed report of TCB measurements", "locator": "TEE attributes; attestation" }, { "source": "S-0005", "supports": "an extracted private key would let a chip's location be spoofed", "locator": "On-chip mechanisms for location verification" } ], "related": [ "K-0005", "K-0006", "K-0019" ], "type": "concept", "url": "https://trustbutveri.fyi/concepts/remote-attestation/", "source_file": "content/concepts/remote-attestation.md", "flags_all": [], "body_markdown": "Remote attestation is a process in which a device produces evidence about its own state, signed with keys it protects, so that a remote party can decide whether to consider it trustworthy [[S-1603]] [[S-1600]].\n\nThe IETF RATS architecture names three roles:\n- **Attester.** It produces evidence: claims that may include configuration data, measurements and telemetry, typically signed with its key material [[S-1603]].\n- **Verifier.** It appraises the evidence against reference values and an appraisal policy, and produces attestation results [[S-1603]].\n- **Relying party.** It uses the results to decide whether to treat the attester as trustworthy [[S-1603]].\n\nIn the Trusted Computing Group's model, a [[K-0005|root of trust]] for reporting gives attested platform characteristics their authenticity and non-repudiation [[S-1602]]. A nonce sent by the appraising party and signed into the evidence shows that the evidence is fresh rather than replayed [[S-1603]]. Attestation is distinct from isolation. A [[K-0006|trusted execution environment]] protects a workload, while attestation lets a remote party check what a genuine TEE loaded, and confidential computing requires both [[S-1604]]. In AI verification, attestation underlies [[M-0008|TEE remote attestation for AI workloads]] and [[M-0018|chip location verification]]. For location, Scher and Thiergart note that if the chip's private key were extracted, its location could be spoofed [[S-0005]].", "body_text": "Remote attestation is a process in which a device produces evidence about its own state, signed with keys it protects, so that a remote party can decide whether to consider it trustworthy [S-1603] [S-1600]. The IETF RATS architecture names three roles: - Attester. It produces evidence: claims that may include configuration data, measurements and telemetry, typically signed with its key material [S-1603]. - Verifier. It appraises the evidence against reference values and an appraisal policy, and produces attestation results [S-1603]. - Relying party. It uses the results to decide whether to treat the attester as trustworthy [S-1603]. In the Trusted Computing Group's model, a root of trust for reporting gives attested platform characteristics their authenticity and non-repudiation [S-1602]. A nonce sent by the appraising party and signed into the evidence shows that the evidence is fresh rather than replayed [S-1603]. Attestation is distinct from isolation. A trusted execution environment protects a workload, while attestation lets a remote party check what a genuine TEE loaded, and confidential computing requires both [S-1604]. In AI verification, attestation underlies TEE remote attestation for AI workloads and chip location verification. For location, Scher and Thiergart note that if the chip's private key were extracted, its location could be spoofed [S-0005].", "referenced_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/" }, { "id": "I-0007", "title": "Attestable Audits", "url": "https://trustbutveri.fyi/implementations/attestable-audits/" }, { "id": "I-0009", "title": "Lucid sovereignty (location) certificates", "url": "https://trustbutveri.fyi/implementations/lucid-location-certificates/" }, { "id": "I-0017", "title": "PySyft double-blind evaluations", "url": "https://trustbutveri.fyi/implementations/pysyft-double-blind-evaluations/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" }, { "id": "K-0019", "title": "Evidence binding", "url": "https://trustbutveri.fyi/concepts/evidence-binding/" }, { "id": "K-0007", "title": "Hardware-enabled mechanism (HEM)", "url": "https://trustbutveri.fyi/concepts/hardware-enabled-mechanism/" }, { "id": "K-0001", "title": "Prover", "url": "https://trustbutveri.fyi/concepts/prover/" }, { "id": "K-0005", "title": "Root of trust", "url": "https://trustbutveri.fyi/concepts/root-of-trust/" }, { "id": "K-0006", "title": "Trusted execution environment (TEE)", "url": "https://trustbutveri.fyi/concepts/trusted-execution-environment/" }, { "id": "K-0002", "title": "Verifier", "url": "https://trustbutveri.fyi/concepts/verifier/" } ] }, { "id": "K-0005", "slug": "root-of-trust", "title": "Root of trust", "aliases": [ "RoT", "hardware root of trust" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "A component that anchors security functions such as measurement, storage and reporting, and must be trusted because its misbehaviour cannot be detected.", "sources": [ { "source": "S-1601", "supports": "definition of RoT and its security-specific functions; must always behave as expected because its misbehaviour cannot be detected; first element of a chain of trust anchoring more complex functionality", "locator": "§3.3, pp. 11–12" }, { "source": "S-1600", "supports": "roots of trust as highly reliable hardware, firmware and software components that must be secure by design", "locator": "term: roots_of_trust (NIST SP 800-172r3; NISTIR 8320)" }, { "source": "S-1602", "supports": "RTM and RTR definitions", "locator": "p. 9" }, { "source": "S-0029", "supports": "hardware roots of trust on ML chips; open-source RoT designs so verifiers can check for backdoors", "locator": "§6.2" }, { "source": "S-0014", "supports": "key provenance as the most difficult part of TEE hardware to accept; vendors generate and fuse keys; holder or certifier of a hardware key can produce valid attestation reports", "locator": "Limitations" } ], "related": [ "K-0004", "K-0006", "K-0007", "K-0015" ], "type": "concept", "url": "https://trustbutveri.fyi/concepts/root-of-trust/", "source_file": "content/concepts/root-of-trust.md", "flags_all": [], "body_markdown": "A root of trust is a hardware, firmware or software element that forms the basis of security functions such as measurement, storage, reporting, recovery, verification and update, and that must always behave as expected because its misbehaviour cannot be detected [[S-1601]] [[S-1600]].\n\nIt is typically the first element in a chain of trust and anchors more complex functions built on it [[S-1601]]. Because it is trusted by assumption, it must be secure by design [[S-1600]]. The Trusted Computing Group distinguishes a root of trust for measurement, which makes the first integrity measurement and adds it to a tamper-resistant log, from a root of trust for reporting, which vouches for platform characteristics during [[K-0004|remote attestation]] [[S-1602]]. Shavit's framework needs hardware roots of trust on ML chips, and calls for open-source designs so that independent verifiers can check them for backdoors [[S-0029]]. A 2026 MIRI Technical Governance Team post on [[M-0008|TEE-based attestation]] calls key provenance the most difficult part of TEE hardware to accept, because a few hardware vendors generated the keys and fused them onto the chips, and whoever holds a hardware key or can certify one can in principle produce valid attestation reports [[S-0014]].", "body_text": "A root of trust is a hardware, firmware or software element that forms the basis of security functions such as measurement, storage, reporting, recovery, verification and update, and that must always behave as expected because its misbehaviour cannot be detected [S-1601] [S-1600]. It is typically the first element in a chain of trust and anchors more complex functions built on it [S-1601]. Because it is trusted by assumption, it must be secure by design [S-1600]. The Trusted Computing Group distinguishes a root of trust for measurement, which makes the first integrity measurement and adds it to a tamper-resistant log, from a root of trust for reporting, which vouches for platform characteristics during remote attestation [S-1602]. Shavit's framework needs hardware roots of trust on ML chips, and calls for open-source designs so that independent verifiers can check them for backdoors [S-0029]. A 2026 MIRI Technical Governance Team post on TEE-based attestation calls key provenance the most difficult part of TEE hardware to accept, because a few hardware vendors generated the keys and fused them onto the chips, and whoever holds a hardware key or can certify one can in principle produce valid attestation reports [S-0014].", "referenced_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/" }, { "id": "I-0009", "title": "Lucid sovereignty (location) certificates", "url": "https://trustbutveri.fyi/implementations/lucid-location-certificates/" }, { "id": "I-0010", "title": "RAND secure inference data center (SIDC) design", "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" }, { "id": "K-0019", "title": "Evidence binding", "url": "https://trustbutveri.fyi/concepts/evidence-binding/" }, { "id": "K-0007", "title": "Hardware-enabled mechanism (HEM)", "url": "https://trustbutveri.fyi/concepts/hardware-enabled-mechanism/" }, { "id": "K-0004", "title": "Remote attestation", "url": "https://trustbutveri.fyi/concepts/remote-attestation/" }, { "id": "K-0015", "title": "Tamper evidence and tamper resistance", "url": "https://trustbutveri.fyi/concepts/tamper-evidence/" }, { "id": "K-0018", "title": "Threat model", "url": "https://trustbutveri.fyi/concepts/threat-model/" }, { "id": "K-0006", "title": "Trusted execution environment (TEE)", "url": "https://trustbutveri.fyi/concepts/trusted-execution-environment/" } ] }, { "id": "K-0020", "slug": "sampling-and-assurance", "title": "Sampling and assurance", "aliases": [ "random sampling", "spot checks", "statistical assurance" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Checking a random sample of accelerators, workload segments or outputs rather than all of them, so that violations are caught with a calculable probability.", "sources": [ { "source": "S-0029", "supports": "number of chips to sample per monitoring period to catch at least one chip from a violating run with probability p; fewer samples for larger runs; the Prover cannot predict which chips are inspected; sampled segment recomputation", "locator": "§3.1–3.2, Equation 1, Table 1; §5.1" }, { "source": "S-0017", "supports": "hash commitment to sampled weights before the prover learns whether a step will be audited", "locator": "pre-training scheme" }, { "source": "S-0067", "supports": "recomputation server re-runs random samples of workload packets", "locator": "Concrete inference-only retrofitting proposal" }, { "source": "S-0018", "supports": "a covert adversary is caught if it fails to stay hidden even once; random sampling needs to catch only a single instance of cheating; physical security and randomly sampled inspections as the likely primary defences", "locator": "threat model; defence layers" } ], "related": [ "K-0009", "K-0024", "K-0003", "K-0018" ], "type": "concept", "url": "https://trustbutveri.fyi/concepts/sampling-and-assurance/", "source_file": "content/concepts/sampling-and-assurance.md", "flags_all": [], "body_markdown": "Sampling-based assurance checks a random sample of accelerators, workload segments or outputs instead of all of them, and chooses the sample size so that a violation is caught with a desired probability [[S-0029]].\n\nShavit gives a formula for how many accelerators a verifier must inspect in each monitoring period to find at least one accelerator used in a rule-violating training run with a chosen probability [[S-0029]]. The required number falls as the run occupies a larger share of the prover's accelerators, so larger runs need fewer inspections [[S-0029]]. Sampling works only if the prover cannot predict what will be checked [[S-0029]] or change its records once it knows. In one scheme, the prover [[K-0024|commits]] a hash of sampled weights at each training step before it learns whether that step will be audited [[S-0017]]. The same logic applies to [[K-0009|recomputation]] of random training segments in [[M-0006|proof-of-learning]] [[S-0029]] and of random workload samples in [[M-0003|reproducible computation packets]] [[S-0067]]. Against a covert adversary, sampling works through deterrence: one system overview notes that such an adversary is caught if it fails to stay hidden even once, and expects physical security and randomly sampled inspections to be the primary defences [[S-0018]].", "body_text": "Sampling-based assurance checks a random sample of accelerators, workload segments or outputs instead of all of them, and chooses the sample size so that a violation is caught with a desired probability [S-0029]. Shavit gives a formula for how many accelerators a verifier must inspect in each monitoring period to find at least one accelerator used in a rule-violating training run with a chosen probability [S-0029]. The required number falls as the run occupies a larger share of the prover's accelerators, so larger runs need fewer inspections [S-0029]. Sampling works only if the prover cannot predict what will be checked [S-0029] or change its records once it knows. In one scheme, the prover commits a hash of sampled weights at each training step before it learns whether that step will be audited [S-0017]. The same logic applies to recomputation of random training segments in proof-of-learning [S-0029] and of random workload samples in reproducible computation packets [S-0067]. Against a covert adversary, sampling works through deterrence: one system overview notes that such an adversary is caught if it fails to stay hidden even once, and expects physical security and randomly sampled inspections to be the primary defences [S-0018].", "referenced_by": [ { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "M-0005", "title": "Zero-knowledge proofs of training constraints", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-training-constraints/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "I-0005", "title": "Attestable zero-knowledge inference prover", "url": "https://trustbutveri.fyi/implementations/attestable-zk-inference/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "I-0008", "title": "SASH confidential network logger", "url": "https://trustbutveri.fyi/implementations/sash-confidential-network-logger/" }, { "id": "C-0001", "title": "Compute stock is at most a declared amount", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0024", "title": "Cryptographic commitment", "url": "https://trustbutveri.fyi/concepts/cryptographic-commitment/" }, { "id": "K-0003", "title": "Positive and negative claims", "url": "https://trustbutveri.fyi/concepts/positive-and-negative-claims/" }, { "id": "K-0009", "title": "Recomputation", "url": "https://trustbutveri.fyi/concepts/recomputation/" }, { "id": "K-0016", "title": "Undeclared compute", "url": "https://trustbutveri.fyi/concepts/undeclared-compute/" }, { "id": "K-0002", "title": "Verifier", "url": "https://trustbutveri.fyi/concepts/verifier/" } ] }, { "id": "K-0013", "slug": "side-channel", "title": "Side channel", "aliases": [ "side-channel attack", "covert channel" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "An unintended path by which information leaks from a system through effects of its operation, such as timing, power draw or electromagnetic emissions.", "sources": [ { "source": "S-1600", "supports": "NIST definitions of side-channel attack and covert channel", "locator": "terms: side_channel_attack (NIST SP 1800-21B; SP 800-63-4); covert_channel (CNSSI 4009-2015; NIST SP 800-53 Rev. 5)" }, { "source": "S-0043", "supports": "correlation electromagnetic analysis recovers neural-network parameters on an edge GPU device", "locator": "abstract" }, { "source": "S-0014", "supports": "microarchitectural side channels as a TEE limitation", "locator": "Limitations" }, { "source": "S-0038", "supports": "physical channels could bypass network monitoring; electromagnetic, acoustic, conducted power-line and fibre-optic channels in a data centre; shielded enclosures, jamming with sensing, inspection and filtering as defences", "locator": "sections on channels and defences" }, { "source": "S-0018", "supports": "suppressing covert bandwidth below kilobits per second judged much more achievable than zero", "locator": "§5.3.1" }, { "source": "S-0005", "supports": "utilization and power draw often differ between training and inference", "locator": "Workload classification with high-level chip measures" } ], "related": [ "K-0015", "K-0022", "K-0006", "K-0017" ], "type": "concept", "url": "https://trustbutveri.fyi/concepts/side-channel/", "source_file": "content/concepts/side-channel.md", "flags_all": [], "body_markdown": "A side channel is an unintended path through which information leaks from a system via observable effects of its operation, such as timing, power consumption, and electromagnetic or acoustic emissions [[S-1600]]. NIST defines a side-channel attack as one enabled by such leakage from a physical cryptosystem [[S-1600]].\n\nA covert channel is related but deliberate. Two cooperating parties use an unintended or unauthorized channel to transfer information in a way that violates a system's security policy [[S-1600]].\n\nSide channels can leak secrets that verification designs aim to protect. BarraCUDA used electromagnetic analysis to recover the parameters of neural networks running on an edge GPU [[S-0043]]. A 2026 MIRI Technical Governance Team post describes microarchitectural side channels as a limitation of [[K-0006|trusted execution environments]] [[S-0014]].\n\nCovert channels threaten the checks themselves. Another post from the same team describes electromagnetic, acoustic, power-line and fibre-optic channels that could carry data around a data centre's monitored network links. It proposes shielded enclosures, jamming combined with sensing, inspection and filtering, the approach of [[M-0022|side-channel suppression]] [[S-0038]]. The same author's low-trust system design judges that keeping covert bandwidth below kilobits per second is much more achievable than eliminating it [[S-0018]].\n\nPhysical signals can also serve the verifier. Training and inference often differ in accelerator utilization and power draw, which [[M-0021|workload classification]] uses [[S-0005]].", "body_text": "A side channel is an unintended path through which information leaks from a system via observable effects of its operation, such as timing, power consumption, and electromagnetic or acoustic emissions [S-1600]. NIST defines a side-channel attack as one enabled by such leakage from a physical cryptosystem [S-1600]. A covert channel is related but deliberate. Two cooperating parties use an unintended or unauthorized channel to transfer information in a way that violates a system's security policy [S-1600]. Side channels can leak secrets that verification designs aim to protect. BarraCUDA used electromagnetic analysis to recover the parameters of neural networks running on an edge GPU [S-0043]. A 2026 MIRI Technical Governance Team post describes microarchitectural side channels as a limitation of trusted execution environments [S-0014]. Covert channels threaten the checks themselves. Another post from the same team describes electromagnetic, acoustic, power-line and fibre-optic channels that could carry data around a data centre's monitored network links. It proposes shielded enclosures, jamming combined with sensing, inspection and filtering, the approach of side-channel suppression [S-0038]. The same author's low-trust system design judges that keeping covert bandwidth below kilobits per second is much more achievable than eliminating it [S-0018]. Physical signals can also serve the verifier. Training and inference often differ in accelerator utilization and power draw, which workload classification uses [S-0005].", "referenced_by": [ { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0017", "title": "Compartmentalization", "url": "https://trustbutveri.fyi/concepts/compartmentalization/" }, { "id": "K-0015", "title": "Tamper evidence and tamper resistance", "url": "https://trustbutveri.fyi/concepts/tamper-evidence/" }, { "id": "K-0006", "title": "Trusted execution environment (TEE)", "url": "https://trustbutveri.fyi/concepts/trusted-execution-environment/" }, { "id": "K-0022", "title": "Weight exfiltration", "url": "https://trustbutveri.fyi/concepts/weight-exfiltration/" } ] }, { "id": "K-0015", "slug": "tamper-evidence", "title": "Tamper evidence and tamper resistance", "aliases": [ "tamper evidence", "tamper resistance", "tamper response", "anti-tamper" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Tamper evidence makes interference detectable; tamper resistance makes it difficult or costly; tamper response reacts to it, often by erasing secrets.", "sources": [ { "source": "S-1605", "supports": "definitions of tamper evidence, tamper detection and tamper response; physical security Levels 2–4; superseded by FIPS 140-3", "locator": "§2.1 Glossary; §4.5; CSRC status page" }, { "source": "S-1600", "supports": "tamper resistant: makes alterations difficult, costly or both (definition written for data)", "locator": "term: tamper_resistant (NISTIR 8202)" }, { "source": "S-0029", "supports": "unlimited physical access can undermine attestation; inspections detect hard-to-hide hardware attacks", "locator": "§3.1" }, { "source": "S-0035", "supports": "flexHEG secure enclosure providing physical tamper protection", "locator": "abstract" }, { "source": "S-0067", "supports": "tamper-evident enclosures named among promising future directions and existing methods for physical security of AI computing infrastructure", "locator": "physical security measures" }, { "source": "S-1317", "supports": "94 seals studied; 1–3 low-tech defeats demonstrated for each, 132 in total; mean defeat time 4.3 minutes by one practised person", "locator": "abstract; results" } ], "related": [ "K-0005", "K-0007", "K-0013" ], "type": "concept", "url": "https://trustbutveri.fyi/concepts/tamper-evidence/", "source_file": "content/concepts/tamper-evidence.md", "flags_all": [], "body_markdown": "Tamper evidence is an external indication that someone has tried to compromise a device's physical security; tamper resistance makes such attempts difficult, costly or both; and tamper response is an automatic action, at minimum erasing plaintext keys, taken when tampering is detected [[S-1605]] [[S-1600]].\n\nThe US standard for cryptographic modules, FIPS 140-2, since superseded by FIPS 140-3, layers these properties [[S-1605]]:\n- **Level 2** requires evidence of tampering, such as tamper-evident coatings or seals, or pick-resistant locks on covers and doors [[S-1605]].\n- **Level 3** adds detection and response circuitry that zeroizes plaintext secret and private keys when covers or doors are opened [[S-1605]].\n- **Level 4** requires a complete envelope of protection intended to detect and respond to all unauthorized attempts at physical access [[S-1605]].\n\nThese properties matter because the prover usually controls the hardware: Shavit notes that unlimited physical access could undermine a chip's attestation, and relies on inspections to find hardware attacks that damage chips in ways that are hard to hide [[S-0029]]. The flexHEG proposal houses its guarantee processor in a secure enclosure that provides physical tamper protection ([[M-0009]]) [[S-0035]]. One verification plan names tamper-evident enclosures among \"promising future directions and existing methods\" for physical security of AI computing infrastructure [[S-0067]]. Protecting verifier equipment in the prover's facility, such as network taps and recomputation servers, is the subject of [[M-0017|tamper evidence for verifier devices]]. Seals can be defeated with simple methods: a 1996 Los Alamos study demonstrated low-tech defeats for each of the 94 passive and electronic seals it examined, with a mean defeat time of 4.3 minutes for one practised person [[S-1317]].", "body_text": "Tamper evidence is an external indication that someone has tried to compromise a device's physical security; tamper resistance makes such attempts difficult, costly or both; and tamper response is an automatic action, at minimum erasing plaintext keys, taken when tampering is detected [S-1605] [S-1600]. The US standard for cryptographic modules, FIPS 140-2, since superseded by FIPS 140-3, layers these properties [S-1605]: - Level 2 requires evidence of tampering, such as tamper-evident coatings or seals, or pick-resistant locks on covers and doors [S-1605]. - Level 3 adds detection and response circuitry that zeroizes plaintext secret and private keys when covers or doors are opened [S-1605]. - Level 4 requires a complete envelope of protection intended to detect and respond to all unauthorized attempts at physical access [S-1605]. These properties matter because the prover usually controls the hardware: Shavit notes that unlimited physical access could undermine a chip's attestation, and relies on inspections to find hardware attacks that damage chips in ways that are hard to hide [S-0029]. The flexHEG proposal houses its guarantee processor in a secure enclosure that provides physical tamper protection (Hardware-enabled guarantees (flexHEG) and guarantee processors) [S-0035]. One verification plan names tamper-evident enclosures among \"promising future directions and existing methods\" for physical security of AI computing infrastructure [S-0067]. Protecting verifier equipment in the prover's facility, such as network taps and recomputation servers, is the subject of tamper evidence for verifier devices. Seals can be defeated with simple methods: a 1996 Los Alamos study demonstrated low-tech defeats for each of the 94 passive and electronic seals it examined, with a mean defeat time of 4.3 minutes for one practised person [S-1317].", "referenced_by": [ { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "I-0010", "title": "RAND secure inference data center (SIDC) design", "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "K-0007", "title": "Hardware-enabled mechanism (HEM)", "url": "https://trustbutveri.fyi/concepts/hardware-enabled-mechanism/" }, { "id": "K-0005", "title": "Root of trust", "url": "https://trustbutveri.fyi/concepts/root-of-trust/" }, { "id": "K-0013", "title": "Side channel", "url": "https://trustbutveri.fyi/concepts/side-channel/" } ] }, { "id": "K-0018", "slug": "threat-model", "title": "Threat model", "aliases": [ "adversary model" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "An explicit statement of who might attack a system, what they can do, and which threats the design covers or leaves out of scope.", "sources": [ { "source": "S-0072", "supports": "threat model describes the capabilities an attacker is assumed to deploy, including information, computing capability and control of the system; purpose is to identify threats of concern and rule others out of scope; nearly every system is vulnerable to a sufficiently dedicated and resourceful attacker", "locator": "§3" }, { "source": "S-1600", "supports": "NIST definition of threat modeling as a form of risk assessment modelling attack and defence sides", "locator": "term: threat_modeling (NIST SP 800-53 Rev. 5)" }, { "source": "S-0029", "supports": "Prover as covert adversary; unlimited physical access undermines attestation and signed firmware; physical inspections after the fact may detect such tampering", "locator": "§2; §3.1" }, { "source": "S-0004", "supports": "states seeking to demonstrate compliance while seeking ways to circumvent verification, a framing the report says is sometimes termed the covert adversary", "locator": "§1.1.2, fn. 40" }, { "source": "S-0018", "supports": "nation-state adversaries; no single chain of trust; redundancy across devices trusted unilaterally by each party; malicious verifier", "locator": "threat model section" } ], "related": [ "K-0001", "K-0002", "K-0005" ], "type": "concept", "url": "https://trustbutveri.fyi/concepts/threat-model/", "source_file": "content/concepts/threat-model.md", "flags_all": [], "body_markdown": "A threat model describes the capabilities an attacker is assumed to be able to use against a system, such as the information, computing power and control of the system available to it [[S-0072]].\n\nIts purpose is to identify the threats a design must withstand and to rule others explicitly out of scope, since nearly every security system is vulnerable to a sufficiently dedicated and resourceful attacker [[S-0072]]. NIST treats threat modelling as a form of risk assessment that models both the attack and the defence side of a system [[S-1600]]. Threat models used in AI verification differ in how far each party is trusted:\n- **Covert adversary.** Shavit models the [[K-0001|prover]] as willing to break the rules only if it expects not to be detected [[S-0029]]. The Oxford Martin report uses the same framing for states that seek to demonstrate compliance to each other while also looking for ways to circumvent verification [[S-0004]].\n- **Mutual distrust.** One low-trust system overview assumes nation-state adversaries on both sides, including a [[K-0002|verifier]] that may try to exfiltrate the prover's secrets, and relies on redundant checks across devices that each party trusts unilaterally, instead of a single chain of trust [[S-0018]].\n\nPhysical access is a recurring issue: Shavit notes that a prover with unlimited physical access to a chip could undermine its attestation and signed-firmware protections, and proposes physical inspections after the fact to detect such tampering [[S-0029]].", "body_text": "A threat model describes the capabilities an attacker is assumed to be able to use against a system, such as the information, computing power and control of the system available to it [S-0072]. Its purpose is to identify the threats a design must withstand and to rule others explicitly out of scope, since nearly every security system is vulnerable to a sufficiently dedicated and resourceful attacker [S-0072]. NIST treats threat modelling as a form of risk assessment that models both the attack and the defence side of a system [S-1600]. Threat models used in AI verification differ in how far each party is trusted: - Covert adversary. Shavit models the prover as willing to break the rules only if it expects not to be detected [S-0029]. The Oxford Martin report uses the same framing for states that seek to demonstrate compliance to each other while also looking for ways to circumvent verification [S-0004]. - Mutual distrust. One low-trust system overview assumes nation-state adversaries on both sides, including a verifier that may try to exfiltrate the prover's secrets, and relies on redundant checks across devices that each party trusts unilaterally, instead of a single chain of trust [S-0018]. Physical access is a recurring issue: Shavit notes that a prover with unlimited physical access to a chip could undermine its attestation and signed-firmware protections, and proposes physical inspections after the fact to detect such tampering [S-0029].", "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" }, { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" }, { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "I-0009", "title": "Lucid sovereignty (location) certificates", "url": "https://trustbutveri.fyi/implementations/lucid-location-certificates/" }, { "id": "I-0017", "title": "PySyft double-blind evaluations", "url": "https://trustbutveri.fyi/implementations/pysyft-double-blind-evaluations/" }, { "id": "I-0010", "title": "RAND secure inference data center (SIDC) design", "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/" }, { "id": "K-0001", "title": "Prover", "url": "https://trustbutveri.fyi/concepts/prover/" }, { "id": "K-0020", "title": "Sampling and assurance", "url": "https://trustbutveri.fyi/concepts/sampling-and-assurance/" }, { "id": "K-0002", "title": "Verifier", "url": "https://trustbutveri.fyi/concepts/verifier/" } ] }, { "id": "K-0006", "slug": "trusted-execution-environment", "title": "Trusted execution environment (TEE)", "aliases": [ "TEE", "enclave", "confidential computing", "confidential VM" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "An execution area protected by the processor that keeps the data inside confidential and unaltered, and the code unaltered, even from the host's own software.", "sources": [ { "source": "S-1600", "supports": "NIST definition of TEE as an area or enclave protected by a system processor", "locator": "term: trusted_execution_environment (NISTIR 8320)" }, { "source": "S-1604", "supports": "confidential computing defined as computation in a hardware-based, attested TEE; core TEE attributes (data confidentiality, data integrity, code integrity); optional attributes including code confidentiality and attestability; attestation as validation of a hardware-signed report of TCB measurements; confidential VMs protected from the hypervisor and host OS", "locator": "definitions; TEE attributes; attestation; confidential VMs" }, { "source": "S-0012", "supports": "property attestation using confidential VMs with Intel TDX and NVIDIA H100 GPUs", "locator": "abstract" }, { "source": "S-0009", "supports": "TEE-based verifiable benchmarks keeping model and test data confidential", "locator": "abstract" }, { "source": "S-0014", "supports": "attestation-key holder can produce valid reports; microarchitectural side channels and physical attacks such as bus interposition as limitations", "locator": "Limitations" } ], "related": [ "K-0004", "K-0005", "K-0013" ], "type": "concept", "url": "https://trustbutveri.fyi/concepts/trusted-execution-environment/", "source_file": "content/concepts/trusted-execution-environment.md", "flags_all": [], "body_markdown": "A trusted execution environment (TEE) is an execution area that the processor protects, so that software outside it, including the host operating system and hypervisor, cannot read the data inside or alter its code and data [[S-1600]] [[S-1604]].\n\nThe Confidential Computing Consortium lists data confidentiality, data integrity and code integrity as the core attributes of a TEE, and code confidentiality and attestability as optional [[S-1604]]. Isolation and [[K-0004|remote attestation]] are separate properties. The TEE protects a workload, and attestation, the validation of a hardware-signed report of what was loaded, lets a remote party check it [[S-1604]]. The consortium defines confidential computing as computation in a hardware-based, attested TEE, so it requires both [[S-1604]]. A TEE can cover a whole confidential virtual machine and extend to accelerators [[S-1604]]. PAL\\*M, for example, pairs Intel TDX confidential VMs with NVIDIA H100 GPUs to attest properties of generative models [[S-0012]]. In AI verification, TEEs underpin [[M-0008|TEE remote attestation for AI workloads]] and [[M-0025|confidential multi-party verification]], such as running safety benchmarks while keeping both the model and the test data confidential [[S-0009]]. A 2026 MIRI Technical Governance Team post notes that whoever holds the hardware's attestation key can produce valid reports, and that microarchitectural [[K-0013|side channels]] and physical attacks such as bus interposition remain concerns [[S-0014]].", "body_text": "A trusted execution environment (TEE) is an execution area that the processor protects, so that software outside it, including the host operating system and hypervisor, cannot read the data inside or alter its code and data [S-1600] [S-1604]. The Confidential Computing Consortium lists data confidentiality, data integrity and code integrity as the core attributes of a TEE, and code confidentiality and attestability as optional [S-1604]. Isolation and remote attestation are separate properties. The TEE protects a workload, and attestation, the validation of a hardware-signed report of what was loaded, lets a remote party check it [S-1604]. The consortium defines confidential computing as computation in a hardware-based, attested TEE, so it requires both [S-1604]. A TEE can cover a whole confidential virtual machine and extend to accelerators [S-1604]. PAL*M, for example, pairs Intel TDX confidential VMs with NVIDIA H100 GPUs to attest properties of generative models [S-0012]. In AI verification, TEEs underpin TEE remote attestation for AI workloads and confidential multi-party verification, such as running safety benchmarks while keeping both the model and the test data confidential [S-0009]. A 2026 MIRI Technical Governance Team post notes that whoever holds the hardware's attestation key can produce valid reports, and that microarchitectural side channels and physical attacks such as bus interposition remain concerns [S-0014].", "referenced_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/" }, { "id": "I-0007", "title": "Attestable Audits", "url": "https://trustbutveri.fyi/implementations/attestable-audits/" }, { "id": "I-0009", "title": "Lucid sovereignty (location) certificates", "url": "https://trustbutveri.fyi/implementations/lucid-location-certificates/" }, { "id": "I-0017", "title": "PySyft double-blind evaluations", "url": "https://trustbutveri.fyi/implementations/pysyft-double-blind-evaluations/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" }, { "id": "K-0004", "title": "Remote attestation", "url": "https://trustbutveri.fyi/concepts/remote-attestation/" }, { "id": "K-0005", "title": "Root of trust", "url": "https://trustbutveri.fyi/concepts/root-of-trust/" }, { "id": "K-0013", "title": "Side channel", "url": "https://trustbutveri.fyi/concepts/side-channel/" } ] }, { "id": "K-0016", "slug": "undeclared-compute", "title": "Undeclared compute", "aliases": [ "dark compute", "hidden compute", "covert compute" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "AI-relevant hardware, or uses of declared hardware, that a prover has not reported, and that verification must therefore detect or rule out.", "sources": [ { "source": "S-0002", "supports": "Subgoal 2: no undeclared uses of declared clusters (2.A) and no undeclared clusters in known data centres or standalone (2.B)", "locator": "§3.2, Figure 4" }, { "source": "S-0029", "supports": "hundreds of thousands of ML chips already sold, many lacking the required security features; supply-chain monitoring and chip-owner directory", "locator": "§1.2; §6; §6.1" }, { "source": "S-0005", "supports": "millions of AI-relevant chips exist, with no central tracking to the authors' knowledge; detecting secret data centres unlikely to be load-bearing because AI compute will likely be too easy to hide", "locator": "Verifying the location of AI compute" }, { "source": "S-0053", "supports": "algorithmic efficiency and decentralized training could undermine compute detectability", "locator": "limitations of compute governance" }, { "source": "S-0062", "supports": "national technical means (remote sensing, energy monitoring, customs, financial intelligence) and whistleblowers", "locator": "Verification methods; Table 1" }, { "source": "S-0018", "supports": "memory wiping to remove residual capacity for hidden workloads", "locator": "§5.1.2" } ], "related": [ "K-0003", "K-0020", "K-0023" ], "type": "concept", "url": "https://trustbutveri.fyi/concepts/undeclared-compute/", "source_file": "content/concepts/undeclared-compute.md", "flags_all": [], "body_markdown": "Undeclared compute is AI-relevant hardware, or use of declared hardware, that a [[K-0001|prover]] has not reported to the verifier [[S-0002]].\n\nRAND's verification framework separates two cases: undeclared uses of declared clusters, and undeclared clusters, whether inside known data centres or standalone [[S-0002]]. The problem is sharpest for hardware that predates tracking. Shavit notes that hundreds of thousands of ML chips had already been sold, many lacking the security features his framework needs [[S-0029]], and Scher and Thiergart write that millions of AI-relevant chips exist and that, to their knowledge, no central tracking of them has taken place [[S-0005]]. They judge that searching for secret data centres may help but is unlikely to carry a verification regime, because it will likely be too easy to hide AI compute among other compute or to build secret data centres [[S-0005]]. Sastry and colleagues caution that more efficient algorithms and more viable decentralized training could reduce how much compute, or how concentrated, a prohibited activity needs [[S-0053]]. Proposed responses include:\n- **Tracking hardware.** Monitoring the chip supply chain and keeping a directory of chip owners [[S-0029]], as in [[M-0019|chip registries and manufacturing records]].\n- **Finding facilities.** National technical means such as remote sensing, energy monitoring, customs data and financial intelligence, alongside whistleblowers [[S-0062]], as in [[M-0020|remote detection of data centres]].\n- **Bounding declared capacity.** Wiping memory to remove residual capacity for hidden workloads on declared hardware [[S-0018]], as in [[M-0015|memory wiping and proofs of secure erasure]].", "body_text": "Undeclared compute is AI-relevant hardware, or use of declared hardware, that a prover has not reported to the verifier [S-0002]. RAND's verification framework separates two cases: undeclared uses of declared clusters, and undeclared clusters, whether inside known data centres or standalone [S-0002]. The problem is sharpest for hardware that predates tracking. Shavit notes that hundreds of thousands of ML chips had already been sold, many lacking the security features his framework needs [S-0029], and Scher and Thiergart write that millions of AI-relevant chips exist and that, to their knowledge, no central tracking of them has taken place [S-0005]. They judge that searching for secret data centres may help but is unlikely to carry a verification regime, because it will likely be too easy to hide AI compute among other compute or to build secret data centres [S-0005]. Sastry and colleagues caution that more efficient algorithms and more viable decentralized training could reduce how much compute, or how concentrated, a prohibited activity needs [S-0053]. Proposed responses include: - Tracking hardware. Monitoring the chip supply chain and keeping a directory of chip owners [S-0029], as in chip registries and manufacturing records. - Finding facilities. National technical means such as remote sensing, energy monitoring, customs data and financial intelligence, alongside whistleblowers [S-0062], as in remote detection of data centres. - Bounding declared capacity. Wiping memory to remove residual capacity for hidden workloads on declared hardware [S-0018], as in memory wiping and proofs of secure erasure.", "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/" }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "C-0001", "title": "Compute stock is at most a declared amount", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" }, { "id": "K-0003", "title": "Positive and negative claims", "url": "https://trustbutveri.fyi/concepts/positive-and-negative-claims/" }, { "id": "K-0012", "title": "Proof of space", "url": "https://trustbutveri.fyi/concepts/proof-of-space/" }, { "id": "K-0011", "title": "Proof of (useful) work", "url": "https://trustbutveri.fyi/concepts/proof-of-useful-work/" } ] }, { "id": "K-0002", "slug": "verifier", "title": "Verifier", "aliases": [], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "The party that examines evidence supplied by, or collected about, a prover and decides whether the prover's claim holds.", "sources": [ { "source": "S-0004", "supports": "Verifier defined as an institution, such as another state or an international institution, that assesses the Prover's compliance", "locator": "fn. 20" }, { "source": "S-1603", "supports": "Verifier appraises evidence and produces attestation results; Relying Party acts on them", "locator": "§4 Terminology" }, { "source": "S-0002", "supports": "Verifier may be a government body or a third party", "locator": "§3.1" }, { "source": "S-1606", "supports": "soundness: if the statement is false, a verifier following the protocol will not be convinced", "locator": "p. 1" }, { "source": "S-0029", "supports": "Verifier randomly samples and inspects chips and re-runs selected training segments", "locator": "§3.1–3.2; §5.1" }, { "source": "S-0018", "supports": "a malicious verifier may try to exfiltrate confidential information; its devices are verifier-trusted and prover-distrusted", "locator": "threat model section" } ], "related": [ "K-0001", "K-0004", "K-0018", "K-0020" ], "type": "concept", "url": "https://trustbutveri.fyi/concepts/verifier/", "source_file": "content/concepts/verifier.md", "flags_all": [], "body_markdown": "A verifier is the party that examines evidence from or about a [[K-0001|prover]] and decides whether the prover's claim holds [[S-0004]] [[S-1603]].\n\nUnder an international agreement the verifier may be another state, an international institution, a government body or a third party [[S-0004]] [[S-0002]]. In proof systems, soundness protects the verifier: if a statement is false, a verifier that follows the protocol will not be convinced [[S-1606]]. The IETF remote-attestation architecture splits the role in two: a verifier appraises evidence and produces attestation results, and a separate relying party acts on them [[S-1603]]. In Shavit's framework the verifier randomly samples and physically inspects accelerators, and re-runs selected training segments as in [[M-0006|proof-of-learning]] [[S-0029]]. In low-trust settings the verifier is itself a potential adversary: one system design assumes it may try to exfiltrate the prover's secrets, so the prover distrusts the devices the verifier relies on [[S-0018]].", "body_text": "A verifier is the party that examines evidence from or about a prover and decides whether the prover's claim holds [S-0004] [S-1603]. Under an international agreement the verifier may be another state, an international institution, a government body or a third party [S-0004] [S-0002]. In proof systems, soundness protects the verifier: if a statement is false, a verifier that follows the protocol will not be convinced [S-1606]. The IETF remote-attestation architecture splits the role in two: a verifier appraises evidence and produces attestation results, and a separate relying party acts on them [S-1603]. In Shavit's framework the verifier randomly samples and physically inspects accelerators, and re-runs selected training segments as in proof-of-learning [S-0029]. In low-trust settings the verifier is itself a potential adversary: one system design assumes it may try to exfiltrate the prover's secrets, so the prover distrusts the devices the verifier relies on [S-0018].", "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0014", "title": "EZKL", "url": "https://trustbutveri.fyi/implementations/ezkl/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "I-0003", "title": "zkLLM", "url": "https://trustbutveri.fyi/implementations/zkllm/" }, { "id": "K-0001", "title": "Prover", "url": "https://trustbutveri.fyi/concepts/prover/" }, { "id": "K-0009", "title": "Recomputation", "url": "https://trustbutveri.fyi/concepts/recomputation/" }, { "id": "K-0018", "title": "Threat model", "url": "https://trustbutveri.fyi/concepts/threat-model/" }, { "id": "K-0010", "title": "Zero-knowledge proof", "url": "https://trustbutveri.fyi/concepts/zero-knowledge-proof/" }, { "id": "O-0205", "title": "Oxford Martin AI Governance Initiative", "url": "https://trustbutveri.fyi/organizations/oxford-martin-ai-governance-initiative/" } ] }, { "id": "K-0022", "slug": "weight-exfiltration", "title": "Weight exfiltration", "aliases": [ "model weight theft", "data exfiltration" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Unauthorized copying of a model's trained parameters out of the environment meant to contain them, by theft or through covert channels.", "sources": [ { "source": "S-1610", "supports": "model weights as a target for theft; 38 attack vectors; 5 security levels; adversaries from opportunistic criminals to nation-state operations", "locator": "summary" }, { "source": "S-0015", "supports": "exfiltration by hiding weights in model responses (steganography); security game; verification against a reference; on MoE-Qwen-30B, exfiltratable information under 0.5% at a false-positive rate under 0.01%", "locator": "abstract" }, { "source": "S-0038", "supports": "physical side channels can bypass network monitoring", "locator": "side channels of concern" }, { "source": "S-0018", "supports": "malicious verifier may aim to exfiltrate prover secrets; checkpoints committed, not revealed, to an independent governing body; only hashes sent to a verifier outside the facility", "locator": "threat model; weights" } ], "related": [ "K-0013", "K-0014", "K-0021" ], "type": "concept", "url": "https://trustbutveri.fyi/concepts/weight-exfiltration/", "source_file": "content/concepts/weight-exfiltration.md", "flags_all": [], "body_markdown": "Weight exfiltration is the unauthorized copying of a model's trained parameters, its weights, out of the environment meant to contain them [[S-1610]] [[S-0015]].\n\nRAND researchers identified 38 meaningfully distinct attack vectors for stealing frontier model weights, and defined five security levels for defending against actors ranging from opportunistic criminals to highly resourced nation-state operations [[S-1610]]. Exfiltration can be covert. An attacker who controls an inference server could hide weights inside ordinary model responses using steganography [[S-0015]]. Rinberg and colleagues verify inference outputs against a reference to limit what responses can carry. On the MoE-Qwen-30B model, under benign prompts, their detector reduced exfiltratable information to under 0.5% at a false-positive rate below 0.01% [[S-0015]], the approach of [[M-0024|bounding unexplained information in outputs]]. Physical [[K-0013|side channels]] offer another route past network monitoring [[S-0038]], the target of [[M-0022|side-channel suppression]]. In verification the concern runs both ways. The verifier's equipment could leak the prover's secrets, so one low-trust design commits model checkpoints to an independent governing body rather than revealing them, and sends only hashes to the verifier outside the facility [[S-0018]].", "body_text": "Weight exfiltration is the unauthorized copying of a model's trained parameters, its weights, out of the environment meant to contain them [S-1610] [S-0015]. RAND researchers identified 38 meaningfully distinct attack vectors for stealing frontier model weights, and defined five security levels for defending against actors ranging from opportunistic criminals to highly resourced nation-state operations [S-1610]. Exfiltration can be covert. An attacker who controls an inference server could hide weights inside ordinary model responses using steganography [S-0015]. Rinberg and colleagues verify inference outputs against a reference to limit what responses can carry. On the MoE-Qwen-30B model, under benign prompts, their detector reduced exfiltratable information to under 0.5% at a false-positive rate below 0.01% [S-0015], the approach of bounding unexplained information in outputs. Physical side channels offer another route past network monitoring [S-0038], the target of side-channel suppression. In verification the concern runs both ways. The verifier's equipment could leak the prover's secrets, so one low-trust design commits model checkpoints to an independent governing body rather than revealing them, and sends only hashes to the verifier outside the facility [S-0018].", "referenced_by": [ { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "I-0007", "title": "Attestable Audits", "url": "https://trustbutveri.fyi/implementations/attestable-audits/" }, { "id": "I-0010", "title": "RAND secure inference data center (SIDC) design", "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0013", "title": "Side channel", "url": "https://trustbutveri.fyi/concepts/side-channel/" } ] }, { "id": "K-0010", "slug": "zero-knowledge-proof", "title": "Zero-knowledge proof", "aliases": [ "ZKP", "zkSNARK" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "A cryptographic protocol by which a prover convinces a verifier that a statement is true while revealing nothing beyond the fact that it is true.", "sources": [ { "source": "S-1600", "supports": "NIST definition of zero-knowledge proof", "locator": "term: zero_knowledge_proof (NISTIR 8301)" }, { "source": "S-1606", "supports": "completeness, soundness and zero knowledge; instance and witness; proofs sound against unbounded provers vs arguments sound against bounded provers; SNARK as succinct non-interactive argument of knowledge", "locator": "pp. 1–2; p. 21" }, { "source": "S-0023", "supports": "one-time weight commitment and per-query proof; LLaMa-2-13B forward pass of 2,048 tokens proved in 803 s on one A100 GPU with a 188 kB proof; fixed-point approximation", "locator": "abstract; §3; §8, Table 1" }, { "source": "S-0025", "supports": "recent analyses judge ZK verification of frontier training currently impractical; authors argue this is not fundamental and propose an architecture", "locator": "abstract" } ], "related": [ "K-0024", "K-0001", "K-0002" ], "type": "concept", "url": "https://trustbutveri.fyi/concepts/zero-knowledge-proof/", "source_file": "content/concepts/zero-knowledge-proof.md", "flags_all": [], "body_markdown": "A zero-knowledge proof (ZKP) is a cryptographic protocol by which a [[K-0001|prover]] convinces a [[K-0002|verifier]] that a statement is true without revealing any more information than that fact [[S-1600]].\n\nIt has three properties:\n- **Completeness.** If the statement is true and both parties follow the protocol, the verifier accepts [[S-1606]].\n- **Soundness.** If the statement is false, a verifier that follows the protocol is not convinced [[S-1606]].\n- **Zero knowledge.** If the statement is true and the prover follows the protocol, the verifier learns no confidential information beyond the fact that the statement is true [[S-1606]].\n\nThe statement combines a public instance, known to both parties, with a private witness known only to the prover [[S-1606]]. Strictly, a proof must be sound even against a computationally unbounded prover; an argument, such as a SNARK (succinct non-interactive argument of knowledge), need only be sound against bounded ones [[S-1606]]. In [[M-0004|zero-knowledge proofs of inference]] the witness is the model's weights: the prover publishes a [[K-0024|commitment]] to them once, then proves for each query that the output equals the committed model applied to the input [[S-0023]]. zkLLM proved one 2,048-token forward pass of LLaMa-2-13B in 803 seconds on one A100 GPU, producing a 188 kB proof of a fixed-point approximation of the model [[S-0023]]. For [[M-0005|zero-knowledge proofs of training constraints]], a 2026 proposal notes that governance analyses judge such proofs currently impractical at frontier scale, and argues that this limit is not fundamental [[S-0025]].", "body_text": "A zero-knowledge proof (ZKP) is a cryptographic protocol by which a prover convinces a verifier that a statement is true without revealing any more information than that fact [S-1600]. It has three properties: - Completeness. If the statement is true and both parties follow the protocol, the verifier accepts [S-1606]. - Soundness. If the statement is false, a verifier that follows the protocol is not convinced [S-1606]. - Zero knowledge. If the statement is true and the prover follows the protocol, the verifier learns no confidential information beyond the fact that the statement is true [S-1606]. The statement combines a public instance, known to both parties, with a private witness known only to the prover [S-1606]. Strictly, a proof must be sound even against a computationally unbounded prover; an argument, such as a SNARK (succinct non-interactive argument of knowledge), need only be sound against bounded ones [S-1606]. In zero-knowledge proofs of inference the witness is the model's weights: the prover publishes a commitment to them once, then proves for each query that the output equals the committed model applied to the input [S-0023]. zkLLM proved one 2,048-token forward pass of LLaMa-2-13B in 803 seconds on one A100 GPU, producing a 188 kB proof of a fixed-point approximation of the model [S-0023]. For zero-knowledge proofs of training constraints, a 2026 proposal notes that governance analyses judge such proofs currently impractical at frontier scale, and argues that this limit is not fundamental [S-0025].", "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "M-0005", "title": "Zero-knowledge proofs of training constraints", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-training-constraints/" }, { "id": "I-0005", "title": "Attestable zero-knowledge inference prover", "url": "https://trustbutveri.fyi/implementations/attestable-zk-inference/" }, { "id": "I-0014", "title": "EZKL", "url": "https://trustbutveri.fyi/implementations/ezkl/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "I-0003", "title": "zkLLM", "url": "https://trustbutveri.fyi/implementations/zkllm/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0024", "title": "Cryptographic commitment", "url": "https://trustbutveri.fyi/concepts/cryptographic-commitment/" } ] }, { "id": "O-0201", "slug": "ai-futures-project", "title": "AI Futures Project", "aliases": [], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "kind": "research-org", "homepage": "https://www.aifutures.org/", "one_liner": "A small research group forecasting the future of AI; publisher of the AI 2040 scenario, including an inference-only verification plan.", "sources": [ { "source": "S-3608", "supports": "small research group forecasting the future of AI funded by donations and grants" }, { "source": "S-0067", "supports": "AI 2040 verification plan: isolated inference units, taps, recomputation, memory wipes" }, { "source": "S-1511", "supports": "companion page: component status, open problems, organizations working on verification" }, { "source": "S-1410", "supports": "covert AI projects supplement: waste heat, detection, upstream supply-chain audits" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/ai-futures-project/", "source_file": "content/organizations/ai-futures-project.md", "flags_all": [], "body_markdown": "The AI Futures Project describes itself as \"a small research group forecasting the future of AI, funded by charitable donations and grants\" [[S-3608]]. Its AI 2040 scenario has supplements on verification:\n\n- Dean's verification plan would convert data centres into isolated inference units, copy their traffic with network taps, recompute random samples and wipe memory periodically, so that only verified inference outputs persist [[S-0067]]. See [[I-0011]], [[M-0014]], [[M-0013]], [[M-0001]] and [[M-0015]].\n- A companion \"Get involved\" page tracks the status and open problems of each component, and lists organizations working on verification [[S-1511]].\n- Halstead and Larsen discuss how a covert AI project might be hidden and detected, treating waste heat as the main constraint on hiding a facility [[S-1410]]; see [[M-0020]]. They propose auditing the semiconductor supply chain upstream to reconstruct past chip production [[S-1410]]; see [[M-0019]].", "body_text": "The AI Futures Project describes itself as \"a small research group forecasting the future of AI, funded by charitable donations and grants\" [S-3608]. Its AI 2040 scenario has supplements on verification: - Dean's verification plan would convert data centres into isolated inference units, copy their traffic with network taps, recompute random samples and wipe memory periodically, so that only verified inference outputs persist [S-0067]. See AI 2040 inference-only verification stack, Bandwidth limits and compartmentalization, Network taps and certifiers, Sampled inference recomputation and Memory wiping and proofs of secure erasure. - A companion \"Get involved\" page tracks the status and open problems of each component, and lists organizations working on verification [S-1511]. - Halstead and Larsen discuss how a covert AI project might be hidden and detected, treating waste heat as the main constraint on hiding a facility [S-1410]; see Remote detection of data centres. They propose auditing the semiconductor supply chain upstream to reconstruct past chip production [S-1410]; see Chip registries and manufacturing records.", "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/" }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "S-1511", "title": "Get Involved in Verification", "url": "https://trustbutveri.fyi/sources/ai-futures-get-involved-verification/" }, { "id": "S-3608", "title": "AI Futures Project homepage", "url": "https://trustbutveri.fyi/sources/ai-futures-homepage/" }, { "id": "S-0067", "title": "Verification Plan", "url": "https://trustbutveri.fyi/sources/dean-verification-plan/" }, { "id": "S-1410", "title": "Covert AI Projects", "url": "https://trustbutveri.fyi/sources/halstead-covert-ai-projects/" } ] }, { "id": "O-0101", "slug": "amodo-design", "title": "Amodo Design", "aliases": [], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "kind": "company", "homepage": "https://amododesign.com/", "one_liner": "A hardware engineering company that builds and publishes prototypes for verifying AI agreements, including inference recomputation, network taps and memory wiping.", "sources": [ { "source": "S-3601", "supports": "engineering company building tools for research" }, { "source": "S-0017", "supports": "example recomputation schemes for inference and pre-training" }, { "source": "S-1006", "supports": "re-implementation of DiFR (Karvonen et al.) in an asynchronous recomputation prototype; test scale" }, { "source": "S-1007", "supports": "prototype code" }, { "source": "S-1309", "supports": "first passive tap test" }, { "source": "S-1310", "supports": "survey of taps for fast optical links" }, { "source": "S-1311", "supports": "traffic-hashing benchmarks" }, { "source": "S-1312", "supports": "tap fitted to the inference-verification prototype" }, { "source": "S-1313", "supports": "DPU-enforced rate limits on 400G links for weight security" }, { "source": "S-1302", "supports": "memory-filling step of a proof of secure erasure and its benchmarks; challenge phase still to come" }, { "source": "S-1303", "supports": "disk-wiping benchmarks; integration into a full verification scheme still needed" }, { "source": "S-1321", "supports": "disk-wiping code" }, { "source": "S-0048", "supports": "analysis of data-centre power delivery" }, { "source": "S-1511", "supports": "listed as engineering the inference-only retrofit" }, { "source": "S-1008", "supports": "status page grading the plan's workstreams" }, { "source": "S-1512", "supports": "team post: no component past a proof-of-principle prototype" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/amodo-design/", "source_file": "content/organizations/amodo-design.md", "flags_all": [], "body_markdown": "Amodo Design builds engineering tools for research [[S-3601]]. Its public verification work is a series of design notes, prototypes and code:\n\n- **Recomputation.** Amodo sets out example schemes for checking inference and pre-training by recomputation [[S-0017]] ([[M-0001]], [[M-0003]]). It re-implemented [[I-0002|DiFR]], an inference-checking method published by Karvonen et al., in an asynchronous recomputation prototype [[S-1006]]. It tested the prototype with models of 1.5 to 120 billion parameters on H100 and H200 GPUs, and published the code [[S-1006]] [[S-1007]].\n- **Network taps.** Its notes cover a first passive tap test, a survey of taps for fast optical links, benchmarks of traffic hashing, and a tap fitted to its inference-verification prototype [[S-1309]] [[S-1310]] [[S-1311]] [[S-1312]]; see [[M-0013]].\n- **Bandwidth boundary.** For weight security, Amodo replaced a node's network cards with DPUs that enforce rate limits on 400G links [[S-1313]]; see [[M-0014]].\n- **Memory wiping.** It implemented the memory-filling step of a proof of secure erasure, benchmarked label generation on GPUs, CPUs and NVMe drives, and published code for the disk-wiping path [[S-1302]] [[S-1303]] [[S-1321]]. Integrating the wipe into a full verification scheme, with its timed challenges, is still to come [[S-1302]] [[S-1303]]; see [[M-0015]].\n- **Power monitoring.** A note on data-centre power delivery argues that each stage from voltage regulator to grid filters out compute-level detail, so that measurements lower in the chain keep more of it [[S-0048]]; see [[M-0021]].\n- **AI 2040 plan.** The AI Futures Project lists Amodo as engineering the inference-only retrofit in its [[I-0011|verification plan]] [[S-1511]]. Amodo publishes a status page that grades each workstream [[S-1008]], and its team reports that none of the plan's components is yet past a proof-of-principle prototype [[S-1512]].", "body_text": "Amodo Design builds engineering tools for research [S-3601]. Its public verification work is a series of design notes, prototypes and code: - Recomputation. Amodo sets out example schemes for checking inference and pre-training by recomputation [S-0017] (Sampled inference recomputation, Whole-workload recomputation (reproducible packets)). It re-implemented DiFR, an inference-checking method published by Karvonen et al., in an asynchronous recomputation prototype [S-1006]. It tested the prototype with models of 1.5 to 120 billion parameters on H100 and H200 GPUs, and published the code [S-1006] [S-1007]. - Network taps. Its notes cover a first passive tap test, a survey of taps for fast optical links, benchmarks of traffic hashing, and a tap fitted to its inference-verification prototype [S-1309] [S-1310] [S-1311] [S-1312]; see Network taps and certifiers. - Bandwidth boundary. For weight security, Amodo replaced a node's network cards with DPUs that enforce rate limits on 400G links [S-1313]; see Bandwidth limits and compartmentalization. - Memory wiping. It implemented the memory-filling step of a proof of secure erasure, benchmarked label generation on GPUs, CPUs and NVMe drives, and published code for the disk-wiping path [S-1302] [S-1303] [S-1321]. Integrating the wipe into a full verification scheme, with its timed challenges, is still to come [S-1302] [S-1303]; see Memory wiping and proofs of secure erasure. - Power monitoring. A note on data-centre power delivery argues that each stage from voltage regulator to grid filters out compute-level detail, so that measurements lower in the chain keep more of it [S-0048]; see Workload classification from telemetry and side channels. - AI 2040 plan. The AI Futures Project lists Amodo as engineering the inference-only retrofit in its verification plan [S-1511]. Amodo publishes a status page that grades each workstream [S-1008], and its team reports that none of the plan's components is yet past a proof-of-principle prototype [S-1512].", "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" }, { "id": "S-3601", "title": "Amodo Design: About Us", "url": "https://trustbutveri.fyi/sources/amodo-design-about/" }, { "id": "S-0017", "title": "Example Schemes for Verifying High-Stakes AI Agreements", "url": "https://trustbutveri.fyi/sources/amodo-example-schemes-high-stakes-ai-agreements/" }, { "id": "S-1303", "title": "Improving Disk Wiping Speed for Memory Wipes", "url": "https://trustbutveri.fyi/sources/amodo-improving-disk-wiping-speed/" }, { "id": "S-1007", "title": "Amodo-Design/Inference-Recomputation-Prototype (GitHub repository)", "url": "https://trustbutveri.fyi/sources/amodo-inference-recomputation-prototype-code/" }, { "id": "S-3001", "title": "An Inference Verification Prototype — Stage 1", "url": "https://trustbutveri.fyi/sources/amodo-inference-verification-prototype-stage-1/" }, { "id": "S-1302", "title": "Memory Wipes - Performance Analysis", "url": "https://trustbutveri.fyi/sources/amodo-memory-wipes-performance-analysis/" }, { "id": "S-1312", "title": "Fitting a Network TAP to our Inference Verification Prototype", "url": "https://trustbutveri.fyi/sources/amodo-network-tap-inference-verification-prototype/" }, { "id": "S-1310", "title": "Network Tapping for AI Verification: A Technical Assessment", "url": "https://trustbutveri.fyi/sources/amodo-network-tapping-technical-assessment/" }, { "id": "S-1309", "title": "Network Taps — A First Test", "url": "https://trustbutveri.fyi/sources/amodo-network-taps-first-test/" }, { "id": "S-1311", "title": "Network Traffic Hashing", "url": "https://trustbutveri.fyi/sources/amodo-network-traffic-hashing/" }, { "id": "S-1008", "title": "AI 2040 Plan A — Verification SITREP", "url": "https://trustbutveri.fyi/sources/amodo-plan-a-verification-sitrep/" }, { "id": "S-1321", "title": "Amodo-Design/PoSE-Memory-Wiping (GitHub repository)", "url": "https://trustbutveri.fyi/sources/amodo-pose-memory-wiping-code/" }, { "id": "S-1006", "title": "Scaling Recomputation Inference Verification", "url": "https://trustbutveri.fyi/sources/amodo-scaling-recomputation-inference-verification/" }, { "id": "S-1313", "title": "The Tray as a Bandwidth Boundary", "url": "https://trustbutveri.fyi/sources/amodo-tray-bandwidth-boundary/" }, { "id": "S-0048", "title": "Understanding Data Center Power Delivery", "url": "https://trustbutveri.fyi/sources/amodo-understanding-data-center-power-delivery/" }, { "id": "S-1512", "title": "Verifying international AI deals: Plan A, the state-of-play, and what you can do to help", "url": "https://trustbutveri.fyi/sources/milton-verifying-international-ai-deals/" } ] }, { "id": "O-0120", "slug": "attestable", "title": "Attestable", "aliases": [], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "kind": "company", "homepage": "https://attestable.com/", "one_liner": "A company developing zero-knowledge proofs for AI systems; it reports a prover for large language model inference and proposes proof-based compute accounting.", "sources": [ { "source": "S-3602", "supports": "development of zero-knowledge proofs for AI systems" }, { "source": "S-1101", "supports": "zero-knowledge prover for production-scale transformer inference; hash-only security (provider-reported)" }, { "source": "S-1102", "supports": "proposal to combine inference proofs with proof-of-work accounting; need for a compute estimate" }, { "source": "S-1103", "supports": "verification-firewall proposal with randomly selected proofs" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/attestable/", "source_file": "content/organizations/attestable.md", "flags_all": [ "provider-reported" ], "body_markdown": "Attestable says it develops zero-knowledge proofs for AI systems [[S-3602]]. It has published the following on verification:\n\n- Attestable reports a zero-knowledge prover for production-scale transformer inference whose security rests only on cryptographic hash functions [[S-1101]]. See [[I-0005]] and [[M-0004|zero-knowledge proofs of inference]].\n- It proposes combining inference proofs with proof-of-work accounting, and notes that the verifier \"needs a credible estimate of the compute available\" to the actor [[S-1102]]; see [[M-0007]].\n- It proposes a verification firewall at the external interfaces of an environment that holds model weights, which would select outputs for proof at random after each output is committed [[S-1103]].", "body_text": "Attestable says it develops zero-knowledge proofs for AI systems [S-3602]. It has published the following on verification: - Attestable reports a zero-knowledge prover for production-scale transformer inference whose security rests only on cryptographic hash functions [S-1101]. See Attestable zero-knowledge inference prover and zero-knowledge proofs of inference. - It proposes combining inference proofs with proof-of-work accounting, and notes that the verifier \"needs a credible estimate of the compute available\" to the actor [S-1102]; see Proofs of useful work for capacity accounting. - It proposes a verification firewall at the external interfaces of an environment that holds model weights, which would select outputs for proof at random after each output is committed [S-1103].", "referenced_by": [ { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0005", "title": "Attestable zero-knowledge inference prover", "url": "https://trustbutveri.fyi/implementations/attestable-zk-inference/" }, { "id": "S-3602", "title": "Attestable homepage", "url": "https://trustbutveri.fyi/sources/attestable-homepage/" }, { "id": "S-1103", "title": "From Verifiability to Model-Weight Security", "url": "https://trustbutveri.fyi/sources/attestable-model-weight-security/" }, { "id": "S-1102", "title": "Pacing AI Requires Proof", "url": "https://trustbutveri.fyi/sources/attestable-pacing-ai-requires-proof/" }, { "id": "S-1101", "title": "Proving LLMs at Scale", "url": "https://trustbutveri.fyi/sources/attestable-proving-llms-at-scale/" } ] }, { "id": "O-0207", "slug": "center-for-a-new-american-security", "title": "Center for a New American Security", "aliases": [ "CNAS" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "kind": "research-org", "homepage": "https://www.cnas.org/", "one_liner": "A nonprofit national-security policy organization; publisher of a report proposing on-chip mechanisms, including location verification, for governing AI chips.", "sources": [ { "source": "S-3612", "supports": "independent bipartisan nonprofit producing national-security and defense policy" }, { "source": "S-0056", "supports": "Secure, Governable Chips: security module, staged rollout, hardening, development effort, location verification, ownership tracking" }, { "source": "S-3382", "supports": "2025 working paper: smuggling estimate for 2024; recommended location verification and notification of exports and ownership transfers" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/center-for-a-new-american-security/", "source_file": "content/organizations/center-for-a-new-american-security.md", "flags_all": [], "body_markdown": "CNAS describes itself as an independent, bipartisan, nonprofit organization that develops national security and defense policies [[S-3612]]. Its 2024 report *Secure, Governable Chips*, by Aarne, Fist and Withers, proposes on-chip mechanisms for governing AI chips [[S-0056]]:\n\n- **Hardened security module.** The report proposes a module that enforces valid firmware and up-to-date operating licenses and supports remote attestation, rolled out in stages from firmware changes to tamper-evident and then tamper-proof hardware [[S-0056]]. See [[M-0009]], [[M-0011]] and [[M-0008]].\n- **Hardening and effort.** It judges that existing on-chip features must be hardened before they can be relied on in adversarial settings, and estimates that leading firms could build the required functionality with 18 months to 4 years of effort [[S-0056]].\n- **Location verification.** It illustrates ping-based checks with a landmark server in Paris: a reply within 9 ms would place a chip inside a circle that excludes countries subject to export restrictions [[S-0056]]. It expects hundreds of landmarks worldwide [[S-0056]]. See [[M-0018]].\n- **Ownership tracking.** It writes that on-chip mechanisms would need a way to track who owns data-centre AI chips, supported by supply-chain tracking and know-your-customer policies [[S-0056]]; see [[M-0019]].\n\nA 2025 CNAS working paper by Grunewald, of IAPS, and Fist estimates that between 10,000 and several hundred thousand AI chips, with a median estimate of about 140,000, may have been smuggled to China in 2024 [[S-3382]]. It recommends that chip designers implement software-based location verification, and that the US Bureau of Industry and Security require notification of exports, re-exports and ownership transfers of controlled AI chips [[S-3382]].", "body_text": "CNAS describes itself as an independent, bipartisan, nonprofit organization that develops national security and defense policies [S-3612]. Its 2024 report Secure, Governable Chips, by Aarne, Fist and Withers, proposes on-chip mechanisms for governing AI chips [S-0056]: - Hardened security module. The report proposes a module that enforces valid firmware and up-to-date operating licenses and supports remote attestation, rolled out in stages from firmware changes to tamper-evident and then tamper-proof hardware [S-0056]. See Hardware-enabled guarantees (flexHEG) and guarantee processors, Hardware performance throttling and licensing and TEE remote attestation for AI workloads. - Hardening and effort. It judges that existing on-chip features must be hardened before they can be relied on in adversarial settings, and estimates that leading firms could build the required functionality with 18 months to 4 years of effort [S-0056]. - Location verification. It illustrates ping-based checks with a landmark server in Paris: a reply within 9 ms would place a chip inside a circle that excludes countries subject to export restrictions [S-0056]. It expects hundreds of landmarks worldwide [S-0056]. See Chip location verification. - Ownership tracking. It writes that on-chip mechanisms would need a way to track who owns data-centre AI chips, supported by supply-chain tracking and know-your-customer policies [S-0056]; see Chip registries and manufacturing records. A 2025 CNAS working paper by Grunewald, of IAPS, and Fist estimates that between 10,000 and several hundred thousand AI chips, with a median estimate of about 140,000, may have been smuggled to China in 2024 [S-3382]. It recommends that chip designers implement software-based location verification, and that the US Bureau of Industry and Security require notification of exports, re-exports and ownership transfers of controlled AI chips [S-3382].", "referenced_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" }, { "id": "S-0056", "title": "Secure, Governable Chips: Using On-Chip Mechanisms to Manage National Security Risks from AI & Advanced Computing", "url": "https://trustbutveri.fyi/sources/aarne-secure-governable-chips/" }, { "id": "S-3612", "title": "Center for a New American Security: Mission", "url": "https://trustbutveri.fyi/sources/cnas-mission/" }, { "id": "S-3382", "title": "Countering AI Chip Smuggling Has Become a National Security Priority", "url": "https://trustbutveri.fyi/sources/grunewald-countering-ai-chip-smuggling/" } ] }, { "id": "O-0211", "slug": "centre-for-the-governance-of-ai", "title": "Centre for the Governance of AI", "aliases": [ "GovAI" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "kind": "research-org", "homepage": "https://www.governance.ai/", "one_liner": "An AI governance think tank whose research includes compute governance.", "sources": [ { "source": "S-1818", "supports": "self-description as a think tank; founding at Yale, Oxford years, independence in 2021; technical governance as a focus area" }, { "source": "S-0053", "supports": "compute governance paper: AI-relevant compute is detectable, excludable and quantifiable; three policy uses", "locator": "abstract" }, { "source": "S-0061", "supports": "jurisdictional certification proposal with import restrictions and export controls on hardware", "locator": "abstract" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/centre-for-the-governance-of-ai/", "source_file": "content/organizations/centre-for-the-governance-of-ai.md", "flags_all": [], "body_markdown": "The Centre for the Governance of AI (GovAI) describes itself as a think tank that aims to help decision-makers in government and industry navigate the transition to a world with advanced AI [[S-1818]]. It was founded at Yale University in 2016, was an academic centre at the University of Oxford from 2018, and became an independent nonprofit in 2021 [[S-1818]]. Technical governance is one of its focus areas [[S-1818]].\n\nTwo of its policy papers bear on verification:\n\n- A 2024 paper by Sastry, Heim and colleagues argues that AI-relevant compute is \"detectable, excludable, and quantifiable\" and produced through an extremely concentrated supply chain [[S-0053]]. It describes compute as a lever for regulatory visibility, for allocating resources and for enforcing restrictions on AI development and use [[S-0053]]. The claims on [[C-0001|compute stock]], [[C-0004|inference versus training]] and [[C-0008|bandwidth between compute groups]] draw on it.\n- A 2023 report by Trager and colleagues proposes an International AI Organization that certifies state jurisdictions, not firms, for compliance with international oversight standards [[S-0061]]. States would back the standards by restricting imports from uncertified jurisdictions and by controlling exports of inputs such as specialised hardware [[S-0061]].", "body_text": "The Centre for the Governance of AI (GovAI) describes itself as a think tank that aims to help decision-makers in government and industry navigate the transition to a world with advanced AI [S-1818]. It was founded at Yale University in 2016, was an academic centre at the University of Oxford from 2018, and became an independent nonprofit in 2021 [S-1818]. Technical governance is one of its focus areas [S-1818]. Two of its policy papers bear on verification: - A 2024 paper by Sastry, Heim and colleagues argues that AI-relevant compute is \"detectable, excludable, and quantifiable\" and produced through an extremely concentrated supply chain [S-0053]. It describes compute as a lever for regulatory visibility, for allocating resources and for enforcing restrictions on AI development and use [S-0053]. The claims on compute stock, inference versus training and bandwidth between compute groups draw on it. - A 2023 report by Trager and colleagues proposes an International AI Organization that certifies state jurisdictions, not firms, for compliance with international oversight standards [S-0061]. States would back the standards by restricting imports from uncertified jurisdictions and by controlling exports of inputs such as specialised hardware [S-0061].", "referenced_by": [ { "id": "S-1818", "title": "About us: Centre for the Governance of AI (GovAI)", "url": "https://trustbutveri.fyi/sources/govai-about/" }, { "id": "S-0053", "title": "Computing Power and the Governance of Artificial Intelligence", "url": "https://trustbutveri.fyi/sources/sastry-computing-power-governance-ai/" }, { "id": "S-0061", "title": "International Governance of Civilian AI: A Jurisdictional Certification Approach", "url": "https://trustbutveri.fyi/sources/trager-international-governance-civilian-ai/" } ] }, { "id": "O-0208", "slug": "epoch-ai", "title": "Epoch AI", "aliases": [ "Epoch" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "kind": "research-org", "homepage": "https://epoch.ai/", "one_liner": "A research institute studying the trajectory of AI; it runs the Frontier Data Centers Hub, which estimates AI data-centre capacity from satellite imagery and permits.", "sources": [ { "source": "S-3613", "supports": "data-first research nonprofit investigating the future of AI" }, { "source": "S-1411", "supports": "Frontier Data Centers Hub: method, coverage at launch, licence" }, { "source": "S-3301", "supports": "coverage estimate as of September 2026; mostly US sites" }, { "source": "S-1409", "supports": "FAS satellite-tracking report draws on the hub" }, { "source": "S-1410", "supports": "Epoch's methods named among ways to locate AI data centres" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/epoch-ai/", "source_file": "content/organizations/epoch-ai.md", "flags_all": [ "provider-reported" ], "body_markdown": "Epoch AI describes itself as a data-first research nonprofit investigating the future of AI [[S-3613]]. Its public dataset of large AI data centres is an example of [[M-0020|remote detection of data centres]]:\n\n- Epoch's Frontier Data Centers Hub identifies chillers and cooling towers in satellite images and corroborates them with permits and public disclosures to infer each facility's power capacity [[S-1411]]. It then estimates the compute installed, in H100-equivalents [[S-1411]].\n- At its launch in November 2025 the hub tracked 13 large US data centres [[S-1411]]. Epoch estimated that they held about 2.5 million, or about 15%, of the roughly 15 million H100-equivalents delivered to customers worldwide in the preceding several years [[S-1411]]. The data are released under a Creative Commons Attribution licence [[S-1411]].\n- By September 2026 the database still mostly covered US sites, and Epoch estimated its coverage of global deployed AI computing capacity at 44% (90% CI 24–81%) [[S-3301]].\n- A Federation of American Scientists report on tracking AI data centres with satellite imagery draws on the hub, to which its author contributed [[S-1409]]. An AI 2040 supplement on covert AI projects names Epoch's methods among those that intelligence services could use to locate AI data centres that are not specifically concealed [[S-1410]].", "body_text": "Epoch AI describes itself as a data-first research nonprofit investigating the future of AI [S-3613]. Its public dataset of large AI data centres is an example of remote detection of data centres: - Epoch's Frontier Data Centers Hub identifies chillers and cooling towers in satellite images and corroborates them with permits and public disclosures to infer each facility's power capacity [S-1411]. It then estimates the compute installed, in H100-equivalents [S-1411]. - At its launch in November 2025 the hub tracked 13 large US data centres [S-1411]. Epoch estimated that they held about 2.5 million, or about 15%, of the roughly 15 million H100-equivalents delivered to customers worldwide in the preceding several years [S-1411]. The data are released under a Creative Commons Attribution licence [S-1411]. - By September 2026 the database still mostly covered US sites, and Epoch estimated its coverage of global deployed AI computing capacity at 44% (90% CI 24–81%) [S-3301]. - A Federation of American Scientists report on tracking AI data centres with satellite imagery draws on the hub, to which its author contributed [S-1409]. An AI 2040 supplement on covert AI projects names Epoch's methods among those that intelligence services could use to locate AI data centres that are not specifically concealed [S-1410].", "referenced_by": [ { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/" }, { "id": "S-3613", "title": "About Epoch AI", "url": "https://trustbutveri.fyi/sources/epoch-ai-about/" }, { "id": "S-3301", "title": "AI Data Centers Documentation – Methodology", "url": "https://trustbutveri.fyi/sources/epoch-data-centers-methodology/" }, { "id": "S-1411", "title": "Introducing the Frontier Data Centers Hub", "url": "https://trustbutveri.fyi/sources/epoch-frontier-data-centers-hub/" } ] }, { "id": "O-0206", "slug": "future-of-life-institute", "title": "Future of Life Institute", "aliases": [ "FLI" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "kind": "nonprofit", "homepage": "https://futureoflife.org/", "one_liner": "A nonprofit working on extreme risks from transformative technology; it built secure-hardware proofs of concept for AI governance and partners on a network-logger prototype.", "sources": [ { "source": "S-3611", "supports": "self-description as independent nonprofit and mission to steer transformative technology away from extreme risks" }, { "source": "S-1203", "supports": "Intel SGX proof of concept with Mithril Security: design aims and stated limits" }, { "source": "S-3381", "supports": "AICert proof of concept with Mithril Security: TPM-based binding of weights to training inputs; stated limits" }, { "source": "S-1511", "supports": "listed with SASH as building a confidential network logger prototype" }, { "source": "S-1320", "supports": "named by SASH as an early partner" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/future-of-life-institute/", "source_file": "content/organizations/future-of-life-institute.md", "flags_all": [ "provider-reported" ], "body_markdown": "FLI describes itself as a non-profit whose mission is to steer transformative technology towards benefiting life and away from extreme, large-scale risks [[S-3611]]. Its verification work includes:\n\n- With Mithril Security, FLI built a secure-hardware proof of concept on Intel SGX and the BlindAI framework in which a model owner leases weights to an untrusted party [[S-1203]]. It was designed to keep the weights inaccessible to the borrower, report how much computing is done and provide an off-switch [[S-1203]]. FLI calls it \"not necessarily deployable as is\", citing performance and hardware attacks that need mitigation [[S-1203]]. See [[M-0008]].\n- A 2024 FLI post with Mithril Security reports AICert, a proof of concept in which Trusted Platform Modules bind a model's weights to the code and data used to train it [[S-3381]]. The authors state that it covers fine-tuning only, has not been audited by a third party and does not detect poisoned models or datasets [[S-3381]].\n- The AI Futures Project's verification page lists FLI and SASH as building a confidential network logger prototype [[S-1511]], and SASH names FLI as an early partner in that work [[S-1320]]. See [[I-0008]].", "body_text": "FLI describes itself as a non-profit whose mission is to steer transformative technology towards benefiting life and away from extreme, large-scale risks [S-3611]. Its verification work includes: - With Mithril Security, FLI built a secure-hardware proof of concept on Intel SGX and the BlindAI framework in which a model owner leases weights to an untrusted party [S-1203]. It was designed to keep the weights inaccessible to the borrower, report how much computing is done and provide an off-switch [S-1203]. FLI calls it \"not necessarily deployable as is\", citing performance and hardware attacks that need mitigation [S-1203]. See TEE remote attestation for AI workloads. - A 2024 FLI post with Mithril Security reports AICert, a proof of concept in which Trusted Platform Modules bind a model's weights to the code and data used to train it [S-3381]. The authors state that it covers fine-tuning only, has not been audited by a third party and does not detect poisoned models or datasets [S-3381]. - The AI Futures Project's verification page lists FLI and SASH as building a confidential network logger prototype [S-1511], and SASH names FLI as an early partner in that work [S-1320]. See SASH confidential network logger.", "referenced_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0008", "title": "SASH confidential network logger", "url": "https://trustbutveri.fyi/implementations/sash-confidential-network-logger/" }, { "id": "O-0160", "title": "Singapore AI Safety Hub (SASH)", "url": "https://trustbutveri.fyi/organizations/singapore-ai-safety-hub/" }, { "id": "S-3611", "title": "Future of Life Institute: Global Institutions Governing AI", "url": "https://trustbutveri.fyi/sources/fli-governing-ai-grant/" }, { "id": "S-1203", "title": "Exploration of secure hardware solutions for safe AI deployment", "url": "https://trustbutveri.fyi/sources/fli-hardware-backed-compute-governance/" }, { "id": "S-3381", "title": "Verifiable Training of AI Models", "url": "https://trustbutveri.fyi/sources/fli-verifiable-training-ai-models/" } ] }, { "id": "O-0212", "slug": "gensyn", "title": "Gensyn", "aliases": [], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "kind": "company", "homepage": "https://www.gensyn.ai/", "one_liner": "A company building AI systems that forecast events; developer of the Verde verification protocol and the REE runtime for reproducible model inference.", "sources": [ { "source": "S-3560", "supports": "self-description; Delphi and REE as products", "locator": "homepage" }, { "source": "S-1809", "supports": "Verde paper, most authors at Gensyn; dispute protocol and RepOps" }, { "source": "S-1812", "supports": "REE: reproducible LLM inference with receipts; MIT-licensed SDK, proprietary binaries", "locator": "README" }, { "source": "S-1810", "supports": "Verde and RepOps deployed in Judge (provider-reported)" }, { "source": "S-1811", "supports": "Judge, an evaluation service built on Verde and REE (provider-reported)" }, { "source": "S-3022", "supports": "Delphi live; markets settled inside REE produce re-runnable receipts (provider-reported)" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/gensyn/", "source_file": "content/organizations/gensyn.md", "flags_all": [ "provider-reported" ], "body_markdown": "Gensyn builds AI forecasting systems [[S-3560]]. Its verification work includes [[I-0015|Verde and RepOps]], which check machine-learning jobs delegated to compute providers [[S-1809]]:\n\n- Most of the authors of the paper that introduces Verde are at Gensyn [[S-1809]]. In Verde, several providers run the same machine-learning job, and a dispute between them is narrowed to a single operation that a referee re-runs [[S-1809]]. The paper's RepOps library fixes the order of floating-point operations so that results match across hardware [[S-1809]].\n- Gensyn publishes REE, its Reproducible Execution Environment, which it reports runs LLM inference reproducibly and writes receipts that let others re-run it [[S-1812]]. The SDK is MIT-licensed, and the compiler and operator binaries ship under a proprietary licence [[S-1812]]. See [[M-0002|deterministic and bit-exact inference]].\n- Gensyn reports that Verde and RepOps are deployed in Judge, its AI evaluation service [[S-1810]] [[S-1811]]. It reports that Delphi, its information-market app, is live, and that markets settled by open models inside REE produce receipts that anyone can re-run [[S-3022]].", "body_text": "Gensyn builds AI forecasting systems [S-3560]. Its verification work includes Verde and RepOps, which check machine-learning jobs delegated to compute providers [S-1809]: - Most of the authors of the paper that introduces Verde are at Gensyn [S-1809]. In Verde, several providers run the same machine-learning job, and a dispute between them is narrowed to a single operation that a referee re-runs [S-1809]. The paper's RepOps library fixes the order of floating-point operations so that results match across hardware [S-1809]. - Gensyn publishes REE, its Reproducible Execution Environment, which it reports runs LLM inference reproducibly and writes receipts that let others re-run it [S-1812]. The SDK is MIT-licensed, and the compiler and operator binaries ship under a proprietary licence [S-1812]. See deterministic and bit-exact inference. - Gensyn reports that Verde and RepOps are deployed in Judge, its AI evaluation service [S-1810] [S-1811]. It reports that Delphi, its information-market app, is live, and that markets settled by open models inside REE produce receipts that anyone can re-run [S-3022].", "referenced_by": [ { "id": "I-0015", "title": "Verde and RepOps (Gensyn)", "url": "https://trustbutveri.fyi/implementations/gensyn-verde-repops/" }, { "id": "S-1809", "title": "Verde: Verification via Refereed Delegation for Machine Learning Programs", "url": "https://trustbutveri.fyi/sources/arun-verde-refereed-delegation/" }, { "id": "S-3022", "title": "Building Delphi: Pricing, Settlement, and Agentic Trading", "url": "https://trustbutveri.fyi/sources/gensyn-building-delphi/" }, { "id": "S-3560", "title": "Gensyn: machines that predict the future", "url": "https://trustbutveri.fyi/sources/gensyn-homepage/" }, { "id": "S-1811", "title": "Introducing Judge", "url": "https://trustbutveri.fyi/sources/gensyn-introducing-judge/" }, { "id": "S-1812", "title": "gensyn-ai/ree: Gensyn Reproducible Execution Environment (GitHub repository)", "url": "https://trustbutveri.fyi/sources/gensyn-ree-code/" }, { "id": "S-3023", "title": "Reproducible Execution Environment (REE) (Gensyn documentation)", "url": "https://trustbutveri.fyi/sources/gensyn-ree-docs/" }, { "id": "S-1810", "title": "Verde Verification System In Production", "url": "https://trustbutveri.fyi/sources/gensyn-verde-in-production/" } ] }, { "id": "O-0204", "slug": "institute-for-ai-policy-and-strategy", "title": "Institute for AI Policy and Strategy", "aliases": [ "IAPS" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "kind": "research-org", "homepage": "https://www.iaps.ai/", "one_liner": "A nonpartisan think tank on AI policy; its work on AI chips includes a design and prototype for delay-based location verification.", "sources": [ { "source": "S-3610", "supports": "nonpartisan think tank producing AI policy research" }, { "source": "S-1400", "supports": "location verification report: delay-based design, adversary classes, costs, registry recommendation" }, { "source": "S-1401", "supports": "issue brief: H100 prototype and landmark running costs" }, { "source": "S-1402", "supports": "co-written by an IAPS researcher: verification methods for AI chip exports" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/institute-for-ai-policy-and-strategy/", "source_file": "content/organizations/institute-for-ai-policy-and-strategy.md", "flags_all": [], "body_markdown": "IAPS describes itself as a nonpartisan think tank that produces policy research on the implications of AI [[S-3610]]. Its verification work concerns AI chips:\n\n- **Location verification report.** Brass and Aarne write that delay-based schemes \"appear to be the most promising approach to location verification\": a trusted landmark server sends the chip a challenge, and the round-trip time bounds the chip's distance from it [[S-1400]]. The report defines three adversary classes and estimates that a solution would cost less than $1 million to set up and maintain for several years [[S-1400]]. See [[M-0018]].\n- **Prototype brief.** A May 2025 IAPS brief reports a rudimentary prototype on NVIDIA H100 chips, with a video demonstration, and estimates $2.5 million to $12.5 million a year to run 100 to 500 landmark servers [[S-1401]].\n- **Registry.** The report recommends combining location checks with a centralized chip registry [[S-1400]]; see [[M-0019]].\n- **Export verification.** A 2026 preprint co-written by an IAPS researcher reviews near-term methods for verifying AI chip exports, including how a regulator could run landmark-based checks and a registry of serial numbers and ownership records [[S-1402]].", "body_text": "IAPS describes itself as a nonpartisan think tank that produces policy research on the implications of AI [S-3610]. Its verification work concerns AI chips: - Location verification report. Brass and Aarne write that delay-based schemes \"appear to be the most promising approach to location verification\": a trusted landmark server sends the chip a challenge, and the round-trip time bounds the chip's distance from it [S-1400]. The report defines three adversary classes and estimates that a solution would cost less than $1 million to set up and maintain for several years [S-1400]. See Chip location verification. - Prototype brief. A May 2025 IAPS brief reports a rudimentary prototype on NVIDIA H100 chips, with a video demonstration, and estimates $2.5 million to $12.5 million a year to run 100 to 500 landmark servers [S-1401]. - Registry. The report recommends combining location checks with a centralized chip registry [S-1400]; see Chip registries and manufacturing records. - Export verification. A 2026 preprint co-written by an IAPS researcher reviews near-term methods for verifying AI chip exports, including how a regulator could run landmark-based checks and a registry of serial numbers and ownership records [S-1402].", "referenced_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "S-1402", "title": "Near-Term Verification Methods for AI Chip Exports", "url": "https://trustbutveri.fyi/sources/avellar-near-term-verification-ai-chip-exports/" }, { "id": "S-1400", "title": "Location Verification for AI Chips", "url": "https://trustbutveri.fyi/sources/brass-location-verification-ai-chips/" }, { "id": "S-1401", "title": "Location Verification for AI Chips (issue brief)", "url": "https://trustbutveri.fyi/sources/brass-location-verification-issue-brief/" }, { "id": "S-3382", "title": "Countering AI Chip Smuggling Has Become a National Security Priority", "url": "https://trustbutveri.fyi/sources/grunewald-countering-ai-chip-smuggling/" }, { "id": "S-3610", "title": "Institute for AI Policy and Strategy homepage", "url": "https://trustbutveri.fyi/sources/iaps-homepage/" } ] }, { "id": "O-0210", "slug": "intelligence-security-laboratories", "title": "Intelligence Security Laboratories", "aliases": [ "ISL" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "kind": "research-org", "homepage": "https://intelligencesecuritylaboratories.org/", "one_liner": "A nonprofit lab that researches and builds high-security AI systems, including secure data centres.", "sources": [ { "source": "S-1706", "supports": "nonprofit research lab and 501(c)(3) status; aim; threat of tampering and theft; integrator role; STPA-Sec and formal methods; hand-off of knowledge and artifacts; refers to the RAND report for STPA-Sec" }, { "source": "S-1707", "supports": "executive director, staff director and technical director; executive director's earlier RAND fellowship" }, { "source": "S-1510", "supports": "G. Kulp is a co-author of the RAND secure inference data center report" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/intelligence-security-laboratories/", "source_file": "content/organizations/intelligence-security-laboratories.md", "flags_all": [], "body_markdown": "Intelligence Security Laboratories (ISL) is a 501(c)(3) nonprofit research lab that describes its focus as \"implementation-driven R&D for high-security AI systems\" [[S-1706]]. It aims to build and demonstrate the security that critical AI deployments will need, and names tampering with or theft of a model by nation-state actors, other AIs or the model itself as the threats [[S-1706]]; see [[C-0009]].\n\nIts stated approach:\n\n- ISL argues that studying each component in depth will not show that a data centre is secure as a whole system, so it aims to be the integrator for the whole problem [[S-1706]].\n- It applies STPA-Sec (Systems-Theoretic Process Analysis for Security), a system design method that emphasises unknown-unknown attack vectors, and uses formal methods wherever it can [[S-1706]]. For details of STPA-Sec, it points to RAND's [[I-0010|secure inference data center design]] [[S-1706]].\n- It plans to hand the knowledge and technical artifacts it produces to other actors, and states that it does not expect to become the world's producer of secure compute [[S-1706]].\n\nIts executive director, Gabriel Kulp, is a co-author of the RAND report [[S-1510]] [[S-1707]]. Before founding ISL, he was a fellow at RAND working on hardware-enabled governance mechanisms for GPU export controls and on international verification of agreements [[S-1707]]. Tom Gardiner is ISL's staff director and Paul Murley its technical director [[S-1707]].", "body_text": "Intelligence Security Laboratories (ISL) is a 501(c)(3) nonprofit research lab that describes its focus as \"implementation-driven R&D for high-security AI systems\" [S-1706]. It aims to build and demonstrate the security that critical AI deployments will need, and names tampering with or theft of a model by nation-state actors, other AIs or the model itself as the threats [S-1706]; see Model weights have not left the facility. Its stated approach: - ISL argues that studying each component in depth will not show that a data centre is secure as a whole system, so it aims to be the integrator for the whole problem [S-1706]. - It applies STPA-Sec (Systems-Theoretic Process Analysis for Security), a system design method that emphasises unknown-unknown attack vectors, and uses formal methods wherever it can [S-1706]. For details of STPA-Sec, it points to RAND's secure inference data center design [S-1706]. - It plans to hand the knowledge and technical artifacts it produces to other actors, and states that it does not expect to become the world's producer of secure compute [S-1706]. Its executive director, Gabriel Kulp, is a co-author of the RAND report [S-1510] [S-1707]. Before founding ISL, he was a fellow at RAND working on hardware-enabled governance mechanisms for GPU export controls and on international verification of agreements [S-1707]. Tom Gardiner is ISL's staff director and Paul Murley its technical director [S-1707].", "referenced_by": [ { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" }, { "id": "I-0010", "title": "RAND secure inference data center (SIDC) design", "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/" }, { "id": "S-1706", "title": "Intelligence Security Laboratories: Building secure infrastructure for transformative AI", "url": "https://trustbutveri.fyi/sources/intelligence-security-laboratories-home/" }, { "id": "S-1707", "title": "Our Team: Intelligence Security Laboratories", "url": "https://trustbutveri.fyi/sources/intelligence-security-laboratories-team/" } ] }, { "id": "O-0180", "slug": "lucid-computing", "title": "Lucid Computing", "aliases": [], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "kind": "company", "homepage": "https://lucidcomputing.ai/", "one_liner": "A company offering attested, confidential-computing AI clusters; it hosts the Sovereignty Certificates location specification and runs a verification research programme.", "sources": [ { "source": "S-1406", "supports": "self-description: verifiable, confidential-computing clusters; links to Sovereignty Certificates", "locator": "homepage" }, { "source": "S-1404", "supports": "Sovereignty Certificates draft specification hosted in Lucid's GitHub organization; short-lived location certificates", "locator": "README" }, { "source": "S-1407", "supports": "developer platform: data-sovereignty auditor" }, { "source": "S-1301", "supports": "traffic-shaping design brief; not yet implemented or red-teamed" }, { "source": "S-1701", "supports": "research cluster with the Verifiable Compute Foundation; red-teaming programmes; standards in development", "locator": "overview page" }, { "source": "S-1702", "supports": "cluster operated on the Verifiable Compute Foundation's behalf; opens in November", "locator": "experimentation page" }, { "source": "S-3363", "supports": "adversarial testbed in a government-funded research institute; two H100 servers; three claims; host sets the test plan" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/lucid-computing/", "source_file": "content/organizations/lucid-computing.md", "flags_all": [ "provider-reported" ], "body_markdown": "Lucid Computing describes its offering as \"Verifiable AI. Proven in hardware.\": privacy-preserving, verifiable compute clusters built on trusted execution environments [[S-1406]]. Its verification work includes:\n\n- **Location certificates.** Lucid hosts, in its GitHub organization, the draft Sovereignty Certificates specification, which packages ping-based [[M-0018|chip location verification]] as short-lived certificates [[S-1404]]; its homepage links to the effort [[S-1406]]. See [[I-0009]].\n- **Developer platform.** Its documentation lists a \"Data Sovereignty & Localization\" auditor, described as \"Ensuring data remains within approved geographic jurisdictions\" [[S-1407]].\n- **Traffic shaping.** A Lucid design brief proposes capping the external traffic of each pod of accelerators, on the grounds that inference sends little external traffic while training moves large volumes between machines [[S-1301]]. The brief says the design is not yet implemented or red-teamed [[S-1301]]. See [[M-0014]].\n- **Research cluster.** Lucid reports that it builds and operates a bare-metal GPU research cluster on behalf of the [[O-0203|Verifiable Compute Foundation]], which decides who gets access [[S-1701]] [[S-1702]]. Its cluster page says the cluster opens to the community in November [[S-1702]].\n- **Red-teaming and standards.** Lucid reports running red-teaming programmes with national security agencies and government research institutes, in which the teams attacking its architecture are not the teams that built it [[S-1701]]. It reports building a testbed inside an unnamed government-funded research institute, whose first configuration retrofits two eight-GPU H100 servers to test claims about where compute runs, how it is used and which model is loaded [[S-3363]]. Lucid says the institute sets the test plan [[S-3363]]. It lists standards in development for compute identity, attested audits and hardware-rooted compute accounting [[S-1701]].", "body_text": "Lucid Computing describes its offering as \"Verifiable AI. Proven in hardware.\": privacy-preserving, verifiable compute clusters built on trusted execution environments [S-1406]. Its verification work includes: - Location certificates. Lucid hosts, in its GitHub organization, the draft Sovereignty Certificates specification, which packages ping-based chip location verification as short-lived certificates [S-1404]; its homepage links to the effort [S-1406]. See Lucid sovereignty (location) certificates. - Developer platform. Its documentation lists a \"Data Sovereignty & Localization\" auditor, described as \"Ensuring data remains within approved geographic jurisdictions\" [S-1407]. - Traffic shaping. A Lucid design brief proposes capping the external traffic of each pod of accelerators, on the grounds that inference sends little external traffic while training moves large volumes between machines [S-1301]. The brief says the design is not yet implemented or red-teamed [S-1301]. See Bandwidth limits and compartmentalization. - Research cluster. Lucid reports that it builds and operates a bare-metal GPU research cluster on behalf of the Verifiable Compute Foundation, which decides who gets access [S-1701] [S-1702]. Its cluster page says the cluster opens to the community in November [S-1702]. - Red-teaming and standards. Lucid reports running red-teaming programmes with national security agencies and government research institutes, in which the teams attacking its architecture are not the teams that built it [S-1701]. It reports building a testbed inside an unnamed government-funded research institute, whose first configuration retrofits two eight-GPU H100 servers to test claims about where compute runs, how it is used and which model is loaded [S-3363]. Lucid says the institute sets the test plan [S-3363]. It lists standards in development for compute identity, attested audits and hardware-rooted compute accounting [S-1701].", "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "I-0009", "title": "Lucid sovereignty (location) certificates", "url": "https://trustbutveri.fyi/implementations/lucid-location-certificates/" }, { "id": "O-0203", "title": "Verifiable Compute Foundation", "url": "https://trustbutveri.fyi/organizations/verifiable-compute-foundation/" }, { "id": "S-3363", "title": "Building an Adversarial Testbed for AI Verification in Europe", "url": "https://trustbutveri.fyi/sources/lucid-adversarial-testbed-europe/" }, { "id": "S-1406", "title": "Lucid Computing: Verifiable AI. Proven in hardware.", "url": "https://trustbutveri.fyi/sources/lucid-computing-homepage/" }, { "id": "S-1407", "title": "Lucid Developer Platform documentation", "url": "https://trustbutveri.fyi/sources/lucid-developer-platform-docs/" }, { "id": "S-1702", "title": "Experiments: Lucid Labs", "url": "https://trustbutveri.fyi/sources/lucid-experimentation-cluster/" }, { "id": "S-1701", "title": "Lucid Labs: the verification flywheel", "url": "https://trustbutveri.fyi/sources/lucid-labs-overview/" }, { "id": "S-1301", "title": "Traffic Shaping for Workload Classification", "url": "https://trustbutveri.fyi/sources/lucid-traffic-shaping-workload-classification/" }, { "id": "S-1404", "title": "Sovereignty Certificates: draft specification, version 0.1.0", "url": "https://trustbutveri.fyi/sources/sovereignty-certificates-specification/" } ] }, { "id": "O-0202", "slug": "machine-intelligence-research-institute", "title": "Machine Intelligence Research Institute", "aliases": [ "MIRI" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "kind": "research-org", "homepage": "https://intelligence.org/", "one_liner": "A nonprofit focused on preventing human extinction from artificial superintelligence; its Technical Governance Team publishes designs and analyses for verifying AI agreements.", "sources": [ { "source": "S-3609", "supports": "nonprofit focus on preventing human extinction from artificial superintelligence" }, { "source": "S-0018", "supports": "low-trust compute verification system overview" }, { "source": "S-0038", "supports": "side-channel suppression with retrofitted defences" }, { "source": "S-0014", "supports": "analysis of TEEs for privacy-preserving monitoring (Technical Governance Fellowship)" }, { "source": "S-0005", "supports": "survey of mechanisms to verify international agreements; interconnect bandwidth limits" }, { "source": "S-0063", "supports": "draft international agreement: monitored facilities and tracking of chip production" }, { "source": "S-3380", "supports": "six TGT papers at the ICML 2026 TAIGR workshop; best-paper award for bit-exact inference verification" }, { "source": "S-0020", "supports": "bit-exact inference verification paper" }, { "source": "S-0037", "supports": "detecting hidden training from GPU telemetry" }, { "source": "S-3220", "supports": "interconnect-limit monitoring prototype on two nodes with four A100 GPUs" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/", "source_file": "content/organizations/machine-intelligence-research-institute.md", "flags_all": [], "body_markdown": "MIRI describes itself as \"a nonprofit focused on preventing human extinction from artificial superintelligence\" [[S-3609]]. Its publications on verifying AI agreements, several of them from its Technical Governance Team, include:\n\n- **Low-trust verification system.** Cankaya's system overview combines network taps, sampled recomputation, memory challenges and facility monitoring in one reference architecture for near-term, low-trust compute verification [[S-0018]]. See [[I-0012]], [[M-0013]], [[M-0001]] and [[M-0016]].\n- **Side-channel suppression.** A post on suppressing side channels in an untrusted data centre with retrofitted defences sets out channel classes, attenuation targets and costs [[S-0038]]; see [[M-0022]].\n- **TEEs for monitoring.** A post written in MIRI's Technical Governance Fellowship analyses trusted execution environments for privacy-preserving monitoring [[S-0014]]. It contrasts the usual confidential-computing adversary, a dishonest operator, with a state that has physical access to data centres [[S-0014]]. See [[M-0008]], [[M-0023]] and [[M-0025]].\n- **Mechanisms survey.** Scher and Thiergart survey mechanisms for verifying international agreements about AI development, with a detailed analysis of [[M-0014|interconnect bandwidth limits]] [[S-0005]].\n- **Draft agreement.** A draft international agreement would prohibit concentrations of more than 16 H100-equivalents outside monitored facilities and would track new chip production [[S-0063]]; see [[C-0001]].\n- **Workshop papers.** MIRI reports that six Technical Governance Team papers appeared at the ICML 2026 Workshop on Technical AI Governance Research [[S-3380]]. They include bit-exact inference verification, which won the workshop's best-paper award [[S-3380]] [[S-0020]], and detection of hidden training from GPU telemetry [[S-0037]]. See [[M-0002]] and [[M-0021]].\n- **Interconnect prototype.** A 2026 post reports a prototype, built under Scher's supervision, that monitors inter-node traffic against a threshold on a two-node cluster with four A100 GPUs [[S-3220]]. See [[M-0014]].", "body_text": "MIRI describes itself as \"a nonprofit focused on preventing human extinction from artificial superintelligence\" [S-3609]. Its publications on verifying AI agreements, several of them from its Technical Governance Team, include: - Low-trust verification system. Cankaya's system overview combines network taps, sampled recomputation, memory challenges and facility monitoring in one reference architecture for near-term, low-trust compute verification [S-0018]. See Low-trust AI compute verification system overview, Network taps and certifiers, Sampled inference recomputation and Timed challenge-response and memory-occupation challenges. - Side-channel suppression. A post on suppressing side channels in an untrusted data centre with retrofitted defences sets out channel classes, attenuation targets and costs [S-0038]; see Side-channel suppression for isolated facilities. - TEEs for monitoring. A post written in MIRI's Technical Governance Fellowship analyses trusted execution environments for privacy-preserving monitoring [S-0014]. It contrasts the usual confidential-computing adversary, a dishonest operator, with a state that has physical access to data centres [S-0014]. See TEE remote attestation for AI workloads, Safeguard attestation and Confidential multi-party verification. - Mechanisms survey. Scher and Thiergart survey mechanisms for verifying international agreements about AI development, with a detailed analysis of interconnect bandwidth limits [S-0005]. - Draft agreement. A draft international agreement would prohibit concentrations of more than 16 H100-equivalents outside monitored facilities and would track new chip production [S-0063]; see Compute stock is at most a declared amount. - Workshop papers. MIRI reports that six Technical Governance Team papers appeared at the ICML 2026 Workshop on Technical AI Governance Research [S-3380]. They include bit-exact inference verification, which won the workshop's best-paper award [S-3380] [S-0020], and detection of hidden training from GPU telemetry [S-0037]. See Deterministic and bit-exact inference and Workload classification from telemetry and side channels. - Interconnect prototype. A 2026 post reports a prototype, built under Scher's supervision, that monitors inter-node traffic against a threshold on a two-node cluster with four A100 GPUs [S-3220]. See Bandwidth limits and compartmentalization.", "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "S-0038", "title": "Suppressing Side Channels in an Untrusted Data Center via Retrofitted Defenses", "url": "https://trustbutveri.fyi/sources/cankaya-suppressing-side-channels/" }, { "id": "S-0018", "title": "A System Overview for Near-Term, Low-Trust AI Compute Verification", "url": "https://trustbutveri.fyi/sources/cankaya-system-overview-low-trust-compute-verification/" }, { "id": "S-3609", "title": "About MIRI", "url": "https://trustbutveri.fyi/sources/miri-about/" }, { "id": "S-3380", "title": "Summary: TGT's 2026 ICML Papers", "url": "https://trustbutveri.fyi/sources/miri-tgt-2026-icml-papers/" }, { "id": "S-0063", "title": "An International Agreement to Prevent the Premature Creation of Artificial Superintelligence", "url": "https://trustbutveri.fyi/sources/scher-agreement-prevent-premature-asi/" }, { "id": "S-3220", "title": "De-risking Interconnect Limits for AI Verification", "url": "https://trustbutveri.fyi/sources/scher-derisking-interconnect-limits/" }, { "id": "S-0005", "title": "Mechanisms to Verify International Agreements About AI Development", "url": "https://trustbutveri.fyi/sources/scher-mechanisms-verify-ai-agreements/" }, { "id": "S-0014", "title": "On TEEs for Privacy-Preserving Monitoring in AI Governance", "url": "https://trustbutveri.fyi/sources/zhao-tees-privacy-preserving-monitoring/" } ] }, { "id": "O-0140", "slug": "nvidia", "title": "NVIDIA", "aliases": [], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "kind": "company", "homepage": "https://www.nvidia.com/", "one_liner": "An accelerated-computing company whose Hopper and Blackwell data-centre GPUs offer a confidential-computing mode with hardware attestation.", "sources": [ { "source": "S-3604", "supports": "self-description as pioneer of accelerated computing" }, { "source": "S-1200", "supports": "confidential computing on Hopper and Blackwell GPUs; performance counters disabled in confidential-computing mode", "locator": "pp. 6-18" }, { "source": "S-1201", "supports": "H100 confidential-computing launch: on-die root of trust, device identity key, SPDM attestation" }, { "source": "S-1206", "supports": "NVIDIA GPUs in confidential-computing mode listed as supported hardware by Tinfoil" }, { "source": "S-1413", "supports": "opt-in, customer-installed fleet-management service with read-only telemetry; statement on tracking, kill switches and backdoors (provider-reported)" }, { "source": "S-3180", "supports": "Fleet Intelligence: open-source read-only agent, telemetry and attestation on Blackwell and Vera Rubin (provider-reported)" }, { "source": "S-1402", "supports": "NVIDIA reportedly developing delay-based location verification with NVIDIA-run servers (citing Reuters)" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/nvidia/", "source_file": "content/organizations/nvidia.md", "flags_all": [ "provider-reported" ], "body_markdown": "NVIDIA describes itself as having \"pioneered accelerated computing\" [[S-3604]]. Its public material relevant to verification concerns its data-centre GPUs:\n\n- **Confidential-computing mode.** NVIDIA documents a confidential-computing mode for its Hopper and Blackwell data-centre GPUs, in which the GPU is paired with a CPU trusted execution environment and attests its identity and firmware [[S-1200]]; see [[M-0008]].\n- **H100 launch.** NVIDIA engineers describe the feature's launch on the H100, with an on-die root of trust, a fused device identity key and SPDM attestation [[S-1201]].\n- **Counters in confidential mode.** NVIDIA disables performance counters in full confidential-computing mode, stating that they could provide an avenue for side-channel attacks [[S-1200]]; see [[M-0010]].\n- **Fleet telemetry.** NVIDIA reports an opt-in, customer-installed fleet-management service that provides read-only GPU telemetry [[S-1413]]. Its Fleet Intelligence service uses a read-only, open-source agent that sends GPU power, performance, health and configuration data to an NVIDIA cloud service [[S-3180]]. The service supports attestation only on Blackwell and Vera Rubin GPUs [[S-3180]]. NVIDIA states that its GPUs \"do not have hardware tracking technology, kill switches and backdoors\" [[S-1413]]. See [[M-0010]].\n- **Location verification.** Avellar and Grunewald report, citing Reuters, that NVIDIA has confirmed it is developing location verification that estimates a chip's location from communication delays with NVIDIA-run servers [[S-1402]]; see [[M-0018]].\n- **Use by others.** Tinfoil's documentation lists NVIDIA GPUs in confidential-computing mode as supported hardware for its enclaves [[S-1206]]; see [[I-0006]].", "body_text": "NVIDIA describes itself as having \"pioneered accelerated computing\" [S-3604]. Its public material relevant to verification concerns its data-centre GPUs: - Confidential-computing mode. NVIDIA documents a confidential-computing mode for its Hopper and Blackwell data-centre GPUs, in which the GPU is paired with a CPU trusted execution environment and attests its identity and firmware [S-1200]; see TEE remote attestation for AI workloads. - H100 launch. NVIDIA engineers describe the feature's launch on the H100, with an on-die root of trust, a fused device identity key and SPDM attestation [S-1201]. - Counters in confidential mode. NVIDIA disables performance counters in full confidential-computing mode, stating that they could provide an avenue for side-channel attacks [S-1200]; see On-chip telemetry from timing, memory and performance counters. - Fleet telemetry. NVIDIA reports an opt-in, customer-installed fleet-management service that provides read-only GPU telemetry [S-1413]. Its Fleet Intelligence service uses a read-only, open-source agent that sends GPU power, performance, health and configuration data to an NVIDIA cloud service [S-3180]. The service supports attestation only on Blackwell and Vera Rubin GPUs [S-3180]. NVIDIA states that its GPUs \"do not have hardware tracking technology, kill switches and backdoors\" [S-1413]. See On-chip telemetry from timing, memory and performance counters. - Location verification. Avellar and Grunewald report, citing Reuters, that NVIDIA has confirmed it is developing location verification that estimates a chip's location from communication delays with NVIDIA-run servers [S-1402]; see Chip location verification. - Use by others. Tinfoil's documentation lists NVIDIA GPUs in confidential-computing mode as supported hardware for its enclaves [S-1206]; see Tinfoil model identity (Modelwrap).", "referenced_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "S-1201", "title": "Confidential Computing on NVIDIA H100 GPUs for Secure and Trustworthy AI", "url": "https://trustbutveri.fyi/sources/apsey-confidential-computing-h100-gpus/" }, { "id": "S-1611", "title": "Boost GPU Memory Performance with No Code Changes Using NVIDIA CUDA MPS", "url": "https://trustbutveri.fyi/sources/nassernia-cuda-mps-gpu-memory-performance/" }, { "id": "S-3604", "title": "About NVIDIA", "url": "https://trustbutveri.fyi/sources/nvidia-about/" }, { "id": "S-1815", "title": "Now in General Availability: NVIDIA H100 GPUs in Microsoft Azure Confidential Virtual Machines", "url": "https://trustbutveri.fyi/sources/nvidia-azure-confidential-h100-ga/" }, { "id": "S-3180", "title": "Introducing NVIDIA Fleet Intelligence for Real-Time GPU Fleet Visibility and Optimization", "url": "https://trustbutveri.fyi/sources/nvidia-fleet-intelligence-technical-blog/" }, { "id": "S-1413", "title": "Opt-In NVIDIA Software Enables Data Center Fleet Management", "url": "https://trustbutveri.fyi/sources/nvidia-opt-in-fleet-management-software/" }, { "id": "S-1200", "title": "NVIDIA Secure AI with Blackwell and Hopper GPUs (White Paper)", "url": "https://trustbutveri.fyi/sources/nvidia-secure-ai-blackwell-hopper-whitepaper/" }, { "id": "S-3122", "title": "NVIDIA Trusted Computing Solutions Release Notes (R595 TRD1)", "url": "https://trustbutveri.fyi/sources/nvidia-trusted-computing-r595-release-notes/" }, { "id": "S-3162", "title": "No Backdoors. No Kill Switches. No Spyware.", "url": "https://trustbutveri.fyi/sources/reber-no-backdoors-no-kill-switches/" } ] }, { "id": "O-0215", "slug": "openmined", "title": "OpenMined", "aliases": [], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "kind": "nonprofit", "homepage": "https://openmined.org/", "one_liner": "A nonprofit developing PySyft, software for joint analysis of private data, including confidential AI evaluations.", "sources": [ { "source": "S-3564", "supports": "nonprofit status, open-source software for secure work across organizations, PySyft research agreement", "locator": "first paragraph" }, { "source": "S-3563", "supports": "nonprofit status, PySyft development and role in double-blind evaluations", "locator": "Executive Summary; Details of the pilots; page footer" }, { "source": "S-3320", "supports": "PySyft in the 2026 pilot and OpenMined author affiliations", "locator": "abstract; §2.5; §3" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/openmined/", "source_file": "content/organizations/openmined.md", "flags_all": [ "provider-reported" ], "body_markdown": "OpenMined is a nonprofit that develops open-source software for work across private datasets [[S-3564]]. Its work on this map is [[I-0017|PySyft double-blind evaluations]]:\n\n- OpenMined reports that PySyft coordinated the submission, approval and execution of private code and assets in GPU enclaves for 2026 evaluations by AVERI and Singapore AISI [[S-3563]].\n- The pilot's technical report describes a run of Gemini 2.5 Flash Lite on private MLCommons AILuminate prompts in a Google Cloud confidential VM with an H100, using PySyft v0.10.x [[S-3320]].", "body_text": "OpenMined is a nonprofit that develops open-source software for work across private datasets [S-3564]. Its work on this map is PySyft double-blind evaluations: - OpenMined reports that PySyft coordinated the submission, approval and execution of private code and assets in GPU enclaves for 2026 evaluations by AVERI and Singapore AISI [S-3563]. - The pilot's technical report describes a run of Gemini 2.5 Flash Lite on private MLCommons AILuminate prompts in a Google Cloud confidential VM with an H100, using PySyft v0.10.x [S-3320].", "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "I-0017", "title": "PySyft double-blind evaluations", "url": "https://trustbutveri.fyi/implementations/pysyft-double-blind-evaluations/" }, { "id": "S-3563", "title": "PySyft used for first double-blind evaluation of a proprietary, frontier-class AI model", "url": "https://trustbutveri.fyi/sources/openmined-double-blind-evaluation/" }, { "id": "S-3320", "title": "Double Blind Evals: Resolving the Dual Confidentiality Dilemma in AI Safety Auditing", "url": "https://trustbutveri.fyi/sources/trask-double-blind-evals/" } ] }, { "id": "O-0209", "slug": "oxford-hardware-ai-governance-lab", "title": "Hardware AI Governance Lab", "aliases": [ "HAIGL", "Oxford Hardware AI Governance Lab" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "kind": "academic-lab", "homepage": "https://aigi.ox.ac.uk/hardware-ai-governance-lab/", "one_liner": "A University of Oxford lab, hosted by the Oxford Martin AI Governance Initiative, that studies how computer hardware can support AI governance and international coordination.", "sources": [ { "source": "S-1703", "supports": "hosted by AIGI; co-directors and research lead; research focus and aims; first design profile expected in late 2026; lists S-1300 and S-1704 among its publications" }, { "source": "S-1300", "supports": "cluster I/O fingerprinting design and its aim; first author affiliated with the lab" }, { "source": "S-1704", "supports": "chip manufacturing threat model, defence in depth and tiered assurance; two authors affiliated with the lab" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/oxford-hardware-ai-governance-lab/", "source_file": "content/organizations/oxford-hardware-ai-governance-lab.md", "flags_all": [], "body_markdown": "The Hardware AI Governance Lab (HAIGL) is an interdisciplinary initiative at the University of Oxford, hosted by the [[O-0205|Oxford Martin AI Governance Initiative]] (AIGI) [[S-1703]]. Its co-directors are Amro Awad and Robert Trager, and Ben Harack is its co-founder and research lead [[S-1703]].\n\nThe lab states that it treats hardware architecture as a foundational layer for embedding oversight, verification and control mechanisms [[S-1703]]. Its stated aims are:\n\n- It aims to turn high-level governance concepts into hardware design proposals, and then into early-stage feasibility prototypes [[S-1703]].\n- It explores ways to verify how AI systems behave, and whether some governance constraints can be enforced directly in hardware [[S-1703]].\n- It studies how agreements between states could be structured and implemented when the parties have limited trust in each other [[S-1703]].\n\nThe lab expects to release its first hardware governance design profile in late 2026 [[S-1703]]. It lists two 2026 papers among its publications [[S-1703]]:\n\n- A design for fingerprinting all of an AI cluster's I/O without processors that both parties must trust. It aims to make it infeasible to covertly exfiltrate the results of undisclosed workloads through the tapped links [[S-1300]]. Its first author lists the lab as an affiliation [[S-1300]]. See [[M-0013]] and [[I-0012]].\n- A survey of verifiable semiconductor manufacturing that develops a threat model spanning chip design through manufacturing. It argues that defence in depth is required, with assurance tiered by chip criticality, and that neutral third-party verification authorities can help overcome industry opacity [[S-1704]]. Two of its four authors list the lab as an affiliation [[S-1704]].", "body_text": "The Hardware AI Governance Lab (HAIGL) is an interdisciplinary initiative at the University of Oxford, hosted by the Oxford Martin AI Governance Initiative (AIGI) [S-1703]. Its co-directors are Amro Awad and Robert Trager, and Ben Harack is its co-founder and research lead [S-1703]. The lab states that it treats hardware architecture as a foundational layer for embedding oversight, verification and control mechanisms [S-1703]. Its stated aims are: - It aims to turn high-level governance concepts into hardware design proposals, and then into early-stage feasibility prototypes [S-1703]. - It explores ways to verify how AI systems behave, and whether some governance constraints can be enforced directly in hardware [S-1703]. - It studies how agreements between states could be structured and implemented when the parties have limited trust in each other [S-1703]. The lab expects to release its first hardware governance design profile in late 2026 [S-1703]. It lists two 2026 papers among its publications [S-1703]: - A design for fingerprinting all of an AI cluster's I/O without processors that both parties must trust. It aims to make it infeasible to covertly exfiltrate the results of undisclosed workloads through the tapped links [S-1300]. Its first author lists the lab as an affiliation [S-1300]. See Network taps and certifiers and Low-trust AI compute verification system overview. - A survey of verifiable semiconductor manufacturing that develops a threat model spanning chip design through manufacturing. It argues that defence in depth is required, with assurance tiered by chip criticality, and that neutral third-party verification authorities can help overcome industry opacity [S-1704]. Two of its four authors list the lab as an affiliation [S-1704].", "referenced_by": [ { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "O-0205", "title": "Oxford Martin AI Governance Initiative", "url": "https://trustbutveri.fyi/organizations/oxford-martin-ai-governance-initiative/" }, { "id": "S-1300", "title": "Fingerprinting All AI Cluster I/O Without Mutually Trusted Processors", "url": "https://trustbutveri.fyi/sources/cankaya-fingerprinting-ai-cluster-io/" }, { "id": "S-1704", "title": "Verifiable Semiconductor Manufacturing", "url": "https://trustbutveri.fyi/sources/ilhan-verifiable-semiconductor-manufacturing/" }, { "id": "S-1703", "title": "Hardware AI Governance Lab", "url": "https://trustbutveri.fyi/sources/oxford-hardware-ai-governance-lab/" } ] }, { "id": "O-0205", "slug": "oxford-martin-ai-governance-initiative", "title": "Oxford Martin AI Governance Initiative", "aliases": [ "AIGI", "Oxford Martin AIGI" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "kind": "academic-lab", "homepage": "https://aigi.ox.ac.uk/", "one_liner": "A research programme at the University of Oxford that studies AI governance from technical and policy angles and hosts the Hardware AI Governance Lab.", "sources": [ { "source": "S-1705", "supports": "housed in the Oxford Martin School; technical and policy focus; co-led by Robert Trager and Maike Osborne" }, { "source": "S-0004", "supports": "Verification for International AI Governance: prover and verifier, presence versus absence, model fingerprint attestation, traffic commitments" }, { "source": "S-0058", "supports": "compute providers as verifiers; workload type and compute from non-confidential information" }, { "source": "S-1703", "supports": "hosts the Hardware AI Governance Lab; the lab's research focus" }, { "source": "S-1320", "supports": "co-launched SASH's international verification collaboration; named as an early partner" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/oxford-martin-ai-governance-initiative/", "source_file": "content/organizations/oxford-martin-ai-governance-initiative.md", "flags_all": [], "body_markdown": "The Oxford Martin AI Governance Initiative (AIGI) is a research programme housed in the Oxford Martin School at the University of Oxford [[S-1705]]. It describes itself as one of the few centres in the world that study the governance of AI from both technical and policy perspectives [[S-1705]]. It is co-led by Robert Trager, a social scientist who specialises in international relations and frontier AI regulation, and Maike Osborne, a machine-learning specialist [[S-1705]].\n\nIts verification work includes:\n\n- A 2025 AIGI report examines how states could verify compliance with international AI agreements, framed around a [[K-0001|prover]] and a [[K-0002|verifier]] [[S-0004]]. It observes that demonstrating that something exists is often easier than demonstrating that it does not [[S-0004]]. It includes an appendix on model fingerprint attestation, and describes networking hardware that makes cryptographic commitments about all traffic [[S-0004]].\n- A 2024 policy paper argues that compute providers could be verifiers of their customers' activities [[S-0058]]. It describes how information that providers already hold, without confidential data, can indicate a workload's type, such as large-scale training or inference, and the compute it consumed [[S-0058]]; see [[C-0004]].\n- AIGI hosts the [[O-0209|Hardware AI Governance Lab]], which studies how computer hardware architecture can support AI governance, oversight and international coordination [[S-1703]].\n- SASH reports launching, with AIGI and a team of researchers, an international collaboration to build tools that make AI verifiable across borders, and names AIGI as an early partner [[S-1320]]. The collaboration's first project is a confidential network logger [[S-1320]]; see [[I-0008]].", "body_text": "The Oxford Martin AI Governance Initiative (AIGI) is a research programme housed in the Oxford Martin School at the University of Oxford [S-1705]. It describes itself as one of the few centres in the world that study the governance of AI from both technical and policy perspectives [S-1705]. It is co-led by Robert Trager, a social scientist who specialises in international relations and frontier AI regulation, and Maike Osborne, a machine-learning specialist [S-1705]. Its verification work includes: - A 2025 AIGI report examines how states could verify compliance with international AI agreements, framed around a prover and a verifier [S-0004]. It observes that demonstrating that something exists is often easier than demonstrating that it does not [S-0004]. It includes an appendix on model fingerprint attestation, and describes networking hardware that makes cryptographic commitments about all traffic [S-0004]. - A 2024 policy paper argues that compute providers could be verifiers of their customers' activities [S-0058]. It describes how information that providers already hold, without confidential data, can indicate a workload's type, such as large-scale training or inference, and the compute it consumed [S-0058]; see This compute runs inference, not training. - AIGI hosts the Hardware AI Governance Lab, which studies how computer hardware architecture can support AI governance, oversight and international coordination [S-1703]. - SASH reports launching, with AIGI and a team of researchers, an international collaboration to build tools that make AI verifiable across borders, and names AIGI as an early partner [S-1320]. The collaboration's first project is a confidential network logger [S-1320]; see SASH confidential network logger.", "referenced_by": [ { "id": "O-0209", "title": "Hardware AI Governance Lab", "url": "https://trustbutveri.fyi/organizations/oxford-hardware-ai-governance-lab/" }, { "id": "O-0160", "title": "Singapore AI Safety Hub (SASH)", "url": "https://trustbutveri.fyi/organizations/singapore-ai-safety-hub/" }, { "id": "S-0004", "title": "Verification for International AI Governance", "url": "https://trustbutveri.fyi/sources/harack-verification-international-ai-governance/" }, { "id": "S-0058", "title": "Governing Through the Cloud: The Intermediary Role of Compute Providers in AI Regulation", "url": "https://trustbutveri.fyi/sources/heim-governing-through-cloud/" }, { "id": "S-1704", "title": "Verifiable Semiconductor Manufacturing", "url": "https://trustbutveri.fyi/sources/ilhan-verifiable-semiconductor-manufacturing/" }, { "id": "S-1705", "title": "About: Oxford Martin AIGI", "url": "https://trustbutveri.fyi/sources/oxford-martin-aigi-about/" }, { "id": "S-0053", "title": "Computing Power and the Governance of Artificial Intelligence", "url": "https://trustbutveri.fyi/sources/sastry-computing-power-governance-ai/" }, { "id": "S-0061", "title": "International Governance of Civilian AI: A Jurisdictional Certification Approach", "url": "https://trustbutveri.fyi/sources/trager-international-governance-civilian-ai/" } ] }, { "id": "O-0121", "slug": "pearl-research", "title": "Pearl Research Labs", "aliases": [], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "kind": "company", "homepage": "https://pearlresearch.ai/", "one_liner": "A company building AI inference infrastructure and Pearl, a blockchain whose proof-of-useful-work mining is a by-product of GPU matrix multiplication.", "sources": [ { "source": "S-3603", "supports": "infrastructure and algorithms for AI inference" }, { "source": "S-1105", "supports": "FP8 proof-of-useful-work protocol specification" }, { "source": "S-1106", "supports": "integer matrix-multiplication whitepaper; mining as a side-effect of AI workloads" }, { "source": "S-1107", "supports": "public monorepo: full node, GPU miner, proof-of-work circuit and verifier; README cites the Komargodski-Weinstein proposal" }, { "source": "S-1609", "supports": "proof of useful work from arbitrary matrix multiplication (Komargodski and Weinstein)" }, { "source": "S-1010", "supports": "Hawkeye (MLSys 2026): two of four authors at Pearl Research Labs; bit-exact CPU replay of GPU matrix multiplication" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/pearl-research/", "source_file": "content/organizations/pearl-research.md", "flags_all": [ "provider-reported" ], "body_markdown": "Pearl Research Labs describes itself as building infrastructure and algorithms for AI inference [[S-3603]]. It develops the [[I-0004|Pearl blockchain]], an instance of [[M-0007|proofs of useful work]]:\n\n- **Protocol specification.** Pearl specifies a proof-of-useful-work protocol in which the unit of mining work is FP8 matrix multiplication on GPUs [[S-1105]].\n- **Integer whitepaper.** Its earlier whitepaper presents mining as a side-effect of the matrix multiplications in AI training and inference workloads [[S-1106]].\n- **Open-source network.** Pearl publishes a monorepo with a reference full node, a vLLM-based GPU miner and a proof-of-work circuit and verifier [[S-1107]]. Its README says the mining follows the proof of useful work from arbitrary matrix multiplication proposed by Komargodski and Weinstein [[S-1107]] [[S-1609]].\n- **Reproducible GPU arithmetic.** Two Pearl researchers, with two Stanford co-authors, published Hawkeye at MLSys 2026. It re-executes on a CPU, bit for bit, the matrix multiplications that NVIDIA Ampere, Hopper and Ada Lovelace GPUs perform in FP16, BF16 and FP8 [[S-1010]]; see [[M-0002]].", "body_text": "Pearl Research Labs describes itself as building infrastructure and algorithms for AI inference [S-3603]. It develops the Pearl blockchain, an instance of proofs of useful work: - Protocol specification. Pearl specifies a proof-of-useful-work protocol in which the unit of mining work is FP8 matrix multiplication on GPUs [S-1105]. - Integer whitepaper. Its earlier whitepaper presents mining as a side-effect of the matrix multiplications in AI training and inference workloads [S-1106]. - Open-source network. Pearl publishes a monorepo with a reference full node, a vLLM-based GPU miner and a proof-of-work circuit and verifier [S-1107]. Its README says the mining follows the proof of useful work from arbitrary matrix multiplication proposed by Komargodski and Weinstein [S-1107] [S-1609]. - Reproducible GPU arithmetic. Two Pearl researchers, with two Stanford co-authors, published Hawkeye at MLSys 2026. It re-executes on a CPU, bit for bit, the matrix multiplications that NVIDIA Ampere, Hopper and Ada Lovelace GPUs perform in FP16, BF16 and FP8 [S-1010]; see Deterministic and bit-exact inference.", "referenced_by": [ { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" }, { "id": "S-1010", "title": "Hawkeye: Reproducing GPU-Level Non-Determinism", "url": "https://trustbutveri.fyi/sources/badash-hawkeye/" }, { "id": "S-1105", "title": "Pearl Floating Point Scheme Specification", "url": "https://trustbutveri.fyi/sources/pearl-floating-point-scheme-specification/" }, { "id": "S-1106", "title": "Pearl INT Whitepaper", "url": "https://trustbutveri.fyi/sources/pearl-int-whitepaper/" }, { "id": "S-1107", "title": "pearl: Monorepo for the Pearl network", "url": "https://trustbutveri.fyi/sources/pearl-network-monorepo/" }, { "id": "S-3571", "title": "Proof of Useful Work from the Ground Up", "url": "https://trustbutveri.fyi/sources/pearl-proof-of-useful-work-blog/" }, { "id": "S-3603", "title": "Pearl Research Labs homepage", "url": "https://trustbutveri.fyi/sources/pearl-research-homepage/" } ] }, { "id": "O-0102", "slug": "planet-labs", "title": "Planet Labs", "aliases": [ "Planet", "Planet Labs PBC" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "kind": "company", "homepage": "https://www.planet.com/", "one_liner": "A company that builds and operates Earth-imaging satellites and sells their imagery; it is one commercial source used to track AI data-centre construction.", "sources": [ { "source": "S-1700", "supports": "company description; renewed data-centre and chip-fab monitoring agreement with an AI developer (provider-reported)" }, { "source": "S-1409", "supports": "named among commonly used providers of paid satellite imagery in a data-centre tracking report" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/planet-labs/", "source_file": "content/organizations/planet-labs.md", "flags_all": [ "provider-reported" ], "body_markdown": "Planet Labs PBC designs, builds and operates a fleet of Earth-observation satellites and sells satellite imagery and geospatial data; it is a public benefit corporation listed on the New York Stock Exchange [[S-1700]]. Its imagery is one of the commercial inputs available for [[M-0020|remote detection of data centres]]:\n\n- A Federation of American Scientists report on tracking AI data centres with satellite imagery names Planet Labs, with Airbus and Vantor, as commonly used sources of paid commercial imagery [[S-1409]].\n- Planet reports that in August 2026 it renewed an agreement with a \"hyperscaler AI developer\" for global monitoring of data centres and of semiconductor-fab construction [[S-1700]]. It says its high-resolution Pelican imagery tracks construction milestones at these sites, which it calls indicators of \"supply chain health and computing capacity\" [[S-1700]].", "body_text": "Planet Labs PBC designs, builds and operates a fleet of Earth-observation satellites and sells satellite imagery and geospatial data; it is a public benefit corporation listed on the New York Stock Exchange [S-1700]. Its imagery is one of the commercial inputs available for remote detection of data centres: - A Federation of American Scientists report on tracking AI data centres with satellite imagery names Planet Labs, with Airbus and Vantor, as commonly used sources of paid commercial imagery [S-1409]. - Planet reports that in August 2026 it renewed an agreement with a \"hyperscaler AI developer\" for global monitoring of data centres and of semiconductor-fab construction [S-1700]. It says its high-resolution Pelican imagery tracks construction milestones at these sites, which it calls indicators of \"supply chain health and computing capacity\" [S-1700].", "referenced_by": [ { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/" }, { "id": "S-1700", "title": "Planet Reports Financial Results for Second Quarter of Fiscal Year 2027", "url": "https://trustbutveri.fyi/sources/planet-q2-fy2027-financial-results/" } ] }, { "id": "O-0100", "slug": "prime-intellect", "title": "Prime Intellect", "aliases": [], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "kind": "company", "homepage": "https://www.primeintellect.ai/", "one_liner": "An AI compute, training and inference company; developer of TOPLOC, a hashing scheme for verifying LLM inference.", "sources": [ { "source": "S-3600", "supports": "integrated compute, training, inference and sandbox stack" }, { "source": "S-1000", "supports": "TOPLOC paper; seven of the eight authors at Prime Intellect; what the method checks" }, { "source": "S-1001", "supports": "open-source TOPLOC package (MIT licence)" }, { "source": "S-1002", "supports": "vLLM and SGLang integrations (provider-reported)" }, { "source": "S-1003", "supports": "TOPLOC proofs in the INTELLECT-2 training run (provider-reported)" }, { "source": "S-1004", "supports": "TOPLOC v2 extends the scheme to pipeline-parallel inference and sampling checks (provider-reported)" }, { "source": "S-3000", "supports": "TOPLOC v2 checks in the SYNTHETIC-2 run: 1,250+ GPUs, false-positive rate over 4 million samples (provider-reported)" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/prime-intellect/", "source_file": "content/organizations/prime-intellect.md", "flags_all": [ "provider-reported" ], "body_markdown": "Prime Intellect describes itself as offering an integrated compute, training, inference and sandbox stack for AI models [[S-3600]]. Its verification work centres on [[I-0001|TOPLOC]], a form of [[M-0001|sampled inference recomputation]]:\n\n- Seven of the eight authors of the ICML 2025 paper that introduces TOPLOC are at Prime Intellect [[S-1000]]. An inference provider using it records the largest activations of the last hidden layer, so that a verifier who re-runs the sequence can check the claimed model, prompt and precision [[S-1000]].\n- Prime Intellect publishes TOPLOC as an MIT-licensed Python package [[S-1001]], and reports integrations with vLLM and a fork of SGLang [[S-1002]].\n- Prime Intellect reports that each inference worker in its INTELLECT-2 decentralized training run generated a TOPLOC proof for every sequence [[S-1003]]. It reports that a later version, TOPLOC v2, extends the scheme to pipeline-parallel inference and adds checks of token sampling [[S-1004]]. It reports using TOPLOC v2 to check the inference workers in its SYNTHETIC-2 data-generation run, on more than 1,250 GPUs, with a false-positive rate of 0.000925% over 4 million samples [[S-3000]].", "body_text": "Prime Intellect describes itself as offering an integrated compute, training, inference and sandbox stack for AI models [S-3600]. Its verification work centres on TOPLOC, a form of sampled inference recomputation: - Seven of the eight authors of the ICML 2025 paper that introduces TOPLOC are at Prime Intellect [S-1000]. An inference provider using it records the largest activations of the last hidden layer, so that a verifier who re-runs the sequence can check the claimed model, prompt and precision [S-1000]. - Prime Intellect publishes TOPLOC as an MIT-licensed Python package [S-1001], and reports integrations with vLLM and a fork of SGLang [S-1002]. - Prime Intellect reports that each inference worker in its INTELLECT-2 decentralized training run generated a TOPLOC proof for every sequence [S-1003]. It reports that a later version, TOPLOC v2, extends the scheme to pipeline-parallel inference and adds checks of token sampling [S-1004]. It reports using TOPLOC v2 to check the inference workers in its SYNTHETIC-2 data-generation run, on more than 1,250 GPUs, with a false-positive rate of 0.000925% over 4 million samples [S-3000].", "referenced_by": [ { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "S-1000", "title": "TOPLOC: A Locality Sensitive Hashing Scheme for Trustless Verifiable Inference", "url": "https://trustbutveri.fyi/sources/ong-toploc/" }, { "id": "S-3600", "title": "Prime Intellect homepage", "url": "https://trustbutveri.fyi/sources/prime-intellect-homepage/" }, { "id": "S-1003", "title": "INTELLECT-2: A Reasoning Model Trained Through Globally Decentralized Reinforcement Learning", "url": "https://trustbutveri.fyi/sources/primeintellect-intellect-2/" }, { "id": "S-3000", "title": "SYNTHETIC-2 Release: Four Million Collaboratively Generated Reasoning Traces", "url": "https://trustbutveri.fyi/sources/primeintellect-synthetic-2-release/" }, { "id": "S-1004", "title": "SYNTHETIC-2", "url": "https://trustbutveri.fyi/sources/primeintellect-synthetic-2/" }, { "id": "S-1002", "title": "TOPLOC: A Locality Sensitive Hashing Scheme for Trustless Verifiable Inference (blog post)", "url": "https://trustbutveri.fyi/sources/primeintellect-toploc-blog/" }, { "id": "S-1001", "title": "PrimeIntellect-ai/toploc (GitHub repository)", "url": "https://trustbutveri.fyi/sources/primeintellect-toploc-code/" } ] }, { "id": "O-0200", "slug": "rand", "title": "RAND", "aliases": [], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "kind": "research-org", "homepage": "https://www.rand.org/", "one_liner": "A nonprofit, nonpartisan research organization; its reports cover verification of international AI agreements, hardware-enabled governance mechanisms and secure inference data centres.", "sources": [ { "source": "S-3607", "supports": "nonprofit nonpartisan research organization and evidence-based decisions" }, { "source": "S-0002", "supports": "six layers of verification; AI chip registry; satellite imagery as a supplementary mechanism" }, { "source": "S-0057", "supports": "hardware-enabled governance mechanisms: offline licensing and fixed set" }, { "source": "S-1510", "supports": "secure inference data center design by the Center on AI, Security, and Technology" }, { "source": "S-1610", "supports": "attack vectors and security levels for model weights" }, { "source": "S-1511", "supports": "RAND CAST listed as doing research on AI verification" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/rand/", "source_file": "content/organizations/rand.md", "flags_all": [], "body_markdown": "RAND describes itself as a nonprofit, nonpartisan research organization that provides leaders with the information they need to make evidence-based decisions [[S-3607]]. Its reports on verification include:\n\n- **Six layers of verification.** Baker et al. set out six layers of verification for rules on large-scale AI development and deployment [[S-0002]]. They describe an AI chip registry with sampled chain-of-custody checks ([[M-0019]]), and list satellite imagery among supplementary mechanisms that they call \"less robust\" than their main layers ([[M-0020]]) [[S-0002]].\n- **Hardware-enabled governance mechanisms.** Kulp et al. define such mechanisms as controls built into AI hardware that enable \"enforcement and compliance verification\" [[S-0057]]. They analyse offline licensing, which lets a GPU run a set amount of work before it refuses or slows further work, and a \"fixed set\" that limits high-bandwidth links to a pod of pre-authorized chips [[S-0057]]. See [[M-0011]], [[M-0009]] and [[M-0010]].\n- **Secure inference data centres.** A report by RAND's Center on AI, Security, and Technology designs a highly secure, vertically integrated inference data centre [[S-1510]]; see [[I-0010]].\n- **Model-weight security.** Nevo et al. identify 38 attack vectors against model weights and define five security levels, for defending against actors up to well-resourced nation-states [[S-1610]]; see [[C-0009]].\n- **Field listing.** The AI Futures Project's verification page lists RAND's Center on AI, Security, and Technology as doing \"foundational technical and policy research on AI verification\" [[S-1511]].", "body_text": "RAND describes itself as a nonprofit, nonpartisan research organization that provides leaders with the information they need to make evidence-based decisions [S-3607]. Its reports on verification include: - Six layers of verification. Baker et al. set out six layers of verification for rules on large-scale AI development and deployment [S-0002]. They describe an AI chip registry with sampled chain-of-custody checks (Chip registries and manufacturing records), and list satellite imagery among supplementary mechanisms that they call \"less robust\" than their main layers (Remote detection of data centres) [S-0002]. - Hardware-enabled governance mechanisms. Kulp et al. define such mechanisms as controls built into AI hardware that enable \"enforcement and compliance verification\" [S-0057]. They analyse offline licensing, which lets a GPU run a set amount of work before it refuses or slows further work, and a \"fixed set\" that limits high-bandwidth links to a pod of pre-authorized chips [S-0057]. See Hardware performance throttling and licensing, Hardware-enabled guarantees (flexHEG) and guarantee processors and On-chip telemetry from timing, memory and performance counters. - Secure inference data centres. A report by RAND's Center on AI, Security, and Technology designs a highly secure, vertically integrated inference data centre [S-1510]; see RAND secure inference data center (SIDC) design. - Model-weight security. Nevo et al. identify 38 attack vectors against model weights and define five security levels, for defending against actors up to well-resourced nation-states [S-1610]; see Model weights have not left the facility. - Field listing. The AI Futures Project's verification page lists RAND's Center on AI, Security, and Technology as doing \"foundational technical and policy research on AI verification\" [S-1511].", "referenced_by": [ { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" }, { "id": "I-0010", "title": "RAND secure inference data center (SIDC) design", "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/" }, { "id": "S-0002", "title": "Verifying International Agreements on AI: Six Layers of Verification for Rules on Large-Scale AI Development and Deployment", "url": "https://trustbutveri.fyi/sources/baker-verifying-international-agreements-ai/" }, { "id": "S-1510", "title": "Highly Secure Inference Data Centers: A Vertically Integrated Strategy for Security Engineering", "url": "https://trustbutveri.fyi/sources/comer-highly-secure-inference-data-centers/" }, { "id": "S-0057", "title": "Hardware-Enabled Governance Mechanisms: Developing Technical Solutions to Exempt Items Otherwise Classified Under Export Control Classification Numbers 3A090 and 4A090", "url": "https://trustbutveri.fyi/sources/kulp-hardware-enabled-governance-mechanisms/" }, { "id": "S-1610", "title": "Securing AI Model Weights: Preventing Theft and Misuse of Frontier Models", "url": "https://trustbutveri.fyi/sources/nevo-securing-ai-model-weights/" }, { "id": "S-3607", "title": "About RAND", "url": "https://trustbutveri.fyi/sources/rand-about/" } ] }, { "id": "O-0160", "slug": "singapore-ai-safety-hub", "title": "Singapore AI Safety Hub (SASH)", "aliases": [ "SASH" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "kind": "nonprofit", "homepage": "https://www.aisafety.sg/", "one_liner": "A Singapore-based AI safety research and field-building organization; with international partners, it develops a confidential network logger for inference verification.", "sources": [ { "source": "S-3606", "supports": "research and field-building organisation rooted in Singapore" }, { "source": "S-1320", "supports": "international collaboration and partners; confidential network logger design and planned prototype" }, { "source": "S-1319", "supports": "prototype code (MIT licence)" }, { "source": "S-1511", "supports": "listed with FLI as building the logger prototype" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/singapore-ai-safety-hub/", "source_file": "content/organizations/singapore-ai-safety-hub.md", "flags_all": [], "body_markdown": "SASH describes itself as a research and field-building organisation rooted in Singapore [[S-3606]]. Its verification work is an international collaboration on inference verification:\n\n- **Collaboration.** SASH reports launching, with the [[O-0205|Oxford Martin AI Governance Initiative]] and a team of researchers, an international collaboration to build tools that make AI verifiable across borders, and names the [[O-0206|Future of Life Institute]] as another early partner [[S-1320]].\n- **Confidential network logger.** The first project asks whether confidential network loggers can help distinguish inference from training [[S-1320]]. SASH describes loggers that copy the traffic leaving a data centre; in its planned next prototype, a random sample of requests goes to a separate cluster that re-runs them and raises an alarm on a mismatch [[S-1320]]. See [[I-0008]], [[M-0013]] and [[M-0001]].\n- **Code.** The first prototype's code is public under the MIT licence [[S-1319]].\n- **Field listing.** The AI Futures Project's verification page lists FLI and SASH as building the logger prototype [[S-1511]].", "body_text": "SASH describes itself as a research and field-building organisation rooted in Singapore [S-3606]. Its verification work is an international collaboration on inference verification: - Collaboration. SASH reports launching, with the Oxford Martin AI Governance Initiative and a team of researchers, an international collaboration to build tools that make AI verifiable across borders, and names the Future of Life Institute as another early partner [S-1320]. - Confidential network logger. The first project asks whether confidential network loggers can help distinguish inference from training [S-1320]. SASH describes loggers that copy the traffic leaving a data centre; in its planned next prototype, a random sample of requests goes to a separate cluster that re-runs them and raises an alarm on a mismatch [S-1320]. See SASH confidential network logger, Network taps and certifiers and Sampled inference recomputation. - Code. The first prototype's code is public under the MIT licence [S-1319]. - Field listing. The AI Futures Project's verification page lists FLI and SASH as building the logger prototype [S-1511].", "referenced_by": [ { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "I-0008", "title": "SASH confidential network logger", "url": "https://trustbutveri.fyi/implementations/sash-confidential-network-logger/" }, { "id": "S-3606", "title": "Singapore AI Safety Hub: About", "url": "https://trustbutveri.fyi/sources/sash-about/" }, { "id": "S-3200", "title": "Field Notes on International AI Verification from Shanghai, Seoul, and Sydney", "url": "https://trustbutveri.fyi/sources/sash-field-notes-international-ai-verification/" }, { "id": "S-1319", "title": "inference-verification: Inference Verification Prototype", "url": "https://trustbutveri.fyi/sources/sash-inference-verification-repo/" }, { "id": "S-1320", "title": "Internationalising AI Verification", "url": "https://trustbutveri.fyi/sources/sash-internationalising-ai-verification/" } ] }, { "id": "O-0214", "slug": "thinking-machines-lab", "title": "Thinking Machines Lab", "aliases": [], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "kind": "company", "homepage": "https://thinkingmachines.ai/", "one_liner": "An AI research and product company; developer of batch-invariant kernels that make language-model outputs independent of batch size.", "sources": [ { "source": "S-3562", "supports": "self-description", "locator": "homepage" }, { "source": "S-1009", "supports": "batch size as the main cause of nondeterminism; batch-invariant kernels; Qwen3-235B result (provider-reported)" }, { "source": "S-1813", "supports": "MIT-licensed batch_invariant_ops library", "locator": "README" }, { "source": "S-1012", "supports": "SGLang's deterministic mode built on the kernels" }, { "source": "S-1814", "supports": "vLLM developers' statement that batch invariance is based on the post" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/thinking-machines-lab/", "source_file": "content/organizations/thinking-machines-lab.md", "flags_all": [ "provider-reported" ], "body_markdown": "Thinking Machines Lab describes itself as \"an artificial intelligence research and product company\" [[S-3562]]. Its work relevant here is [[I-0016]]:\n\n- In September 2025 it published batch-invariant kernels for LLM inference, arguing that varying batch sizes are the main reason LLM endpoints give nondeterministic outputs [[S-1009]]. Its MIT-licensed batch_invariant_ops library replaces several PyTorch operations with batch-invariant versions [[S-1813]]. See [[M-0002|deterministic and bit-exact inference]].\n- It reports that, with the kernels, 1,000 temperature-zero completions from Qwen3-235B were identical [[S-1009]]. SGLang built its deterministic mode on the kernels [[S-1012]], and vLLM's developers state that its batch-invariant mode is based on the same work [[S-1814]].", "body_text": "Thinking Machines Lab describes itself as \"an artificial intelligence research and product company\" [S-3562]. Its work relevant here is Batch-invariant inference kernels (Thinking Machines): - In September 2025 it published batch-invariant kernels for LLM inference, arguing that varying batch sizes are the main reason LLM endpoints give nondeterministic outputs [S-1009]. Its MIT-licensed batchinvariantops library replaces several PyTorch operations with batch-invariant versions [S-1813]. See deterministic and bit-exact inference. - It reports that, with the kernels, 1,000 temperature-zero completions from Qwen3-235B were identical [S-1009]. SGLang built its deterministic mode on the kernels [S-1012], and vLLM's developers state that its batch-invariant mode is based on the same work [S-1814].", "referenced_by": [ { "id": "I-0016", "title": "Batch-invariant inference kernels (Thinking Machines)", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/" }, { "id": "S-1009", "title": "Defeating Nondeterminism in LLM Inference", "url": "https://trustbutveri.fyi/sources/he-defeating-nondeterminism-llm-inference/" }, { "id": "S-1813", "title": "thinking-machines-lab/batch_invariant_ops (GitHub repository)", "url": "https://trustbutveri.fyi/sources/thinking-machines-batch-invariant-ops-code/" }, { "id": "S-3562", "title": "Thinking Machines Lab", "url": "https://trustbutveri.fyi/sources/thinking-machines-homepage/" } ] }, { "id": "O-0141", "slug": "tinfoil", "title": "Tinfoil", "aliases": [], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "kind": "company", "homepage": "https://tinfoil.sh/", "one_liner": "A company offering AI inference inside secure hardware enclaves, with remote attestation that clients can check; developer of the Modelwrap model-identity tool.", "sources": [ { "source": "S-3605", "supports": "AI models running inside secure hardware enclaves" }, { "source": "S-0013", "supports": "Modelwrap binds model weights to enclave attestation (provider-reported)" }, { "source": "S-1209", "supports": "Modelwrap code: dm-verity commitment to model weights" }, { "source": "S-1207", "supports": "measured boot, reproducible builds, Sigstore measurements, client verification" }, { "source": "S-1208", "supports": "connection-time verification and transparency logs" }, { "source": "S-1206", "supports": "supported hardware (AMD SEV-SNP, Intel TDX, NVIDIA confidential-computing mode) and documented limitations" }, { "source": "S-3361", "supports": "Pour Demain's interpretability evaluations of GLM-5.1 on Tinfoil Containers (Intel TDX, eight H200 GPUs)" }, { "source": "S-3360", "supports": "the lab supplies the model and the auditor the code; weights stay in the enclave (provider-reported)" }, { "source": "S-3362", "supports": "safeguard models run inside enclaves; pipeline code public and attested (provider-reported)" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/tinfoil/", "source_file": "content/organizations/tinfoil.md", "flags_all": [ "provider-reported" ], "body_markdown": "Tinfoil says it runs AI models inside secure hardware enclaves [[S-3605]]. Its documentation and code describe how a client checks what runs in the enclave:\n\n- Tinfoil developed Modelwrap, which it reports binds model weights to enclave attestation [[S-0013]]. The open-source tool commits the weights to a dm-verity root hash [[S-1209]]. See [[I-0006]] and [[M-0012]].\n- Tinfoil documents a measured boot chain, reproducible builds, publication of expected measurements through Sigstore, and verification by the client [[S-1207]], with checks at connection time and transparency logs [[S-1208]]; see [[M-0008]].\n- Its documentation lists AMD SEV-SNP and Intel TDX confidential virtual machines, and NVIDIA GPUs in confidential-computing mode, as supported hardware [[S-1206]]. The same page documents limitations, including physical attacks, side channels, I/O leakage, denial of service, supply-chain risks and rollback [[S-1206]].\n- Pour Demain reports running interpretability evaluations of GLM-5.1, an open-weights model of 744 billion parameters, on Tinfoil Containers with Intel TDX and eight NVIDIA H200 GPUs [[S-3361]]. Tinfoil describes the setup as one in which the lab supplies the model, the auditor supplies the evaluation code, and the weights stay inside the enclave [[S-3360]]. See [[M-0025]].\n- Tinfoil reports running safeguard models inside its enclaves that output only a flag, and says the pipeline code is public and its enforcement \"verifiable through attestation\" [[S-3362]]; see [[M-0023]].", "body_text": "Tinfoil says it runs AI models inside secure hardware enclaves [S-3605]. Its documentation and code describe how a client checks what runs in the enclave: - Tinfoil developed Modelwrap, which it reports binds model weights to enclave attestation [S-0013]. The open-source tool commits the weights to a dm-verity root hash [S-1209]. See Tinfoil model identity (Modelwrap) and Model identity attestation. - Tinfoil documents a measured boot chain, reproducible builds, publication of expected measurements through Sigstore, and verification by the client [S-1207], with checks at connection time and transparency logs [S-1208]; see TEE remote attestation for AI workloads. - Its documentation lists AMD SEV-SNP and Intel TDX confidential virtual machines, and NVIDIA GPUs in confidential-computing mode, as supported hardware [S-1206]. The same page documents limitations, including physical attacks, side channels, I/O leakage, denial of service, supply-chain risks and rollback [S-1206]. - Pour Demain reports running interpretability evaluations of GLM-5.1, an open-weights model of 744 billion parameters, on Tinfoil Containers with Intel TDX and eight NVIDIA H200 GPUs [S-3361]. Tinfoil describes the setup as one in which the lab supplies the model, the auditor supplies the evaluation code, and the weights stay inside the enclave [S-3360]. See Confidential multi-party verification. - Tinfoil reports running safeguard models inside its enclaves that output only a flag, and says the pipeline code is public and its enforcement \"verifiable through attestation\" [S-3362]; see Safeguard attestation.", "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" }, { "id": "S-3360", "title": "Auditing a Frontier Model Without Seeing its Weights", "url": "https://trustbutveri.fyi/sources/tinfoil-auditing-frontier-model-without-weights/" }, { "id": "S-1207", "title": "Backend infrastructure", "url": "https://trustbutveri.fyi/sources/tinfoil-docs-attestation-architecture/" }, { "id": "S-1206", "title": "A primer on secure enclaves", "url": "https://trustbutveri.fyi/sources/tinfoil-docs-secure-enclave-primer/" }, { "id": "S-1208", "title": "How verification works in Tinfoil", "url": "https://trustbutveri.fyi/sources/tinfoil-docs-verification-in-tinfoil/" }, { "id": "S-3605", "title": "Tinfoil homepage", "url": "https://trustbutveri.fyi/sources/tinfoil-homepage/" }, { "id": "S-1209", "title": "modelwrap: Reproducible dm-verity read-only image of Huggingface models", "url": "https://trustbutveri.fyi/sources/tinfoil-modelwrap-code/" }, { "id": "S-0013", "title": "How Tinfoil Proves Exactly What Model Is Running", "url": "https://trustbutveri.fyi/sources/tinfoil-proving-model-identity/" }, { "id": "S-3362", "title": "Safety Without Compromising on Privacy", "url": "https://trustbutveri.fyi/sources/tinfoil-safety-without-compromising-privacy/" } ] }, { "id": "O-0142", "slug": "university-of-cambridge", "title": "University of Cambridge", "aliases": [], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "kind": "academic-lab", "homepage": "https://www.cam.ac.uk/", "one_liner": "A UK university; researchers in its Department of Computer Science and Technology developed Attestable Audits, AI benchmarks run and attested inside trusted execution environments.", "sources": [ { "source": "S-0009", "supports": "Attestable Audits: author affiliations, audit and inference protocols, prototype" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/university-of-cambridge/", "source_file": "content/organizations/university-of-cambridge.md", "flags_all": [], "body_markdown": "All four authors of Attestable Audits list the University of Cambridge's Department of Computer Science and Technology as their affiliation [[S-0009]]:\n\n- The Attestable Audits paper runs AI safety benchmarks inside a trusted execution environment and publishes an attestation that binds the model hash, the audit code and data, and the result to a transparency log [[S-0009]]; see [[I-0007]] and [[M-0025]].\n- In the paper's inference protocol, each response carries an attestation that links the model, its earlier audit result, the prompt and the response [[S-0009]]; see [[M-0023]] and [[M-0012]].\n- A prototype ran a 4-bit, 8-billion-parameter model in CPU-only AWS Nitro Enclaves [[S-0009]]; see [[M-0008]].", "body_text": "All four authors of Attestable Audits list the University of Cambridge's Department of Computer Science and Technology as their affiliation [S-0009]: - The Attestable Audits paper runs AI safety benchmarks inside a trusted execution environment and publishes an attestation that binds the model hash, the audit code and data, and the result to a transparency log [S-0009]; see Attestable Audits and Confidential multi-party verification. - In the paper's inference protocol, each response carries an attestation that links the model, its earlier audit result, the prompt and the response [S-0009]; see Safeguard attestation and Model identity attestation. - A prototype ran a 4-bit, 8-billion-parameter model in CPU-only AWS Nitro Enclaves [S-0009]; see TEE remote attestation for AI workloads.", "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0007", "title": "Attestable Audits", "url": "https://trustbutveri.fyi/implementations/attestable-audits/" }, { "id": "S-0009", "title": "Attestable Audits: Verifiable AI Safety Benchmarks Using Trusted Execution Environments", "url": "https://trustbutveri.fyi/sources/schnabl-attestable-audits/" } ] }, { "id": "O-0122", "slug": "university-of-waterloo", "title": "University of Waterloo", "aliases": [], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "kind": "academic-lab", "homepage": "https://uwaterloo.ca/", "one_liner": "A university in Waterloo, Ontario, Canada, whose researchers developed zkLLM, a zero-knowledge proof system for large language model inference.", "sources": [ { "source": "S-0023", "supports": "zkLLM paper; all three authors at the University of Waterloo; view on proofs of training" }, { "source": "S-1108", "supports": "zkLLM code and artifact; repository archived and no longer maintained" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/university-of-waterloo/", "source_file": "content/organizations/university-of-waterloo.md", "flags_all": [], "body_markdown": "The University of Waterloo is in Waterloo, Ontario, Canada; all three authors of zkLLM list it as their affiliation [[S-0023]]. Their work includes:\n\n- Their CCS 2024 paper presents [[I-0003|zkLLM]], a zero-knowledge proof system for large language model inference [[S-0023]]; see [[M-0004]].\n- The authors released the zkLLM code as an artifact after the conference's artifact evaluation, archived on Zenodo [[S-1108]]. The GitHub repository was made read-only in July 2025, and its author states that the project is no longer actively maintained [[S-1108]].\n- The authors wrote that zero-knowledge proofs of LLM training \"may pose insurmountable challenges\" [[S-0023]]; see [[M-0005]].", "body_text": "The University of Waterloo is in Waterloo, Ontario, Canada; all three authors of zkLLM list it as their affiliation [S-0023]. Their work includes: - Their CCS 2024 paper presents zkLLM, a zero-knowledge proof system for large language model inference [S-0023]; see Zero-knowledge proofs of inference. - The authors released the zkLLM code as an artifact after the conference's artifact evaluation, archived on Zenodo [S-1108]. The GitHub repository was made read-only in July 2025, and its author states that the project is no longer actively maintained [S-1108]. - The authors wrote that zero-knowledge proofs of LLM training \"may pose insurmountable challenges\" [S-0023]; see Zero-knowledge proofs of training constraints.", "referenced_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0003", "title": "zkLLM", "url": "https://trustbutveri.fyi/implementations/zkllm/" }, { "id": "S-1108", "title": "zkllm-ccs2024: code for zkLLM: Zero Knowledge Proofs for Large Language Models", "url": "https://trustbutveri.fyi/sources/sun-zkllm-code/" }, { "id": "S-0023", "title": "zkLLM: Zero Knowledge Proofs for Large Language Models", "url": "https://trustbutveri.fyi/sources/sun-zkllm/" } ] }, { "id": "O-0203", "slug": "verifiable-compute-foundation", "title": "Verifiable Compute Foundation", "aliases": [ "VCF" ], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [ "provider-reported" ], "kind": "nonprofit", "homepage": "https://verifiablecompute.ai/", "one_liner": "Lucid Computing describes VCF as a nonprofit that will allocate free research access to a bare-metal AI-verification cluster.", "sources": [ { "source": "S-1701", "supports": "described as an independent nonprofit building a toolbox of AI verification techniques; Lucid builds and operates the cluster on its behalf", "locator": "overview page" }, { "source": "S-1702", "supports": "cluster hardware, research areas, free access, allocation by the foundation, opening in November", "locator": "experimentation page" }, { "source": "S-1406", "supports": "research cluster managed by the foundation, at no cost to qualifying groups", "locator": "homepage, research section" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/verifiable-compute-foundation/", "source_file": "content/organizations/verifiable-compute-foundation.md", "flags_all": [ "provider-reported" ], "body_markdown": "[[O-0180|Lucid Computing]] describes the Verifiable Compute Foundation (VCF) as \"an independent nonprofit working to create a toolbox of AI verification techniques\" [[S-1701]]. Lucid's pages describe the foundation's role in a research cluster:\n\n- Lucid reports that the foundation's planned cluster includes NVIDIA H100, H200 and B300 and AMD Instinct MI355X accelerators, with firmware-level access for researchers [[S-1701]] [[S-1702]]. Lucid says it builds and operates the hardware on the foundation's behalf [[S-1701]] [[S-1702]].\n- According to Lucid, the cluster is \"free for qualifying safety & verification research\", the foundation allocates it through a fair-share process, and Lucid \"takes no part in allocation decisions\" [[S-1702]] [[S-1406]]. In September 2026, Lucid's page said the cluster opens to the community in November [[S-1702]].\n- Lucid's cluster page lists the research areas as workload forensics from power and emission signatures, probing of TEEs and confidential computing, network observation, firmware and root-of-trust inspection, and in-path hardware such as DPUs and FPGAs [[S-1702]]. These areas bear on [[M-0021|workload classification from side channels]], [[M-0008|TEE remote attestation]] and [[M-0013|network taps]].", "body_text": "Lucid Computing describes the Verifiable Compute Foundation (VCF) as \"an independent nonprofit working to create a toolbox of AI verification techniques\" [S-1701]. Lucid's pages describe the foundation's role in a research cluster: - Lucid reports that the foundation's planned cluster includes NVIDIA H100, H200 and B300 and AMD Instinct MI355X accelerators, with firmware-level access for researchers [S-1701] [S-1702]. Lucid says it builds and operates the hardware on the foundation's behalf [S-1701] [S-1702]. - According to Lucid, the cluster is \"free for qualifying safety & verification research\", the foundation allocates it through a fair-share process, and Lucid \"takes no part in allocation decisions\" [S-1702] [S-1406]. In September 2026, Lucid's page said the cluster opens to the community in November [S-1702]. - Lucid's cluster page lists the research areas as workload forensics from power and emission signatures, probing of TEEs and confidential computing, network observation, firmware and root-of-trust inspection, and in-path hardware such as DPUs and FPGAs [S-1702]. These areas bear on workload classification from side channels, TEE remote attestation and network taps.", "referenced_by": [ { "id": "O-0180", "title": "Lucid Computing", "url": "https://trustbutveri.fyi/organizations/lucid-computing/" } ] }, { "id": "O-0213", "slug": "zkonduit", "title": "Zkonduit", "aliases": [], "status": "published", "last_reviewed": "2026-09-25", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "kind": "company", "homepage": "https://ezkl.xyz/", "one_liner": "The company that makes EZKL, a library for proving in zero knowledge that a neural network produced a given output.", "sources": [ { "source": "S-3561", "supports": "self-description of EZKL; Zkonduit copyright", "locator": "documentation home" }, { "source": "S-1806", "supports": "EZKL: ONNX models compiled to halo2 circuits; public or private model and data; Zkonduit Inc. as maintainer", "locator": "README" }, { "source": "S-0024", "supports": "two EZKL authors; verifiable evaluations with EZKL; model sizes up to about a million parameters", "locator": "author list; §6.1 Table 1" }, { "source": "S-0070", "supports": "Trail of Bits audit and limited fix review: 34 findings, 8 high resolved, 3 other findings partially resolved, 2 unresolved" } ], "type": "organization", "url": "https://trustbutveri.fyi/organizations/zkonduit/", "source_file": "content/organizations/zkonduit.md", "flags_all": [], "body_markdown": "Zkonduit Inc. develops EZKL, which its documentation describes as \"a developer-friendly system for verifiable AI and analytics\" [[S-3561]]. Its verification work is [[I-0014]]:\n\n- EZKL compiles a model exported in the ONNX format into a halo2 circuit, so a prover can show that a model produced an output while keeping either the model or the data private [[S-1806]]. See [[M-0004|zero-knowledge proofs of inference]].\n- Two of the authors of South et al. are from EZKL. The paper uses EZKL to attest that a model with private weights reaches a stated score, with proofs for models of up to about a million parameters [[S-0024]].\n- Trail of Bits reviewed EZKL for Zkonduit in January 2025 and reported 34 findings, 8 of high severity. Its March fix review marked all eight high-severity findings resolved, three other findings partially resolved and two unresolved [[S-0070]].", "body_text": "Zkonduit Inc. develops EZKL, which its documentation describes as \"a developer-friendly system for verifiable AI and analytics\" [S-3561]. Its verification work is EZKL: - EZKL compiles a model exported in the ONNX format into a halo2 circuit, so a prover can show that a model produced an output while keeping either the model or the data private [S-1806]. See zero-knowledge proofs of inference. - Two of the authors of South et al. are from EZKL. The paper uses EZKL to attest that a model with private weights reaches a stated score, with proofs for models of up to about a million parameters [S-0024]. - Trail of Bits reviewed EZKL for Zkonduit in January 2025 and reported 34 findings, 8 of high severity. Its March fix review marked all eight high-severity findings resolved, three other findings partially resolved and two unresolved [S-0070].", "referenced_by": [ { "id": "I-0014", "title": "EZKL", "url": "https://trustbutveri.fyi/implementations/ezkl/" }, { "id": "S-3561", "title": "EZKL documentation (overview)", "url": "https://trustbutveri.fyi/sources/ezkl-docs-overview/" }, { "id": "S-0024", "title": "Verifiable evaluations of machine learning models using zkSNARKs", "url": "https://trustbutveri.fyi/sources/south-verifiable-evaluations-zksnarks/" }, { "id": "S-1806", "title": "zkonduit/ezkl (GitHub repository)", "url": "https://trustbutveri.fyi/sources/zkonduit-ezkl-code/" } ] }, { "id": "S-1205", "slug": "aarne-international-security-applications-flexheg", "status": "published", "risk_flags": [], "title": "International Security Applications of Flexible Hardware-Enabled Guarantees", "authors": [ "O. Aarne", "J. Petrie" ], "year": 2025, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/aarne-international-security-applications-flexheg/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2506.15100", "arxiv": "2506.15100", "version_note": "v1 (2025-06-18), still the only version and with no journal reference, read on 2026-09-25.", "note": "The report calls itself the third and final part of the flexHEG series commissioned by ARIA. The arXiv abstract page does not state the series number.", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2506.15100", "source_file": "content/sources/aarne-international-security-applications-flexheg.yaml", "flags_all": [], "cited_by": [ { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" } ], "referenced_by": [ { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" } ] }, { "id": "S-0056", "slug": "aarne-secure-governable-chips", "status": "published", "risk_flags": [], "title": "Secure, Governable Chips: Using On-Chip Mechanisms to Manage National Security Risks from AI & Advanced Computing", "authors": [ "O. Aarne", "T. Fist", "C. Withers" ], "year": 2024, "venue": "Center for a New American Security", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/aarne-secure-governable-chips/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Motivations and policy proposals\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [ "O-0207" ], "source_type": "tech-report", "source_url": "https://www.cnas.org/publications/reports/secure-governable-chips", "source_file": "content/sources/aarne-secure-governable-chips.yaml", "flags_all": [], "cited_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "K-0007", "title": "Hardware-enabled mechanism (HEM)", "url": "https://trustbutveri.fyi/concepts/hardware-enabled-mechanism/" }, { "id": "O-0207", "title": "Center for a New American Security", "url": "https://trustbutveri.fyi/organizations/center-for-a-new-american-security/" } ], "referenced_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "K-0007", "title": "Hardware-enabled mechanism (HEM)", "url": "https://trustbutveri.fyi/concepts/hardware-enabled-mechanism/" }, { "id": "O-0207", "title": "Center for a New American Security", "url": "https://trustbutveri.fyi/organizations/center-for-a-new-american-security/" } ] }, { "id": "S-1110", "slug": "abbaszadeh-zero-knowledge-proofs-of-training", "status": "published", "risk_flags": [], "title": "Zero-Knowledge Proofs of Training for Deep Neural Networks", "authors": [ "K. Abbaszadeh", "C. Pappas", "J. Katz", "D. Papadopoulos" ], "year": 2024, "venue": "2024 ACM SIGSAC Conference on Computer and Communications Security (CCS 2024), pp. 4316-4330", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/abbaszadeh-zero-knowledge-proofs-of-training/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.1145/3658644.3670316", "arxiv": null, "version_note": "Read the IACR ePrint version (last revised 22 July 2024). Venue and DOI from the HKUST research portal record.", "note": "Describes the Kaizen zkPoT system.", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://eprint.iacr.org/2024/162", "source_file": "content/sources/abbaszadeh-zero-knowledge-proofs-of-training.yaml", "flags_all": [], "cited_by": [ { "id": "M-0005", "title": "Zero-knowledge proofs of training constraints", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-training-constraints/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" } ], "referenced_by": [ { "id": "M-0005", "title": "Zero-knowledge proofs of training constraints", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-training-constraints/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" } ] }, { "id": "S-1506", "slug": "abdelghafar-minimal-information-disclosure", "status": "published", "risk_flags": [], "title": "Privacy-Preserving AI Verification via Minimal Information Disclosure", "authors": [ "S. Abdelghafar", "G. Kulp" ], "year": 2026, "date": "2026-08-03", "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/abdelghafar-minimal-information-disclosure/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2608.02774", "version_note": "v1, the only version on 2026-09-25, read via the arXiv HTML rendering. The HTML fetch ends before the reference list, and the PDF returned HTTP 429, so the bibliography entries were not read.", "note": "Authors affiliated with Rice University (Abdelghafar) and Intelligence Security Laboratories (Kulp).", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2608.02774", "source_file": "content/sources/abdelghafar-minimal-information-disclosure.yaml", "flags_all": [], "cited_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" } ], "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" } ] }, { "id": "S-2003", "slug": "abm-treaty-1972", "status": "published", "risk_flags": [], "title": "Treaty between the United States of America and the Union of Soviet Socialist Republics on the Limitation of Anti-Ballistic Missile Systems", "authors": [ "United States of America", "Union of Soviet Socialist Republics" ], "year": 1972, "date": "1972-05-26", "venue": "United Nations Treaty Series, vol. 944, No. 13446", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/abm-treaty-1972/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Article XII covers national technical means of verification and non-interference with them.", "organizations": [], "source_type": "gov-doc", "source_url": "https://treaties.un.org/doc/Publication/UNTS/Volume%20944/volume-944-I-13446-English.pdf", "source_file": "content/sources/abm-treaty-1972.yaml", "flags_all": [], "cited_by": [], "referenced_by": [] }, { "id": "S-1511", "slug": "ai-futures-get-involved-verification", "status": "published", "risk_flags": [], "title": "Get Involved in Verification", "authors": [ "AI Futures Project" ], "year": 2026, "date": "2026-07-09", "venue": "AI 2040", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/ai-futures-get-involved-verification/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "The page states it was updated on 9 July 2026. Content as viewed on 2026-09-23; the workstream statuses were unchanged on 2026-09-25, and the site changelog lists only changes to event and contact links on the page since then.", "note": "Companion page to S-0067 listing open technical problems for the plan's verification components.", "organizations": [ "O-0201" ], "source_type": "blog", "source_url": "https://ai-2040.com/supplements/verification-plan/get-involved", "source_file": "content/sources/ai-futures-get-involved-verification.yaml", "flags_all": [], "cited_by": [ { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "O-0201", "title": "AI Futures Project", "url": "https://trustbutveri.fyi/organizations/ai-futures-project/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" }, { "id": "O-0206", "title": "Future of Life Institute", "url": "https://trustbutveri.fyi/organizations/future-of-life-institute/" }, { "id": "O-0200", "title": "RAND", "url": "https://trustbutveri.fyi/organizations/rand/" }, { "id": "O-0160", "title": "Singapore AI Safety Hub (SASH)", "url": "https://trustbutveri.fyi/organizations/singapore-ai-safety-hub/" } ], "referenced_by": [ { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "O-0201", "title": "AI Futures Project", "url": "https://trustbutveri.fyi/organizations/ai-futures-project/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" }, { "id": "O-0206", "title": "Future of Life Institute", "url": "https://trustbutveri.fyi/organizations/future-of-life-institute/" }, { "id": "O-0200", "title": "RAND", "url": "https://trustbutveri.fyi/organizations/rand/" }, { "id": "O-0160", "title": "Singapore AI Safety Hub (SASH)", "url": "https://trustbutveri.fyi/organizations/singapore-ai-safety-hub/" } ] }, { "id": "S-3608", "slug": "ai-futures-homepage", "status": "published", "risk_flags": [], "title": "AI Futures Project homepage", "authors": [], "year": 2026, "venue": "AI Futures Project", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/ai-futures-homepage/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Official organization page, read 2026-09-25; the year is the access year because the page is undated. Supports the organization's attributed self-description: small research group forecasting the future of AI funded by donations and grants.", "organizations": [ "O-0201" ], "source_type": "docs", "source_url": "https://www.aifutures.org/", "source_file": "content/sources/ai-futures-homepage.yaml", "flags_all": [], "cited_by": [ { "id": "O-0201", "title": "AI Futures Project", "url": "https://trustbutveri.fyi/organizations/ai-futures-project/" } ], "referenced_by": [ { "id": "O-0201", "title": "AI Futures Project", "url": "https://trustbutveri.fyi/organizations/ai-futures-project/" } ] }, { "id": "S-0008", "slug": "ai-security-forum-rfds", "status": "published", "risk_flags": [], "title": "AI Security RFDs", "authors": [ "AI Security Forum" ], "year": 2025, "venue": "AI Security Forum", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/ai-security-forum-rfds/", "archive_url": null, "accessed": "2026-09-23", "doi": null, "arxiv": null, "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Research agendas and surveys\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "forum", "source_url": "https://projects.aisecurity.forum/", "source_file": "content/sources/ai-security-forum-rfds.yaml", "flags_all": [], "cited_by": [], "referenced_by": [] }, { "id": "S-1213", "slug": "amd-sb-3020-rmp-initialization", "status": "published", "risk_flags": [], "title": "SEV-SNP RMP Initialization Vulnerability (AMD-SB-3020)", "authors": [ "AMD" ], "year": 2025, "date": "2025-10-13", "venue": "AMD product security bulletin", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/amd-sb-3020-rmp-initialization/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Initial publication 2025-10-13; read in the revision last updated 2026-02-23.", "note": "AMD's own bulletin on CVE-2025-0033 (RMPocalypse, S-1212); supports only \"AMD reports\" statements. Rates the issue CVSS 6.0 (medium) and lists SEV firmware, microcode or platform-initialisation firmware updates for affected EPYC 7003, 8004, 9004 and 9005 server series and embedded series, with release dates from June 2025 to February 2026.", "organizations": [], "source_type": "docs", "source_url": "https://www.amd.com/en/resources/product-security/bulletin/amd-sb-3020.html", "source_file": "content/sources/amd-sb-3020-rmp-initialization.yaml", "flags_all": [], "cited_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" } ], "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" } ] }, { "id": "S-3128", "slug": "amd-sb-3034-sev-snp-routing", "status": "published", "risk_flags": [], "title": "SEV-SNP Routing Misconfiguration (AMD-SB-3034)", "authors": [ "AMD" ], "year": 2026, "date": "2026-04-14", "venue": "AMD product security bulletin", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/amd-sb-3034-sev-snp-routing/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Initial publication 2026-04-14; read in the revision of 2026-09-24.", "note": "AMD's own bulletin on CVE-2025-54510 (Fabricked, S-3127); supports only \"AMD reports\" statements. Credits Schlüter, Wech and Shinde of ETH Zurich. Rates the issue CVSS 5.9 (medium). Lists platform-initialisation firmware updates for EPYC 7003, 8004, 9004 and 9005 server series released 2025-11-26 to 2025-12-15, and for the embedded series to 2026-04-13.", "organizations": [], "source_type": "docs", "source_url": "https://www.amd.com/en/resources/product-security/bulletin/amd-sb-3034.html", "source_file": "content/sources/amd-sb-3034-sev-snp-routing.yaml", "flags_all": [], "cited_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ], "referenced_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ] }, { "id": "S-3131", "slug": "amd-sb-3045-milanlaunchy", "status": "published", "risk_flags": [], "title": "MilanLaunchy Firmware Loader (AMD-SB-3045)", "authors": [ "AMD" ], "year": 2026, "date": "2026-05-12", "venue": "AMD product security bulletin", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/amd-sb-3045-milanlaunchy/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "AMD's own bulletin on MilanLaunchy (S-3130); supports only \"AMD reports\" statements. Lists CVE-2021-26315, CVE-2024-21944 and CVE-2024-21981, rates the report informational, and calls it \"a legacy attack that was previously mitigated in 2021\". Mitigation: MilanPI 1.0.0.3 or later for EPYC 7003.", "organizations": [], "source_type": "docs", "source_url": "https://www.amd.com/en/resources/product-security/bulletin/amd-sb-3045.html", "source_file": "content/sources/amd-sb-3045-milanlaunchy.yaml", "flags_all": [], "cited_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ], "referenced_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ] }, { "id": "S-0066", "slug": "ammann-faster-ai-diffusion-hardware-verification", "status": "published", "risk_flags": [], "title": "Faster AI Diffusion Through Hardware-Based Verification", "authors": [ "N. Ammann", "D. Dalrymple" ], "year": 2025, "venue": "Institute for Progress", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/ammann-faster-ai-diffusion-hardware-verification/", "archive_url": null, "accessed": "2026-09-23", "doi": null, "arxiv": null, "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Motivations and policy proposals\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "blog", "source_url": "https://ifp.org/faster-ai-diffusion-through-hardware-based-verification/", "source_file": "content/sources/ammann-faster-ai-diffusion-hardware-verification.yaml", "flags_all": [], "cited_by": [], "referenced_by": [] }, { "id": "S-3601", "slug": "amodo-design-about", "status": "published", "risk_flags": [], "title": "Amodo Design: About Us", "authors": [], "year": 2026, "venue": "Amodo Design", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/amodo-design-about/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Official organization page, read 2026-09-25; the year is the access year because the page is undated. Supports the organization's attributed self-description: engineering company building tools for research.", "organizations": [ "O-0101" ], "source_type": "docs", "source_url": "https://amododesign.com/about-us/", "source_file": "content/sources/amodo-design-about.yaml", "flags_all": [], "cited_by": [ { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ], "referenced_by": [ { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ] }, { "id": "S-0017", "slug": "amodo-example-schemes-high-stakes-ai-agreements", "status": "published", "risk_flags": [], "title": "Example Schemes for Verifying High-Stakes AI Agreements", "authors": [ "Amodo Design" ], "year": 2026, "venue": "Amodo Design", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/amodo-example-schemes-high-stakes-ai-agreements/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Inference verification\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [ "O-0101" ], "source_type": "blog", "source_url": "https://amododesign.com/notes/2026-06-23-verification-algorithms/", "source_file": "content/sources/amodo-example-schemes-high-stakes-ai-agreements.yaml", "flags_all": [], "cited_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "K-0024", "title": "Cryptographic commitment", "url": "https://trustbutveri.fyi/concepts/cryptographic-commitment/" }, { "id": "K-0019", "title": "Evidence binding", "url": "https://trustbutveri.fyi/concepts/evidence-binding/" }, { "id": "K-0009", "title": "Recomputation", "url": "https://trustbutveri.fyi/concepts/recomputation/" }, { "id": "K-0020", "title": "Sampling and assurance", "url": "https://trustbutveri.fyi/concepts/sampling-and-assurance/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ], "referenced_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "K-0024", "title": "Cryptographic commitment", "url": "https://trustbutveri.fyi/concepts/cryptographic-commitment/" }, { "id": "K-0019", "title": "Evidence binding", "url": "https://trustbutveri.fyi/concepts/evidence-binding/" }, { "id": "K-0009", "title": "Recomputation", "url": "https://trustbutveri.fyi/concepts/recomputation/" }, { "id": "K-0020", "title": "Sampling and assurance", "url": "https://trustbutveri.fyi/concepts/sampling-and-assurance/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ] }, { "id": "S-1303", "slug": "amodo-improving-disk-wiping-speed", "status": "published", "risk_flags": [], "title": "Improving Disk Wiping Speed for Memory Wipes", "authors": [ "Amodo Design" ], "year": 2026, "date": "2026-09-14", "venue": "Amodo Design", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/amodo-improving-disk-wiping-speed/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "No individual byline on the page.", "organizations": [ "O-0101" ], "source_type": "blog", "source_url": "https://amododesign.com/notes/2026-09-14-disk-wiping-speed/", "source_file": "content/sources/amodo-improving-disk-wiping-speed.yaml", "flags_all": [], "cited_by": [ { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ], "referenced_by": [ { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ] }, { "id": "S-1007", "slug": "amodo-inference-recomputation-prototype-code", "status": "published", "risk_flags": [], "title": "Amodo-Design/Inference-Recomputation-Prototype (GitHub repository)", "authors": [ "Amodo Design" ], "year": 2026, "venue": "GitHub", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/amodo-inference-recomputation-prototype-code/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Pinned to commit 4bc6872 (4bc687218d7af8aba3db6d373d4700b0f7ff746c, 10 Sep 2026), the single commit on main per the GitHub API on 2026-09-23; no tags.", "license_note": "MIT licence; the README describes it as research prototype code, not production software.", "note": "The README describes its DiFR module as a vendored, modified copy of the upstream difr library.", "organizations": [ "O-0101" ], "source_type": "code", "source_url": "https://github.com/Amodo-Design/Inference-Recomputation-Prototype", "source_file": "content/sources/amodo-inference-recomputation-prototype-code.yaml", "flags_all": [], "cited_by": [ { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ], "referenced_by": [ { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ] }, { "id": "S-3001", "slug": "amodo-inference-verification-prototype-stage-1", "status": "published", "risk_flags": [], "title": "An Inference Verification Prototype — Stage 1", "authors": [ "Amodo Design" ], "year": 2026, "date": "2026-06-29", "venue": "Amodo Design", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/amodo-inference-verification-prototype-stage-1/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "No individual byline. Describes the authors' first prototype, which reuses the DiFR library, and its tests against two LoRA fine-tuning attacks on an 8×H100 server. Results are provider-reported.", "organizations": [ "O-0101" ], "source_type": "blog", "source_url": "https://amododesign.com/notes/2026-06-29-inference-verification-prototype/", "source_file": "content/sources/amodo-inference-verification-prototype-stage-1.yaml", "flags_all": [], "cited_by": [ { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" } ], "referenced_by": [ { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" } ] }, { "id": "S-1302", "slug": "amodo-memory-wipes-performance-analysis", "status": "published", "risk_flags": [], "title": "Memory Wipes - Performance Analysis", "authors": [ "Amodo Design" ], "year": 2026, "date": "2026-07-01", "venue": "Amodo Design", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/amodo-memory-wipes-performance-analysis/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Page carries a correction note saying the original post's wipe-time results were revised; figures used here are the corrected ones.", "note": "No individual byline on the page.", "organizations": [ "O-0101" ], "source_type": "blog", "source_url": "https://amododesign.com/notes/2026-07-01-memory-wiping/", "source_file": "content/sources/amodo-memory-wipes-performance-analysis.yaml", "flags_all": [], "cited_by": [ { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ], "referenced_by": [ { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ] }, { "id": "S-1312", "slug": "amodo-network-tap-inference-verification-prototype", "status": "published", "risk_flags": [], "title": "Fitting a Network TAP to our Inference Verification Prototype", "authors": [ "Amodo Design" ], "year": 2026, "date": "2026-09-15", "venue": "Amodo Design", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/amodo-network-tap-inference-verification-prototype/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "No individual byline and no source-code link on the page.", "organizations": [ "O-0101" ], "source_type": "blog", "source_url": "https://amododesign.com/notes/2026-09-15-network-tap-inference-verification/", "source_file": "content/sources/amodo-network-tap-inference-verification-prototype.yaml", "flags_all": [], "cited_by": [ { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ], "referenced_by": [ { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ] }, { "id": "S-1310", "slug": "amodo-network-tapping-technical-assessment", "status": "published", "risk_flags": [], "title": "Network Tapping for AI Verification: A Technical Assessment", "authors": [ "Amodo Design" ], "year": 2026, "date": "2026-05-03", "venue": "Amodo Design", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/amodo-network-tapping-technical-assessment/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "No individual byline on the page. Amodo republished the note on its Substack on 2026-05-14 as \"Tech note: network taps for AI verification\" (https://amodo.substack.com/p/tech-note-network-taps-for-ai-verification), with the byline Thomas Milton and Sam Reynolds.", "organizations": [ "O-0101" ], "source_type": "blog", "source_url": "https://amododesign.com/notes/2026-05-03-network-tapping/", "source_file": "content/sources/amodo-network-tapping-technical-assessment.yaml", "flags_all": [], "cited_by": [ { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ], "referenced_by": [ { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ] }, { "id": "S-1309", "slug": "amodo-network-taps-first-test", "status": "published", "risk_flags": [], "title": "Network Taps — A First Test", "authors": [ "Amodo Design" ], "year": 2026, "date": "2026-03-20", "venue": "Amodo Design", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/amodo-network-taps-first-test/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "No individual byline on the page.", "organizations": [ "O-0101" ], "source_type": "blog", "source_url": "https://amododesign.com/notes/2026-03-20-network-taps-first-test/", "source_file": "content/sources/amodo-network-taps-first-test.yaml", "flags_all": [], "cited_by": [ { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ], "referenced_by": [ { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ] }, { "id": "S-1311", "slug": "amodo-network-traffic-hashing", "status": "published", "risk_flags": [], "title": "Network Traffic Hashing", "authors": [ "Amodo Design" ], "year": 2026, "date": "2026-07-03", "venue": "Amodo Design", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/amodo-network-traffic-hashing/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "No individual byline on the page.", "organizations": [ "O-0101" ], "source_type": "blog", "source_url": "https://amododesign.com/notes/2026-07-03-network-traffic-hashing/", "source_file": "content/sources/amodo-network-traffic-hashing.yaml", "flags_all": [], "cited_by": [ { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ], "referenced_by": [ { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ] }, { "id": "S-1008", "slug": "amodo-plan-a-verification-sitrep", "status": "published", "risk_flags": [], "title": "AI 2040 Plan A — Verification SITREP", "authors": [ "Amodo Design" ], "year": 2026, "venue": "Amodo Design", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/amodo-plan-a-verification-sitrep/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Undated status page with no byline; its status labels (\"Active effort\", \"Not started\", \"Not on track\", \"Uncertain\") may change. Content as viewed on 2026-09-23; the recomputation items were unchanged on 2026-09-25.", "organizations": [ "O-0101" ], "source_type": "blog", "source_url": "https://amododesign.com/ai-verification/plan-a-sitrep/", "source_file": "content/sources/amodo-plan-a-verification-sitrep.yaml", "flags_all": [], "cited_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" }, { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ], "referenced_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" }, { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ] }, { "id": "S-1321", "slug": "amodo-pose-memory-wiping-code", "status": "published", "risk_flags": [], "title": "Amodo-Design/PoSE-Memory-Wiping (GitHub repository)", "authors": [ "Amodo Design" ], "year": 2026, "venue": "GitHub", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/amodo-pose-memory-wiping-code/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Pinned to commit fae0935d37a4e2b983334e5b999cbc2c4b6edf67 (head of main on 2026-09-23, committed 2026-09-14; 2 commits, no tags).", "license_note": "MIT licence (\"Copyright (c) 2026 Amodo Design Ltd.\"), per the repository.", "note": "Linked from Amodo's note \"Improving Disk Wiping Speed for Memory Wipes\" (S-1303). The README says the repository is the disk-wiping path only and excludes the verifier and the RAM and GPU-HBM session code.", "organizations": [ "O-0101" ], "source_type": "code", "source_url": "https://github.com/Amodo-Design/PoSE-Memory-Wiping", "source_file": "content/sources/amodo-pose-memory-wiping-code.yaml", "flags_all": [], "cited_by": [ { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ], "referenced_by": [ { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ] }, { "id": "S-1006", "slug": "amodo-scaling-recomputation-inference-verification", "status": "published", "risk_flags": [], "title": "Scaling Recomputation Inference Verification", "authors": [ "Amodo Design" ], "year": 2026, "date": "2026-09-02", "venue": "Amodo Design", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/amodo-scaling-recomputation-inference-verification/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "No individual byline. Describes the authors' own prototype; performance statements are provider-reported.", "organizations": [ "O-0101" ], "source_type": "blog", "source_url": "https://amododesign.com/notes/2026-09-02-scaling-recomputation-inference-verification/", "source_file": "content/sources/amodo-scaling-recomputation-inference-verification.yaml", "flags_all": [], "cited_by": [ { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ], "referenced_by": [ { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ] }, { "id": "S-1313", "slug": "amodo-tray-bandwidth-boundary", "status": "published", "risk_flags": [], "title": "The Tray as a Bandwidth Boundary", "authors": [ "Amodo Design" ], "year": 2026, "date": "2026-03-16", "venue": "Amodo Design", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/amodo-tray-bandwidth-boundary/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "No individual byline on the page.", "organizations": [ "O-0101" ], "source_type": "blog", "source_url": "https://amododesign.com/notes/2026-03-16-dpu-bandwidth-limiter/", "source_file": "content/sources/amodo-tray-bandwidth-boundary.yaml", "flags_all": [], "cited_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ], "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ] }, { "id": "S-0048", "slug": "amodo-understanding-data-center-power-delivery", "status": "published", "risk_flags": [], "title": "Understanding Data Center Power Delivery", "authors": [ "Amodo Design" ], "year": 2026, "date": "2026-04-02", "venue": "Amodo Design", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/amodo-understanding-data-center-power-delivery/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "The URL imported from the Hodgkins bibliography (https://amododesign.com/data-center-power-delivery/) redirects to the notes page above, which was read on 2026-09-23. The page credits \"Amodo Design\" and names no individual authors; the bibliography's list of three individual authors could not be confirmed on the page.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Power telemetry and side-channel attacks\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [ "O-0101" ], "source_type": "blog", "source_url": "https://amododesign.com/notes/2026-04-02-data-center-power-delivery/", "source_file": "content/sources/amodo-understanding-data-center-power-delivery.yaml", "flags_all": [], "cited_by": [ { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ], "referenced_by": [ { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ] }, { "id": "S-0007", "slug": "ansari-hardware-level-governance-ai-compute", "status": "published", "risk_flags": [], "title": "Hardware-Level Governance of AI Compute: A Feasibility Taxonomy for Regulatory Compliance and Treaty Verification", "authors": [ "S. Ansari" ], "year": 2026, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/ansari-hardware-level-governance-ai-compute/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2604.04712", "arxiv": "2604.04712", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Research agendas and surveys\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2604.04712", "source_file": "content/sources/ansari-hardware-level-governance-ai-compute.yaml", "flags_all": [], "cited_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/" }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" } ], "referenced_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/" }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" } ] }, { "id": "S-3221", "slug": "anthropic-activating-asl3-protections", "status": "published", "risk_flags": [], "title": "Activating AI Safety Level 3 Protections", "authors": [ "Anthropic" ], "year": 2025, "date": "2025-05-22", "venue": "Anthropic", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/anthropic-activating-asl3-protections/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Developer's announcement of its own security measures. Describes egress bandwidth controls that limit the rate of outbound network traffic from environments holding model weights. Supports \"Anthropic reports\" statements only.", "organizations": [], "source_type": "blog", "source_url": "https://www.anthropic.com/news/activating-asl3-protections", "source_file": "content/sources/anthropic-activating-asl3-protections.yaml", "flags_all": [], "cited_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" } ], "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" } ] }, { "id": "S-1817", "slug": "anthropic-confidential-inference-trusted-vms", "status": "published", "risk_flags": [], "title": "Confidential Inference via Trusted Virtual Machines", "authors": [ "Anthropic", "Pattern Labs" ], "year": 2025, "date": "2025-06-18", "venue": "Anthropic research", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/anthropic-confidential-inference-trusted-vms/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Published in collaboration with Pattern Labs, with a companion report (\"Confidential Inference Systems: Design principles and security risks\", June 2025). Sketches a small, attested model loader and invoker that decrypts data only inside a trusted environment and passes it to the accelerator, for protecting model weights and user data; states the work is early. Developer's description of its own research; supports \"Anthropic reports\" statements.", "organizations": [], "source_type": "blog", "source_url": "https://www.anthropic.com/research/confidential-inference-trusted-vms", "source_file": "content/sources/anthropic-confidential-inference-trusted-vms.yaml", "flags_all": [], "cited_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ], "referenced_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ] }, { "id": "S-0055", "slug": "anwar-foundational-challenges-assuring-alignment-safety", "status": "published", "risk_flags": [], "title": "Foundational Challenges in Assuring Alignment and Safety of Large Language Models", "authors": [ "U. Anwar", "A. Saparov", "J. Rando", "D. Paleka", "M. Turpin", "P. Hase", "E. S. Lubana", "E. Jenner", "S. Casper", "O. Sourbut", "B. L. Edelman", "Z. Zhang", "M. Günther", "A. Korinek", "J. Hernandez-Orallo", "L. Hammond", "E. Bigelow", "A. Pan", "L. Langosco", "T. Korbak", "H. Zhang", "R. Zhong", "S. Ó hÉigeartaigh", "G. Recchia", "G. Corsi", "A. Chan", "M. Anderljung", "L. Edwards", "A. Petrov", "C. Schroeder de Witt", "S. R. Motwani", "Y. Bengio", "D. Chen", "P. H. S. Torr", "S. Albanie", "T. Maharaj", "J. Foerster", "F. Tramèr", "H. He", "A. Kasirzadeh", "Y. Choi", "D. Krueger" ], "year": 2024, "venue": "Transactions on Machine Learning Research", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/anwar-foundational-challenges-assuring-alignment-safety/", "archive_url": null, "accessed": "2026-09-23", "doi": null, "arxiv": "2404.09932", "version_note": "Published in TMLR (2024); arXiv preprint 2404.09932. Venue and author list checked 2026-09-24 against the ML Anthology record of the TMLR paper, which gives \"Sumeet Ramesh Motwani\" (the arXiv metadata reads \"Motwan\").", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Motivations and policy proposals\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://openreview.net/forum?id=oVTkOs8Pka", "source_file": "content/sources/anwar-foundational-challenges-assuring-alignment-safety.yaml", "flags_all": [], "cited_by": [], "referenced_by": [] }, { "id": "S-3125", "slug": "apple-developer-private-cloud-compute", "status": "published", "risk_flags": [], "title": "Private Cloud Compute (Apple Developer)", "authors": [ "Apple" ], "year": 2026, "venue": "Apple Developer", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/apple-developer-private-cloud-compute/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Undated page, read on 2026-09-25. The access was announced at WWDC26 in June 2026 (\"What's new in the Foundation Models framework\", https://developer.apple.com/videos/play/wwdc2026/241/, which names the PrivateCloudComputeLanguageModel API).", "note": "Apple's description of its own developer programme; supports only \"Apple reports\" statements. Developers enrolled in the App Store Small Business Program, with fewer than 2 million first-time downloads across their apps and the Private Cloud Compute entitlement assigned to their account, can use Apple Foundation Models on PCC in their apps at no cloud API cost. Developers who exceed the threshold must migrate within six months.", "organizations": [], "source_type": "docs", "source_url": "https://developer.apple.com/private-cloud-compute/", "source_file": "content/sources/apple-developer-private-cloud-compute.yaml", "flags_all": [], "cited_by": [ { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/" } ], "referenced_by": [ { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/" } ] }, { "id": "S-1802", "slug": "apple-expanding-pcc-2026", "status": "published", "risk_flags": [], "title": "Expanding Private Cloud Compute", "authors": [ "Apple Security Engineering and Architecture (SEAR)" ], "year": 2026, "date": "2026-06-08", "venue": "Apple Security Research blog", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/apple-expanding-pcc-2026/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Apple's announcement that PCC extends to Google Cloud systems with NVIDIA GPUs, using NVIDIA Confidential Computing, Intel CPUs with TDX and Google's Titan chip; for components that could be abused to exfiltrate user data if compromised, attestation \"rooted in at least two separate roots of trust from independent vendors\"; attested keys held in a separate confidential VM; binaries to be published and live nodes available in research mode through the Apple Security Bounty. States that PCC on Google Cloud \"will be gradually ramping towards the complete set of protections throughout the summer preview period\". Supports only \"Apple reports\" statements.", "organizations": [], "source_type": "blog", "source_url": "https://security.apple.com/blog/expanding-pcc/", "source_file": "content/sources/apple-expanding-pcc-2026.yaml", "flags_all": [], "cited_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/" } ], "referenced_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/" } ] }, { "id": "S-1801", "slug": "apple-pcc-security-research-2024", "status": "published", "risk_flags": [], "title": "Security research on Private Cloud Compute", "authors": [ "Apple Security Engineering and Architecture (SEAR)" ], "year": 2024, "date": "2024-10-24", "venue": "Apple Security Research blog", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/apple-pcc-security-research-2024/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Apple's announcement of its PCC research resources: the PCC Security Guide, the Virtual Research Environment (runs PCC node software in a VM on an Apple silicon Mac; lists and inspects releases, checks the transparency log, runs inference against demonstration models), source code for CloudAttestation, Thimble, splunkloggingd and srd_tools under a limited-use licence (github.com/apple/security-pcc), and PCC bounty categories of up to $1,000,000. Supports only \"Apple reports\" statements. The Security Guide itself (security.apple.com/documentation/private-cloud-compute) needs JavaScript and was not read.", "organizations": [], "source_type": "blog", "source_url": "https://security.apple.com/blog/pcc-security-research/", "source_file": "content/sources/apple-pcc-security-research-2024.yaml", "flags_all": [], "cited_by": [ { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/" } ], "referenced_by": [ { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/" } ] }, { "id": "S-1800", "slug": "apple-private-cloud-compute-2024", "status": "published", "risk_flags": [], "title": "Private Cloud Compute: A new frontier for AI privacy in the cloud", "authors": [ "Apple Security Engineering and Architecture (SEAR)" ], "year": 2024, "date": "2024-06-10", "venue": "Apple Security Research blog", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/apple-private-cloud-compute-2024/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Byline: SEAR with Apple's User Privacy, Core Operating Systems, Services Engineering, and Machine Learning and AI teams. Apple's description of its own system; supports only \"Apple reports\" statements. Covers the five core requirements, Apple silicon servers with the Secure Enclave and Secure Boot, the rule that devices wrap request keys only to nodes whose attested measurements match a release in the public transparency log, publication of production images within 90 days of log inclusion, integrity protection of code and model assets, and a threat model that includes attackers with physical access to a node.", "organizations": [], "source_type": "blog", "source_url": "https://security.apple.com/blog/private-cloud-compute/", "source_file": "content/sources/apple-private-cloud-compute-2024.yaml", "flags_all": [], "cited_by": [ { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/" } ], "referenced_by": [ { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/" } ] }, { "id": "S-1201", "slug": "apsey-confidential-computing-h100-gpus", "status": "published", "risk_flags": [], "title": "Confidential Computing on NVIDIA H100 GPUs for Secure and Trustworthy AI", "authors": [ "E. Apsey", "P. Rogers", "M. O'Connor", "R. Nertney" ], "year": 2023, "date": "2023-08-03", "venue": "NVIDIA Technical Blog", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/apsey-confidential-computing-h100-gpus/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "NVIDIA engineers describing the H100 confidential-computing launch: on-die root of trust, fused device identity key, SPDM attestation, and performance at launch. Supports only \"NVIDIA reports\" statements.", "organizations": [ "O-0140" ], "source_type": "blog", "source_url": "https://developer.nvidia.com/blog/confidential-computing-on-h100-gpus-for-secure-and-trustworthy-ai/", "source_file": "content/sources/apsey-confidential-computing-h100-gpus.yaml", "flags_all": [], "cited_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "O-0140", "title": "NVIDIA", "url": "https://trustbutveri.fyi/organizations/nvidia/" } ], "referenced_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "O-0140", "title": "NVIDIA", "url": "https://trustbutveri.fyi/organizations/nvidia/" } ] }, { "id": "S-1809", "slug": "arun-verde-refereed-delegation", "status": "published", "risk_flags": [], "title": "Verde: Verification via Refereed Delegation for Machine Learning Programs", "authors": [ "A. Arun", "A. St. Arnaud", "A. Titov", "B. Wilcox", "V. Kolobaric", "M. Brinkmann", "O. Ersoy", "B. Fielding", "J. Bonneau" ], "year": 2025, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/arun-verde-refereed-delegation/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2502.19405", "version_note": "v1 (26 Feb 2025) read in HTML. dblp lists only the arXiv version.", "note": "Most authors at Gensyn; Arun and Bonneau at New York University (Bonneau also a16z crypto research). Describes the Verde dispute protocol (correct result if at least one provider is honest) and RepOps; evaluation reports RepOps overheads on T4, RTX 3090 and A100 GPUs for DistilBERT and Llama models (Table 2: Llama-8B on A100 80 GB, 98% inference, 126% LoRA fine-tuning). RepOps supports FP32 and one GPU per setup.", "organizations": [ "O-0212" ], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2502.19405", "source_file": "content/sources/arun-verde-refereed-delegation.yaml", "flags_all": [], "cited_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" }, { "id": "I-0016", "title": "Batch-invariant inference kernels (Thinking Machines)", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/" }, { "id": "I-0015", "title": "Verde and RepOps (Gensyn)", "url": "https://trustbutveri.fyi/implementations/gensyn-verde-repops/" }, { "id": "O-0212", "title": "Gensyn", "url": "https://trustbutveri.fyi/organizations/gensyn/" } ], "referenced_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" }, { "id": "I-0016", "title": "Batch-invariant inference kernels (Thinking Machines)", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/" }, { "id": "I-0015", "title": "Verde and RepOps (Gensyn)", "url": "https://trustbutveri.fyi/implementations/gensyn-verde-repops/" }, { "id": "O-0212", "title": "Gensyn", "url": "https://trustbutveri.fyi/organizations/gensyn/" } ] }, { "id": "S-3602", "slug": "attestable-homepage", "status": "published", "risk_flags": [], "title": "Attestable homepage", "authors": [], "year": 2026, "venue": "Attestable", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/attestable-homepage/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Official organization page, read 2026-09-25; the year is the access year because the page is undated. Supports the organization's attributed self-description: development of zero-knowledge proofs for AI systems.", "organizations": [ "O-0120" ], "source_type": "docs", "source_url": "https://attestable.com/", "source_file": "content/sources/attestable-homepage.yaml", "flags_all": [], "cited_by": [ { "id": "O-0120", "title": "Attestable", "url": "https://trustbutveri.fyi/organizations/attestable/" } ], "referenced_by": [ { "id": "O-0120", "title": "Attestable", "url": "https://trustbutveri.fyi/organizations/attestable/" } ] }, { "id": "S-1103", "slug": "attestable-model-weight-security", "status": "published", "risk_flags": [], "title": "From Verifiability to Model-Weight Security", "authors": [ "Attestable" ], "year": 2026, "venue": "Attestable blog", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/attestable-model-weight-security/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Developer's own proposal for a \"verification firewall\" that checks sampled outputs with zero-knowledge proofs. No visible date; the year comes from the site's copyright notice.", "organizations": [ "O-0120" ], "source_type": "blog", "source_url": "https://attestable.com/blog/model-weights-security", "source_file": "content/sources/attestable-model-weight-security.yaml", "flags_all": [], "cited_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0005", "title": "Attestable zero-knowledge inference prover", "url": "https://trustbutveri.fyi/implementations/attestable-zk-inference/" }, { "id": "O-0120", "title": "Attestable", "url": "https://trustbutveri.fyi/organizations/attestable/" } ], "referenced_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0005", "title": "Attestable zero-knowledge inference prover", "url": "https://trustbutveri.fyi/implementations/attestable-zk-inference/" }, { "id": "O-0120", "title": "Attestable", "url": "https://trustbutveri.fyi/organizations/attestable/" } ] }, { "id": "S-1102", "slug": "attestable-pacing-ai-requires-proof", "status": "published", "risk_flags": [], "title": "Pacing AI Requires Proof", "authors": [ "Attestable" ], "year": 2026, "venue": "Attestable blog", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/attestable-pacing-ai-requires-proof/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Developer's own proposal to combine zero-knowledge inference proofs with proof-of-work accounting. No visible byline or date; the year comes from page metadata and should be checked.", "organizations": [ "O-0120" ], "source_type": "blog", "source_url": "https://attestable.com/blog/pacing-ai-requires-proof", "source_file": "content/sources/attestable-pacing-ai-requires-proof.yaml", "flags_all": [], "cited_by": [ { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0005", "title": "Attestable zero-knowledge inference prover", "url": "https://trustbutveri.fyi/implementations/attestable-zk-inference/" }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "K-0011", "title": "Proof of (useful) work", "url": "https://trustbutveri.fyi/concepts/proof-of-useful-work/" }, { "id": "O-0120", "title": "Attestable", "url": "https://trustbutveri.fyi/organizations/attestable/" } ], "referenced_by": [ { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0005", "title": "Attestable zero-knowledge inference prover", "url": "https://trustbutveri.fyi/implementations/attestable-zk-inference/" }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "K-0011", "title": "Proof of (useful) work", "url": "https://trustbutveri.fyi/concepts/proof-of-useful-work/" }, { "id": "O-0120", "title": "Attestable", "url": "https://trustbutveri.fyi/organizations/attestable/" } ] }, { "id": "S-1101", "slug": "attestable-proving-llms-at-scale", "status": "published", "risk_flags": [], "title": "Proving LLMs at Scale", "authors": [ "Attestable" ], "year": 2026, "date": "2026-08-17", "venue": "Attestable blog", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/attestable-proving-llms-at-scale/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Developer's own description of its zero-knowledge inference prover; supports only \"Attestable reports\" statements. No byline on the page; the date is taken from the Attestable resources listing. No paper, code or docs are linked.", "organizations": [ "O-0120" ], "source_type": "blog", "source_url": "https://attestable.com/blog/proving-llms-scale", "source_file": "content/sources/attestable-proving-llms-at-scale.yaml", "flags_all": [], "cited_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0005", "title": "Attestable zero-knowledge inference prover", "url": "https://trustbutveri.fyi/implementations/attestable-zk-inference/" }, { "id": "O-0120", "title": "Attestable", "url": "https://trustbutveri.fyi/organizations/attestable/" } ], "referenced_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0005", "title": "Attestable zero-knowledge inference prover", "url": "https://trustbutveri.fyi/implementations/attestable-zk-inference/" }, { "id": "O-0120", "title": "Attestable", "url": "https://trustbutveri.fyi/organizations/attestable/" } ] }, { "id": "S-1402", "slug": "avellar-near-term-verification-ai-chip-exports", "status": "published", "risk_flags": [], "title": "Near-Term Verification Methods for AI Chip Exports", "authors": [ "B. Avellar", "E. Grunewald" ], "year": 2026, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/avellar-near-term-verification-ai-chip-exports/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2609.07637", "version_note": "v1. Read via the arXiv HTML rendering (arxiv.org/html/2609.07637); the abstract page returned no extractable text to the fetch tool. Affiliations as shown: B. Avellar (independent researcher), E. Grunewald (Institute for AI Policy and Strategy).", "organizations": [ "O-0204" ], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2609.07637", "source_file": "content/sources/avellar-near-term-verification-ai-chip-exports.yaml", "flags_all": [], "cited_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "O-0204", "title": "Institute for AI Policy and Strategy", "url": "https://trustbutveri.fyi/organizations/institute-for-ai-policy-and-strategy/" }, { "id": "O-0140", "title": "NVIDIA", "url": "https://trustbutveri.fyi/organizations/nvidia/" } ], "referenced_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "O-0204", "title": "Institute for AI Policy and Strategy", "url": "https://trustbutveri.fyi/organizations/institute-for-ai-policy-and-strategy/" }, { "id": "O-0140", "title": "NVIDIA", "url": "https://trustbutveri.fyi/organizations/nvidia/" } ] }, { "id": "S-1010", "slug": "badash-hawkeye", "status": "published", "risk_flags": [], "title": "Hawkeye: Reproducing GPU-Level Non-Determinism", "authors": [ "E. Badash", "D. Boneh", "I. Komargodski", "M. Srivastava" ], "year": 2026, "venue": "Proceedings of Machine Learning and Systems 8 (MLSys 2026)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/badash-hawkeye/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2603.20421", "version_note": "Full text read from arXiv HTML v2 (15 May 2026); MLSys proceedings abstract page checked. The arXiv abstract page could not be rendered by the fetch tool, so the v1 date was not confirmed.", "note": "Code at github.com/badasherez/gpu-simulator (MIT; release tag \"MLSys Release\", 4 Apr 2026). Badash and Komargodski are at Pearl Research Labs, Boneh and Srivastava at Stanford University.", "organizations": [ "O-0121" ], "source_type": "peer-reviewed", "source_url": "https://proceedings.mlsys.org/paper_files/paper/2026/hash/e217c271a57c365a246b0ad39e668ba8-Abstract-Conference.html", "source_file": "content/sources/badash-hawkeye.yaml", "flags_all": [], "cited_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" }, { "id": "K-0008", "title": "Numerical nondeterminism", "url": "https://trustbutveri.fyi/concepts/numerical-nondeterminism/" }, { "id": "O-0121", "title": "Pearl Research Labs", "url": "https://trustbutveri.fyi/organizations/pearl-research/" } ], "referenced_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" }, { "id": "K-0008", "title": "Numerical nondeterminism", "url": "https://trustbutveri.fyi/concepts/numerical-nondeterminism/" }, { "id": "O-0121", "title": "Pearl Research Labs", "url": "https://trustbutveri.fyi/organizations/pearl-research/" } ] }, { "id": "S-2001", "slug": "baker-nuclear-arms-control-lessons-ai-treaties", "status": "published", "risk_flags": [], "title": "Nuclear Arms Control Verification and Lessons for AI Treaties", "authors": [ "M. Baker" ], "year": 2023, "date": "2023-04-08", "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/baker-nuclear-arms-control-lessons-ai-treaties/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2304.04123", "arxiv": "2304.04123", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2304.04123", "source_file": "content/sources/baker-nuclear-arms-control-lessons-ai-treaties.yaml", "flags_all": [], "cited_by": [], "referenced_by": [] }, { "id": "S-0002", "slug": "baker-verifying-international-agreements-ai", "status": "published", "risk_flags": [], "title": "Verifying International Agreements on AI: Six Layers of Verification for Rules on Large-Scale AI Development and Deployment", "authors": [ "M. Baker", "G. Kulp", "O. Marks", "M. Brundage", "L. Heim" ], "year": 2025, "venue": "RAND Corporation", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/baker-verifying-international-agreements-ai/", "archive_url": "https://web.archive.org/web/20260918125709/https://www.rand.org/pubs/working_papers/WRA4077-1.html", "accessed": "2026-09-25", "doi": "10.7249/WRA4077-1", "arxiv": "2507.15916", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Research agendas and surveys\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [ "O-0200" ], "source_type": "tech-report", "source_url": "https://www.rand.org/pubs/working_papers/WRA4077-1.html", "source_file": "content/sources/baker-verifying-international-agreements-ai.yaml", "flags_all": [], "cited_by": [ { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0001", "title": "Compute stock is at most a declared amount", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0017", "title": "Compartmentalization", "url": "https://trustbutveri.fyi/concepts/compartmentalization/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" }, { "id": "K-0025", "title": "Inference and training workloads", "url": "https://trustbutveri.fyi/concepts/inference-and-training-workloads/" }, { "id": "K-0014", "title": "Network tap", "url": "https://trustbutveri.fyi/concepts/network-tap/" }, { "id": "K-0003", "title": "Positive and negative claims", "url": "https://trustbutveri.fyi/concepts/positive-and-negative-claims/" }, { "id": "K-0001", "title": "Prover", "url": "https://trustbutveri.fyi/concepts/prover/" }, { "id": "K-0016", "title": "Undeclared compute", "url": "https://trustbutveri.fyi/concepts/undeclared-compute/" }, { "id": "K-0002", "title": "Verifier", "url": "https://trustbutveri.fyi/concepts/verifier/" }, { "id": "O-0200", "title": "RAND", "url": "https://trustbutveri.fyi/organizations/rand/" } ], "referenced_by": [ { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0001", "title": "Compute stock is at most a declared amount", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0017", "title": "Compartmentalization", "url": "https://trustbutveri.fyi/concepts/compartmentalization/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" }, { "id": "K-0025", "title": "Inference and training workloads", "url": "https://trustbutveri.fyi/concepts/inference-and-training-workloads/" }, { "id": "K-0014", "title": "Network tap", "url": "https://trustbutveri.fyi/concepts/network-tap/" }, { "id": "K-0003", "title": "Positive and negative claims", "url": "https://trustbutveri.fyi/concepts/positive-and-negative-claims/" }, { "id": "K-0001", "title": "Prover", "url": "https://trustbutveri.fyi/concepts/prover/" }, { "id": "K-0016", "title": "Undeclared compute", "url": "https://trustbutveri.fyi/concepts/undeclared-compute/" }, { "id": "K-0002", "title": "Verifier", "url": "https://trustbutveri.fyi/concepts/verifier/" }, { "id": "O-0200", "title": "RAND", "url": "https://trustbutveri.fyi/organizations/rand/" } ] }, { "id": "S-1608", "slug": "ball-proofs-of-useful-work", "status": "published", "risk_flags": [], "title": "Proofs of Useful Work", "authors": [ "M. Ball", "A. Rosen", "M. Sabin", "P. N. Vasudevan" ], "year": 2017, "venue": "IACR Cryptology ePrint Archive 2017/203", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/ball-proofs-of-useful-work/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "The ePrint page describes it as a major revision of an IACR publication in CRYPTO 2018.", "organizations": [], "source_type": "preprint", "source_url": "https://eprint.iacr.org/2017/203", "source_file": "content/sources/ball-proofs-of-useful-work.yaml", "flags_all": [], "cited_by": [ { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "K-0011", "title": "Proof of (useful) work", "url": "https://trustbutveri.fyi/concepts/proof-of-useful-work/" } ], "referenced_by": [ { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "K-0011", "title": "Proof of (useful) work", "url": "https://trustbutveri.fyi/concepts/proof-of-useful-work/" } ] }, { "id": "S-0071", "slug": "basu-usefulness-gap-pearl", "status": "published", "risk_flags": [], "title": "The Usefulness Gap in Proof-of-Useful-Work: An Empirical Study of Pearl's cuPOW Protocol", "authors": [ "A. Basu" ], "year": 2026, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/basu-usefulness-gap-pearl/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2606.04819", "version_note": "Read arXiv HTML v2 (5 June 2026), which describes itself as under review; v1 was read on 2026-09-23. The arXiv abstract page could not be rendered by the fetch tool.", "note": "Independent measurement study of Pearl's mainnet; the author (NIELIT New Delhi; IIIT Allahabad) is not affiliated with Pearl Research Labs.", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2606.04819", "source_file": "content/sources/basu-usefulness-gap-pearl.yaml", "flags_all": [], "cited_by": [ { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" }, { "id": "K-0011", "title": "Proof of (useful) work", "url": "https://trustbutveri.fyi/concepts/proof-of-useful-work/" } ], "referenced_by": [ { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" }, { "id": "K-0011", "title": "Proof of (useful) work", "url": "https://trustbutveri.fyi/concepts/proof-of-useful-work/" } ] }, { "id": "S-0054", "slug": "bengio-international-ai-safety-report-2025", "status": "published", "risk_flags": [], "title": "International AI Safety Report", "authors": [ "Y. Bengio", "S. Mindermann", "D. Privitera", "T. Besiroglu", "R. Bommasani", "S. Casper", "Y. Choi", "P. Fox", "B. Garfinkel", "D. Goldfarb", "H. Heidari", "A. Ho", "S. Kapoor", "L. Khalatbari", "S. Longpre", "S. Manning", "V. Mavroudis", "M. Mazeika", "J. Michael", "J. Newman", "K. Y. Ng", "C. T. Okolo", "D. Raji", "G. Sastry", "E. Seger", "T. Skeadas", "T. South", "E. Strubell", "F. Tramèr", "L. Velasco", "N. Wheeler", "D. Acemoglu", "O. Adekanmbi", "D. Dalrymple", "T. G. Dietterich", "E. W. Felten", "P. Fung", "P.-O. Gourinchas", "F. Heintz", "G. Hinton", "N. Jennings", "A. Krause", "S. Leavy", "P. Liang", "T. Ludermir", "V. Marda", "H. Margetts", "J. McDermid", "J. Munga", "A. Narayanan", "A. Nelson", "C. Neppel", "A. Oh", "G. Ramchurn", "S. Russell", "M. Schaake", "B. Schölkopf", "D. Song", "A. Soto", "L. Tiedrich", "G. Varoquaux", "A. Yao", "Y.-Q. Zhang", "F. Albalawi", "M. Alserkal", "O. Ajala", "G. Avrin", "C. Busch", "A. C. P. de Leon Ferreira de Carvalho", "B. Fox", "A. S. Gill", "A. H. Hatip", "J. Heikkilä", "G. Jolly", "Z. Katzir", "H. Kitano", "A. Krüger", "C. Johnson", "S. M. Khan", "K. M. Lee", "D. V. Ligot", "O. Molchanovskyi", "A. Monti", "N. Mwamanzi", "M. Nemer", "N. Oliver", "J. R. López Portillo", "B. Ravindran", "R. Pezoa Rivera", "H. Riza", "C. Rugege", "C. Seoighe", "J. Sheehan", "H. Sheikh", "D. Wong", "Y. Zeng" ], "year": 2025, "venue": "International AI Safety Report", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/bengio-international-ai-safety-report-2025/", "archive_url": null, "accessed": "2026-09-23", "doi": "10.48550/arXiv.2501.17805", "arxiv": "2501.17805", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Motivations and policy proposals\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "tech-report", "source_url": "https://internationalaisafetyreport.org/publication/international-ai-safety-report-2025", "source_file": "content/sources/bengio-international-ai-safety-report-2025.yaml", "flags_all": [], "cited_by": [], "referenced_by": [] }, { "id": "S-1603", "slug": "birkholz-rats-architecture-rfc-9334", "status": "published", "risk_flags": [], "title": "Remote ATtestation procedureS (RATS) Architecture (RFC 9334)", "authors": [ "H. Birkholz", "D. Thaler", "M. Richardson", "N. Smith", "W. Pan" ], "year": 2023, "venue": "Internet Engineering Task Force (RATS Working Group)", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/birkholz-rats-architecture-rfc-9334/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.17487/RFC9334", "arxiv": null, "version_note": "IETF Informational RFC, January 2023. Not a Standards Track specification; the header states it represents the consensus of the IETF community. Recorded as a technical report (tier B) rather than a standard for that reason.", "organizations": [], "source_type": "tech-report", "source_url": "https://www.rfc-editor.org/rfc/rfc9334.html", "source_file": "content/sources/birkholz-rats-architecture-rfc-9334.yaml", "flags_all": [], "cited_by": [ { "id": "K-0019", "title": "Evidence binding", "url": "https://trustbutveri.fyi/concepts/evidence-binding/" }, { "id": "K-0001", "title": "Prover", "url": "https://trustbutveri.fyi/concepts/prover/" }, { "id": "K-0004", "title": "Remote attestation", "url": "https://trustbutveri.fyi/concepts/remote-attestation/" }, { "id": "K-0002", "title": "Verifier", "url": "https://trustbutveri.fyi/concepts/verifier/" } ], "referenced_by": [ { "id": "K-0019", "title": "Evidence binding", "url": "https://trustbutveri.fyi/concepts/evidence-binding/" }, { "id": "K-0001", "title": "Prover", "url": "https://trustbutveri.fyi/concepts/prover/" }, { "id": "K-0004", "title": "Remote attestation", "url": "https://trustbutveri.fyi/concepts/remote-attestation/" }, { "id": "K-0002", "title": "Verifier", "url": "https://trustbutveri.fyi/concepts/verifier/" } ] }, { "id": "S-1400", "slug": "brass-location-verification-ai-chips", "status": "published", "risk_flags": [], "title": "Location Verification for AI Chips", "authors": [ "A. Brass", "O. Aarne" ], "year": 2024, "venue": "Institute for AI Policy and Strategy", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/brass-location-verification-ai-chips/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "The IAPS page links the full report PDF (static1.squarespace.com/static/64edf8e7f2b10d716b5ba0e1/t/6670467ebe2a477eb1554f40/1718634112482/Location%2BVerification%2Bfor%2BAI%2BChips.pdf), which was read on 2026-09-23. The PDF text as fetched shows no byline or date. The current IAPS landing page presents the May 2025 issue brief by Asher Brass. Authors are taken from the IAPS issue brief (S-1401), which names Asher Brass and Onni Aarne and refers to a \"full 2024 report\", and from citations in S-1402 and S-0007. S-1402 cites it as April 2024; the month is not confirmed on the IAPS page itself.", "organizations": [ "O-0204" ], "source_type": "tech-report", "source_url": "https://www.iaps.ai/research/location-verification-for-ai-chips", "source_file": "content/sources/brass-location-verification-ai-chips.yaml", "flags_all": [], "cited_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "I-0009", "title": "Lucid sovereignty (location) certificates", "url": "https://trustbutveri.fyi/implementations/lucid-location-certificates/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "O-0204", "title": "Institute for AI Policy and Strategy", "url": "https://trustbutveri.fyi/organizations/institute-for-ai-policy-and-strategy/" } ], "referenced_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "I-0009", "title": "Lucid sovereignty (location) certificates", "url": "https://trustbutveri.fyi/implementations/lucid-location-certificates/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "O-0204", "title": "Institute for AI Policy and Strategy", "url": "https://trustbutveri.fyi/organizations/institute-for-ai-policy-and-strategy/" } ] }, { "id": "S-1401", "slug": "brass-location-verification-issue-brief", "status": "published", "risk_flags": [], "title": "Location Verification for AI Chips (issue brief)", "authors": [ "A. Brass" ], "year": 2025, "date": "2025-05-16", "venue": "Institute for AI Policy and Strategy", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/brass-location-verification-issue-brief/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Two-page IAPS issue brief. The document says \"Issue brief written May 2025, full 2024 report by Asher Brass & Onni Aarne\"; the IAPS page that presents the brief (iaps.ai/research/location-verification-for-ai-chips) reads \"Written By Asher Brass\" and is dated May 16 (2025). It reports an H100 prototype with a video demonstration at ping-location.info. The site's technical details are separately recorded in S-3570.", "organizations": [ "O-0204" ], "source_type": "tech-report", "source_url": "https://static1.squarespace.com/static/64edf8e7f2b10d716b5ba0e1/t/6827b67275666f3757f134ea/1747433075281/Location+Verification+two-pager.pdf", "source_file": "content/sources/brass-location-verification-issue-brief.yaml", "flags_all": [], "cited_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "O-0204", "title": "Institute for AI Policy and Strategy", "url": "https://trustbutveri.fyi/organizations/institute-for-ai-policy-and-strategy/" } ], "referenced_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "O-0204", "title": "Institute for AI Policy and Strategy", "url": "https://trustbutveri.fyi/organizations/institute-for-ai-policy-and-strategy/" } ] }, { "id": "S-0003", "slug": "brundage-frontier-ai-auditing", "status": "published", "risk_flags": [], "title": "Frontier AI Auditing: Toward Rigorous Third-Party Assessment of Safety and Security Practices at Leading AI Companies", "authors": [ "M. Brundage", "N. Dreksler", "A. Homewood", "S. McGregor", "P. Paskov", "C. Stosz", "G. Sastry", "A. F. Cooper", "G. Balston", "S. Adler", "S. Casper", "M. Anderljung", "G. Werner", "S. Mindermann", "V. Mavroudis", "B. Bucknall", "C. Stix", "J. Freund", "L. Pacchiardi", "J. Hernandez-Orallo", "M. Pistillo", "M. Chen", "C. Painter", "D. W. Ball", "C. O'Keefe", "G. Weil", "B. Harack", "G. Finley", "R. Hassan", "S. Emmons", "C. Foster", "A. Reuel", "B. Treece", "Y. Bengio", "D. Reti", "R. Bommasani", "C. Trout", "A. S. Shamsabadi", "R. Dattani", "A. Weller", "R. Trager", "J. Sevilla", "L. Wagner", "L. Soder", "K. Ramakrishnan", "H. Papadatos", "M. Murray", "R. Tovcimak" ], "year": 2026, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/brundage-frontier-ai-auditing/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2601.11699", "arxiv": "2601.11699", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Research agendas and surveys\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2601.11699", "source_file": "content/sources/brundage-frontier-ai-auditing.yaml", "flags_all": [], "cited_by": [ { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" } ], "referenced_by": [ { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" } ] }, { "id": "S-0032", "slug": "bursuc-software-based-memory-erasure", "status": "published", "risk_flags": [], "title": "Software-Based Memory Erasure with Relaxed Isolation Requirements", "authors": [ "S. Bursuc", "R. Gil-Pons", "S. Mauw", "R. Trujillo-Rasua" ], "year": 2024, "venue": "2024 IEEE 37th Computer Security Foundations Symposium (CSF 2024)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/bursuc-software-based-memory-erasure/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.1109/CSF61375.2024.00022", "arxiv": "2401.06626", "version_note": "The record previously pointed to the arXiv extended version, \"Software-Based Memory Erasure with relaxed isolation requirements: Extended Version\" (https://arxiv.org/abs/2401.06626); section locators in citing records refer to that version. Venue and DOI from the Crossref record for the CSF 2024 paper, checked 2026-09-24.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Network and memory telemetry\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://ieeexplore.ieee.org/document/10664348/", "source_file": "content/sources/bursuc-software-based-memory-erasure.yaml", "flags_all": [], "cited_by": [ { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "K-0012", "title": "Proof of space", "url": "https://trustbutveri.fyi/concepts/proof-of-space/" } ], "referenced_by": [ { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "K-0012", "title": "Proof of space", "url": "https://trustbutveri.fyi/concepts/proof-of-space/" } ] }, { "id": "S-3541", "slug": "cai-auditing-model-substitution-llm-apis", "status": "published", "risk_flags": [], "title": "Are You Getting What You Pay For? Auditing Model Substitution in LLM APIs", "authors": [ "W. Cai", "T. Shi", "X. Zhao", "D. Song" ], "year": 2025, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/cai-auditing-model-substitution-llm-apis/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2504.04715", "arxiv": "2504.04715", "version_note": "arXiv v2 (29 September 2025); v1 was posted on 7 April 2025. An OpenReview forum exists (https://openreview.net/forum?id=3DZeEUTwhq) but could not be read by the fetch tool, so no venue or decision is recorded.", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2504.04715", "source_file": "content/sources/cai-auditing-model-substitution-llm-apis.yaml", "flags_all": [], "cited_by": [ { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" } ], "referenced_by": [ { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" } ] }, { "id": "S-3565", "slug": "cai-shortcut-connected-expert-parallelism", "status": "published", "risk_flags": [], "title": "Shortcut-connected Expert Parallelism for Accelerating Mixture of Experts", "authors": [ "W. Cai", "J. Jiang", "L. Qin", "J. Cui", "S. Kim", "J. Huang" ], "year": 2025, "venue": "ICML 2025, Proceedings of Machine Learning Research 267", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/cai-shortcut-connected-expert-parallelism/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "The paper reports all-to-all expert-parallel communication as a bottleneck in both MoE training and inference.", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://proceedings.mlr.press/v267/cai25c.html", "source_file": "content/sources/cai-shortcut-connected-expert-parallelism.yaml", "flags_all": [], "cited_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "K-0017", "title": "Compartmentalization", "url": "https://trustbutveri.fyi/concepts/compartmentalization/" }, { "id": "K-0025", "title": "Inference and training workloads", "url": "https://trustbutveri.fyi/concepts/inference-and-training-workloads/" }, { "id": "K-0021", "title": "Interconnect bandwidth", "url": "https://trustbutveri.fyi/concepts/interconnect-bandwidth/" } ], "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "K-0017", "title": "Compartmentalization", "url": "https://trustbutveri.fyi/concepts/compartmentalization/" }, { "id": "K-0025", "title": "Inference and training workloads", "url": "https://trustbutveri.fyi/concepts/inference-and-training-workloads/" }, { "id": "K-0021", "title": "Interconnect bandwidth", "url": "https://trustbutveri.fyi/concepts/interconnect-bandwidth/" } ] }, { "id": "S-3544", "slug": "california-sb-53-frontier-ai-transparency", "status": "published", "risk_flags": [], "title": "California Senate Bill 53 (2025): Transparency in Frontier Artificial Intelligence Act", "authors": [ "California State Legislature" ], "year": 2025, "date": "2025-09-29", "venue": "Statutes of 2025, Chapter 138 (Business and Professions Code §22757.10 et seq.)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/california-sb-53-frontier-ai-transparency/", "archive_url": "https://web.archive.org/web/20260919100303/https://legiscan.com/CA/text/SB53/id/3271094", "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Chaptered text, approved by the Governor and filed with the Secretary of State on 29 September 2025. The official page (https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=202520260SB53) disallows the fetch tool, so the chaptered text was read on LegiScan. §22757.11(i) defines a frontier model by training compute greater than 10^26 integer or floating-point operations, including later fine-tuning, reinforcement learning or other material modifications; §22757.12 sets the frontier AI framework and transparency-report duties.", "organizations": [], "source_type": "gov-doc", "source_url": "https://legiscan.com/CA/text/SB53/id/3271094", "source_file": "content/sources/california-sb-53-frontier-ai-transparency.yaml", "flags_all": [], "cited_by": [ { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" } ], "referenced_by": [ { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" } ] }, { "id": "S-0020", "slug": "cankaya-bit-exact-inference-verification", "status": "published", "risk_flags": [], "title": "Bit-Exact AI Inference Verification Without Performance Tradeoffs", "authors": [ "N. Cankaya" ], "year": 2026, "venue": "ICML 2026 Workshop on Technical AI Governance Research", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/cankaya-bit-exact-inference-verification/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2606.00279", "arxiv": "2606.00279", "version_note": "arXiv v1 29 May 2026; v2 5 June 2026. The arXiv comments field reads \"Best paper award, ICML 2026 TAIGR workshop\"; the paper is listed in the workshop's poster session (https://icml.cc/virtual/2026/workshop/54084). Checked 2026-09-24.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Inference verification\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2606.00279", "source_file": "content/sources/cankaya-bit-exact-inference-verification.yaml", "flags_all": [], "cited_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" }, { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0008", "title": "Numerical nondeterminism", "url": "https://trustbutveri.fyi/concepts/numerical-nondeterminism/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" } ], "referenced_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" }, { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0008", "title": "Numerical nondeterminism", "url": "https://trustbutveri.fyi/concepts/numerical-nondeterminism/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" } ] }, { "id": "S-1300", "slug": "cankaya-fingerprinting-ai-cluster-io", "status": "published", "risk_flags": [], "title": "Fingerprinting All AI Cluster I/O Without Mutually Trusted Processors", "authors": [ "N. Cankaya", "J. Kryś", "J. Ng", "L. Marks", "F. Krückel" ], "year": 2026, "date": "2026-06-09", "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/cankaya-fingerprinting-ai-cluster-io/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2606.10724", "version_note": "v1, submitted 9 June 2026; read via the arXiv HTML rendering. The Oxford Martin AI Governance Initiative published an earlier version dated April 2026 (https://aigi.ox.ac.uk/wp-content/uploads/2026/04/Fingerprinting_All_AI_Cluster_IO.pdf; publication page dated 28 April 2026) before arXiv v1. Both versions give the first author's affiliations as MATS and the Oxford Hardware AI Governance Lab. arXiv still lists only v1, with no journal reference, on 2026-09-25.", "organizations": [ "O-0209" ], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2606.10724", "source_file": "content/sources/cankaya-fingerprinting-ai-cluster-io.yaml", "flags_all": [], "cited_by": [ { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "O-0209", "title": "Hardware AI Governance Lab", "url": "https://trustbutveri.fyi/organizations/oxford-hardware-ai-governance-lab/" } ], "referenced_by": [ { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "O-0209", "title": "Hardware AI Governance Lab", "url": "https://trustbutveri.fyi/organizations/oxford-hardware-ai-governance-lab/" } ] }, { "id": "S-0031", "slug": "cankaya-secure-network-taps", "status": "published", "risk_flags": [], "title": "The Fundamentals and Feasibility of Secure Network Taps for Verifying AI Datacenter Use", "authors": [ "N. Cankaya" ], "year": 2026, "date": "2026-04-30", "venue": "The Datacenter Lie Detector", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/cankaya-secure-network-taps/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Network and memory telemetry\" in the Hodgkins bibliography (CC BY 4.0). The post's byline names N. Cankaya only; A. Friedman and M. Baker are thanked for \"substantial contributions\" (checked 2026-09-23).", "organizations": [], "source_type": "blog", "source_url": "https://nacicankaya.substack.com/p/research-note-the-fundamentals-and", "source_file": "content/sources/cankaya-secure-network-taps.yaml", "flags_all": [], "cited_by": [ { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0019", "title": "Evidence binding", "url": "https://trustbutveri.fyi/concepts/evidence-binding/" }, { "id": "K-0014", "title": "Network tap", "url": "https://trustbutveri.fyi/concepts/network-tap/" } ], "referenced_by": [ { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0019", "title": "Evidence binding", "url": "https://trustbutveri.fyi/concepts/evidence-binding/" }, { "id": "K-0014", "title": "Network tap", "url": "https://trustbutveri.fyi/concepts/network-tap/" } ] }, { "id": "S-0038", "slug": "cankaya-suppressing-side-channels", "status": "published", "risk_flags": [], "title": "Suppressing Side Channels in an Untrusted Data Center via Retrofitted Defenses", "authors": [ "N. Cankaya" ], "year": 2026, "date": "2026-05-29", "venue": "MIRI Technical Governance Team", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/cankaya-suppressing-side-channels/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Power telemetry and side-channel attacks\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [ "O-0202" ], "source_type": "blog", "source_url": "https://techgov.intelligence.org/blog/suppressing-side-channels-in-an-untrusted-data-center-via-retrofitted-defenses", "source_file": "content/sources/cankaya-suppressing-side-channels.yaml", "flags_all": [], "cited_by": [ { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0017", "title": "Compartmentalization", "url": "https://trustbutveri.fyi/concepts/compartmentalization/" }, { "id": "K-0013", "title": "Side channel", "url": "https://trustbutveri.fyi/concepts/side-channel/" }, { "id": "K-0022", "title": "Weight exfiltration", "url": "https://trustbutveri.fyi/concepts/weight-exfiltration/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" } ], "referenced_by": [ { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0017", "title": "Compartmentalization", "url": "https://trustbutveri.fyi/concepts/compartmentalization/" }, { "id": "K-0013", "title": "Side channel", "url": "https://trustbutveri.fyi/concepts/side-channel/" }, { "id": "K-0022", "title": "Weight exfiltration", "url": "https://trustbutveri.fyi/concepts/weight-exfiltration/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" } ] }, { "id": "S-0018", "slug": "cankaya-system-overview-low-trust-compute-verification", "status": "published", "risk_flags": [], "title": "A System Overview for Near-Term, Low-Trust AI Compute Verification", "authors": [ "N. Cankaya" ], "year": 2026, "venue": "Machine Intelligence Research Institute", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/cankaya-system-overview-low-trust-compute-verification/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Inference verification\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [ "O-0202" ], "source_type": "tech-report", "source_url": "https://intelligence.org/wp-content/uploads/2026/06/A-system-overview-for-near-term-low-trust-AI-compute-verification.pdf", "source_file": "content/sources/cankaya-system-overview-low-trust-compute-verification.yaml", "flags_all": [], "cited_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "I-0003", "title": "zkLLM", "url": "https://trustbutveri.fyi/implementations/zkllm/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0017", "title": "Compartmentalization", "url": "https://trustbutveri.fyi/concepts/compartmentalization/" }, { "id": "K-0024", "title": "Cryptographic commitment", "url": "https://trustbutveri.fyi/concepts/cryptographic-commitment/" }, { "id": "K-0019", "title": "Evidence binding", "url": "https://trustbutveri.fyi/concepts/evidence-binding/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" }, { "id": "K-0025", "title": "Inference and training workloads", "url": "https://trustbutveri.fyi/concepts/inference-and-training-workloads/" }, { "id": "K-0021", "title": "Interconnect bandwidth", "url": "https://trustbutveri.fyi/concepts/interconnect-bandwidth/" }, { "id": "K-0014", "title": "Network tap", "url": "https://trustbutveri.fyi/concepts/network-tap/" }, { "id": "K-0008", "title": "Numerical nondeterminism", "url": "https://trustbutveri.fyi/concepts/numerical-nondeterminism/" }, { "id": "K-0003", "title": "Positive and negative claims", "url": "https://trustbutveri.fyi/concepts/positive-and-negative-claims/" }, { "id": "K-0012", "title": "Proof of space", "url": "https://trustbutveri.fyi/concepts/proof-of-space/" }, { "id": "K-0009", "title": "Recomputation", "url": "https://trustbutveri.fyi/concepts/recomputation/" }, { "id": "K-0020", "title": "Sampling and assurance", "url": "https://trustbutveri.fyi/concepts/sampling-and-assurance/" }, { "id": "K-0013", "title": "Side channel", "url": "https://trustbutveri.fyi/concepts/side-channel/" }, { "id": "K-0018", "title": "Threat model", "url": "https://trustbutveri.fyi/concepts/threat-model/" }, { "id": "K-0016", "title": "Undeclared compute", "url": "https://trustbutveri.fyi/concepts/undeclared-compute/" }, { "id": "K-0002", "title": "Verifier", "url": "https://trustbutveri.fyi/concepts/verifier/" }, { "id": "K-0022", "title": "Weight exfiltration", "url": "https://trustbutveri.fyi/concepts/weight-exfiltration/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" } ], "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0012", "title": "Low-trust AI compute verification system overview", "url": "https://trustbutveri.fyi/implementations/low-trust-compute-verification-system-overview/" }, { "id": "I-0003", "title": "zkLLM", "url": "https://trustbutveri.fyi/implementations/zkllm/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0017", "title": "Compartmentalization", "url": "https://trustbutveri.fyi/concepts/compartmentalization/" }, { "id": "K-0024", "title": "Cryptographic commitment", "url": "https://trustbutveri.fyi/concepts/cryptographic-commitment/" }, { "id": "K-0019", "title": "Evidence binding", "url": "https://trustbutveri.fyi/concepts/evidence-binding/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" }, { "id": "K-0025", "title": "Inference and training workloads", "url": "https://trustbutveri.fyi/concepts/inference-and-training-workloads/" }, { "id": "K-0021", "title": "Interconnect bandwidth", "url": "https://trustbutveri.fyi/concepts/interconnect-bandwidth/" }, { "id": "K-0014", "title": "Network tap", "url": "https://trustbutveri.fyi/concepts/network-tap/" }, { "id": "K-0008", "title": "Numerical nondeterminism", "url": "https://trustbutveri.fyi/concepts/numerical-nondeterminism/" }, { "id": "K-0003", "title": "Positive and negative claims", "url": "https://trustbutveri.fyi/concepts/positive-and-negative-claims/" }, { "id": "K-0012", "title": "Proof of space", "url": "https://trustbutveri.fyi/concepts/proof-of-space/" }, { "id": "K-0009", "title": "Recomputation", "url": "https://trustbutveri.fyi/concepts/recomputation/" }, { "id": "K-0020", "title": "Sampling and assurance", "url": "https://trustbutveri.fyi/concepts/sampling-and-assurance/" }, { "id": "K-0013", "title": "Side channel", "url": "https://trustbutveri.fyi/concepts/side-channel/" }, { "id": "K-0018", "title": "Threat model", "url": "https://trustbutveri.fyi/concepts/threat-model/" }, { "id": "K-0016", "title": "Undeclared compute", "url": "https://trustbutveri.fyi/concepts/undeclared-compute/" }, { "id": "K-0002", "title": "Verifier", "url": "https://trustbutveri.fyi/concepts/verifier/" }, { "id": "K-0022", "title": "Weight exfiltration", "url": "https://trustbutveri.fyi/concepts/weight-exfiltration/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" } ] }, { "id": "S-1408", "slug": "cankaya-tsmc-golden-record", "status": "published", "risk_flags": [], "title": "TSMC most definitely has a golden record of all AI chips it made", "authors": [ "N. Cankaya" ], "year": 2025, "date": "2025-12-15", "venue": "The Datacenter Lie Detector (Substack)", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/cankaya-tsmc-golden-record/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "organizations": [], "source_type": "blog", "source_url": "https://nacicankaya.substack.com/p/tsmc-most-definitely-has-a-golden", "source_file": "content/sources/cankaya-tsmc-golden-record.yaml", "flags_all": [], "cited_by": [ { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" } ], "referenced_by": [ { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" } ] }, { "id": "S-0064", "slug": "carlsmith-restraining-ai-development-safety", "status": "published", "risk_flags": [], "title": "On restraining AI development for the sake of safety", "authors": [ "J. Carlsmith" ], "year": 2026, "venue": "Joseph Carlsmith", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/carlsmith-restraining-ai-development-safety/", "archive_url": null, "accessed": "2026-09-23", "doi": null, "arxiv": null, "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Motivations and policy proposals\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "blog", "source_url": "https://joecarlsmith.com/2026/03/19/on-restraining-ai-development-for-the-sake-of-safety/", "source_file": "content/sources/carlsmith-restraining-ai-development-safety.yaml", "flags_all": [], "cited_by": [], "referenced_by": [] }, { "id": "S-1308", "slug": "castelluccia-difficulty-software-based-attestation", "status": "published", "risk_flags": [], "title": "On the Difficulty of Software-Based Attestation of Embedded Devices", "authors": [ "C. Castelluccia", "A. Francillon", "D. Perito", "C. Soriente" ], "year": 2009, "venue": "Proceedings of the 16th ACM Conference on Computer and Communications Security (CCS 2009), pp. 400–409", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/castelluccia-difficulty-software-based-attestation/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.1145/1653662.1653711", "arxiv": null, "version_note": "Read the authors' PDF hosted by EURECOM; the ACM page could not be fetched, and the DOI, title and pages were confirmed in Crossref on 2026-09-25.", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://s3.eurecom.fr/docs/ccs09_Castelluccia.pdf", "source_file": "content/sources/castelluccia-difficulty-software-based-attestation.yaml", "flags_all": [], "cited_by": [ { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" } ], "referenced_by": [ { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" } ] }, { "id": "S-1604", "slug": "ccc-common-terminology-confidential-computing", "status": "published", "risk_flags": [], "title": "Common Terminology for Confidential Computing", "authors": [ "Confidential Computing Consortium" ], "year": 2022, "venue": "Confidential Computing Consortium", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/ccc-common-terminology-confidential-computing/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "The document gives its date as December 2022.", "organizations": [], "source_type": "tech-report", "source_url": "https://confidentialcomputing.io/wp-content/uploads/sites/10/2023/03/Common-Terminology-for-Confidential-Computing.pdf", "source_file": "content/sources/ccc-common-terminology-confidential-computing.yaml", "flags_all": [], "cited_by": [ { "id": "K-0004", "title": "Remote attestation", "url": "https://trustbutveri.fyi/concepts/remote-attestation/" }, { "id": "K-0006", "title": "Trusted execution environment (TEE)", "url": "https://trustbutveri.fyi/concepts/trusted-execution-environment/" } ], "referenced_by": [ { "id": "K-0004", "title": "Remote attestation", "url": "https://trustbutveri.fyi/concepts/remote-attestation/" }, { "id": "K-0006", "title": "Trusted execution environment (TEE)", "url": "https://trustbutveri.fyi/concepts/trusted-execution-environment/" } ] }, { "id": "S-0065", "slug": "chan-components-frontier-ai-slowdown", "status": "published", "risk_flags": [], "title": "Components of a Frontier AI Slowdown", "authors": [ "A. Chan" ], "year": 2026, "venue": "A Strange Attractor", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/chan-components-frontier-ai-slowdown/", "archive_url": null, "accessed": "2026-09-23", "doi": null, "arxiv": null, "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Motivations and policy proposals\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "blog", "source_url": "https://astrangeattractor.substack.com/p/components-of-a-frontier-ai-slowdown", "source_file": "content/sources/chan-components-frontier-ai-slowdown.yaml", "flags_all": [], "cited_by": [], "referenced_by": [] }, { "id": "S-0012", "slug": "chantasantitam-palm", "status": "published", "risk_flags": [], "title": "PAL*M: Property Attestation for Large Generative Models", "authors": [ "P. Chantasantitam", "A. I. Caulfield", "V. Duddu", "L. J. Gunn", "N. Asokan" ], "year": 2026, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/chantasantitam-palm/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2601.16199", "arxiv": "2601.16199", "version_note": "arXiv v3 (2026-04-30) read on 2026-09-25; still a preprint, with code \"to be released after peer review\".", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Trusted execution and attestation\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2601.16199", "source_file": "content/sources/chantasantitam-palm.yaml", "flags_all": [], "cited_by": [ { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" }, { "id": "K-0019", "title": "Evidence binding", "url": "https://trustbutveri.fyi/concepts/evidence-binding/" }, { "id": "K-0006", "title": "Trusted execution environment (TEE)", "url": "https://trustbutveri.fyi/concepts/trusted-execution-environment/" } ], "referenced_by": [ { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" }, { "id": "K-0019", "title": "Evidence binding", "url": "https://trustbutveri.fyi/concepts/evidence-binding/" }, { "id": "K-0006", "title": "Trusted execution environment (TEE)", "url": "https://trustbutveri.fyi/concepts/trusted-execution-environment/" } ] }, { "id": "S-0039", "slug": "chen-detecting-anomalies-hardware-telemetry", "status": "published", "risk_flags": [], "title": "Detecting Anomalies in Machine Learning Infrastructure via Hardware Telemetry", "authors": [ "Z. Chen", "S. Chien", "P. Qian", "N. Zilberman" ], "year": 2025, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/chen-detecting-anomalies-hardware-telemetry/", "archive_url": null, "accessed": "2026-09-23", "doi": "10.48550/arXiv.2510.26008", "arxiv": "2510.26008", "version_note": "Title and author order follow the arXiv listing (abstract-page title, also used on the Oxford Computing Infrastructure Group publications page). The paper's own header in the arXiv HTML (v1 and v2) and PDF reads \"Detecting Anomalies in Systems for AI Using Hardware Telemetry\", with authors in the order Z. Chen, P. Qian, S. W. D. Chien, N. Zilberman, all University of Oxford; the system is named Reveal. Checked 2026-09-23.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Power telemetry and side-channel attacks\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2510.26008", "source_file": "content/sources/chen-detecting-anomalies-hardware-telemetry.yaml", "flags_all": [], "cited_by": [ { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" } ], "referenced_by": [ { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" } ] }, { "id": "S-0021", "slug": "chen-zkml", "status": "published", "risk_flags": [], "title": "ZKML: An Optimizing System for ML Inference in Zero-Knowledge Proofs", "authors": [ "B.-J. Chen", "S. Waiwitlikhit", "I. Stoica", "D. Kang" ], "year": 2024, "venue": "19th European Conference on Computer Systems (EuroSys 2024)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/chen-zkml/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.1145/3627703.3650088", "arxiv": null, "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Zero-knowledge proofs\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://doi.org/10.1145/3627703.3650088", "source_file": "content/sources/chen-zkml.yaml", "flags_all": [], "cited_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" } ], "referenced_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" } ] }, { "id": "S-0030", "slug": "choi-tools-verifying-training-data", "status": "published", "risk_flags": [], "title": "Tools for Verifying Neural Models' Training Data", "authors": [ "D. Choi", "Y. Shavit", "D. K. Duvenaud" ], "year": 2023, "venue": "Advances in Neural Information Processing Systems 36 (NeurIPS 2023)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/choi-tools-verifying-training-data/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2307.00682", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Proof of learning and training\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://proceedings.neurips.cc/paper_files/paper/2023/hash/03e33e1f62e3302b47fe1d38a235921e-Abstract-Conference.html", "source_file": "content/sources/choi-tools-verifying-training-data.yaml", "flags_all": [], "cited_by": [ { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" } ], "referenced_by": [ { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" } ] }, { "id": "S-1202", "slug": "chuang-tee-fail", "status": "published", "risk_flags": [], "title": "TEE.fail: Breaking Trusted Execution Environments via DDR5 Memory Bus Interposition", "authors": [ "J. Chuang", "A. Seto", "N. Berrios", "S. van Schaik", "C. Garman", "D. Genkin" ], "year": 2026, "venue": "2026 IEEE Symposium on Security and Privacy (SP)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/chuang-tee-fail/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Read the authors' site and the paper PDF linked from it (https://tee.fail/files/paper.pdf) on 2026-09-23.", "note": "Independent attack paper. Venue confirmed on 2026-09-23: the authors' site lists the 47th IEEE Symposium on Security and Privacy (IEEE S&P '26), co-author C. Garman's publication list gives IEEE Symposium on Security and Privacy (SP) 2026, and the IEEE Computer Society Digital Library lists it among the SP 2026 proceedings articles (https://www.computer.org/csdl/proceedings-article/sp/2026/606500b876/2bojwtctN28; page content not machine-readable, so no DOI is recorded). The site (re-read 2026-09-25) links an Intel security announcement of 2025-10-28 and AMD bulletin AMD-SB-3040; it states that both vendors treat interposer attacks as out of scope and that there are no mitigations on the NVIDIA side. Cited by S-0012 (as TEE.fail) and discussed in S-0014 and the Tinfoil documentation.", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://tee.fail/", "source_file": "content/sources/chuang-tee-fail.yaml", "flags_all": [], "cited_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" } ], "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" } ] }, { "id": "S-3612", "slug": "cnas-mission", "status": "published", "risk_flags": [], "title": "Center for a New American Security: Mission", "authors": [], "year": 2026, "venue": "Center for a New American Security", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/cnas-mission/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Official organization page, read 2026-09-25; the year is the access year because the page is undated. Supports the organization's attributed self-description: independent bipartisan nonprofit producing national-security and defense policy.", "organizations": [ "O-0207" ], "source_type": "docs", "source_url": "https://www.cnas.org/mission", "source_file": "content/sources/cnas-mission.yaml", "flags_all": [], "cited_by": [ { "id": "O-0207", "title": "Center for a New American Security", "url": "https://trustbutveri.fyi/organizations/center-for-a-new-american-security/" } ], "referenced_by": [ { "id": "O-0207", "title": "Center for a New American Security", "url": "https://trustbutveri.fyi/organizations/center-for-a-new-american-security/" } ] }, { "id": "S-1510", "slug": "comer-highly-secure-inference-data-centers", "status": "published", "risk_flags": [], "title": "Highly Secure Inference Data Centers: A Vertically Integrated Strategy for Security Engineering", "authors": [ "S. F. Comer", "H. Pavela", "V. Gandhi", "K. Siler-Evans", "E. Devendorf", "B. Kelley", "J. Gimbi", "J. Aguirre", "G. Kulp", "M. Stalczynski", "M. J. Malone" ], "year": 2026, "date": "2026-08-04", "venue": "RAND Corporation (Research Report RR-A4827-1)", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/comer-highly-secure-inference-data-centers/", "archive_url": "https://web.archive.org/web/20260818130032/https://www.rand.org/pubs/research_reports/RRA4827-1.html", "accessed": "2026-09-25", "doi": "10.7249/RRA4827-1", "arxiv": null, "version_note": "Read the landing page and the PDF (https://www.rand.org/content/dam/rand/pubs/research_reports/RRA4800/RRA4827-1/RAND_RRA4827-1.pdf), 51 pages. Author list taken from the PDF title page.", "note": "Produced by RAND's Center on AI, Security, and Technology (CAST).", "organizations": [ "O-0200" ], "source_type": "tech-report", "source_url": "https://www.rand.org/pubs/research_reports/RRA4827-1.html", "source_file": "content/sources/comer-highly-secure-inference-data-centers.yaml", "flags_all": [], "cited_by": [ { "id": "I-0010", "title": "RAND secure inference data center (SIDC) design", "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "O-0210", "title": "Intelligence Security Laboratories", "url": "https://trustbutveri.fyi/organizations/intelligence-security-laboratories/" }, { "id": "O-0200", "title": "RAND", "url": "https://trustbutveri.fyi/organizations/rand/" } ], "referenced_by": [ { "id": "I-0010", "title": "RAND secure inference data center (SIDC) design", "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "O-0210", "title": "Intelligence Security Laboratories", "url": "https://trustbutveri.fyi/organizations/intelligence-security-laboratories/" }, { "id": "O-0200", "title": "RAND", "url": "https://trustbutveri.fyi/organizations/rand/" } ] }, { "id": "S-2005", "slug": "ctbto-international-monitoring-system", "status": "published", "risk_flags": [], "title": "The International Monitoring System", "authors": [ "CTBTO Preparatory Commission" ], "year": 2026, "venue": "CTBTO", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/ctbto-international-monitoring-system/", "archive_url": "https://web.archive.org/web/20260825083709/https://www.ctbto.org/our-work/international-monitoring-system", "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "The page is undated; the year is the year it was read.", "organizations": [], "source_type": "gov-doc", "source_url": "https://www.ctbto.org/our-work/international-monitoring-system", "source_file": "content/sources/ctbto-international-monitoring-system.yaml", "flags_all": [], "cited_by": [], "referenced_by": [] }, { "id": "S-2006", "slug": "ctbto-the-treaty", "status": "published", "risk_flags": [], "title": "The Comprehensive Nuclear-Test-Ban Treaty (CTBT)", "authors": [ "CTBTO Preparatory Commission" ], "year": 2026, "venue": "CTBTO", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/ctbto-the-treaty/", "archive_url": "https://web.archive.org/web/20260918195255/https://www.ctbto.org/our-mission/the-treaty", "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "The page is undated; the year is the year it was read.", "organizations": [], "source_type": "gov-doc", "source_url": "https://www.ctbto.org/our-mission/the-treaty", "source_file": "content/sources/ctbto-the-treaty.yaml", "flags_all": [], "cited_by": [], "referenced_by": [] }, { "id": "S-1805", "slug": "cve-2026-20685", "status": "published", "risk_flags": [], "title": "CVE-2026-20685 (Apple Private Cloud Compute Server Software)", "authors": [ "Apple (CVE Numbering Authority)" ], "year": 2026, "date": "2026-05-18", "venue": "CVE Program", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/cve-2026-20685/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Read through the CVE Services API (https://cveawg.mitre.org/api/cve/CVE-2026-20685) because the cve.org page needs JavaScript. Affected product: Private Cloud Compute Server Software before 5E290.3.", "note": "Vendor-assigned description: \"An attacker in a privileged network position may be able to leak sensitive information. A path handling issue was addressed with improved validation.\"", "organizations": [], "source_type": "docs", "source_url": "https://www.cve.org/CVERecord?id=CVE-2026-20685", "source_file": "content/sources/cve-2026-20685.yaml", "flags_all": [], "cited_by": [ { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/" } ], "referenced_by": [ { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/" } ] }, { "id": "S-1210", "slug": "de-meulemeester-battering-ram", "status": "published", "risk_flags": [], "title": "Battering RAM: Low-Cost Interposer Attacks on Confidential Computing via Dynamic Memory Aliasing", "authors": [ "J. De Meulemeester", "D. Oswald", "I. Verbauwhede", "J. Van Bulck" ], "year": 2026, "venue": "47th IEEE Symposium on Security and Privacy (S&P 2026)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/de-meulemeester-battering-ram/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Read the authors' site and the paper PDF linked from it (https://batteringram.eu/batteringram.pdf) on 2026-09-24. Venue from the BibTeX entry on the authors' site (47th IEEE S&P, May 2026); no DOI is printed in the PDF or on the site.", "note": "Independent attack paper (KU Leuven, University of Birmingham and Durham University). DDR4 memory interposer with a bill of materials of $47.62. Reports arbitrary plaintext access to Intel Scalable SGX enclaves, extraction of SGX's platform provisioning key, and a full attestation breach on up-to-date AMD SEV-SNP by replaying launch digests. The site states that Intel and AMD acknowledged the findings but consider physical attacks on DRAM out of scope, and links Intel's guidance and AMD bulletin AMD-SB-3024. Hardware schematics and proof-of-concept code at https://github.com/batteringramattack/batteringram.", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://batteringram.eu/", "source_file": "content/sources/de-meulemeester-battering-ram.yaml", "flags_all": [], "cited_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" } ], "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" } ] }, { "id": "S-3126", "slug": "de-meulemeester-ddrop", "status": "published", "risk_flags": [], "title": "DDRop: Active Memory Interposer Attacks on Confidential VMs by Dropping DDR5 Writes", "authors": [ "J. De Meulemeester", "S. Gloor", "P. Jattke", "D. Moghimi", "D. Oswald", "M. Thompson", "K. Razavi", "I. Verbauwhede", "J. Van Bulck" ], "year": 2026, "venue": "2026 ACM SIGSAC Conference on Computer and Communications Security (CCS '26)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/de-meulemeester-ddrop/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Read the authors' site and the paper PDF linked from it (https://ddropattack.eu/ddrop.pdf) on 2026-09-25. The PDF names CCS '26 (The Hague); the conference is in November 2026, and no DOI is printed. Disclosed on 2026-09-14.", "note": "Independent attack paper (KU Leuven, ETH Zurich, Durham University and Google). An active DDR5 RDIMM interposer with a bill of materials of $159 injects parity errors so that cache-line writebacks are silently dropped, which the missing freshness protection in scalable memory encryption does not detect. Attacker model: brief physical access to install the interposer, plus control of host software and BIOS. Demonstrates forcing any TD into debug mode and forging attestation reports on an up-to-date Intel TDX platform, and integrity breaks of Scalable SGX and AMD SEV-SNP (no SEV-SNP attestation forgery reported). The site states that Intel and AMD consider physical attacks on DRAM out of scope for current products and issued security advisories on disclosure (AMD-SB-3048).", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://ddropattack.eu/", "source_file": "content/sources/de-meulemeester-ddrop.yaml", "flags_all": [], "cited_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/" } ], "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/" } ] }, { "id": "S-0067", "slug": "dean-verification-plan", "status": "published", "risk_flags": [], "title": "Verification Plan", "authors": [ "R. Dean" ], "year": 2026, "venue": "AI 2040", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/dean-verification-plan/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Motivations and policy proposals\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [ "O-0201" ], "source_type": "blog", "source_url": "https://ai-2040.com/supplements/verification-plan", "source_file": "content/sources/dean-verification-plan.yaml", "flags_all": [], "cited_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "K-0020", "title": "Sampling and assurance", "url": "https://trustbutveri.fyi/concepts/sampling-and-assurance/" }, { "id": "K-0015", "title": "Tamper evidence and tamper resistance", "url": "https://trustbutveri.fyi/concepts/tamper-evidence/" }, { "id": "O-0201", "title": "AI Futures Project", "url": "https://trustbutveri.fyi/organizations/ai-futures-project/" } ], "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "K-0020", "title": "Sampling and assurance", "url": "https://trustbutveri.fyi/concepts/sampling-and-assurance/" }, { "id": "K-0015", "title": "Tamper evidence and tamper resistance", "url": "https://trustbutveri.fyi/concepts/tamper-evidence/" }, { "id": "O-0201", "title": "AI Futures Project", "url": "https://trustbutveri.fyi/organizations/ai-futures-project/" } ] }, { "id": "S-1011", "slug": "deepseek-v4", "status": "published", "risk_flags": [], "title": "DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence", "authors": [ "DeepSeek-AI" ], "year": 2026, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/deepseek-v4/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2606.19348", "version_note": "Read v1 (PDF), section 3.3 on batch-invariant and deterministic kernel libraries. The abstract page gives a v1 timestamp of 26 Apr 2026, which does not match the 2606 identifier; date left unset.", "note": "Developer's report on its own systems; kernel statements are provider-reported. Several hundred authors; listed under the organisational author.", "organizations": [], "source_type": "tech-report", "source_url": "https://arxiv.org/abs/2606.19348", "source_file": "content/sources/deepseek-v4.yaml", "flags_all": [], "cited_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" } ], "referenced_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" } ] }, { "id": "S-0010", "slug": "dhar-guardain", "status": "published", "risk_flags": [], "title": "GuardAIn: Protecting Emerging Generative AI Workloads on Heterogeneous NPU", "authors": [ "A. Dhar", "C. Thorens", "L. M. Lazier", "L. Cavigelli" ], "year": 2025, "venue": "2025 IEEE Symposium on Security and Privacy", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/dhar-guardain/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.1109/SP61157.2025.00221", "arxiv": null, "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Trusted execution and attestation\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://www.computer.org/csdl/proceedings-article/sp/2025/223600d823/26hiVLrJZi8", "source_file": "content/sources/dhar-guardain.yaml", "flags_all": [], "cited_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ], "referenced_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ] }, { "id": "S-1505", "slug": "ding-cove-code", "status": "published", "risk_flags": [], "title": "Cove: Compositional Multi-Party Confidential Workflows for Verifiable AI Governance (reference implementation)", "authors": [ "covehub" ], "year": 2026, "venue": "GitHub", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/ding-cove-code/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Pinned to commit a3e4904926d7c9c160d232222d95f5db316627ca (main as of 2026-09-23, obtained with git ls-remote). Read README.md, docs/internal/architecture.md and docs/internal/security_model.md at that commit via raw.githubusercontent.com; the files were identical to main on that date. On 2026-09-25 security_model.md was re-read at the pinned commit and README.md on main; the github.com repository page returned 404 to the fetch tool, while raw files on main still loaded.", "note": "Reference implementation of S-0011; developer documentation of its own system supports \"reports\" statements only.", "organizations": [], "source_type": "code", "source_url": "https://github.com/covehub/cove", "source_file": "content/sources/ding-cove-code.yaml", "flags_all": [], "cited_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" } ], "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" } ] }, { "id": "S-0011", "slug": "ding-cove", "status": "published", "risk_flags": [], "title": "Cove: Compositional Multi-Party Confidential Workflows for Verifiable AI Governance", "authors": [ "S. Ding", "E. Lee", "R. Cheng", "D. Kang" ], "year": 2026, "venue": "ICML 2026 Workshop on Technical AI Governance Research", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/ding-cove/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Only the abstract was read, on the ICML 2026 virtual poster page (https://icml.cc/virtual/2026/78562), on 2026-09-23 and 2026-09-25; the OpenReview forum and PDF were not reachable with the fetch tool. No arXiv version was found.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Trusted execution and attestation\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "preprint", "source_url": "https://openreview.net/forum?id=t1dz06Vcee", "source_file": "content/sources/ding-cove.yaml", "flags_all": [], "cited_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" } ], "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" } ] }, { "id": "S-0045", "slug": "ding-moecho", "status": "published", "risk_flags": [], "title": "MoEcho: Exploiting Side-Channel Attacks to Compromise User Privacy in Mixture-of-Experts LLMs", "authors": [ "R. Ding", "T. Xu", "X. Shen", "A. A. Ding", "Y. Fei" ], "year": 2025, "venue": "2025 ACM SIGSAC Conference on Computer and Communications Security (CCS 2025)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/ding-moecho/", "archive_url": null, "accessed": "2026-09-23", "doi": "10.1145/3719027.3765174", "arxiv": "2508.15036", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Power telemetry and side-channel attacks\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://doi.org/10.1145/3719027.3765174", "source_file": "content/sources/ding-moecho.yaml", "flags_all": [], "cited_by": [], "referenced_by": [] }, { "id": "S-1803", "slug": "dittmar-unlocking-apple-pcc", "status": "published", "risk_flags": [], "title": "Unlocking Apple's Private Cloud Compute: An Analysis of Privacy-Preserving Artificial Intelligence", "authors": [ "Y. Dittmar", "M. J. Stephan", "T. Völkl", "M. Hollick", "J. Classen" ], "year": 2026, "venue": "Proceedings of the 19th ACM Conference on Security and Privacy in Wireless and Mobile Networks (WiSec '26)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/dittmar-unlocking-apple-pcc/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.1145/3765613.3811691", "arxiv": "2605.24239", "version_note": "Read the arXiv HTML v1 (22 May 2026), whose ACM reference format names WiSec '26 (30 June–3 July 2026, Saarbrücken) and prints the DOI above (doi.org returned 403 to the fetch tool, so the DOI was not resolved). Authors at the Hasso Plattner Institute, TU Darmstadt and IMDEA Networks; none at Apple.", "note": "Independent analysis that reverse-engineers PCC's client on macOS 15.5 and iOS 26.2 and queries the production service. Reports that there are no reproducible builds or symbols, request-token handling that departs from Apple's documentation (reusable one-time tokens, skipped signature validation, linkable salts), disclosure to Apple, and that Apple only clarified its documentation. It does not test attestation.", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://arxiv.org/abs/2605.24239", "source_file": "content/sources/dittmar-unlocking-apple-pcc.yaml", "flags_all": [], "cited_by": [ { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/" } ], "referenced_by": [ { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/" } ] }, { "id": "S-3383", "slug": "doj-operation-gatekeeper-2025", "status": "published", "risk_flags": [], "title": "U.S. Authorities Shut Down Major China-Linked AI Tech Smuggling Network", "authors": [ "U.S. Department of Justice" ], "year": 2025, "date": "2025-12-08", "venue": "U.S. Department of Justice, Office of Public Affairs", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/doj-operation-gatekeeper-2025/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Separates the guilty pleas of Hsu and Hao Global from charges against Gong and Yuan. The alleged removal and replacement of NVIDIA GPU labels concerns Gong's charged scheme, not the guilty pleas.", "organizations": [], "source_type": "gov-doc", "source_url": "https://www.justice.gov/opa/pr/us-authorities-shut-down-major-china-linked-ai-tech-smuggling-network", "source_file": "content/sources/doj-operation-gatekeeper-2025.yaml", "flags_all": [], "cited_by": [ { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" } ], "referenced_by": [ { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" } ] }, { "id": "S-1314", "slug": "douillard-diloco", "status": "published", "risk_flags": [], "title": "DiLoCo: Distributed Low-Communication Training of Language Models", "authors": [ "A. Douillard", "Q. Feng", "A. A. Rusu", "R. Chhaparia", "Y. Donchev", "A. Kuncoro", "M. Ranzato", "A. Szlam", "J. Shen" ], "year": 2024, "venue": "ICML 2024 Workshop on Advancing Neural Network Training (WANT)", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/douillard-diloco/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2311.08105", "version_note": "v1 submitted 14 November 2023; v3 revised 23 September 2024. Poster at the 2nd Workshop on Advancing Neural Network Training (WANT@ICML 2024), per the ICML 2024 page (https://icml.cc/virtual/2024/37144), checked 2026-09-24.", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2311.08105", "source_file": "content/sources/douillard-diloco.yaml", "flags_all": [], "cited_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" } ], "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" } ] }, { "id": "S-1607", "slug": "dziembowski-proofs-of-space", "status": "published", "risk_flags": [], "title": "Proofs of Space", "authors": [ "S. Dziembowski", "S. Faust", "V. Kolmogorov", "K. Pietrzak" ], "year": 2015, "venue": "CRYPTO 2015 (IACR Cryptology ePrint Archive 2013/796)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/dziembowski-proofs-of-space/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "The ePrint version states it is a minor revision of the CRYPTO 2015 publication.", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://eprint.iacr.org/2013/796", "source_file": "content/sources/dziembowski-proofs-of-space.yaml", "flags_all": [], "cited_by": [ { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "K-0024", "title": "Cryptographic commitment", "url": "https://trustbutveri.fyi/concepts/cryptographic-commitment/" }, { "id": "K-0012", "title": "Proof of space", "url": "https://trustbutveri.fyi/concepts/proof-of-space/" }, { "id": "K-0011", "title": "Proof of (useful) work", "url": "https://trustbutveri.fyi/concepts/proof-of-useful-work/" } ], "referenced_by": [ { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "K-0024", "title": "Cryptographic commitment", "url": "https://trustbutveri.fyi/concepts/cryptographic-commitment/" }, { "id": "K-0012", "title": "Proof of space", "url": "https://trustbutveri.fyi/concepts/proof-of-space/" }, { "id": "K-0011", "title": "Proof of (useful) work", "url": "https://trustbutveri.fyi/concepts/proof-of-useful-work/" } ] }, { "id": "S-3543", "slug": "ec-guidelines-gpai-provider-obligations", "status": "published", "risk_flags": [], "title": "Guidelines on the scope of the obligations for general-purpose AI models established by Regulation (EU) 2024/1689 (AI Act)", "authors": [ "European Commission" ], "year": 2025, "date": "2025-07-18", "venue": "European Commission, Communication C(2025) 5045 final", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/ec-guidelines-gpai-provider-obligations/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "The landing page gives the publication date and the entry into application on 2 August 2025. The text (§2.3.1 on the 10^25 FLOP presumption, §2.3.2 on notification within two weeks) was read in the annex PDF on the AI Act Service Desk.", "organizations": [], "source_type": "gov-doc", "source_url": "https://digital-strategy.ec.europa.eu/en/library/guidelines-scope-obligations-providers-general-purpose-ai-models-under-ai-act", "source_file": "content/sources/ec-guidelines-gpai-provider-obligations.yaml", "flags_all": [], "cited_by": [ { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" } ], "referenced_by": [ { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" } ] }, { "id": "S-3021", "slug": "eigencloud-eigenai-launch", "status": "published", "risk_flags": [], "title": "EigenCloud Brings Verifiable AI to Mass Market with EigenAI and EigenCompute Launches", "authors": [ "EigenCloud" ], "year": 2025, "date": "2025-09-30", "venue": "Eigen Labs blog", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/eigencloud-eigenai-launch/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Developer's launch announcement (the blog.eigencloud.xyz address redirects here). States that EigenAI and EigenCompute launched \"on mainnet alpha\", that the backing stake \"is not exposed to slashing risk yet during the alpha phase\", that code will be open-sourced so users can verify responses, and that the API first serves gpt-oss-120b-f16. Supports only \"Eigen Labs reports\" statements.", "organizations": [], "source_type": "blog", "source_url": "https://www.eigenlabs.org/blog/eigencloud-brings-verifiable-ai-to-mass-market-with-eigenai-and-eigencompute-launches/", "source_file": "content/sources/eigencloud-eigenai-launch.yaml", "flags_all": [], "cited_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" } ], "referenced_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" } ] }, { "id": "S-0069", "slug": "eo-14148-initial-rescissions", "status": "published", "risk_flags": [], "title": "Executive Order 14148: Initial Rescissions of Harmful Executive Orders and Actions", "authors": [ "Executive Office of the President" ], "year": 2025, "date": "2025-01-20", "venue": "Federal Register, 90 FR 8237 (document 2025-01901, published 2025-01-28)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/eo-14148-initial-rescissions/", "archive_url": null, "accessed": "2026-09-23", "doi": null, "arxiv": null, "version_note": "Signed 2025-01-20. Section 2(ggg) lists Executive Order 14110 of October 30, 2023 (Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence) among the orders revoked.", "organizations": [], "source_type": "gov-doc", "source_url": "https://www.federalregister.gov/documents/2025/01/28/2025-01901/initial-rescissions-of-harmful-executive-orders-and-actions", "source_file": "content/sources/eo-14148-initial-rescissions.yaml", "flags_all": [], "cited_by": [ { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" }, { "id": "K-0021", "title": "Interconnect bandwidth", "url": "https://trustbutveri.fyi/concepts/interconnect-bandwidth/" } ], "referenced_by": [ { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" }, { "id": "K-0021", "title": "Interconnect bandwidth", "url": "https://trustbutveri.fyi/concepts/interconnect-bandwidth/" } ] }, { "id": "S-3613", "slug": "epoch-ai-about", "status": "published", "risk_flags": [], "title": "About Epoch AI", "authors": [], "year": 2026, "venue": "Epoch AI", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/epoch-ai-about/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Official organization page, read 2026-09-25; the year is the access year because the page is undated. Supports the organization's attributed self-description: data-first research nonprofit investigating the future of AI.", "organizations": [ "O-0208" ], "source_type": "docs", "source_url": "https://epoch.ai/about", "source_file": "content/sources/epoch-ai-about.yaml", "flags_all": [], "cited_by": [ { "id": "O-0208", "title": "Epoch AI", "url": "https://trustbutveri.fyi/organizations/epoch-ai/" } ], "referenced_by": [ { "id": "O-0208", "title": "Epoch AI", "url": "https://trustbutveri.fyi/organizations/epoch-ai/" } ] }, { "id": "S-3301", "slug": "epoch-data-centers-methodology", "status": "published", "risk_flags": [], "title": "AI Data Centers Documentation – Methodology", "authors": [ "Epoch AI" ], "year": 2026, "venue": "Epoch AI", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/epoch-data-centers-methodology/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Living documentation for Epoch AI's Frontier Data Centers Hub; the page is undated and was read on 2026-09-25, when it gave the coverage estimate as of that date. Supports only statements attributed to Epoch AI.", "organizations": [ "O-0208" ], "source_type": "docs", "source_url": "https://epoch.ai/data/data-centers-documentation/methodology", "source_file": "content/sources/epoch-data-centers-methodology.yaml", "flags_all": [], "cited_by": [ { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/" }, { "id": "O-0208", "title": "Epoch AI", "url": "https://trustbutveri.fyi/organizations/epoch-ai/" } ], "referenced_by": [ { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/" }, { "id": "O-0208", "title": "Epoch AI", "url": "https://trustbutveri.fyi/organizations/epoch-ai/" } ] }, { "id": "S-1411", "slug": "epoch-frontier-data-centers-hub", "status": "published", "risk_flags": [], "title": "Introducing the Frontier Data Centers Hub", "authors": [ "Epoch AI" ], "year": 2025, "date": "2025-11-04", "venue": "Epoch AI", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/epoch-frontier-data-centers-hub/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Announcement of Epoch AI's public dataset of large AI data centres (epoch.ai/data/data-centers).", "organizations": [ "O-0208" ], "source_type": "blog", "source_url": "https://epoch.ai/latest/introducing-the-frontier-data-centers-hub", "source_file": "content/sources/epoch-frontier-data-centers-hub.yaml", "flags_all": [], "cited_by": [ { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/" }, { "id": "C-0001", "title": "Compute stock is at most a declared amount", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/" }, { "id": "O-0208", "title": "Epoch AI", "url": "https://trustbutveri.fyi/organizations/epoch-ai/" } ], "referenced_by": [ { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/" }, { "id": "C-0001", "title": "Compute stock is at most a declared amount", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/" }, { "id": "O-0208", "title": "Epoch AI", "url": "https://trustbutveri.fyi/organizations/epoch-ai/" } ] }, { "id": "S-3542", "slug": "eu-ai-act-regulation-2024-1689", "status": "published", "risk_flags": [], "title": "Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act)", "authors": [ "European Parliament", "Council of the European Union" ], "year": 2024, "date": "2024-06-13", "venue": "Official Journal of the European Union, OJ L, 2024/1689", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/eu-ai-act-regulation-2024-1689/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "The fetch tool could not render the articles on EUR-Lex, so Articles 51, 52 and 113 were read on the European Commission's AI Act Service Desk (https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-51, article-52 and article-113). Article 51(2) presumes high-impact capabilities above 10^25 FLOP of cumulative training compute; Article 52(1) requires notification of the Commission within two weeks; Article 113(b) applies Chapter V from 2 August 2025.", "organizations": [], "source_type": "gov-doc", "source_url": "https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng", "source_file": "content/sources/eu-ai-act-regulation-2024-1689.yaml", "flags_all": [], "cited_by": [ { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" } ], "referenced_by": [ { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" } ] }, { "id": "S-3561", "slug": "ezkl-docs-overview", "status": "published", "risk_flags": [], "title": "EZKL documentation (overview)", "authors": [ "Zkonduit Inc." ], "year": 2025, "venue": "EZKL documentation", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/ezkl-docs-overview/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Developer's documentation home, read 2026-09-25. Describes EZKL as \"a developer-friendly system for verifiable AI and analytics\" and carries the notice \"© Copyright Zkonduit Inc. 2025\". Zkonduit's GitHub organization (github.com/zkonduit) describes itself as \"Making ezkl\". Supports \"Zkonduit reports\" statements only.", "organizations": [ "O-0213" ], "source_type": "docs", "source_url": "https://docs.ezkl.xyz/", "source_file": "content/sources/ezkl-docs-overview.yaml", "flags_all": [], "cited_by": [ { "id": "O-0213", "title": "Zkonduit", "url": "https://trustbutveri.fyi/organizations/zkonduit/" } ], "referenced_by": [ { "id": "O-0213", "title": "Zkonduit", "url": "https://trustbutveri.fyi/organizations/zkonduit/" } ] }, { "id": "S-0027", "slug": "fang-proof-of-learning-broken", "status": "published", "risk_flags": [], "title": "Proof-of-Learning is Currently More Broken Than You Think", "authors": [ "C. Fang", "H. Jia", "A. Thudi", "M. Yaghini", "C. A. Choquette-Choo", "N. Dullerud", "V. Chandrasekaran", "N. Papernot" ], "year": 2023, "venue": "8th IEEE European Symposium on Security and Privacy (EuroS&P 2023)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/fang-proof-of-learning-broken/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2208.03567", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Proof of learning and training\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://arxiv.org/abs/2208.03567", "source_file": "content/sources/fang-proof-of-learning-broken.yaml", "flags_all": [], "cited_by": [ { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" } ], "referenced_by": [ { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" }, { "id": "M-0003", "title": "Whole-workload recomputation (reproducible packets)", "url": "https://trustbutveri.fyi/mechanisms/reproducible-computation-packets/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" } ] }, { "id": "S-3611", "slug": "fli-governing-ai-grant", "status": "published", "risk_flags": [], "title": "Future of Life Institute: Global Institutions Governing AI", "authors": [], "year": 2026, "venue": "Future of Life Institute", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/fli-governing-ai-grant/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Official organization page, read 2026-09-25; the year is the access year because the page is undated. Supports the organization's attributed self-description: self-description as independent nonprofit and mission to steer transformative technology away from extreme risks.", "organizations": [ "O-0206" ], "source_type": "docs", "source_url": "https://futureoflife.org/grant-program/global-institutions-governing-ai/", "source_file": "content/sources/fli-governing-ai-grant.yaml", "flags_all": [], "cited_by": [ { "id": "O-0206", "title": "Future of Life Institute", "url": "https://trustbutveri.fyi/organizations/future-of-life-institute/" } ], "referenced_by": [ { "id": "O-0206", "title": "Future of Life Institute", "url": "https://trustbutveri.fyi/organizations/future-of-life-institute/" } ] }, { "id": "S-1203", "slug": "fli-hardware-backed-compute-governance", "status": "published", "risk_flags": [], "title": "Exploration of secure hardware solutions for safe AI deployment", "authors": [ "Future of Life Institute" ], "year": 2023, "date": "2023-11-30", "venue": "Future of Life Institute", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/fli-hardware-backed-compute-governance/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Describes an FLI and Mithril Security proof-of-concept built on BlindAI and Intel SGX (code at https://github.com/mithril-security/blindai_drm_fli). The page shows a modification date of 2024-07-31.", "organizations": [ "O-0206" ], "source_type": "blog", "source_url": "https://futureoflife.org/ai-policy/hardware-backed-compute-governance/", "source_file": "content/sources/fli-hardware-backed-compute-governance.yaml", "flags_all": [], "cited_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "O-0206", "title": "Future of Life Institute", "url": "https://trustbutveri.fyi/organizations/future-of-life-institute/" } ], "referenced_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "O-0206", "title": "Future of Life Institute", "url": "https://trustbutveri.fyi/organizations/future-of-life-institute/" } ] }, { "id": "S-3381", "slug": "fli-verifiable-training-ai-models", "status": "published", "risk_flags": [], "title": "Verifiable Training of AI Models", "authors": [ "A. Aguirre", "R. Millet" ], "year": 2024, "date": "2024-07-23", "venue": "Future of Life Institute", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/fli-verifiable-training-ai-models/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "The page shows publication on 2024-07-23 and an update on 2025-05-20.", "note": "FLI (Aguirre) and Mithril Security (Millet) describe AICert, a proof of concept that uses Trusted Platform Modules to bind a model's weights to its training code and data, and state its limits (fine-tuning only, no third-party audit, no detection of poisoned models or datasets). Code at https://github.com/mithril-security/aicert, whose README warns that AICert is still under development and not for production use (read 2026-09-25). Supports \"FLI reports\" statements only.", "organizations": [ "O-0206" ], "source_type": "blog", "source_url": "https://futureoflife.org/ai/verifiable-training-of-ai-models/", "source_file": "content/sources/fli-verifiable-training-ai-models.yaml", "flags_all": [], "cited_by": [ { "id": "O-0206", "title": "Future of Life Institute", "url": "https://trustbutveri.fyi/organizations/future-of-life-institute/" } ], "referenced_by": [ { "id": "O-0206", "title": "Future of Life Institute", "url": "https://trustbutveri.fyi/organizations/future-of-life-institute/" } ] }, { "id": "S-0040", "slug": "gangwal-detecting-covert-cryptomining", "status": "published", "risk_flags": [], "title": "Detecting Covert Cryptomining Using HPC", "authors": [ "A. Gangwal", "S. G. Piazzetta", "G. Lain", "M. Conti" ], "year": 2020, "venue": "Cryptology and Network Security – CANS 2020, LNCS 12579, pp. 344–364", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/gangwal-detecting-covert-cryptomining/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.1007/978-3-030-65411-5_17", "arxiv": "1909.00268", "version_note": "arXiv preprint first posted in 2019 (1909.00268); published at the 19th International Conference on Cryptology and Network Security (CANS 2020). The year is the proceedings year. Venue, pages and DOI from the Springer chapter page, checked 2026-09-24.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Power telemetry and side-channel attacks\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://link.springer.com/chapter/10.1007/978-3-030-65411-5_17", "source_file": "content/sources/gangwal-detecting-covert-cryptomining.yaml", "flags_all": [], "cited_by": [ { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" } ], "referenced_by": [ { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" } ] }, { "id": "S-0046", "slug": "gao-deeptheft", "status": "published", "risk_flags": [], "title": "DeepTheft: Stealing DNN Model Architectures through Power Side Channel", "authors": [ "Y. Gao", "H. Qiu", "Z. Zhang", "B. Wang", "H. Ma", "A. Abuadbba", "M. Xue", "A. Fu", "S. Nepal" ], "year": 2024, "venue": "2024 IEEE Symposium on Security and Privacy", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/gao-deeptheft/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.1109/SP54263.2024.00250", "arxiv": "2309.11894", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Power telemetry and side-channel attacks\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://doi.org/10.1109/SP54263.2024.00250", "source_file": "content/sources/gao-deeptheft.yaml", "flags_all": [], "cited_by": [ { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" } ], "referenced_by": [ { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" } ] }, { "id": "S-3540", "slug": "gao-model-equality-testing", "status": "published", "risk_flags": [], "title": "Model Equality Testing: Which Model Is This API Serving?", "authors": [ "I. Gao", "P. Liang", "C. Guestrin" ], "year": 2025, "venue": "International Conference on Learning Representations (ICLR 2025)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/gao-model-equality-testing/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2410.20247", "version_note": "arXiv v2 (8 April 2025), marked as the ICLR 2025 camera-ready version; v1 was posted on 26 October 2024. The ICLR 2025 publication was confirmed through the conference's slides page and the ML Anthology entry; the OpenReview forum (https://openreview.net/forum?id=QCDdI7X3f9) was not readable by the fetch tool.", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://arxiv.org/abs/2410.20247", "source_file": "content/sources/gao-model-equality-testing.yaml", "flags_all": [], "cited_by": [ { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" } ], "referenced_by": [ { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" } ] }, { "id": "S-1412", "slug": "gargiulo-workload-identification-physical-side-channels", "status": "published", "risk_flags": [], "title": "Workload Identification with Physical Side Channels for AI Governance", "authors": [ "S. Gargiulo", "G. Kulp" ], "year": 2026, "date": "2026-08-31", "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/gargiulo-workload-identification-physical-side-channels/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2609.00309", "arxiv": "2609.00309", "version_note": "v1 (2026-08-31), the version shown on 2026-09-25. Read via the arXiv HTML rendering (arxiv.org/html/2609.00309); the abstract page returned no extractable text to the fetch tool. The paper states that the recordings are released as a Hugging Face dataset; the dataset page returned HTTP 401 on 2026-09-23 and 2026-09-25. Affiliations as shown: S. Gargiulo (Pivotal Research), G. Kulp (Intelligence Security Laboratories).", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2609.00309", "source_file": "content/sources/gargiulo-workload-identification-physical-side-channels.yaml", "flags_all": [], "cited_by": [ { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" } ], "referenced_by": [ { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" } ] }, { "id": "S-3022", "slug": "gensyn-building-delphi", "status": "published", "risk_flags": [], "title": "Building Delphi: Pricing, Settlement, and Agentic Trading", "authors": [ "D. Jedamski" ], "year": 2026, "date": "2026-05-07", "venue": "Gensyn blog", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/gensyn-building-delphi/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Developer's post on Delphi, Gensyn's information-market app. States that \"Delphi is live on Gensyn Mainnet\" and that \"hosted REE settlement is available for partner markets\", and that open-source models running inside REE \"produce a receipt that can be independently rerun to verify the answer\". Describes no dispute process and does not mention Verde. Supports only \"Gensyn reports\" statements.", "organizations": [ "O-0212" ], "source_type": "blog", "source_url": "https://www.gensyn.ai/blog/building-delphi-pricing-settlement-and-agentic-trading", "source_file": "content/sources/gensyn-building-delphi.yaml", "flags_all": [], "cited_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "I-0015", "title": "Verde and RepOps (Gensyn)", "url": "https://trustbutveri.fyi/implementations/gensyn-verde-repops/" }, { "id": "O-0212", "title": "Gensyn", "url": "https://trustbutveri.fyi/organizations/gensyn/" } ], "referenced_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "I-0015", "title": "Verde and RepOps (Gensyn)", "url": "https://trustbutveri.fyi/implementations/gensyn-verde-repops/" }, { "id": "O-0212", "title": "Gensyn", "url": "https://trustbutveri.fyi/organizations/gensyn/" } ] }, { "id": "S-3560", "slug": "gensyn-homepage", "status": "published", "risk_flags": [], "title": "Gensyn: machines that predict the future", "authors": [], "year": 2026, "venue": "Gensyn", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/gensyn-homepage/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Company homepage, read 2026-09-25. The title is the HTML page title. States that \"Gensyn builds AI that forecasts – and keeps improving – by verifiably closing the loop between prediction and reality\", and lists Delphi (information markets) and REE (Reproducible Execution Environment) among its products. Supports \"Gensyn reports\" statements only.", "organizations": [ "O-0212" ], "source_type": "docs", "source_url": "https://www.gensyn.ai/", "source_file": "content/sources/gensyn-homepage.yaml", "flags_all": [], "cited_by": [ { "id": "O-0212", "title": "Gensyn", "url": "https://trustbutveri.fyi/organizations/gensyn/" } ], "referenced_by": [ { "id": "O-0212", "title": "Gensyn", "url": "https://trustbutveri.fyi/organizations/gensyn/" } ] }, { "id": "S-1811", "slug": "gensyn-introducing-judge", "status": "published", "risk_flags": [], "title": "Introducing Judge", "authors": [ "Gensyn" ], "year": 2025, "date": "2025-08-27", "venue": "Gensyn news", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/gensyn-introducing-judge/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Developer's announcement of Judge, an AI evaluation service built on Verde and Gensyn's Reproducible Runtime, showcased with a reasoning task framed as a prediction market. Supports only \"Gensyn reports\" statements.", "organizations": [ "O-0212" ], "source_type": "blog", "source_url": "https://www.gensyn.ai/news/introducing-judge", "source_file": "content/sources/gensyn-introducing-judge.yaml", "flags_all": [], "cited_by": [ { "id": "I-0015", "title": "Verde and RepOps (Gensyn)", "url": "https://trustbutveri.fyi/implementations/gensyn-verde-repops/" }, { "id": "O-0212", "title": "Gensyn", "url": "https://trustbutveri.fyi/organizations/gensyn/" } ], "referenced_by": [ { "id": "I-0015", "title": "Verde and RepOps (Gensyn)", "url": "https://trustbutveri.fyi/implementations/gensyn-verde-repops/" }, { "id": "O-0212", "title": "Gensyn", "url": "https://trustbutveri.fyi/organizations/gensyn/" } ] }, { "id": "S-1812", "slug": "gensyn-ree-code", "status": "published", "risk_flags": [], "title": "gensyn-ai/ree: Gensyn Reproducible Execution Environment (GitHub repository)", "authors": [ "Gensyn" ], "year": 2026, "venue": "GitHub", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/gensyn-ree-code/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Main branch as viewed on 2026-09-25; README patch notes run to v0.4.0 (4 June 2026), and v0.2.0 (20 April 2026) adds pipeline parallelism for models of up to 72B and a threefold cut in reproducible-mode overhead. Gensyn's documentation gives v0.6.0 as the current release (S-3023). The SDK is MIT-licensed; the REE compiler and reproducible-operators binaries ship in Docker images under a proprietary binary licence.", "note": "REE runs LLM inference reproducibly and writes JSON receipts with the model, prompt, output and the metadata needed to reproduce the run, with verify and validate subcommands. Developer's repository.", "organizations": [ "O-0212" ], "source_type": "code", "source_url": "https://github.com/gensyn-ai/ree", "source_file": "content/sources/gensyn-ree-code.yaml", "flags_all": [], "cited_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" }, { "id": "I-0015", "title": "Verde and RepOps (Gensyn)", "url": "https://trustbutveri.fyi/implementations/gensyn-verde-repops/" }, { "id": "O-0212", "title": "Gensyn", "url": "https://trustbutveri.fyi/organizations/gensyn/" } ], "referenced_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" }, { "id": "I-0015", "title": "Verde and RepOps (Gensyn)", "url": "https://trustbutveri.fyi/implementations/gensyn-verde-repops/" }, { "id": "O-0212", "title": "Gensyn", "url": "https://trustbutveri.fyi/organizations/gensyn/" } ] }, { "id": "S-3023", "slug": "gensyn-ree-docs", "status": "published", "risk_flags": [], "title": "Reproducible Execution Environment (REE) (Gensyn documentation)", "authors": [ "Gensyn" ], "year": 2026, "venue": "Gensyn documentation", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/gensyn-ree-docs/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Page as viewed on 2026-09-25, giving v0.6.0 as the current release (no release date shown). The GitHub README (S-1812) carries patch notes only to v0.4.0.", "note": "Developer's documentation of its own runtime. Describes three modes: default (standard PyTorch kernels, no determinism guarantee), deterministic (PyTorch deterministic algorithms, reproducible on the same hardware) and reproducible (RepOp kernels, bitwise-identical across supported hardware). Lists reproducible int8 attention, fused gathered-log-probability and MoE expert GEMM kernels; warns that some v0.5.0 receipts may not re-verify with v0.6.0; states that REE as a whole is not open source. Supports only \"Gensyn reports\" statements.", "organizations": [ "O-0212" ], "source_type": "docs", "source_url": "https://docs.gensyn.ai/tech", "source_file": "content/sources/gensyn-ree-docs.yaml", "flags_all": [], "cited_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "I-0015", "title": "Verde and RepOps (Gensyn)", "url": "https://trustbutveri.fyi/implementations/gensyn-verde-repops/" } ], "referenced_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "I-0015", "title": "Verde and RepOps (Gensyn)", "url": "https://trustbutveri.fyi/implementations/gensyn-verde-repops/" } ] }, { "id": "S-1810", "slug": "gensyn-verde-in-production", "status": "published", "risk_flags": [], "title": "Verde Verification System In Production", "authors": [ "O. Ersoy" ], "year": 2025, "date": "2025-11-24", "venue": "Gensyn research blog", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/gensyn-verde-in-production/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Developer's post (page shows last modified 4 Sep 2026). States that Verde and RepOps are deployed in Judge, that Verde covers training, fine-tuning and inference, that RepOps gives bitwise-reproducible results across hardware, and that Verde guarantees the output came from the declared model and data, not that the model is correct. Supports only \"Gensyn reports\" statements.", "organizations": [ "O-0212" ], "source_type": "blog", "source_url": "https://www.gensyn.ai/research/verde-verification-system-in-production", "source_file": "content/sources/gensyn-verde-in-production.yaml", "flags_all": [], "cited_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "I-0015", "title": "Verde and RepOps (Gensyn)", "url": "https://trustbutveri.fyi/implementations/gensyn-verde-repops/" }, { "id": "O-0212", "title": "Gensyn", "url": "https://trustbutveri.fyi/organizations/gensyn/" } ], "referenced_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "I-0015", "title": "Verde and RepOps (Gensyn)", "url": "https://trustbutveri.fyi/implementations/gensyn-verde-repops/" }, { "id": "O-0212", "title": "Gensyn", "url": "https://trustbutveri.fyi/organizations/gensyn/" } ] }, { "id": "S-3240", "slug": "gil-pons-empirical-evaluation-memory-erasure", "status": "published", "risk_flags": [], "title": "Empirical Evaluation of Memory-Erasure Protocols", "authors": [ "R. Gil-Pons", "S. Mauw", "R. Trujillo-Rasua" ], "year": 2025, "venue": "Proceedings of the 22nd International Conference on Security and Cryptography (SECRYPT 2025), pp. 209–220", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/gil-pons-empirical-evaluation-memory-erasure/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.5220/0013554800003979", "arxiv": "2509.10224", "version_note": "Read the SciTePress PDF and the arXiv HTML preprint (arXiv 2509.10224); venue, pages and DOI checked against Crossref on 2026-09-25.", "note": "Code at gitlab.com/uniluxembourg/fstm/dcs/satoss/memory-erasure-experiments, per the paper.", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://www.scitepress.org/Papers/2025/135548/135548.pdf", "source_file": "content/sources/gil-pons-empirical-evaluation-memory-erasure.yaml", "flags_all": [], "cited_by": [ { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" } ], "referenced_by": [ { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" } ] }, { "id": "S-2007", "slug": "glaser-zero-knowledge-warhead-verification", "status": "published", "risk_flags": [], "title": "A zero-knowledge protocol for nuclear warhead verification", "authors": [ "A. Glaser", "B. Barak", "R. J. Goldston" ], "year": 2014, "venue": "Nature 510, 497–502", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/glaser-zero-knowledge-warhead-verification/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.1038/nature13457", "arxiv": null, "organizations": [], "source_type": "peer-reviewed", "source_url": "https://www.nature.com/articles/nature13457", "source_file": "content/sources/glaser-zero-knowledge-warhead-verification.yaml", "flags_all": [], "cited_by": [], "referenced_by": [] }, { "id": "S-1014", "slug": "gond-llm-42", "status": "published", "risk_flags": [], "title": "LLM-42: Enabling Determinism in LLM Inference with Verified Speculation", "authors": [ "R. Gond", "A. K. Kamath", "R. Ramjee", "A. Panwar" ], "year": 2026, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/gond-llm-42/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2601.17768", "version_note": "Abstract of v2 (30 Jan 2026) read; full text not reviewed. Code listed at github.com/microsoft/llm-42.", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2601.17768", "source_file": "content/sources/gond-llm-42.yaml", "flags_all": [], "cited_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" } ], "referenced_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" } ] }, { "id": "S-1112", "slug": "gong-hollow-llm-attack", "status": "published", "risk_flags": [], "title": "Hollow-LLM Attack: Computationally Trivial Weights in Zero-Knowledge Verification of LLM Inference", "authors": [ "C. Gong", "B. Liu", "M. Li" ], "year": 2026, "date": "2026-07-30", "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/gong-hollow-llm-attack/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2607.28884", "version_note": "v1 (30 July 2026), read via the arXiv HTML and PDF renderings; the abstract page did not render for the fetch tool. Author order is as printed on the paper's title page; a mirror of the arXiv listing (pith.science) lists B. Liu first. A co-author's web page labels the paper \"IEEE S&P 2026\", but no proceedings record was found and a co-author's CV lists it as under submission, so it is recorded as a preprint.", "note": "Independent analysis (University of Southern California) of zero-knowledge proofs of LLM inference. Shows that valid proofs do not bind the computation spent, using \"ghost weights\"; experiments use the proof procedure of zkGPT on a 6-layer, 512-dimensional transformer.", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2607.28884", "source_file": "content/sources/gong-hollow-llm-attack.yaml", "flags_all": [], "cited_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0003", "title": "zkLLM", "url": "https://trustbutveri.fyi/implementations/zkllm/" } ], "referenced_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0003", "title": "zkLLM", "url": "https://trustbutveri.fyi/implementations/zkllm/" } ] }, { "id": "S-3321", "slug": "google-confidential-space-overview", "status": "published", "risk_flags": [], "title": "Confidential Space overview", "authors": [ "Google Cloud" ], "year": 2026, "venue": "Google Cloud documentation", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/google-confidential-space-overview/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Page last updated 2026-09-22 (UTC), read on 2026-09-25.", "note": "Vendor documentation; supports only \"Google reports\" statements. Describes an isolated environment to operate on sensitive data from multiple parties, with data collaborators, workload authors and workload operators; data is released only to workloads whose attestation meets the collaborators' conditions, and the operator has no access to the data. Runs on AMD SEV, Intel TDX, or Intel TDX with NVIDIA Confidential Computing.", "organizations": [], "source_type": "docs", "source_url": "https://docs.cloud.google.com/confidential-computing/confidential-space/docs/confidential-space-overview", "source_file": "content/sources/google-confidential-space-overview.yaml", "flags_all": [], "cited_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" } ], "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" } ] }, { "id": "S-3322", "slug": "google-confidential-space-release-notes", "status": "published", "risk_flags": [], "title": "Confidential Space release notes", "authors": [ "Google Cloud" ], "year": 2026, "venue": "Google Cloud documentation", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/google-confidential-space-release-notes/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Vendor release notes; support only \"Google reports\" statements. Confidential Space generally available on 2023-03-28; on Intel TDX (C3) on 2025-03-31; on H100 GPUs (a3-highgpu-1g) on 2026-04-29; on H100 with Intel Trust Authority attestation on 2026-09-15.", "organizations": [], "source_type": "docs", "source_url": "https://docs.cloud.google.com/confidential-computing/confidential-space/docs/release-notes", "source_file": "content/sources/google-confidential-space-release-notes.yaml", "flags_all": [], "cited_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" } ], "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" } ] }, { "id": "S-1816", "slug": "google-private-ai-compute", "status": "published", "risk_flags": [], "title": "Private AI Compute: our next step in building private and helpful AI", "authors": [ "J. Yagnik" ], "year": 2025, "date": "2025-11-11", "venue": "Google blog (The Keyword)", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/google-private-ai-compute/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Google's announcement of Private AI Compute: Gemini models on Google's TPUs with Titanium Intelligence Enclaves (TIE), and remote attestation and encryption connecting the device to the \"hardware-secured sealed cloud environment\"; first used by Magic Cue and Recorder on Pixel. Vendor post; supports only \"Google reports\" statements.", "organizations": [], "source_type": "blog", "source_url": "https://blog.google/innovation-and-ai/products/google-private-ai-compute/", "source_file": "content/sources/google-private-ai-compute.yaml", "flags_all": [], "cited_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ], "referenced_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ] }, { "id": "S-1818", "slug": "govai-about", "status": "published", "risk_flags": [], "title": "About us: Centre for the Governance of AI (GovAI)", "authors": [], "year": 2026, "venue": "Centre for the Governance of AI", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/govai-about/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "About page of GovAI. States that GovAI is a think tank that aims to help decision-makers in government and industry navigate the transition to a world with advanced AI; that it was founded at Yale University in 2016, became an academic centre at Oxford in 2018 and an independent nonprofit in 2021; that it has offices in London and Washington, DC; and that it is a US 501(c)(3) organization with a UK subsidiary. Self-description; supports \"GovAI states\" statements.", "organizations": [ "O-0211" ], "source_type": "docs", "source_url": "https://www.governance.ai/about-us", "source_file": "content/sources/govai-about.yaml", "flags_all": [], "cited_by": [ { "id": "O-0211", "title": "Centre for the Governance of AI", "url": "https://trustbutveri.fyi/organizations/centre-for-the-governance-of-ai/" } ], "referenced_by": [ { "id": "O-0211", "title": "Centre for the Governance of AI", "url": "https://trustbutveri.fyi/organizations/centre-for-the-governance-of-ai/" } ] }, { "id": "S-1509", "slug": "greenblatt-upload-limits", "status": "published", "risk_flags": [], "title": "Preventing model exfiltration with upload limits", "authors": [ "R. Greenblatt" ], "year": 2024, "date": "2024-02-06", "venue": "AI Alignment Forum", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/greenblatt-upload-limits/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Posted under the username ryan_greenblatt.", "organizations": [], "source_type": "forum", "source_url": "https://www.alignmentforum.org/posts/rf66R4YsrCHgWx9RG/preventing-model-exfiltration-with-upload-limits", "source_file": "content/sources/greenblatt-upload-limits.yaml", "flags_all": [], "cited_by": [ { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" } ], "referenced_by": [ { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" } ] }, { "id": "S-0042", "slug": "gregersen-input-dependent-power-usage-gpus", "status": "published", "risk_flags": [], "title": "Input-Dependent Power Usage in GPUs", "authors": [ "T. Gregersen", "P. Patel", "E. Choukse" ], "year": 2024, "venue": "SC24-W: Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis (Sustainable Supercomputing workshop), pp. 1872–1877", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/gregersen-input-dependent-power-usage-gpus/", "archive_url": null, "accessed": "2026-09-23", "doi": "10.1109/SCW63240.2024.00235", "arxiv": "2409.18324", "version_note": "Workshop paper at Sustainable Supercomputing at SC24 (IEEE Xplore document 10820679); kept at tier B as a workshop paper. The URL points to the arXiv version. Venue, pages and DOI from Crossref, checked 2026-09-24.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Power telemetry and side-channel attacks\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2409.18324", "source_file": "content/sources/gregersen-input-dependent-power-usage-gpus.yaml", "flags_all": [], "cited_by": [ { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" } ], "referenced_by": [ { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" } ] }, { "id": "S-3382", "slug": "grunewald-countering-ai-chip-smuggling", "status": "published", "risk_flags": [], "title": "Countering AI Chip Smuggling Has Become a National Security Priority", "authors": [ "E. Grunewald", "T. Fist" ], "year": 2025, "date": "2025-06-11", "venue": "Center for a New American Security (working paper)", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/grunewald-countering-ai-chip-smuggling/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "CNAS working paper. Affiliations as shown on the CNAS page: E. Grunewald (Institute for AI Policy and Strategy), T. Fist (CNAS). Estimates AI chip smuggling to the PRC in 2024 at 10,000 to several hundred thousand chips, median about 140,000, and recommends software-based location verification and a notification requirement for exports, re-exports and ownership transfers of controlled AI chips.", "organizations": [ "O-0207", "O-0204" ], "source_type": "tech-report", "source_url": "https://www.cnas.org/publications/reports/countering-ai-chip-smuggling-has-become-a-national-security-priority", "source_file": "content/sources/grunewald-countering-ai-chip-smuggling.yaml", "flags_all": [], "cited_by": [ { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "O-0207", "title": "Center for a New American Security", "url": "https://trustbutveri.fyi/organizations/center-for-a-new-american-security/" } ], "referenced_by": [ { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "O-0207", "title": "Center for a New American Security", "url": "https://trustbutveri.fyi/organizations/center-for-a-new-american-security/" } ] }, { "id": "S-3129", "slug": "gu-security-look-nvidia-gpu-cc", "status": "published", "risk_flags": [], "title": "Blueprint, Bootstrap, and Bridge: A Security Look at NVIDIA GPU Confidential Computing", "authors": [ "Z. Gu", "E. Valdez", "S. Ahmed", "J. J. Stephen", "M. V. Le", "H. Jamjoom", "S. Zhao", "Z. Lin" ], "year": 2026, "venue": "Proceedings of the 9th MLSys Conference (MLSys 2026)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/gu-security-look-nvidia-gpu-cc/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2507.02770", "arxiv": "2507.02770", "version_note": "Read arXiv v2 (2026-04-17), whose header names the 9th MLSys Conference (Bellevue, 2026). v1 (2025-07-03) was titled \"NVIDIA GPU Confidential Computing Demystified\".", "note": "Independent security analysis (IBM Research and The Ohio State University) of NVIDIA's GPU confidential computing on Hopper. Concludes that bulk command and data transfers are protected, but some metadata, timing behaviour and coordination signals remain in unprotected shared memory, which can reveal computational behaviour and in some cases allow manipulation of operations (\"a partial loss of integrity\"). Findings were disclosed to NVIDIA PSIRT. No attestation break is reported.", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://arxiv.org/abs/2507.02770", "source_file": "content/sources/gu-security-look-nvidia-gpu-cc.yaml", "flags_all": [], "cited_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ], "referenced_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ] }, { "id": "S-1410", "slug": "halstead-covert-ai-projects", "status": "published", "risk_flags": [], "title": "Covert AI Projects", "authors": [ "B. Halstead", "T. Larsen" ], "year": 2026, "venue": "AI 2040", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/halstead-covert-ai-projects/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Supplement to the AI 2040 scenario (AI Futures Project). The supplement page shows no date; the AI Futures blog announced AI 2040 on 2026-07-09, so the year is inferred from that announcement.", "organizations": [ "O-0201" ], "source_type": "blog", "source_url": "https://ai-2040.com/supplements/covert-ai-projects", "source_file": "content/sources/halstead-covert-ai-projects.yaml", "flags_all": [], "cited_by": [ { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/" }, { "id": "O-0201", "title": "AI Futures Project", "url": "https://trustbutveri.fyi/organizations/ai-futures-project/" }, { "id": "O-0208", "title": "Epoch AI", "url": "https://trustbutveri.fyi/organizations/epoch-ai/" } ], "referenced_by": [ { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/" }, { "id": "O-0201", "title": "AI Futures Project", "url": "https://trustbutveri.fyi/organizations/ai-futures-project/" }, { "id": "O-0208", "title": "Epoch AI", "url": "https://trustbutveri.fyi/organizations/epoch-ai/" } ] }, { "id": "S-2010", "slug": "harack-ai-verification-infrastructure", "status": "published", "risk_flags": [], "title": "AI Verification: Infrastructure for Prosperity, Governance, and Peace", "authors": [ "B. Harack" ], "year": 2026, "date": "2026-04-15", "venue": "Lawfare", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/harack-ai-verification-infrastructure/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "organizations": [], "source_type": "blog", "source_url": "https://www.lawfaremedia.org/article/ai-verification--infrastructure-for-prosperity--governance--and-peace", "source_file": "content/sources/harack-ai-verification-infrastructure.yaml", "flags_all": [], "cited_by": [], "referenced_by": [] }, { "id": "S-0004", "slug": "harack-verification-international-ai-governance", "status": "published", "risk_flags": [], "title": "Verification for International AI Governance", "authors": [ "B. Harack", "R. F. Trager", "A. Reuel", "D. Manheim", "M. Brundage", "O. Aarne", "A. Scher", "Y. Pan", "J. Xiao", "K. Loke", "S. N. Adan", "G. Bas", "N. A. Caputo", "J. C. Morse", "J. Ahuja", "I. Duan", "J. Egan", "B. Bucknall", "B. Rosen", "R. Araujo", "V. Boulanin", "R. Lall", "F. Barez", "S. Alvira", "C. Katzke", "A. Atamli", "A. Awad" ], "year": 2025, "venue": "Oxford Martin AI Governance Initiative", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/harack-verification-international-ai-governance/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Research agendas and surveys\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [ "O-0205" ], "source_type": "tech-report", "source_url": "https://www.oxfordmartin.ox.ac.uk/publications/verification-for-international-ai-governance", "source_file": "content/sources/harack-verification-international-ai-governance.yaml", "flags_all": [], "cited_by": [ { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "K-0024", "title": "Cryptographic commitment", "url": "https://trustbutveri.fyi/concepts/cryptographic-commitment/" }, { "id": "K-0003", "title": "Positive and negative claims", "url": "https://trustbutveri.fyi/concepts/positive-and-negative-claims/" }, { "id": "K-0001", "title": "Prover", "url": "https://trustbutveri.fyi/concepts/prover/" }, { "id": "K-0018", "title": "Threat model", "url": "https://trustbutveri.fyi/concepts/threat-model/" }, { "id": "K-0002", "title": "Verifier", "url": "https://trustbutveri.fyi/concepts/verifier/" }, { "id": "O-0205", "title": "Oxford Martin AI Governance Initiative", "url": "https://trustbutveri.fyi/organizations/oxford-martin-ai-governance-initiative/" } ], "referenced_by": [ { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "K-0024", "title": "Cryptographic commitment", "url": "https://trustbutveri.fyi/concepts/cryptographic-commitment/" }, { "id": "K-0003", "title": "Positive and negative claims", "url": "https://trustbutveri.fyi/concepts/positive-and-negative-claims/" }, { "id": "K-0001", "title": "Prover", "url": "https://trustbutveri.fyi/concepts/prover/" }, { "id": "K-0018", "title": "Threat model", "url": "https://trustbutveri.fyi/concepts/threat-model/" }, { "id": "K-0002", "title": "Verifier", "url": "https://trustbutveri.fyi/concepts/verifier/" }, { "id": "O-0205", "title": "Oxford Martin AI Governance Initiative", "url": "https://trustbutveri.fyi/organizations/oxford-martin-ai-governance-initiative/" } ] }, { "id": "S-1009", "slug": "he-defeating-nondeterminism-llm-inference", "status": "published", "risk_flags": [], "title": "Defeating Nondeterminism in LLM Inference", "authors": [ "H. He", "Thinking Machines Lab" ], "year": 2025, "date": "2025-09-10", "venue": "Thinking Machines Lab: Connectionism", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/he-defeating-nondeterminism-llm-inference/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Byline reads \"Horace He in collaboration with others at Thinking Machines\". Code is published at github.com/thinking-machines-lab/batch_invariant_ops (S-1813).", "organizations": [ "O-0214" ], "source_type": "blog", "source_url": "https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/", "source_file": "content/sources/he-defeating-nondeterminism-llm-inference.yaml", "flags_all": [], "cited_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "I-0016", "title": "Batch-invariant inference kernels (Thinking Machines)", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "K-0008", "title": "Numerical nondeterminism", "url": "https://trustbutveri.fyi/concepts/numerical-nondeterminism/" }, { "id": "O-0214", "title": "Thinking Machines Lab", "url": "https://trustbutveri.fyi/organizations/thinking-machines-lab/" } ], "referenced_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "I-0016", "title": "Batch-invariant inference kernels (Thinking Machines)", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "K-0008", "title": "Numerical nondeterminism", "url": "https://trustbutveri.fyi/concepts/numerical-nondeterminism/" }, { "id": "O-0214", "title": "Thinking Machines Lab", "url": "https://trustbutveri.fyi/organizations/thinking-machines-lab/" } ] }, { "id": "S-0058", "slug": "heim-governing-through-cloud", "status": "published", "risk_flags": [], "title": "Governing Through the Cloud: The Intermediary Role of Compute Providers in AI Regulation", "authors": [ "L. Heim", "T. Fist", "J. Egan", "S. Huang", "S. Zekany", "R. Trager", "M. Osborne", "N. Zilberman" ], "year": 2024, "venue": "Oxford Martin AI Governance Initiative", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/heim-governing-through-cloud/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2403.08501", "arxiv": "2403.08501", "version_note": "Oxford Martin AI Governance Initiative policy paper dated 13 March 2024; also on arXiv (v1 13 March 2024, v2 26 March 2024), with no journal reference as of 2026-09-25.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Motivations and policy proposals\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [ "O-0205" ], "source_type": "tech-report", "source_url": "https://www.oxfordmartin.ox.ac.uk/publications/governing-through-the-cloud-the-intermediary-role-of-compute-providers-in-ai-regulation", "source_file": "content/sources/heim-governing-through-cloud.yaml", "flags_all": [], "cited_by": [ { "id": "O-0205", "title": "Oxford Martin AI Governance Initiative", "url": "https://trustbutveri.fyi/organizations/oxford-martin-ai-governance-initiative/" } ], "referenced_by": [ { "id": "O-0205", "title": "Oxford Martin AI Governance Initiative", "url": "https://trustbutveri.fyi/organizations/oxford-martin-ai-governance-initiative/" } ] }, { "id": "S-3300", "slug": "heim-pilz-limitations-satellite-imagery", "status": "published", "risk_flags": [], "title": "Limitations of Satellite Imagery Analysis for AI-Specific Data Centers", "authors": [ "L. Heim", "K. Pilz" ], "year": 2024, "date": "2024-09-13", "venue": "Lennart Heim's blog", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/heim-pilz-limitations-satellite-imagery/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "organizations": [], "source_type": "blog", "source_url": "https://blog.heim.xyz/limitations-of-satellite-imagery/", "source_file": "content/sources/heim-pilz-limitations-satellite-imagery.yaml", "flags_all": [], "cited_by": [ { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/" } ], "referenced_by": [ { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/" } ] }, { "id": "S-0043", "slug": "horvath-barracuda", "status": "published", "risk_flags": [], "title": "BarraCUDA: Edge GPUs do Leak DNN Weights", "authors": [ "P. Horvath", "L. Chmielewski", "L. Weissbart", "L. Batina", "Y. Yarom" ], "year": 2025, "venue": "34th USENIX Security Symposium", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/horvath-barracuda/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2312.07783", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Power telemetry and side-channel attacks\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://www.usenix.org/conference/usenixsecurity25/presentation/horvath", "source_file": "content/sources/horvath-barracuda.yaml", "flags_all": [], "cited_by": [ { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "K-0013", "title": "Side channel", "url": "https://trustbutveri.fyi/concepts/side-channel/" } ], "referenced_by": [ { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "K-0013", "title": "Side channel", "url": "https://trustbutveri.fyi/concepts/side-channel/" } ] }, { "id": "S-0044", "slug": "horvath-kraken", "status": "published", "risk_flags": [], "title": "Kraken: Higher-order EM Side-Channel Attacks on DNNs in Near and Far Field", "authors": [ "P. Horvath", "I. Shumailov", "L. Chmielewski", "L. Batina", "Y. Yarom" ], "year": 2026, "venue": "IEEE Conference on Secure and Trustworthy Machine Learning (SaTML 2026)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/horvath-kraken/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2603.02891", "arxiv": "2603.02891", "version_note": "arXiv v1 3 Mar 2026, v3 27 Mar 2026. Listed among the accepted papers of IEEE SaTML 2026 (Munich, 23–25 March 2026), https://satml.org/2026/accepted-papers/, checked 2026-09-25.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Power telemetry and side-channel attacks\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://arxiv.org/abs/2603.02891", "source_file": "content/sources/horvath-kraken.yaml", "flags_all": [], "cited_by": [ { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" } ], "referenced_by": [ { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" } ] }, { "id": "S-2004", "slug": "iaea-safeguards-overview", "status": "published", "risk_flags": [], "title": "IAEA Safeguards Overview: Comprehensive Safeguards Agreements and Additional Protocols", "authors": [ "International Atomic Energy Agency" ], "year": 2026, "venue": "IAEA fact sheet", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/iaea-safeguards-overview/", "archive_url": "https://web.archive.org/web/20251027091503/https://www.iaea.org/publications/factsheets/iaea-safeguards-overview", "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "The page is undated; the year is the year it was read.", "organizations": [], "source_type": "gov-doc", "source_url": "https://www.iaea.org/publications/factsheets/iaea-safeguards-overview", "source_file": "content/sources/iaea-safeguards-overview.yaml", "flags_all": [], "cited_by": [], "referenced_by": [] }, { "id": "S-3610", "slug": "iaps-homepage", "status": "published", "risk_flags": [], "title": "Institute for AI Policy and Strategy homepage", "authors": [], "year": 2026, "venue": "Institute for AI Policy and Strategy", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/iaps-homepage/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Official organization page, read 2026-09-25; the year is the access year because the page is undated. Supports the organization's attributed self-description: nonpartisan think tank producing AI policy research.", "organizations": [ "O-0204" ], "source_type": "docs", "source_url": "https://www.iaps.ai/", "source_file": "content/sources/iaps-homepage.yaml", "flags_all": [], "cited_by": [ { "id": "O-0204", "title": "Institute for AI Policy and Strategy", "url": "https://trustbutveri.fyi/organizations/institute-for-ai-policy-and-strategy/" } ], "referenced_by": [ { "id": "O-0204", "title": "Institute for AI Policy and Strategy", "url": "https://trustbutveri.fyi/organizations/institute-for-ai-policy-and-strategy/" } ] }, { "id": "S-0050", "slug": "ibm-4765-security-policy", "status": "published", "risk_flags": [], "title": "IBM 4765 Cryptographic Coprocessor Security Module: Security Policy", "authors": [ "IBM Corporation" ], "year": 2012, "venue": "NIST Cryptographic Module Validation Program", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/ibm-4765-security-policy/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Tamper resistance and detection\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "tech-report", "source_url": "https://csrc.nist.gov/csrc/media/projects/cryptographic-module-validation-program/documents/security-policies/140sp1505.pdf", "source_file": "content/sources/ibm-4765-security-policy.yaml", "flags_all": [], "cited_by": [ { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" } ], "referenced_by": [ { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" } ] }, { "id": "S-1704", "slug": "ilhan-verifiable-semiconductor-manufacturing", "status": "published", "risk_flags": [], "title": "Verifiable Semiconductor Manufacturing", "authors": [ "A. Ilhan", "C. Withers", "H. Gietz", "B. Harack" ], "year": 2026, "date": "2026-04-14", "venue": "Oxford Martin AI Governance Initiative", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/ilhan-verifiable-semiconductor-manufacturing/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Research paper dated April 2026. The landing page and the abstract and author block of the PDF (https://aigi.ox.ac.uk/wp-content/uploads/2026/04/Verifiable-Semiconductor-Manufacturing.pdf) were read; the body was not reviewed.", "note": "Affiliations as printed: A. Ilhan (ERA Cambridge; Oxford Hardware AI Governance Lab), C. Withers (Center for a New American Security), H. Gietz (ERA Cambridge), B. Harack (Oxford Martin AI Governance Initiative; Oxford Hardware AI Governance Lab). Listed among the Hardware AI Governance Lab's publications (S-1703).", "organizations": [ "O-0209", "O-0205" ], "source_type": "tech-report", "source_url": "https://aigi.ox.ac.uk/publications/verifiable-semiconductor-manufacturing/", "source_file": "content/sources/ilhan-verifiable-semiconductor-manufacturing.yaml", "flags_all": [], "cited_by": [ { "id": "O-0209", "title": "Hardware AI Governance Lab", "url": "https://trustbutveri.fyi/organizations/oxford-hardware-ai-governance-lab/" } ], "referenced_by": [ { "id": "O-0209", "title": "Hardware AI Governance Lab", "url": "https://trustbutveri.fyi/organizations/oxford-hardware-ai-governance-lab/" } ] }, { "id": "S-1315", "slug": "immler-secure-physical-enclosures-covers", "status": "published", "risk_flags": [], "title": "Secure Physical Enclosures from Covers with Tamper-Resistance", "authors": [ "V. Immler", "J. Obermaier", "K. K. Ng", "F. X. Ke", "J. Lee", "Y. P. Lim", "W. K. Oh", "K. H. Wee", "G. Sigl" ], "year": 2019, "venue": "IACR Transactions on Cryptographic Hardware and Embedded Systems, 2019(1), 51–96", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/immler-secure-physical-enclosures-covers/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.13154/tches.v2019.i1.51-96", "arxiv": null, "organizations": [], "source_type": "peer-reviewed", "source_url": "https://tches.iacr.org/index.php/TCHES/article/view/7334", "source_file": "content/sources/immler-secure-physical-enclosures-covers.yaml", "flags_all": [], "cited_by": [ { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" } ], "referenced_by": [ { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" } ] }, { "id": "S-1706", "slug": "intelligence-security-laboratories-home", "status": "published", "risk_flags": [], "title": "Intelligence Security Laboratories: Building secure infrastructure for transformative AI", "authors": [], "year": 2026, "venue": "Intelligence Security Laboratories", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/intelligence-security-laboratories-home/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Homepage of Intelligence Security Laboratories (ISL); intseclab.org redirects here. States that ISL is a nonprofit research lab for high-security AI systems, its aim of developing and demonstrating the security that critical AI deployments need, its role as integrator of the secure data centre as a whole system, its use of STPA-Sec and formal methods, and its plan to hand knowledge and artifacts to others; refers to RAND's secure inference data center report (S-1510) for details of STPA-Sec. The footer gives 501(c)(3) nonprofit status. No page date. Self-description; supports \"ISL states\" statements.", "organizations": [ "O-0210" ], "source_type": "docs", "source_url": "https://intelligencesecuritylaboratories.org/", "source_file": "content/sources/intelligence-security-laboratories-home.yaml", "flags_all": [], "cited_by": [ { "id": "I-0010", "title": "RAND secure inference data center (SIDC) design", "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/" }, { "id": "O-0210", "title": "Intelligence Security Laboratories", "url": "https://trustbutveri.fyi/organizations/intelligence-security-laboratories/" } ], "referenced_by": [ { "id": "I-0010", "title": "RAND secure inference data center (SIDC) design", "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/" }, { "id": "O-0210", "title": "Intelligence Security Laboratories", "url": "https://trustbutveri.fyi/organizations/intelligence-security-laboratories/" } ] }, { "id": "S-1707", "slug": "intelligence-security-laboratories-team", "status": "published", "risk_flags": [], "title": "Our Team: Intelligence Security Laboratories", "authors": [], "year": 2026, "venue": "Intelligence Security Laboratories", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/intelligence-security-laboratories-team/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Team page. Lists Gabriel Kulp (Executive Director; before founding ISL, a fellow at RAND working on hardware-enabled governance mechanisms for GPU export controls and on international verification of agreements), Tom Gardiner (Staff Director) and Paul Murley (Technical Director; leads the development and infrastructure teams). No page date. Self-description.", "organizations": [ "O-0210" ], "source_type": "docs", "source_url": "https://intelligencesecuritylaboratories.org/team/", "source_file": "content/sources/intelligence-security-laboratories-team.yaml", "flags_all": [], "cited_by": [ { "id": "I-0010", "title": "RAND secure inference data center (SIDC) design", "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/" }, { "id": "O-0210", "title": "Intelligence Security Laboratories", "url": "https://trustbutveri.fyi/organizations/intelligence-security-laboratories/" } ], "referenced_by": [ { "id": "I-0010", "title": "RAND secure inference data center (SIDC) design", "url": "https://trustbutveri.fyi/implementations/rand-secure-inference-data-centers/" }, { "id": "O-0210", "title": "Intelligence Security Laboratories", "url": "https://trustbutveri.fyi/organizations/intelligence-security-laboratories/" } ] }, { "id": "S-1306", "slug": "ivanov-sage-gpu-attestation", "status": "published", "risk_flags": [], "title": "SAGE: Software-based Attestation for GPU Execution", "authors": [ "A. Ivanov", "B. Rothenberger", "A. Dethise", "M. Canini", "T. Hoefler", "A. Perrig" ], "year": 2023, "venue": "2023 USENIX Annual Technical Conference (USENIX ATC 23), pp. 485–499", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/ivanov-sage-gpu-attestation/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2209.03125", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://www.usenix.org/conference/atc23/presentation/ivanov", "source_file": "content/sources/ivanov-sage-gpu-attestation.yaml", "flags_all": [], "cited_by": [ { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" } ], "referenced_by": [ { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" } ] }, { "id": "S-1111", "slug": "jia-proof-of-learning-code", "status": "published", "risk_flags": [], "title": "Proof-of-Learning: code for Proof-of-Learning: Definitions and Practice", "authors": [ "CleverHans Lab" ], "year": 2021, "venue": "GitHub", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/jia-proof-of-learning-code/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Default branch at commit c286c7c0d2b45d80b4b6a7b0c2995690034927ff (19 October 2021), the latest commit at access (confirmed through the GitHub API by the verifier on 2026-09-23).", "organizations": [], "source_type": "code", "source_url": "https://github.com/cleverhans-lab/Proof-of-Learning", "source_file": "content/sources/jia-proof-of-learning-code.yaml", "flags_all": [], "cited_by": [ { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" } ], "referenced_by": [ { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" } ] }, { "id": "S-0028", "slug": "jia-proof-of-learning-definitions-practice", "status": "published", "risk_flags": [], "title": "Proof-of-Learning: Definitions and Practice", "authors": [ "H. Jia", "M. Yaghini", "C. A. Choquette-Choo", "N. Dullerud", "A. Thudi", "V. Chandrasekaran", "N. Papernot" ], "year": 2021, "venue": "2021 IEEE Symposium on Security and Privacy (SP), pp. 1039-1056", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/jia-proof-of-learning-definitions-practice/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.1109/SP40001.2021.00106", "arxiv": "2103.05633", "version_note": "Read the arXiv version (v1, 9 March 2021). DOI and pages from OpenAlex, checked 2026-09-25.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Proof of learning and training\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://ieeexplore.ieee.org/document/9519402/", "source_file": "content/sources/jia-proof-of-learning-definitions-practice.yaml", "flags_all": [], "cited_by": [ { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" } ], "referenced_by": [ { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" } ] }, { "id": "S-1500", "slug": "jin-proof-of-guardrail", "status": "published", "risk_flags": [], "title": "Proof-of-Guardrail in AI Agents and What (Not) to Trust from It", "authors": [ "X. Jin", "M. Duan", "Q. Lin", "A. Chan", "Z. Chen", "J. Du", "X. Ren" ], "year": 2026, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/jin-proof-of-guardrail/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2603.05786", "version_note": "arXiv v1 (2026-03-06) and v2 (2026-06-26), read via the arXiv HTML renderings on 2026-09-23 and 2026-09-25; the abs page did not render for the fetch tool. v2 keeps the figures the records cite (34% average latency overhead, Table 2 per-step overheads of 24.8–38.0%, F1 0.56 on the unsafe class).", "note": "The paper header names the Trustworthy AI for Good (AI4GOOD) workshop at ICML 2026, with the ICML template's \"PMLR 306\" line; PMLR volume 306 is set aside for the ICML 2026 main conference (github.com/mlresearch/v306), so the paper is recorded as a workshop paper and tier B preprint. Authors are affiliated with Sahara AI and the University of Southern California.", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2603.05786", "source_file": "content/sources/jin-proof-of-guardrail.yaml", "flags_all": [], "cited_by": [ { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" } ], "referenced_by": [ { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" } ] }, { "id": "S-1317", "slug": "johnston-physical-security-tamper-indicating-devices", "status": "published", "risk_flags": [], "title": "Physical Security and Tamper-Indicating Devices", "authors": [ "R. G. Johnston", "A. R. E. Garcia" ], "year": 1996, "venue": "Los Alamos National Laboratory, LA-UR-96-3827", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/johnston-physical-security-tamper-indicating-devices/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "organizations": [], "source_type": "tech-report", "source_url": "https://www.osti.gov/servlets/purl/459707", "source_file": "content/sources/johnston-physical-security-tamper-indicating-devices.yaml", "flags_all": [], "cited_by": [ { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" }, { "id": "K-0015", "title": "Tamper evidence and tamper resistance", "url": "https://trustbutveri.fyi/concepts/tamper-evidence/" } ], "referenced_by": [ { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" }, { "id": "K-0015", "title": "Tamper evidence and tamper resistance", "url": "https://trustbutveri.fyi/concepts/tamper-evidence/" } ] }, { "id": "S-1318", "slug": "johnston-tamper-detection-safeguards-treaty-monitoring", "status": "published", "risk_flags": [], "title": "Tamper Detection for Safeguards and Treaty Monitoring: Fantasies, Realities, and Potentials", "authors": [ "R. G. Johnston" ], "year": 2001, "venue": "The Nonproliferation Review, Spring 2001, pp. 102–114", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/johnston-tamper-detection-safeguards-treaty-monitoring/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "organizations": [], "source_type": "peer-reviewed", "source_url": "https://www.nonproliferation.org/wp-content/uploads/npr/81john.pdf", "source_file": "content/sources/johnston-tamper-detection-safeguards-treaty-monitoring.yaml", "flags_all": [], "cited_by": [ { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" } ], "referenced_by": [ { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" } ] }, { "id": "S-3566", "slug": "joshc-governments-slow-ai-training", "status": "published", "risk_flags": [], "title": "Can governments quickly and cheaply slow AI training?", "authors": [ "joshc" ], "year": 2026, "date": "2026-03-07", "venue": "AI Alignment Forum", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/joshc-governments-slow-ai-training/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Author's public analysis of inference-only verification limits, RL rollouts, work accounting and residual bandwidth.", "organizations": [], "source_type": "blog", "source_url": "https://www.alignmentforum.org/posts/Xzf3eMnhTko7AxnEy/can-governments-quickly-and-cheaply-slow-ai-training", "source_file": "content/sources/joshc-governments-slow-ai-training.yaml", "flags_all": [], "cited_by": [ { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "K-0025", "title": "Inference and training workloads", "url": "https://trustbutveri.fyi/concepts/inference-and-training-workloads/" } ], "referenced_by": [ { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "K-0025", "title": "Inference and training workloads", "url": "https://trustbutveri.fyi/concepts/inference-and-training-workloads/" } ] }, { "id": "S-0047", "slug": "joud-practical-side-channel-extraction", "status": "published", "risk_flags": [], "title": "A Practical Introduction to Side-Channel Extraction of Deep Neural Network Parameters", "authors": [ "R. Joud", "P.-A. Moëllic", "S. Pontié", "J.-B. Rigaud" ], "year": 2023, "venue": "21st International Conference on Smart Card Research and Advanced Applications (CARDIS 2022), LNCS 13820, pp. 45–65", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/joud-practical-side-channel-extraction/", "archive_url": null, "accessed": "2026-09-23", "doi": "10.1007/978-3-031-25319-5_3", "arxiv": "2211.05590", "version_note": "The CARDIS 2022 proceedings (LNCS 13820) were published in 2023 (online 29 January 2023); the year is the proceedings year. Venue, pages and DOI from the Springer chapter page, checked 2026-09-24.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Power telemetry and side-channel attacks\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://link.springer.com/chapter/10.1007/978-3-031-25319-5_3", "source_file": "content/sources/joud-practical-side-channel-extraction.yaml", "flags_all": [], "cited_by": [], "referenced_by": [] }, { "id": "S-1005", "slug": "karvonen-difr-code", "status": "published", "risk_flags": [], "title": "adamkarvonen/difr (GitHub repository)", "authors": [ "A. Karvonen" ], "year": 2025, "venue": "GitHub", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/karvonen-difr-code/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Pinned to commit eecde35 (eecde35f6458d76af86d68afac00594237162cf2, 8 Dec 2025), the head of the main branch per the GitHub API on 2026-09-23; the branch showed 21 commits and no release tags. Unchanged (21 commits, no tags) on 2026-09-25.", "license_note": "MIT licence, per the repository.", "organizations": [], "source_type": "code", "source_url": "https://github.com/adamkarvonen/difr", "source_file": "content/sources/karvonen-difr-code.yaml", "flags_all": [], "cited_by": [ { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" } ], "referenced_by": [ { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" } ] }, { "id": "S-0016", "slug": "karvonen-difr", "status": "published", "risk_flags": [], "title": "DiFR: Inference Verification Despite Nondeterminism", "authors": [ "A. Karvonen", "D. Reuter", "R. Rinberg", "L. Marks", "A. Garriga-Alonso", "K. Warr" ], "year": 2025, "venue": "ICML 2026 Workshop on Technical AI Governance Research", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/karvonen-difr/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2511.20621", "arxiv": "2511.20621", "version_note": "arXiv v1 (25 Nov 2025) is the only version as of 2026-09-25. The paper was a poster at the Second Workshop on Technical AI Governance Research at ICML 2026 (https://icml.cc/virtual/2026/78608). Checked 2026-09-25.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Inference verification\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2511.20621", "source_file": "content/sources/karvonen-difr.yaml", "flags_all": [], "cited_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0016", "title": "Batch-invariant inference kernels (Thinking Machines)", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" }, { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "K-0008", "title": "Numerical nondeterminism", "url": "https://trustbutveri.fyi/concepts/numerical-nondeterminism/" }, { "id": "K-0009", "title": "Recomputation", "url": "https://trustbutveri.fyi/concepts/recomputation/" } ], "referenced_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0016", "title": "Batch-invariant inference kernels (Thinking Machines)", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" }, { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "K-0008", "title": "Numerical nondeterminism", "url": "https://trustbutveri.fyi/concepts/numerical-nondeterminism/" }, { "id": "K-0009", "title": "Recomputation", "url": "https://trustbutveri.fyi/concepts/recomputation/" } ] }, { "id": "S-1507", "slug": "kezins-adversarial-entropy-inflation", "status": "published", "risk_flags": [], "title": "Adversarial Entropy Inflation Against Gumbel-Based Inference Verification", "authors": [ "N. Kezins" ], "year": 2026, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/kezins-adversarial-entropy-inflation/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2608.23375", "version_note": "Read via the arXiv HTML rendering, which is dated 24 Aug 2026 (v1). The abs page did not render for the fetch tool.", "note": "Independent attack paper (author affiliation Delft University of Technology) on the verification scheme of S-0015.", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2608.23375", "source_file": "content/sources/kezins-adversarial-entropy-inflation.yaml", "flags_all": [], "cited_by": [ { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" } ], "referenced_by": [ { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" } ] }, { "id": "S-1609", "slug": "komargodski-proofs-useful-work-matrix-multiplication", "status": "published", "risk_flags": [], "title": "Proofs of Useful Work from Arbitrary Matrix Multiplication", "authors": [ "I. Komargodski", "O. Weinstein" ], "year": 2025, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/komargodski-proofs-useful-work-matrix-multiplication/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2504.09971", "version_note": "v1 14 April 2025; v4 13 November 2025. Also IACR ePrint 2025/685 (last revised 8 December 2025), listed there as a preprint. v3 listed I. Schen as a third author; v4 and the ePrint version list two. Remark, assumption and open-problem numbers follow the ePrint PDF.", "note": "The construction underlying Pearl (I-0004). Pearl Research Labs lists the paper among its protocol research, and Komargodski gives Pearl Research Labs as his affiliation on S-1010. The paper itself names no company.", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2504.09971", "source_file": "content/sources/komargodski-proofs-useful-work-matrix-multiplication.yaml", "flags_all": [], "cited_by": [ { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" }, { "id": "K-0011", "title": "Proof of (useful) work", "url": "https://trustbutveri.fyi/concepts/proof-of-useful-work/" }, { "id": "O-0121", "title": "Pearl Research Labs", "url": "https://trustbutveri.fyi/organizations/pearl-research/" } ], "referenced_by": [ { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" }, { "id": "K-0011", "title": "Proof of (useful) work", "url": "https://trustbutveri.fyi/concepts/proof-of-useful-work/" }, { "id": "O-0121", "title": "Pearl Research Labs", "url": "https://trustbutveri.fyi/organizations/pearl-research/" } ] }, { "id": "S-1409", "slug": "krawec-tracking-hyperscale-satellite-imagery", "status": "published", "risk_flags": [], "title": "Tracking Hyperscale AI Data Center Growth with Satellite Imagery", "authors": [ "C. Krawec" ], "year": 2026, "date": "2026-05-12", "venue": "Federation of American Scientists", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/krawec-tracking-hyperscale-satellite-imagery/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "organizations": [], "source_type": "tech-report", "source_url": "https://fas.org/publication/tracking-hyperscale/", "source_file": "content/sources/krawec-tracking-hyperscale-satellite-imagery.yaml", "flags_all": [], "cited_by": [ { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "O-0208", "title": "Epoch AI", "url": "https://trustbutveri.fyi/organizations/epoch-ai/" }, { "id": "O-0102", "title": "Planet Labs", "url": "https://trustbutveri.fyi/organizations/planet-labs/" } ], "referenced_by": [ { "id": "M-0020", "title": "Remote detection of data centres", "url": "https://trustbutveri.fyi/mechanisms/remote-detection-of-data-centres/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "O-0208", "title": "Epoch AI", "url": "https://trustbutveri.fyi/organizations/epoch-ai/" }, { "id": "O-0102", "title": "Planet Labs", "url": "https://trustbutveri.fyi/organizations/planet-labs/" } ] }, { "id": "S-0057", "slug": "kulp-hardware-enabled-governance-mechanisms", "status": "published", "risk_flags": [], "title": "Hardware-Enabled Governance Mechanisms: Developing Technical Solutions to Exempt Items Otherwise Classified Under Export Control Classification Numbers 3A090 and 4A090", "authors": [ "G. Kulp", "D. Gonzales", "E. Smith", "L. Heim", "P. Puri", "M. J. D. Vermeer", "Z. Winkelman" ], "year": 2024, "venue": "RAND Corporation", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/kulp-hardware-enabled-governance-mechanisms/", "archive_url": "https://web.archive.org/web/20260904141221/https://www.rand.org/pubs/working_papers/WRA3056-1.html", "accessed": "2026-09-25", "doi": "10.7249/WRA3056-1", "arxiv": null, "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Motivations and policy proposals\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [ "O-0200" ], "source_type": "tech-report", "source_url": "https://www.rand.org/pubs/working_papers/WRA3056-1.html", "source_file": "content/sources/kulp-hardware-enabled-governance-mechanisms.yaml", "flags_all": [], "cited_by": [ { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0001", "title": "Compute stock is at most a declared amount", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0017", "title": "Compartmentalization", "url": "https://trustbutveri.fyi/concepts/compartmentalization/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" }, { "id": "K-0007", "title": "Hardware-enabled mechanism (HEM)", "url": "https://trustbutveri.fyi/concepts/hardware-enabled-mechanism/" }, { "id": "O-0200", "title": "RAND", "url": "https://trustbutveri.fyi/organizations/rand/" } ], "referenced_by": [ { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0001", "title": "Compute stock is at most a declared amount", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0017", "title": "Compartmentalization", "url": "https://trustbutveri.fyi/concepts/compartmentalization/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" }, { "id": "K-0007", "title": "Hardware-enabled mechanism (HEM)", "url": "https://trustbutveri.fyi/concepts/hardware-enabled-mechanism/" }, { "id": "O-0200", "title": "RAND", "url": "https://trustbutveri.fyi/organizations/rand/" } ] }, { "id": "S-1808", "slug": "lagrange-deep-prove-code", "status": "published", "risk_flags": [], "title": "Lagrange-Labs/deep-prove (GitHub repository)", "authors": [ "Lagrange Labs" ], "year": 2026, "venue": "GitHub", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/lagrange-deep-prove-code/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Main branch as viewed on 2026-09-25 (1,219 commits, no releases). Licensed under the Lagrange License.", "note": "Developer's repository. README reports end-to-end proofs of GPT-2, Gemma 3 and Llama 2 inference with sumcheck and logup GKR, and on a 24-core, 504 GB CPU server: GPT-2 at 512 tokens in 7.6 min (verify 1.3 s, 10.7 MiB proof) and Gemma 3 at 512 tokens in 19 min; at least 99.6% cosine similarity to floating point at 12-bit quantization (GPT-2).", "organizations": [], "source_type": "code", "source_url": "https://github.com/Lagrange-Labs/deep-prove", "source_file": "content/sources/lagrange-deep-prove-code.yaml", "flags_all": [], "cited_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" } ], "referenced_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" } ] }, { "id": "S-1807", "slug": "lagrange-deepprove-1", "status": "published", "risk_flags": [], "title": "DeepProve-1: The First zkML System to Prove a Full LLM Inference", "authors": [ "Lagrange Labs" ], "year": 2025, "date": "2025-08-18", "venue": "Lagrange blog", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/lagrange-deepprove-1/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Developer's announcement of a zero-knowledge proof of full GPT-2 inference, with GGUF model import, sumcheck techniques and a Basefold commitment that the post says produces large proofs and slow verification. Gives no timings. Supports only \"Lagrange reports\" statements.", "organizations": [], "source_type": "blog", "source_url": "https://lagrange.dev/blog/deepprove-1", "source_file": "content/sources/lagrange-deepprove-1.yaml", "flags_all": [], "cited_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" } ], "referenced_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" } ] }, { "id": "S-0041", "slug": "latif-empirical-ai-training-power-demand", "status": "published", "risk_flags": [], "title": "Single-Node Power Demand During AI Training: Measurements on an 8-GPU NVIDIA H100 System", "authors": [ "I. Latif", "A. C. Newkirk", "M. R. Carbone", "A. Munir", "Y. Lin", "J. Koomey", "X. Yu", "Z. Dong" ], "year": 2025, "venue": "IEEE Access, vol. 13, pp. 61740–61747", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/latif-empirical-ai-training-power-demand/", "archive_url": null, "accessed": "2026-09-23", "doi": "10.1109/ACCESS.2025.3554728", "arxiv": "2412.08602", "version_note": "Published in IEEE Access under a new title. The arXiv preprint (v1 11 December 2024, v2 20 December 2024) is titled \"Empirical Measurements of AI Training Power Demand on a GPU-Accelerated Node\". The published abstract, checked 2026-09-24, states the 8.4 kW peak and the 18% gap to the rated 10.2 kW that citing records use.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Power telemetry and side-channel attacks\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://doi.org/10.1109/ACCESS.2025.3554728", "source_file": "content/sources/latif-empirical-ai-training-power-demand.yaml", "flags_all": [], "cited_by": [ { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" } ], "referenced_by": [ { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" } ] }, { "id": "S-1502", "slug": "levin-prompt-detective", "status": "published", "risk_flags": [], "title": "Has My System Prompt Been Used? Large Language Model Prompt Membership Inference", "authors": [ "R. Levin", "V. Cherepanova", "A. Hans", "A. Schwarzschild", "T. Goldstein" ], "year": 2025, "date": "2025-02-14", "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/levin-prompt-detective/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2502.09974", "version_note": "Abstract read on the arXiv abs page; method and setup read via the v1 HTML rendering. Only v1 exists on arXiv (checked 2026-09-25). ML Anthology lists the paper under the ICLR 2025 BuildingTrust workshop (https://mlanthology.org/iclrw/2025/levin2025iclrw-my/); a workshop paper, so tier B is kept.", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2502.09974", "source_file": "content/sources/levin-prompt-detective.yaml", "flags_all": [], "cited_by": [ { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" } ], "referenced_by": [ { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" } ] }, { "id": "S-3080", "slug": "liao-verilora-zk-lora-fine-tuning", "status": "published", "risk_flags": [], "title": "VeriLoRA: Fine-Tuning Large Language Models with Verifiable Security via Zero-Knowledge Proofs", "authors": [ "G. Liao", "T. Wang", "S. Zhang", "J. Zhang", "L. Shi", "D. Tao" ], "year": 2025, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/liao-verilora-zk-lora-fine-tuning/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2508.21393", "arxiv": "2508.21393", "version_note": "First posted 29 August 2025; arXiv v3 dated 2 December 2025. The abstract and HTML, read on 2026-09-25, use the title VeriLoRA and state that the paper was accepted for publication at NDSS 2026.", "note": "Proves one LoRA fine-tuning iteration (forward pass, backward pass, parameter update) on a single-sample minibatch for LLaMA-3.2 3B and 11B, LLaMA-2 7B and 13B and OPT 6.7B and 13B on one NVIDIA A100 80 GB GPU. The full text read reports 121.93–249.38 s of proving, 156–554 s of commitment generation and 1.87–3.73 s of verification per step. Code linked from the paper at https://github.com/liaoguofu/zkLoRA (MIT licence, README titled VeriLoRA, built on the zkLLM code base); commit not pinned.", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2508.21393", "source_file": "content/sources/liao-verilora-zk-lora-fine-tuning.yaml", "flags_all": [], "cited_by": [ { "id": "M-0005", "title": "Zero-knowledge proofs of training constraints", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-training-constraints/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" } ], "referenced_by": [ { "id": "M-0005", "title": "Zero-knowledge proofs of training constraints", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-training-constraints/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" } ] }, { "id": "S-3363", "slug": "lucid-adversarial-testbed-europe", "status": "published", "risk_flags": [], "title": "Building an Adversarial Testbed for AI Verification in Europe", "authors": [ "Lucid Computing" ], "year": 2026, "date": "2026-07-29", "venue": "Lucid Computing (Substack)", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/lucid-adversarial-testbed-europe/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Lucid's announcement of an adversarial testbed it is \"building and operating\" inside an unnamed \"government-funded research institute that runs clusters for domestic workloads\". The host institute \"holds technical authority over adversarial testing\" and sets the test plan. The first configuration retrofits two eight-GPU H100 servers against three claims (where compute runs, how it is used, what model is loaded), with an 18-month goal. Supports only \"Lucid reports\" statements.", "organizations": [ "O-0180" ], "source_type": "blog", "source_url": "https://lucidcomputing.substack.com/p/building-an-adversarial-testbed-for", "source_file": "content/sources/lucid-adversarial-testbed-europe.yaml", "flags_all": [], "cited_by": [ { "id": "O-0180", "title": "Lucid Computing", "url": "https://trustbutveri.fyi/organizations/lucid-computing/" } ], "referenced_by": [ { "id": "O-0180", "title": "Lucid Computing", "url": "https://trustbutveri.fyi/organizations/lucid-computing/" } ] }, { "id": "S-1406", "slug": "lucid-computing-homepage", "status": "published", "risk_flags": [], "title": "Lucid Computing: Verifiable AI. Proven in hardware.", "authors": [], "year": 2026, "venue": "Lucid Computing", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/lucid-computing-homepage/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Company homepage, read 2026-09-23 and 2026-09-25. The title above is the hero text; the HTML page title is \"Lucid Computing - Provable trust infrastructure for frontier AI\". Supports \"Lucid reports\" statements only.", "organizations": [ "O-0180" ], "source_type": "docs", "source_url": "https://lucidcomputing.ai/", "source_file": "content/sources/lucid-computing-homepage.yaml", "flags_all": [], "cited_by": [ { "id": "I-0009", "title": "Lucid sovereignty (location) certificates", "url": "https://trustbutveri.fyi/implementations/lucid-location-certificates/" }, { "id": "O-0180", "title": "Lucid Computing", "url": "https://trustbutveri.fyi/organizations/lucid-computing/" }, { "id": "O-0203", "title": "Verifiable Compute Foundation", "url": "https://trustbutveri.fyi/organizations/verifiable-compute-foundation/" } ], "referenced_by": [ { "id": "I-0009", "title": "Lucid sovereignty (location) certificates", "url": "https://trustbutveri.fyi/implementations/lucid-location-certificates/" }, { "id": "O-0180", "title": "Lucid Computing", "url": "https://trustbutveri.fyi/organizations/lucid-computing/" }, { "id": "O-0203", "title": "Verifiable Compute Foundation", "url": "https://trustbutveri.fyi/organizations/verifiable-compute-foundation/" } ] }, { "id": "S-1407", "slug": "lucid-developer-platform-docs", "status": "published", "risk_flags": [], "title": "Lucid Developer Platform documentation", "authors": [], "year": 2026, "venue": "Lucid Computing", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/lucid-developer-platform-docs/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Pages read on 2026-09-23 include the index, concepts/architecture.html, concepts/glossary.html and reference/auditor-catalog.html; the auditor catalog and architecture pages were re-read on 2026-09-25. Supports \"Lucid reports\" statements only.", "organizations": [ "O-0180" ], "source_type": "docs", "source_url": "https://docs.lucidcomputing.ai/", "source_file": "content/sources/lucid-developer-platform-docs.yaml", "flags_all": [], "cited_by": [ { "id": "I-0009", "title": "Lucid sovereignty (location) certificates", "url": "https://trustbutveri.fyi/implementations/lucid-location-certificates/" }, { "id": "O-0180", "title": "Lucid Computing", "url": "https://trustbutveri.fyi/organizations/lucid-computing/" } ], "referenced_by": [ { "id": "I-0009", "title": "Lucid sovereignty (location) certificates", "url": "https://trustbutveri.fyi/implementations/lucid-location-certificates/" }, { "id": "O-0180", "title": "Lucid Computing", "url": "https://trustbutveri.fyi/organizations/lucid-computing/" } ] }, { "id": "S-1702", "slug": "lucid-experimentation-cluster", "status": "published", "risk_flags": [], "title": "Experiments: Lucid Labs", "authors": [], "year": 2026, "venue": "Lucid Computing", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/lucid-experimentation-cluster/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Lucid Computing's page on the bare-metal research cluster that it says the Verifiable Compute Foundation provides and allocates: accelerators, research \"layers\", and access terms (\"Free for qualifying safety & verification research\"; Lucid \"takes no part in allocation decisions\"). The page says the cluster opens to the community in November, without a year (2026 by context). No page date; copyright 2026. Supports \"Lucid reports\" statements only.", "organizations": [ "O-0180" ], "source_type": "docs", "source_url": "https://lucidcomputing.ai/experimentation-cluster", "source_file": "content/sources/lucid-experimentation-cluster.yaml", "flags_all": [], "cited_by": [ { "id": "O-0180", "title": "Lucid Computing", "url": "https://trustbutveri.fyi/organizations/lucid-computing/" }, { "id": "O-0203", "title": "Verifiable Compute Foundation", "url": "https://trustbutveri.fyi/organizations/verifiable-compute-foundation/" } ], "referenced_by": [ { "id": "O-0180", "title": "Lucid Computing", "url": "https://trustbutveri.fyi/organizations/lucid-computing/" }, { "id": "O-0203", "title": "Verifiable Compute Foundation", "url": "https://trustbutveri.fyi/organizations/verifiable-compute-foundation/" } ] }, { "id": "S-1701", "slug": "lucid-labs-overview", "status": "published", "risk_flags": [], "title": "Lucid Labs: the verification flywheel", "authors": [], "year": 2026, "venue": "Lucid Computing", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/lucid-labs-overview/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Lucid Computing's page on its research programme: an experimentation cluster provided by the Verifiable Compute Foundation, which it describes as \"an independent nonprofit working to create a toolbox of AI verification techniques\"; red-teaming with national security agencies and government research institutes; deployment; and standards work. No page date; copyright 2026. Supports \"Lucid reports\" statements only.", "organizations": [ "O-0180" ], "source_type": "docs", "source_url": "https://lucidcomputing.ai/overview", "source_file": "content/sources/lucid-labs-overview.yaml", "flags_all": [], "cited_by": [ { "id": "O-0180", "title": "Lucid Computing", "url": "https://trustbutveri.fyi/organizations/lucid-computing/" }, { "id": "O-0203", "title": "Verifiable Compute Foundation", "url": "https://trustbutveri.fyi/organizations/verifiable-compute-foundation/" } ], "referenced_by": [ { "id": "O-0180", "title": "Lucid Computing", "url": "https://trustbutveri.fyi/organizations/lucid-computing/" }, { "id": "O-0203", "title": "Verifiable Compute Foundation", "url": "https://trustbutveri.fyi/organizations/verifiable-compute-foundation/" } ] }, { "id": "S-1301", "slug": "lucid-traffic-shaping-workload-classification", "status": "published", "risk_flags": [], "title": "Traffic Shaping for Workload Classification", "authors": [ "Lucid Computing" ], "year": 2026, "date": "2026-08-04", "venue": "Lucid Computing (Substack)", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/lucid-traffic-shaping-workload-classification/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Design brief; the post states the design is not yet implemented or red-teamed.", "organizations": [ "O-0180" ], "source_type": "blog", "source_url": "https://lucidcomputing.substack.com/p/traffic-shaping-for-workload-classification", "source_file": "content/sources/lucid-traffic-shaping-workload-classification.yaml", "flags_all": [], "cited_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "O-0180", "title": "Lucid Computing", "url": "https://trustbutveri.fyi/organizations/lucid-computing/" } ], "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "O-0180", "title": "Lucid Computing", "url": "https://trustbutveri.fyi/organizations/lucid-computing/" } ] }, { "id": "S-0036", "slug": "ma-hardware-mechanisms-dynamically-throttle-ai", "status": "published", "risk_flags": [], "title": "Hardware Mechanisms to Dynamically Throttle AI Performance", "authors": [ "H. Ma", "J. Forzani", "L. Malek", "D. Wentzlaff" ], "year": 2026, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/ma-hardware-mechanisms-dynamically-throttle-ai/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2607.18069", "arxiv": "2607.18069", "version_note": "Author order follows the current arXiv HTML version (arxiv.org/html/2607.18069), read 2026-09-23: Ma, Forzani, Malek, Wentzlaff, and it was unchanged on 2026-09-25. The v1 HTML (arxiv.org/html/2607.18069v1) lists Malek before Forzani, as the bibliography did.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Network and memory telemetry\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2607.18069", "source_file": "content/sources/ma-hardware-mechanisms-dynamically-throttle-ai.yaml", "flags_all": [], "cited_by": [ { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" } ], "referenced_by": [ { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" } ] }, { "id": "S-3120", "slug": "meta-whatsapp-private-processing-whitepaper", "status": "published", "risk_flags": [], "title": "Private Processing for WhatsApp: Technical White Paper and Security Guide", "authors": [ "Meta" ], "year": 2026, "date": "2026-03-16", "venue": "Meta", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/meta-whatsapp-private-processing-whitepaper/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Version 2, updated 2026-03-16; version 1 was published 2025-06-10.", "note": "Meta's description of its own system; supports only \"Meta reports\" statements. Private Processing runs WhatsApp's AI requests in confidential VMs on AMD EPYC CPUs with SEV-SNP and NVIDIA Hopper GPUs. The client checks the attestation and transparency proofs, including entries in a third-party transparency log run with Cloudflare, before it connects. The threat model lists physical host attacks, with data-centre security, memory encryption and third-party relays as mitigations.", "organizations": [], "source_type": "docs", "source_url": "https://ai.meta.com/static-resource/private-processing-technical-whitepaper", "source_file": "content/sources/meta-whatsapp-private-processing-whitepaper.yaml", "flags_all": [], "cited_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ], "referenced_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ] }, { "id": "S-1512", "slug": "milton-verifying-international-ai-deals", "status": "published", "risk_flags": [], "title": "Verifying international AI deals: Plan A, the state-of-play, and what you can do to help", "authors": [ "T. Milton", "S. Reynolds", "C. Jacobi", "J. Foster" ], "year": 2026, "date": "2026-07-09", "venue": "Amodo (Substack)", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/milton-verifying-international-ai-deals/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Byline shown as \"Amodo, Thomas Milton, Sam Reynolds, and 2 others\"; the post names Connagh Jacobi and Joel Foster as the other contributors. A search-engine listing gave a slightly longer title that includes \"AI 2040\"; the title here is the post's H1 as fetched.", "organizations": [ "O-0101" ], "source_type": "blog", "source_url": "https://amodo.substack.com/p/verifying-international-ai-deals", "source_file": "content/sources/milton-verifying-international-ai-deals.yaml", "flags_all": [], "cited_by": [ { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ], "referenced_by": [ { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "O-0101", "title": "Amodo Design", "url": "https://trustbutveri.fyi/organizations/amodo-design/" } ] }, { "id": "S-3609", "slug": "miri-about", "status": "published", "risk_flags": [], "title": "About MIRI", "authors": [], "year": 2026, "venue": "Machine Intelligence Research Institute", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/miri-about/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Official organization page, read 2026-09-25; the year is the access year because the page is undated. Supports the organization's attributed self-description: nonprofit focus on preventing human extinction from artificial superintelligence.", "organizations": [ "O-0202" ], "source_type": "docs", "source_url": "https://intelligence.org/about/", "source_file": "content/sources/miri-about.yaml", "flags_all": [], "cited_by": [ { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" } ], "referenced_by": [ { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" } ] }, { "id": "S-3380", "slug": "miri-tgt-2026-icml-papers", "status": "published", "risk_flags": [], "title": "Summary: TGT's 2026 ICML Papers", "authors": [ "Machine Intelligence Research Institute" ], "year": 2026, "date": "2026-06-30", "venue": "Machine Intelligence Research Institute", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/miri-tgt-2026-icml-papers/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "MIRI post summarising six papers by its Technical Governance Team (TGT) at the second Workshop on Technical AI Governance Research (TAIGR) at ICML 2026, including S-0020 (marked as the best-paper award winner), S-0037, S-0060 and S-1300. The date is taken from the URL; the page as fetched showed no byline. Self-description; supports \"MIRI reports\" statements.", "organizations": [ "O-0202" ], "source_type": "blog", "source_url": "https://intelligence.org/2026/06/30/summary-tgts-2026-icml-papers/", "source_file": "content/sources/miri-tgt-2026-icml-papers.yaml", "flags_all": [], "cited_by": [ { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" } ], "referenced_by": [ { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" } ] }, { "id": "S-0033", "slug": "monfared-timing-memory-telemetry-gpus", "status": "published", "risk_flags": [], "title": "Timing and Memory Telemetry on GPUs for AI Governance", "authors": [ "S. K. Monfared", "F. Ganji", "D. E. Holcomb", "S. Tajik" ], "year": 2026, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/monfared-timing-memory-telemetry-gpus/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2602.09369", "arxiv": "2602.09369", "version_note": "v2 (2026-02-12) read on 2026-09-25; no later version or peer-reviewed publication found.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Network and memory telemetry\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2602.09369", "source_file": "content/sources/monfared-timing-memory-telemetry-gpus.yaml", "flags_all": [], "cited_by": [ { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" } ], "referenced_by": [ { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" } ] }, { "id": "S-0051", "slug": "mosavirik-impedanceverif", "status": "published", "risk_flags": [], "title": "ImpedanceVerif: On-Chip Impedance Sensing for System-Level Tampering Detection", "authors": [ "T. Mosavirik", "P. Schaumont", "S. Tajik" ], "year": 2023, "venue": "IACR Transactions on Cryptographic Hardware and Embedded Systems, 2023(1), 301–325", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/mosavirik-impedanceverif/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.46586/tches.v2023.i1.301-325", "arxiv": null, "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Tamper resistance and detection\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://eprint.iacr.org/2022/946", "source_file": "content/sources/mosavirik-impedanceverif.yaml", "flags_all": [], "cited_by": [ { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" } ], "referenced_by": [ { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" } ] }, { "id": "S-1611", "slug": "nassernia-cuda-mps-gpu-memory-performance", "status": "published", "risk_flags": [], "title": "Boost GPU Memory Performance with No Code Changes Using NVIDIA CUDA MPS", "authors": [ "S. Nassernia" ], "year": 2025, "date": "2025-12-16", "venue": "NVIDIA Technical Blog", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/nassernia-cuda-mps-gpu-memory-performance/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "NVIDIA technical blog post about MLOPart partitioning in CUDA 13.1; supports only \"NVIDIA states\" statements about its own hardware. Cited for its statement that the Blackwell GPUs in an NVIDIA HGX B200 system \"normally have 148 SMs\".", "organizations": [ "O-0140" ], "source_type": "blog", "source_url": "https://developer.nvidia.com/blog/boost-gpu-memory-performance-with-no-code-changes-using-nvidia-cuda-mps/", "source_file": "content/sources/nassernia-cuda-mps-gpu-memory-performance.yaml", "flags_all": [], "cited_by": [ { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" } ], "referenced_by": [ { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" } ] }, { "id": "S-1610", "slug": "nevo-securing-ai-model-weights", "status": "published", "risk_flags": [], "title": "Securing AI Model Weights: Preventing Theft and Misuse of Frontier Models", "authors": [ "S. Nevo", "D. Lahav", "A. Karpur", "Y. Bar-On", "H. A. Bradley", "J. Alstott" ], "year": 2024, "date": "2024-05-30", "venue": "RAND Corporation", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/nevo-securing-ai-model-weights/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.7249/RRA2849-1", "arxiv": null, "organizations": [ "O-0200" ], "source_type": "tech-report", "source_url": "https://www.rand.org/pubs/research_reports/RRA2849-1.html", "source_file": "content/sources/nevo-securing-ai-model-weights.yaml", "flags_all": [], "cited_by": [ { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0022", "title": "Weight exfiltration", "url": "https://trustbutveri.fyi/concepts/weight-exfiltration/" }, { "id": "O-0200", "title": "RAND", "url": "https://trustbutveri.fyi/organizations/rand/" } ], "referenced_by": [ { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0022", "title": "Weight exfiltration", "url": "https://trustbutveri.fyi/concepts/weight-exfiltration/" }, { "id": "O-0200", "title": "RAND", "url": "https://trustbutveri.fyi/organizations/rand/" } ] }, { "id": "S-3260", "slug": "nist-cmvp-certificate-1505-ibm-4765", "status": "published", "risk_flags": [], "title": "Cryptographic Module Validation Program Certificate #1505: IBM 4765 Cryptographic Coprocessor Security Module", "authors": [ "National Institute of Standards and Technology" ], "year": 2011, "date": "2011-02-24", "venue": "NIST Cryptographic Module Validation Program", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/nist-cmvp-certificate-1505-ibm-4765/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Living certificate page. Read on 2026-09-25: initial validation 2011-02-24, last update 2012-12-21, overall Level 4, status Historical (reason given: RNG transition under SP 800-131A Rev. 1), so the module should not be included by federal agencies in new procurements.", "organizations": [], "source_type": "gov-doc", "source_url": "https://csrc.nist.gov/projects/cryptographic-module-validation-program/certificate/1505", "source_file": "content/sources/nist-cmvp-certificate-1505-ibm-4765.yaml", "flags_all": [], "cited_by": [ { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" } ], "referenced_by": [ { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" } ] }, { "id": "S-1600", "slug": "nist-csrc-glossary", "status": "published", "risk_flags": [], "title": "NIST Computer Security Resource Center (CSRC) Glossary", "authors": [ "National Institute of Standards and Technology" ], "year": 2026, "venue": "NIST Computer Security Resource Center", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/nist-csrc-glossary/", "archive_url": null, "accessed": "2026-09-23", "doi": null, "arxiv": null, "version_note": "Living glossary. Each entry reproduces definitions from named NIST publications; locators in citing records give the term page and the underlying publication (e.g. NISTIR 8320, NIST SP 800-63-4). Entries read on 2026-09-23: roots_of_trust, trusted_execution_environment, attestation, side_channel_attack, tamper_evident, tamper_resistant, threat_modeling, zero_knowledge_proof, verifier, hash_function.", "organizations": [], "source_type": "gov-doc", "source_url": "https://csrc.nist.gov/glossary", "source_file": "content/sources/nist-csrc-glossary.yaml", "flags_all": [], "cited_by": [ { "id": "K-0024", "title": "Cryptographic commitment", "url": "https://trustbutveri.fyi/concepts/cryptographic-commitment/" }, { "id": "K-0004", "title": "Remote attestation", "url": "https://trustbutveri.fyi/concepts/remote-attestation/" }, { "id": "K-0005", "title": "Root of trust", "url": "https://trustbutveri.fyi/concepts/root-of-trust/" }, { "id": "K-0013", "title": "Side channel", "url": "https://trustbutveri.fyi/concepts/side-channel/" }, { "id": "K-0015", "title": "Tamper evidence and tamper resistance", "url": "https://trustbutveri.fyi/concepts/tamper-evidence/" }, { "id": "K-0018", "title": "Threat model", "url": "https://trustbutveri.fyi/concepts/threat-model/" }, { "id": "K-0006", "title": "Trusted execution environment (TEE)", "url": "https://trustbutveri.fyi/concepts/trusted-execution-environment/" }, { "id": "K-0010", "title": "Zero-knowledge proof", "url": "https://trustbutveri.fyi/concepts/zero-knowledge-proof/" } ], "referenced_by": [ { "id": "K-0024", "title": "Cryptographic commitment", "url": "https://trustbutveri.fyi/concepts/cryptographic-commitment/" }, { "id": "K-0004", "title": "Remote attestation", "url": "https://trustbutveri.fyi/concepts/remote-attestation/" }, { "id": "K-0005", "title": "Root of trust", "url": "https://trustbutveri.fyi/concepts/root-of-trust/" }, { "id": "K-0013", "title": "Side channel", "url": "https://trustbutveri.fyi/concepts/side-channel/" }, { "id": "K-0015", "title": "Tamper evidence and tamper resistance", "url": "https://trustbutveri.fyi/concepts/tamper-evidence/" }, { "id": "K-0018", "title": "Threat model", "url": "https://trustbutveri.fyi/concepts/threat-model/" }, { "id": "K-0006", "title": "Trusted execution environment (TEE)", "url": "https://trustbutveri.fyi/concepts/trusted-execution-environment/" }, { "id": "K-0010", "title": "Zero-knowledge proof", "url": "https://trustbutveri.fyi/concepts/zero-knowledge-proof/" } ] }, { "id": "S-1605", "slug": "nist-fips-140-2", "status": "published", "risk_flags": [], "title": "Security Requirements for Cryptographic Modules (FIPS PUB 140-2)", "authors": [ "National Institute of Standards and Technology" ], "year": 2001, "date": "2001-05-25", "venue": "National Institute of Standards and Technology", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/nist-fips-140-2/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Published 25 May 2001 (Change Notice 2, 3 December 2002). The CSRC page lists it as superseded by FIPS 140-3. Full text read at https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.140-2.pdf", "organizations": [], "source_type": "standard", "source_url": "https://csrc.nist.gov/pubs/fips/140-2/upd2/final", "source_file": "content/sources/nist-fips-140-2.yaml", "flags_all": [], "cited_by": [ { "id": "K-0015", "title": "Tamper evidence and tamper resistance", "url": "https://trustbutveri.fyi/concepts/tamper-evidence/" } ], "referenced_by": [ { "id": "K-0015", "title": "Tamper evidence and tamper resistance", "url": "https://trustbutveri.fyi/concepts/tamper-evidence/" } ] }, { "id": "S-3564", "slug": "nist-openmined-crada-2026", "status": "published", "risk_flags": [], "title": "Announcement: CAISI signs CRADA with OpenMined to Enable Secure AI Evaluations", "authors": [ "National Institute of Standards and Technology" ], "year": 2026, "date": "2026-03-27", "venue": "National Institute of Standards and Technology", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/nist-openmined-crada-2026/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "NIST describes OpenMined as a nonprofit developing open-source software for secure computation across organizations and names PySyft in a research agreement on confidential AI evaluations.", "organizations": [], "source_type": "gov-doc", "source_url": "https://www.nist.gov/news-events/news/2026/03/announcement-caisi-signs-crada-openmined-enable-secure-ai-evaluations", "source_file": "content/sources/nist-openmined-crada-2026.yaml", "flags_all": [], "cited_by": [ { "id": "O-0215", "title": "OpenMined", "url": "https://trustbutveri.fyi/organizations/openmined/" } ], "referenced_by": [ { "id": "O-0215", "title": "OpenMined", "url": "https://trustbutveri.fyi/organizations/openmined/" } ] }, { "id": "S-1601", "slug": "nist-sp-800-193-platform-firmware-resiliency", "status": "published", "risk_flags": [], "title": "Platform Firmware Resiliency Guidelines (NIST SP 800-193)", "authors": [ "A. Regenscheid" ], "year": 2018, "date": "2018-05-04", "venue": "National Institute of Standards and Technology", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/nist-sp-800-193-platform-firmware-resiliency/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Final publication. Full text read at https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-193.pdf", "organizations": [], "source_type": "gov-doc", "source_url": "https://csrc.nist.gov/pubs/sp/800/193/final", "source_file": "content/sources/nist-sp-800-193-platform-firmware-resiliency.yaml", "flags_all": [], "cited_by": [ { "id": "K-0005", "title": "Root of trust", "url": "https://trustbutveri.fyi/concepts/root-of-trust/" } ], "referenced_by": [ { "id": "K-0005", "title": "Root of trust", "url": "https://trustbutveri.fyi/concepts/root-of-trust/" } ] }, { "id": "S-2009", "slug": "nti-biological-weapons-convention", "status": "published", "risk_flags": [], "title": "The Biological Weapons Convention", "authors": [ "Nuclear Threat Initiative" ], "year": 2003, "date": "2003-02-28", "venue": "NTI", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/nti-biological-weapons-convention/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "organizations": [], "source_type": "tech-report", "source_url": "https://www.nti.org/analysis/articles/biological-weapons-convention/", "source_file": "content/sources/nti-biological-weapons-convention.yaml", "flags_all": [], "cited_by": [], "referenced_by": [] }, { "id": "S-2011", "slug": "nti-bwc-at-50", "status": "published", "risk_flags": [], "title": "BWC at 50: Taking Bold Steps to Secure the Future", "authors": [ "G. Essix" ], "year": 2024, "date": "2024-11-25", "venue": "NTI", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/nti-bwc-at-50/", "archive_url": "https://web.archive.org/web/20251222091102/https://www.nti.org/risky-business/bwc-at-50-taking-bold-steps-to-secure-the-future/", "accessed": "2026-09-25", "doi": null, "arxiv": null, "organizations": [], "source_type": "blog", "source_url": "https://www.nti.org/risky-business/bwc-at-50-taking-bold-steps-to-secure-the-future/", "source_file": "content/sources/nti-bwc-at-50.yaml", "flags_all": [], "cited_by": [], "referenced_by": [] }, { "id": "S-3604", "slug": "nvidia-about", "status": "published", "risk_flags": [], "title": "About NVIDIA", "authors": [], "year": 2026, "venue": "NVIDIA", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/nvidia-about/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Official organization page, read 2026-09-25; the year is the access year because the page is undated. Supports the organization's attributed self-description: self-description as pioneer of accelerated computing.", "organizations": [ "O-0140" ], "source_type": "docs", "source_url": "https://www.nvidia.com/en-us/about-nvidia/", "source_file": "content/sources/nvidia-about.yaml", "flags_all": [], "cited_by": [ { "id": "O-0140", "title": "NVIDIA", "url": "https://trustbutveri.fyi/organizations/nvidia/" } ], "referenced_by": [ { "id": "O-0140", "title": "NVIDIA", "url": "https://trustbutveri.fyi/organizations/nvidia/" } ] }, { "id": "S-1815", "slug": "nvidia-azure-confidential-h100-ga", "status": "published", "risk_flags": [], "title": "Now in General Availability: NVIDIA H100 GPUs in Microsoft Azure Confidential Virtual Machines", "authors": [ "C. Su" ], "year": 2024, "date": "2024-09-24", "venue": "NVIDIA Blog", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/nvidia-azure-confidential-h100-ga/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "NVIDIA's announcement that Azure NCC H100 v5 confidential VMs are generally available in the Azure East US2 and West Europe regions. Vendor post; supports only \"NVIDIA reports\" statements.", "organizations": [ "O-0140" ], "source_type": "blog", "source_url": "https://blogs.nvidia.com/blog/azure-confidential-vm-h100-general-availability", "source_file": "content/sources/nvidia-azure-confidential-h100-ga.yaml", "flags_all": [], "cited_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ], "referenced_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ] }, { "id": "S-3180", "slug": "nvidia-fleet-intelligence-technical-blog", "status": "published", "risk_flags": [], "title": "Introducing NVIDIA Fleet Intelligence for Real-Time GPU Fleet Visibility and Optimization", "authors": [ "C. Shrauder", "G. Frederick" ], "year": 2026, "date": "2026-05-11", "venue": "NVIDIA Technical Blog", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/nvidia-fleet-intelligence-technical-blog/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "NVIDIA's description of its own fleet-management service; supports only \"NVIDIA reports\" statements. States that the service is generally available at no cost for Hopper, Blackwell and Vera Rubin GPUs; that a read-only, host-based agent (built on GPUd, DCGM and the NVIDIA Attestation SDK) streams power, temperature, performance, health and configuration telemetry to an NVIDIA cloud service; that the agent is released as open source \"for auditability\" (github.com/NVIDIA/Fleet-Intelligence-Agent); and that GPU attestation evidence obtained at run time is signed with on-device certificates rooted in NVIDIA's root of trust and verified with NRAS. It does not say that the telemetry values themselves are signed. Follows the December 2025 announcement in S-1413.", "organizations": [ "O-0140" ], "source_type": "blog", "source_url": "https://developer.nvidia.com/blog/introducing-nvidia-fleet-intelligence-for-real-time-gpu-fleet-visibility-and-optimization/", "source_file": "content/sources/nvidia-fleet-intelligence-technical-blog.yaml", "flags_all": [], "cited_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "O-0140", "title": "NVIDIA", "url": "https://trustbutveri.fyi/organizations/nvidia/" } ], "referenced_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "O-0140", "title": "NVIDIA", "url": "https://trustbutveri.fyi/organizations/nvidia/" } ] }, { "id": "S-1413", "slug": "nvidia-opt-in-fleet-management-software", "status": "published", "risk_flags": [], "title": "Opt-In NVIDIA Software Enables Data Center Fleet Management", "authors": [ "NVIDIA" ], "year": 2025, "date": "2025-12-10", "venue": "NVIDIA Blog", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/nvidia-opt-in-fleet-management-software/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "NVIDIA's announcement of its own fleet-management service (byline: NVIDIA Newsroom); supports only \"NVIDIA states\" or \"NVIDIA reports\" statements. Describes an opt-in, customer-installed service that streams read-only GPU telemetry to an NVIDIA portal, with a client agent slated to be open-sourced, and states that NVIDIA GPUs do not have hardware tracking technology, kill switches and backdoors. It does not describe location verification.", "organizations": [ "O-0140" ], "source_type": "blog", "source_url": "https://blogs.nvidia.com/blog/optional-data-center-fleet-management-software/", "source_file": "content/sources/nvidia-opt-in-fleet-management-software.yaml", "flags_all": [], "cited_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "O-0140", "title": "NVIDIA", "url": "https://trustbutveri.fyi/organizations/nvidia/" } ], "referenced_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "O-0140", "title": "NVIDIA", "url": "https://trustbutveri.fyi/organizations/nvidia/" } ] }, { "id": "S-1200", "slug": "nvidia-secure-ai-blackwell-hopper-whitepaper", "status": "published", "risk_flags": [], "title": "NVIDIA Secure AI with Blackwell and Hopper GPUs (White Paper)", "authors": [ "NVIDIA" ], "year": 2025, "date": "2025-08-14", "venue": "NVIDIA documentation", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/nvidia-secure-ai-blackwell-hopper-whitepaper/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Document WP-12554-001_v1.3, dated 2025-08-14, as read on 2026-09-23 and 2026-09-25 (still the version linked on 2026-09-25). Linked from https://docs.nvidia.com/confidential-computing/.", "note": "Vendor documentation about NVIDIA's own confidential-computing feature; supports only \"NVIDIA reports\" statements. Covers the CPU-TEE pairing, device identity and root of trust, SPDM session, bounce buffers, TDISP/IDE, multi-GPU modes, the in-scope and out-of-scope threat lists (pp. 14-15) and performance counters in CC mode (p. 18).", "organizations": [ "O-0140" ], "source_type": "docs", "source_url": "https://docs.nvidia.com/nvidia-secure-ai-with-blackwell-and-hopper-gpus-whitepaper.pdf", "source_file": "content/sources/nvidia-secure-ai-blackwell-hopper-whitepaper.yaml", "flags_all": [], "cited_by": [ { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0007", "title": "Attestable Audits", "url": "https://trustbutveri.fyi/implementations/attestable-audits/" }, { "id": "O-0140", "title": "NVIDIA", "url": "https://trustbutveri.fyi/organizations/nvidia/" } ], "referenced_by": [ { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0007", "title": "Attestable Audits", "url": "https://trustbutveri.fyi/implementations/attestable-audits/" }, { "id": "O-0140", "title": "NVIDIA", "url": "https://trustbutveri.fyi/organizations/nvidia/" } ] }, { "id": "S-3122", "slug": "nvidia-trusted-computing-r595-release-notes", "status": "published", "risk_flags": [], "title": "NVIDIA Trusted Computing Solutions Release Notes (R595 TRD1)", "authors": [ "NVIDIA" ], "year": 2026, "venue": "NVIDIA documentation", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/nvidia-trusted-computing-r595-release-notes/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Document RN-12817-001_v02, dated April 2026; the newest release notes linked from https://docs.nvidia.com/confidential-computing/ on 2026-09-25.", "note": "Vendor release notes; support only \"NVIDIA reports\" statements. Lists three generally available confidential modes: single-GPU passthrough on Hopper and Blackwell, Hopper multi-GPU passthrough with protected PCIe (NVLink traffic unencrypted), and Blackwell multi-GPU passthrough (encrypted NVLink). Adds HGX B200 and B300 platforms. No multi-node mode is listed.", "organizations": [ "O-0140" ], "source_type": "docs", "source_url": "https://docs.nvidia.com/595trd1-trusted-computing-solutions-release-notes.pdf", "source_file": "content/sources/nvidia-trusted-computing-r595-release-notes.yaml", "flags_all": [], "cited_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ], "referenced_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ] }, { "id": "S-0049", "slug": "obermaier-physical-security-enclosures", "status": "published", "risk_flags": [], "title": "The Past, Present, and Future of Physical Security Enclosures: From Battery-Backed Monitoring to PUF-Based Inherent Security and Beyond", "authors": [ "J. Obermaier", "V. Immler" ], "year": 2018, "venue": "Journal of Hardware and Systems Security", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/obermaier-physical-security-enclosures/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.1007/s41635-018-0045-2", "arxiv": null, "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Tamper resistance and detection\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://link.springer.com/article/10.1007/s41635-018-0045-2", "source_file": "content/sources/obermaier-physical-security-enclosures.yaml", "flags_all": [], "cited_by": [ { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" } ], "referenced_by": [ { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" } ] }, { "id": "S-0006", "slug": "ogara-hardware-enabled-verifying-responsible-ai", "status": "published", "risk_flags": [], "title": "Hardware-Enabled Mechanisms for Verifying Responsible AI Development", "authors": [ "A. O'Gara", "G. Kulp", "W. Hodgkins", "J. Petrie", "V. Immler", "A. Aysu", "K. Basu", "S. Bhasin", "S. Picek", "A. Srivastava" ], "year": 2025, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/ogara-hardware-enabled-verifying-responsible-ai/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2505.03742", "arxiv": "2505.03742", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Research agendas and surveys\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2505.03742", "source_file": "content/sources/ogara-hardware-enabled-verifying-responsible-ai.yaml", "flags_all": [], "cited_by": [ { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "K-0007", "title": "Hardware-enabled mechanism (HEM)", "url": "https://trustbutveri.fyi/concepts/hardware-enabled-mechanism/" } ], "referenced_by": [ { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "K-0007", "title": "Hardware-enabled mechanism (HEM)", "url": "https://trustbutveri.fyi/concepts/hardware-enabled-mechanism/" } ] }, { "id": "S-1000", "slug": "ong-toploc", "status": "published", "risk_flags": [], "title": "TOPLOC: A Locality Sensitive Hashing Scheme for Trustless Verifiable Inference", "authors": [ "J. M. Ong", "M. Di Ferrante", "A. Pazdera", "R. Garner", "S. Jaghouar", "M. Basra", "M. Ryabinin", "J. Hagemann" ], "year": 2025, "venue": "Proceedings of the 42nd International Conference on Machine Learning (PMLR 267), pp. 47196-47211", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/ong-toploc/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2501.16007", "version_note": "Full text read from arXiv v2 (30 May 2025, HTML and PDF); PMLR abstract page checked for venue, pages and author list.", "organizations": [ "O-0100" ], "source_type": "peer-reviewed", "source_url": "https://proceedings.mlr.press/v267/ong25a.html", "source_file": "content/sources/ong-toploc.yaml", "flags_all": [], "cited_by": [ { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "O-0100", "title": "Prime Intellect", "url": "https://trustbutveri.fyi/organizations/prime-intellect/" } ], "referenced_by": [ { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "O-0100", "title": "Prime Intellect", "url": "https://trustbutveri.fyi/organizations/prime-intellect/" } ] }, { "id": "S-2008", "slug": "opcw-declared-stockpiles-destroyed", "status": "published", "risk_flags": [], "title": "OPCW confirms: All declared chemical weapons stockpiles verified as irreversibly destroyed", "authors": [ "Organisation for the Prohibition of Chemical Weapons" ], "year": 2023, "date": "2023-07-07", "venue": "OPCW", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/opcw-declared-stockpiles-destroyed/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "organizations": [], "source_type": "gov-doc", "source_url": "https://www.opcw.org/media-centre/news/2023/07/opcw-confirms-all-declared-chemical-weapons-stockpiles-verified", "source_file": "content/sources/opcw-declared-stockpiles-destroyed.yaml", "flags_all": [], "cited_by": [], "referenced_by": [] }, { "id": "S-3400", "slug": "opcw-syria-undeclared-chemical-weapons-2026", "status": "published", "risk_flags": [], "title": "Deployment of OPCW expert team to Syrian Arab Republic finds chemical weapons previously undeclared to the Organisation", "authors": [ "Organisation for the Prohibition of Chemical Weapons" ], "year": 2026, "date": "2026-05-27", "venue": "OPCW", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/opcw-syria-undeclared-chemical-weapons-2026/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "organizations": [], "source_type": "gov-doc", "source_url": "https://www.opcw.org/media-centre/news/2026/05/deployment-opcw-expert-team-syrian-arab-republic-finds-chemical-weapons", "source_file": "content/sources/opcw-syria-undeclared-chemical-weapons-2026.yaml", "flags_all": [], "cited_by": [], "referenced_by": [] }, { "id": "S-3563", "slug": "openmined-double-blind-evaluation", "status": "published", "risk_flags": [], "title": "PySyft used for first double-blind evaluation of a proprietary, frontier-class AI model", "authors": [ "OpenMined Team" ], "year": 2026, "venue": "OpenMined", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/openmined-double-blind-evaluation/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "OpenMined's account of PySyft and its role in the 2026 evaluations with Google DeepMind, AVERI, MLCommons and Singapore AISI. It describes the 2024 pilot with public stand-ins separately.", "organizations": [ "O-0215" ], "source_type": "blog", "source_url": "https://openmined.org/blog/double-blind-evaluation-frontier-ai-model/", "source_file": "content/sources/openmined-double-blind-evaluation.yaml", "flags_all": [], "cited_by": [ { "id": "I-0017", "title": "PySyft double-blind evaluations", "url": "https://trustbutveri.fyi/implementations/pysyft-double-blind-evaluations/" }, { "id": "O-0215", "title": "OpenMined", "url": "https://trustbutveri.fyi/organizations/openmined/" } ], "referenced_by": [ { "id": "I-0017", "title": "PySyft double-blind evaluations", "url": "https://trustbutveri.fyi/implementations/pysyft-double-blind-evaluations/" }, { "id": "O-0215", "title": "OpenMined", "url": "https://trustbutveri.fyi/organizations/openmined/" } ] }, { "id": "S-1703", "slug": "oxford-hardware-ai-governance-lab", "status": "published", "risk_flags": [], "title": "Hardware AI Governance Lab", "authors": [], "year": 2026, "venue": "Oxford Martin AI Governance Initiative", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/oxford-hardware-ai-governance-lab/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Programme page of the Hardware AI Governance Lab (HAIGL), \"an interdisciplinary initiative hosted by the Oxford Martin AI Governance Initiative\". States the lab's aims, says it expects to release its first hardware governance design profile in late 2026, names Amro Awad and Robert Trager as HAIGL Co-Directors and Ben Harack as Co-founder & Research Lead, and lists recent publications, including S-1300 (dated 28 April 2026 on the page) and S-1704. No page date. Self-description; supports \"the lab states\" statements.", "organizations": [ "O-0209" ], "source_type": "docs", "source_url": "https://aigi.ox.ac.uk/hardware-ai-governance-lab/", "source_file": "content/sources/oxford-hardware-ai-governance-lab.yaml", "flags_all": [], "cited_by": [ { "id": "O-0209", "title": "Hardware AI Governance Lab", "url": "https://trustbutveri.fyi/organizations/oxford-hardware-ai-governance-lab/" }, { "id": "O-0205", "title": "Oxford Martin AI Governance Initiative", "url": "https://trustbutveri.fyi/organizations/oxford-martin-ai-governance-initiative/" } ], "referenced_by": [ { "id": "O-0209", "title": "Hardware AI Governance Lab", "url": "https://trustbutveri.fyi/organizations/oxford-hardware-ai-governance-lab/" }, { "id": "O-0205", "title": "Oxford Martin AI Governance Initiative", "url": "https://trustbutveri.fyi/organizations/oxford-martin-ai-governance-initiative/" } ] }, { "id": "S-1705", "slug": "oxford-martin-aigi-about", "status": "published", "risk_flags": [], "title": "About: Oxford Martin AIGI", "authors": [], "year": 2026, "venue": "Oxford Martin AI Governance Initiative", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/oxford-martin-aigi-about/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "About page of the Oxford Martin AI Governance Initiative (AIGI). States that AIGI is housed in the Martin School of the University of Oxford, studies the governance of AI from both technical and policy perspectives, and is co-led by Robert Trager and Maike Osborne; the AIGI homepage lists both as Co-Director. No page date. Self-description; supports \"AIGI states\" statements.", "organizations": [ "O-0205" ], "source_type": "docs", "source_url": "https://aigi.ox.ac.uk/about/", "source_file": "content/sources/oxford-martin-aigi-about.yaml", "flags_all": [], "cited_by": [ { "id": "O-0205", "title": "Oxford Martin AI Governance Initiative", "url": "https://trustbutveri.fyi/organizations/oxford-martin-ai-governance-initiative/" } ], "referenced_by": [ { "id": "O-0205", "title": "Oxford Martin AI Governance Initiative", "url": "https://trustbutveri.fyi/organizations/oxford-martin-ai-governance-initiative/" } ] }, { "id": "S-1105", "slug": "pearl-floating-point-scheme-specification", "status": "published", "risk_flags": [], "title": "Pearl Floating Point Scheme Specification", "authors": [ "Pearl Research Team" ], "year": 2026, "venue": "Pearl Research Labs", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/pearl-floating-point-scheme-specification/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "First page dated September 2026; 22 pages.", "note": "Developer's own specification of its FP8 proof-of-useful-work protocol; supports only \"Pearl reports\" statements. The Pearl research page lists this PDF as \"The Pearl Protocol: A Proof-of-Useful-Work L1\".", "organizations": [ "O-0121" ], "source_type": "tech-report", "source_url": "https://pearlresearch.ai/Pearl_Whitepaper.pdf", "source_file": "content/sources/pearl-floating-point-scheme-specification.yaml", "flags_all": [], "cited_by": [ { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" }, { "id": "K-0011", "title": "Proof of (useful) work", "url": "https://trustbutveri.fyi/concepts/proof-of-useful-work/" }, { "id": "O-0121", "title": "Pearl Research Labs", "url": "https://trustbutveri.fyi/organizations/pearl-research/" } ], "referenced_by": [ { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" }, { "id": "K-0011", "title": "Proof of (useful) work", "url": "https://trustbutveri.fyi/concepts/proof-of-useful-work/" }, { "id": "O-0121", "title": "Pearl Research Labs", "url": "https://trustbutveri.fyi/organizations/pearl-research/" } ] }, { "id": "S-1106", "slug": "pearl-int-whitepaper", "status": "published", "risk_flags": [], "title": "Pearl INT Whitepaper", "authors": [ "Pearl Research Labs" ], "year": 2026, "venue": "Pearl Research Labs", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/pearl-int-whitepaper/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Developer's original whitepaper for the integer matrix-multiplication mining scheme; supports only \"Pearl reports\" statements. No explicit publication date; the year is from the page's copyright notice.", "organizations": [ "O-0121" ], "source_type": "tech-report", "source_url": "https://pearlresearch.ai/research/int-whitepaper", "source_file": "content/sources/pearl-int-whitepaper.yaml", "flags_all": [], "cited_by": [ { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" }, { "id": "O-0121", "title": "Pearl Research Labs", "url": "https://trustbutveri.fyi/organizations/pearl-research/" } ], "referenced_by": [ { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" }, { "id": "O-0121", "title": "Pearl Research Labs", "url": "https://trustbutveri.fyi/organizations/pearl-research/" } ] }, { "id": "S-1107", "slug": "pearl-network-monorepo", "status": "published", "risk_flags": [], "title": "pearl: Monorepo for the Pearl network", "authors": [ "Pearl Research Labs" ], "year": 2026, "venue": "GitHub", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/pearl-network-monorepo/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Release v1.2.1 (tag v1.2.1, commit e72ad36, 23 July), still the latest release listed on the releases page on 2026-09-25.", "license_note": "ISC license, per the repository.", "note": "Contains the reference full node, a vLLM-based GPU miner and a Plonky2/STARKy-based proof-of-work circuit and verifier.", "organizations": [ "O-0121" ], "source_type": "code", "source_url": "https://github.com/pearl-research-labs/pearl", "source_file": "content/sources/pearl-network-monorepo.yaml", "flags_all": [], "cited_by": [ { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" }, { "id": "O-0121", "title": "Pearl Research Labs", "url": "https://trustbutveri.fyi/organizations/pearl-research/" } ], "referenced_by": [ { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" }, { "id": "O-0121", "title": "Pearl Research Labs", "url": "https://trustbutveri.fyi/organizations/pearl-research/" } ] }, { "id": "S-3571", "slug": "pearl-proof-of-useful-work-blog", "status": "published", "risk_flags": [], "title": "Proof of Useful Work from the Ground Up", "authors": [ "Pearl Research Labs" ], "year": 2026, "venue": "Pearl Research Labs", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/pearl-proof-of-useful-work-blog/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Pearl reports end-to-end serving overhead for its integer proof-of-useful-work kernel against stock serving engines: 5.08% for Llama 70B at DP=4 on four H200 GPUs and 3.9% for DeepSeek V3.2 at DP=8 plus expert parallelism on eight H200 GPUs. The page gives no publication date; year follows its copyright notice. Developer report only.", "organizations": [ "O-0121" ], "source_type": "blog", "source_url": "https://pearlresearch.ai/research/blog/proof-of-useful-work", "source_file": "content/sources/pearl-proof-of-useful-work-blog.yaml", "flags_all": [], "cited_by": [ { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" } ], "referenced_by": [ { "id": "I-0004", "title": "Pearl proof-of-useful-work blockchain", "url": "https://trustbutveri.fyi/implementations/pearl-proof-of-useful-work/" } ] }, { "id": "S-3603", "slug": "pearl-research-homepage", "status": "published", "risk_flags": [], "title": "Pearl Research Labs homepage", "authors": [], "year": 2026, "venue": "Pearl Research Labs", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/pearl-research-homepage/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Official organization page, read 2026-09-25; the year is the access year because the page is undated. Supports the organization's attributed self-description: infrastructure and algorithms for AI inference.", "organizations": [ "O-0121" ], "source_type": "docs", "source_url": "https://pearlresearch.ai/", "source_file": "content/sources/pearl-research-homepage.yaml", "flags_all": [], "cited_by": [ { "id": "O-0121", "title": "Pearl Research Labs", "url": "https://trustbutveri.fyi/organizations/pearl-research/" } ], "referenced_by": [ { "id": "O-0121", "title": "Pearl Research Labs", "url": "https://trustbutveri.fyi/organizations/pearl-research/" } ] }, { "id": "S-0025", "slug": "peigne-zero-knowledge-frontier-training", "status": "published", "risk_flags": [], "title": "Zero knowledge verification for frontier AI training is possible", "authors": [ "P. Peigné", "K. Nguyen", "P. Wang" ], "year": 2026, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/peigne-zero-knowledge-frontier-training/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2606.05433", "arxiv": "2606.05433", "version_note": "v2 (22 August 2026) read; v1 was posted on 3 June 2026.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Zero-knowledge proofs\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2606.05433", "source_file": "content/sources/peigne-zero-knowledge-frontier-training.yaml", "flags_all": [], "cited_by": [ { "id": "M-0005", "title": "Zero-knowledge proofs of training constraints", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-training-constraints/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0024", "title": "Cryptographic commitment", "url": "https://trustbutveri.fyi/concepts/cryptographic-commitment/" }, { "id": "K-0010", "title": "Zero-knowledge proof", "url": "https://trustbutveri.fyi/concepts/zero-knowledge-proof/" } ], "referenced_by": [ { "id": "M-0005", "title": "Zero-knowledge proofs of training constraints", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-training-constraints/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0024", "title": "Cryptographic commitment", "url": "https://trustbutveri.fyi/concepts/cryptographic-commitment/" }, { "id": "K-0010", "title": "Zero-knowledge proof", "url": "https://trustbutveri.fyi/concepts/zero-knowledge-proof/" } ] }, { "id": "S-1503", "slug": "penchas-verifiably-scoped-monitoring", "status": "published", "risk_flags": [], "title": "Enabling Verifiably-Scoped Monitoring through Large Language Models and Trusted Compute", "authors": [ "B. Penchas", "G. Zhao", "R. Rinberg" ], "year": 2026, "venue": "ICML 2026 Workshop on Technical AI Governance Research", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/penchas-verifiably-scoped-monitoring/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Only the abstract was read, on the ICML 2026 virtual poster page. The OpenReview forum (https://openreview.net/forum?id=AWZJq6O7Cj) was not reachable with the fetch tool.", "note": "The second author appears as \"G Zhao\" on the ICML page.", "organizations": [], "source_type": "preprint", "source_url": "https://icml.cc/virtual/2026/78630", "source_file": "content/sources/penchas-verifiably-scoped-monitoring.yaml", "flags_all": [], "cited_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" } ], "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" } ] }, { "id": "S-1100", "slug": "peng-survey-zkp-verifiable-ml", "status": "published", "risk_flags": [], "title": "A Survey of Zero-Knowledge Proof Based Verifiable Machine Learning", "authors": [ "Z. Peng", "C. Zhao", "T. Wang", "G. Liao", "Z. Lin", "Y. Liu", "B. Cao", "L. Shi", "Q. Yang", "S. Zhang" ], "year": 2026, "venue": "Artificial Intelligence Review, vol. 59, no. 7, article 157", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/peng-survey-zkp-verifiable-ml/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.1007/s10462-026-11557-y", "arxiv": "2502.18535", "version_note": "Read arXiv v2 (29 March 2026). Journal venue and DOI from Crossref (published 13 April 2026), checked 2026-09-24.", "note": "Survey of ZKML literature from June 2017 to August 2025, organised into verifiable training, testing and inference.", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://arxiv.org/abs/2502.18535", "source_file": "content/sources/peng-survey-zkp-verifiable-ml.yaml", "flags_all": [], "cited_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "M-0005", "title": "Zero-knowledge proofs of training constraints", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-training-constraints/" } ], "referenced_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "M-0005", "title": "Zero-knowledge proofs of training constraints", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-training-constraints/" } ] }, { "id": "S-1304", "slug": "perito-proofs-of-secure-erasure", "status": "published", "risk_flags": [], "title": "Secure Code Update for Embedded Devices via Proofs of Secure Erasure", "authors": [ "D. Perito", "G. Tsudik" ], "year": 2010, "venue": "Computer Security – ESORICS 2010, LNCS 6345, pp. 643–662", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/perito-proofs-of-secure-erasure/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.1007/978-3-642-15497-3_39", "arxiv": null, "note": "Also available as IACR ePrint 2010/217 (https://eprint.iacr.org/2010/217), which was also read. Pages confirmed in Crossref on 2026-09-25.", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://link.springer.com/chapter/10.1007/978-3-642-15497-3_39", "source_file": "content/sources/perito-proofs-of-secure-erasure.yaml", "flags_all": [], "cited_by": [ { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" } ], "referenced_by": [ { "id": "M-0015", "title": "Memory wiping and proofs of secure erasure", "url": "https://trustbutveri.fyi/mechanisms/memory-wiping-and-secure-erasure/" }, { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" } ] }, { "id": "S-0019", "slug": "petrie-bounding-unexplained-information-exfiltration", "status": "published", "risk_flags": [], "title": "Verifying AI Compute by Bounding Unexplained Information Exfiltration", "authors": [ "J. Petrie", "Y. Mühlhäuser" ], "year": 2026, "venue": "ICML 2026 Workshop on Technical AI Governance Research", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/petrie-bounding-unexplained-information-exfiltration/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "OpenReview returned a browser check on 2026-09-23 and 2026-09-25, so only the title, authors and abstract were read, on the workshop's ICML 2026 virtual page (https://icml.cc/virtual/2026/78565). No arXiv version found as of 2026-09-25.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Inference verification\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "preprint", "source_url": "https://openreview.net/forum?id=qtgG5HZSsk", "source_file": "content/sources/petrie-bounding-unexplained-information-exfiltration.yaml", "flags_all": [], "cited_by": [ { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" } ], "referenced_by": [ { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" } ] }, { "id": "S-3160", "slug": "petrie-embedded-off-switches-ai-compute", "status": "published", "risk_flags": [], "title": "Embedded Off-Switches for AI Compute", "authors": [ "J. Petrie" ], "year": 2025, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/petrie-embedded-off-switches-ai-compute/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2509.07637", "arxiv": "2509.07637", "version_note": "v1 (2025-09-09), the only version, read on 2026-09-25.", "note": "Design for thousands of redundant on-chip security blocks that each block essential accelerator operations unless they hold a recent, signed authorization. The paper gives gate-count and area estimates and reports no built or measured hardware. A proof-of-concept repository by the same author is S-3161.", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2509.07637", "source_file": "content/sources/petrie-embedded-off-switches-ai-compute.yaml", "flags_all": [], "cited_by": [ { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" } ], "referenced_by": [ { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" } ] }, { "id": "S-0035", "slug": "petrie-flexible-hardware-enabled-guarantees", "status": "published", "risk_flags": [], "title": "Flexible Hardware-Enabled Guarantees for AI Compute", "authors": [ "J. Petrie", "O. Aarne", "N. Ammann", "D. Dalrymple" ], "year": 2025, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/petrie-flexible-hardware-enabled-guarantees/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2506.15093", "arxiv": "2506.15093", "version_note": "v1 (2025-06-18), still the only version and with no journal reference, read on 2026-09-25.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Network and memory telemetry\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2506.15093", "source_file": "content/sources/petrie-flexible-hardware-enabled-guarantees.yaml", "flags_all": [], "cited_by": [ { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" }, { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0007", "title": "Hardware-enabled mechanism (HEM)", "url": "https://trustbutveri.fyi/concepts/hardware-enabled-mechanism/" }, { "id": "K-0015", "title": "Tamper evidence and tamper resistance", "url": "https://trustbutveri.fyi/concepts/tamper-evidence/" } ], "referenced_by": [ { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" }, { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0007", "title": "Hardware-enabled mechanism (HEM)", "url": "https://trustbutveri.fyi/concepts/hardware-enabled-mechanism/" }, { "id": "K-0015", "title": "Tamper evidence and tamper resistance", "url": "https://trustbutveri.fyi/concepts/tamper-evidence/" } ] }, { "id": "S-0034", "slug": "petrie-guaranteeable-memory", "status": "published", "risk_flags": [], "title": "Guaranteeable Memory: An HBM-Based Chiplet for Verifiable AI Workloads", "authors": [ "J. Petrie" ], "year": 2025, "venue": "ICML 2025 Workshop on Technical AI Governance", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/petrie-guaranteeable-memory/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Workshop poster. Read via the ICML 2025 virtual-site abstract (https://icml.cc/virtual/2025/48329) on 2026-09-23 and 2026-09-25; the OpenReview page and PDF could not be read.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Network and memory telemetry\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "preprint", "source_url": "https://openreview.net/forum?id=uc79kOv0MV", "source_file": "content/sources/petrie-guaranteeable-memory.yaml", "flags_all": [], "cited_by": [ { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" } ], "referenced_by": [ { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" } ] }, { "id": "S-3161", "slug": "petrie-off-switch-code", "status": "published", "risk_flags": [], "title": "JamesPetrie/off-switch (GitHub repository)", "authors": [ "J. Petrie" ], "year": 2025, "venue": "GitHub", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/petrie-off-switch-code/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Pinned to commit 4911839c11d95496255df626f59bdae973713e9d (head of main on 2026-09-25, committed 2026-09-17; first commit 2025-11-10; 131 commits, no releases or tags), obtained through the GitHub API. The README was read at that commit via raw.githubusercontent.com.", "license_note": "No licence shown on the repository page on 2026-09-25.", "note": "SystemVerilog proof of concept of the security block in S-3160, with Verilator testbenches. Per its README, it gates an example Int8 adder, supports ECDSA (secp256k1) or HSS/LMS signatures, and omits production features including constant-time field arithmetic, a configurable public key and redundant blocks. The repository also has a pynq-z2 folder with an FPGA build script and a Python client; the README reports no FPGA results.", "organizations": [], "source_type": "code", "source_url": "https://github.com/JamesPetrie/off-switch", "source_file": "content/sources/petrie-off-switch-code.yaml", "flags_all": [], "cited_by": [ { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" } ], "referenced_by": [ { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" } ] }, { "id": "S-1204", "slug": "petrie-technical-options-flexheg", "status": "published", "risk_flags": [], "title": "Technical Options for Flexible Hardware-Enabled Guarantees", "authors": [ "J. Petrie", "O. Aarne" ], "year": 2025, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/petrie-technical-options-flexheg/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2506.03409", "arxiv": "2506.03409", "version_note": "v3 (2025-06-18), still the latest version, read on 2026-09-25; v1 was submitted 2025-06-03.", "note": "Part II of the three-part flexHEG series, which the report says was commissioned by ARIA. Part I is S-0035 and Part III is S-1205.", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2506.03409", "source_file": "content/sources/petrie-technical-options-flexheg.yaml", "flags_all": [], "cited_by": [ { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ], "referenced_by": [ { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ] }, { "id": "S-3262", "slug": "physec-seal", "status": "published", "risk_flags": [], "title": "PHYSEC SEAL: Change detection for maximum safety", "authors": [ "PHYSEC GmbH" ], "year": 2026, "venue": "PHYSEC website", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/physec-seal/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Product page, undated; read on 2026-09-25.", "organizations": [], "source_type": "docs", "source_url": "https://www.physec.de/en/solutions/physec-seal/", "source_file": "content/sources/physec-seal.yaml", "flags_all": [], "cited_by": [ { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" } ], "referenced_by": [ { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" } ] }, { "id": "S-1700", "slug": "planet-q2-fy2027-financial-results", "status": "published", "risk_flags": [], "title": "Planet Reports Financial Results for Second Quarter of Fiscal Year 2027", "authors": [ "Planet Labs PBC" ], "year": 2026, "date": "2026-09-03", "venue": "Business Wire (press release)", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/planet-q2-fy2027-financial-results/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Company press release for the quarter ended 31 July 2026, also carried on nasdaq.com. It states that in August 2026 Planet signed a renewal with a \"hyperscaler AI developer\" for global monitoring of data centres and of semiconductor manufacturing facility construction, using its Pelican high-resolution data, and it gives the company's \"About Planet\" description. Supports \"Planet reports\" statements only.", "organizations": [ "O-0102" ], "source_type": "blog", "source_url": "https://www.businesswire.com/news/home/20260903433245/en/Planet-Reports-Financial-Results-for-Second-Quarter-of-Fiscal-Year-2027", "source_file": "content/sources/planet-q2-fy2027-financial-results.yaml", "flags_all": [], "cited_by": [ { "id": "O-0102", "title": "Planet Labs", "url": "https://trustbutveri.fyi/organizations/planet-labs/" } ], "referenced_by": [ { "id": "O-0102", "title": "Planet Labs", "url": "https://trustbutveri.fyi/organizations/planet-labs/" } ] }, { "id": "S-3600", "slug": "prime-intellect-homepage", "status": "published", "risk_flags": [], "title": "Prime Intellect homepage", "authors": [], "year": 2026, "venue": "Prime Intellect", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/prime-intellect-homepage/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Official organization page, read 2026-09-25; the year is the access year because the page is undated. Supports the organization's attributed self-description: integrated compute, training, inference and sandbox stack.", "organizations": [ "O-0100" ], "source_type": "docs", "source_url": "https://www.primeintellect.ai/", "source_file": "content/sources/prime-intellect-homepage.yaml", "flags_all": [], "cited_by": [ { "id": "O-0100", "title": "Prime Intellect", "url": "https://trustbutveri.fyi/organizations/prime-intellect/" } ], "referenced_by": [ { "id": "O-0100", "title": "Prime Intellect", "url": "https://trustbutveri.fyi/organizations/prime-intellect/" } ] }, { "id": "S-1003", "slug": "primeintellect-intellect-2", "status": "published", "risk_flags": [], "title": "INTELLECT-2: A Reasoning Model Trained Through Globally Decentralized Reinforcement Learning", "authors": [ "Prime Intellect Team", "S. Jaghouar", "J. Mattern", "J. M. Ong", "J. Straube", "M. Basra", "A. Pazdera", "K. Thaman", "M. Di Ferrante", "F. Gabriel", "F. Obeid", "K. Erdem", "M. Keiblinger", "J. Hagemann" ], "year": 2025, "date": "2025-05-12", "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/primeintellect-intellect-2/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2505.07291", "version_note": "Read arXiv v1 (HTML), section 2.3 on TOPLOC and section 2.4 on validation.", "note": "Developer's report on its own training run; TOPLOC statements are provider-reported.", "organizations": [ "O-0100" ], "source_type": "tech-report", "source_url": "https://arxiv.org/abs/2505.07291", "source_file": "content/sources/primeintellect-intellect-2.yaml", "flags_all": [], "cited_by": [ { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "O-0100", "title": "Prime Intellect", "url": "https://trustbutveri.fyi/organizations/prime-intellect/" } ], "referenced_by": [ { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "O-0100", "title": "Prime Intellect", "url": "https://trustbutveri.fyi/organizations/prime-intellect/" } ] }, { "id": "S-3000", "slug": "primeintellect-synthetic-2-release", "status": "published", "risk_flags": [], "title": "SYNTHETIC-2 Release: Four Million Collaboratively Generated Reasoning Traces", "authors": [ "Prime Intellect" ], "year": 2025, "date": "2025-07-10", "venue": "Prime Intellect blog", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/primeintellect-synthetic-2-release/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Developer's report on its own data-generation run, bylined with first names only. Describes the TOPLOC v2 checks, the reported false-positive rate and verification cost, and the GPUs that took part. Supports only \"Prime Intellect reports\" statements.", "organizations": [ "O-0100" ], "source_type": "blog", "source_url": "https://www.primeintellect.ai/blog/synthetic-2-release", "source_file": "content/sources/primeintellect-synthetic-2-release.yaml", "flags_all": [], "cited_by": [ { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "O-0100", "title": "Prime Intellect", "url": "https://trustbutveri.fyi/organizations/prime-intellect/" } ], "referenced_by": [ { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "O-0100", "title": "Prime Intellect", "url": "https://trustbutveri.fyi/organizations/prime-intellect/" } ] }, { "id": "S-1004", "slug": "primeintellect-synthetic-2", "status": "published", "risk_flags": [], "title": "SYNTHETIC-2", "authors": [ "Prime Intellect" ], "year": 2025, "date": "2025-06-23", "venue": "Prime Intellect blog", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/primeintellect-synthetic-2/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Developer's blog post; describes its use of \"TOPLOC v2\". Supports only \"Prime Intellect reports\" statements.", "organizations": [ "O-0100" ], "source_type": "blog", "source_url": "https://www.primeintellect.ai/blog/synthetic-2", "source_file": "content/sources/primeintellect-synthetic-2.yaml", "flags_all": [], "cited_by": [ { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "O-0100", "title": "Prime Intellect", "url": "https://trustbutveri.fyi/organizations/prime-intellect/" } ], "referenced_by": [ { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "O-0100", "title": "Prime Intellect", "url": "https://trustbutveri.fyi/organizations/prime-intellect/" } ] }, { "id": "S-1002", "slug": "primeintellect-toploc-blog", "status": "published", "risk_flags": [], "title": "TOPLOC: A Locality Sensitive Hashing Scheme for Trustless Verifiable Inference (blog post)", "authors": [ "Prime Intellect" ], "year": 2025, "date": "2025-01-28", "venue": "Prime Intellect blog", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/primeintellect-toploc-blog/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Developer's announcement of its own method; supports only \"Prime Intellect reports\" statements. Byline gives first names only.", "organizations": [ "O-0100" ], "source_type": "blog", "source_url": "https://www.primeintellect.ai/blog/toploc", "source_file": "content/sources/primeintellect-toploc-blog.yaml", "flags_all": [], "cited_by": [ { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "O-0100", "title": "Prime Intellect", "url": "https://trustbutveri.fyi/organizations/prime-intellect/" } ], "referenced_by": [ { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "O-0100", "title": "Prime Intellect", "url": "https://trustbutveri.fyi/organizations/prime-intellect/" } ] }, { "id": "S-1001", "slug": "primeintellect-toploc-code", "status": "published", "risk_flags": [], "title": "PrimeIntellect-ai/toploc (GitHub repository)", "authors": [ "Prime Intellect" ], "year": 2025, "venue": "GitHub", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/primeintellect-toploc-code/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Pinned to release tag v0.1.6 (commit 7ab7bcd, 10 Apr 2025), the latest release listed on the repository's releases page when viewed on 2026-09-23. Still the latest release on GitHub and PyPI on 2026-09-25.", "license_note": "MIT licence, per the repository.", "organizations": [ "O-0100" ], "source_type": "code", "source_url": "https://github.com/PrimeIntellect-ai/toploc", "source_file": "content/sources/primeintellect-toploc-code.yaml", "flags_all": [], "cited_by": [ { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "O-0100", "title": "Prime Intellect", "url": "https://trustbutveri.fyi/organizations/prime-intellect/" } ], "referenced_by": [ { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0001", "title": "TOPLOC", "url": "https://trustbutveri.fyi/implementations/toploc/" }, { "id": "O-0100", "title": "Prime Intellect", "url": "https://trustbutveri.fyi/organizations/prime-intellect/" } ] }, { "id": "S-3060", "slug": "qu-zkgpt", "status": "published", "risk_flags": [], "title": "zkGPT: An Efficient Non-interactive Zero-knowledge Proof Framework for LLM Inference", "authors": [ "W. Qu", "Y. Sun", "X. Liu", "T. Lu", "Y. Guo", "K. Chen", "J. Zhang" ], "year": 2025, "venue": "34th USENIX Security Symposium (USENIX Security 25), pp. 2045–2063", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/qu-zkgpt/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Proceedings paper read from the USENIX PDF (usenixsecurity25-qu-zkgpt.pdf). Code archived on Zenodo, DOI 10.5281/zenodo.14727819 (v1, 23 January 2025, CC BY 4.0).", "note": "National University of Singapore and Hong Kong University of Science and Technology. Proves GPT-2 inference with GKR, Lasso lookups and Hyrax commitments, made non-interactive with Fiat–Shamir; Table 3 reports 21.8 s proving (32 threads), 0.35 s verification and a 101 KB proof on a 16-core Xeon server with 200 GB of memory.", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://www.usenix.org/conference/usenixsecurity25/presentation/qu-zkgpt", "source_file": "content/sources/qu-zkgpt.yaml", "flags_all": [], "cited_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" } ], "referenced_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" } ] }, { "id": "S-0037", "slug": "rahman-detecting-hidden-ml-training", "status": "published", "risk_flags": [], "title": "Detecting Hidden ML Training With Zero-Overhead Telemetry", "authors": [ "R. Rahman", "S. Tajdari" ], "year": 2026, "venue": "ICML 2026 Workshop on Technical AI Governance Research", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/rahman-detecting-hidden-ml-training/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2606.19262", "arxiv": "2606.19262", "version_note": "v1 (2026-06-17) read on 2026-09-25. The paper header reads \"Proceedings of the Second Workshop on Technical AI Governance Research (TAIGR) at the 43rd International Conference on Machine Learning, Seoul\". Affiliations: R. Rahman (Machine Intelligence Research Institute), S. Tajdari (University of Virginia).", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Power telemetry and side-channel attacks\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2606.19262", "source_file": "content/sources/rahman-detecting-hidden-ml-training.yaml", "flags_all": [], "cited_by": [ { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "K-0025", "title": "Inference and training workloads", "url": "https://trustbutveri.fyi/concepts/inference-and-training-workloads/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" } ], "referenced_by": [ { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "K-0025", "title": "Inference and training workloads", "url": "https://trustbutveri.fyi/concepts/inference-and-training-workloads/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" } ] }, { "id": "S-0060", "slug": "rahman-distributed-training-compute-governance", "status": "published", "risk_flags": [], "title": "Does Distributed Training Undermine Compute Governance?", "authors": [ "R. Rahman" ], "year": 2026, "venue": "ICML 2026 Workshop on Technical AI Governance Research", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/rahman-distributed-training-compute-governance/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2605.29359", "arxiv": "2605.29359", "version_note": "arXiv v1, 28 May 2026. Poster at the Second Workshop on Technical AI Governance Research (ICML 2026), per the workshop page (https://icml.cc/virtual/2026/workshop/54084), checked 2026-09-25.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Motivations and policy proposals\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2605.29359", "source_file": "content/sources/rahman-distributed-training-compute-governance.yaml", "flags_all": [], "cited_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" } ], "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" } ] }, { "id": "S-3607", "slug": "rand-about", "status": "published", "risk_flags": [], "title": "About RAND", "authors": [], "year": 2026, "venue": "RAND", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/rand-about/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Official organization page, read 2026-09-25; the year is the access year because the page is undated. Supports the organization's attributed self-description: nonprofit nonpartisan research organization and evidence-based decisions.", "organizations": [ "O-0200" ], "source_type": "docs", "source_url": "https://www.rand.org/about.html", "source_file": "content/sources/rand-about.yaml", "flags_all": [], "cited_by": [ { "id": "O-0200", "title": "RAND", "url": "https://trustbutveri.fyi/organizations/rand/" } ], "referenced_by": [ { "id": "O-0200", "title": "RAND", "url": "https://trustbutveri.fyi/organizations/rand/" } ] }, { "id": "S-2002", "slug": "reagan-remarks-signing-inf-treaty", "status": "published", "risk_flags": [], "title": "Remarks on Signing the Intermediate-Range Nuclear Forces Treaty", "authors": [ "R. Reagan" ], "year": 1987, "date": "1987-12-08", "venue": "Ronald Reagan Presidential Library and Museum", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/reagan-remarks-signing-inf-treaty/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "organizations": [], "source_type": "gov-doc", "source_url": "https://www.reaganlibrary.gov/archives/speech/remarks-signing-intermediate-range-nuclear-forces-treaty", "source_file": "content/sources/reagan-remarks-signing-inf-treaty.yaml", "flags_all": [], "cited_by": [], "referenced_by": [] }, { "id": "S-3162", "slug": "reber-no-backdoors-no-kill-switches", "status": "published", "risk_flags": [], "title": "No Backdoors. No Kill Switches. No Spyware.", "authors": [ "D. Reber Jr." ], "year": 2025, "date": "2025-08-05", "venue": "NVIDIA Blog", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/reber-no-backdoors-no-kill-switches/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "NVIDIA blog post by David Reber Jr. stating the company's position (the page gives no job title); supports only \"NVIDIA states\" statements. It states that NVIDIA GPUs \"do not and should not have kill switches and backdoors\", and distinguishes optional software features controlled by the user from a kill switch hardwired into a chip.", "organizations": [ "O-0140" ], "source_type": "blog", "source_url": "https://blogs.nvidia.com/blog/no-backdoors-no-kill-switches-no-spyware/", "source_file": "content/sources/reber-no-backdoors-no-kill-switches.yaml", "flags_all": [], "cited_by": [ { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" } ], "referenced_by": [ { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0011", "title": "Hardware performance throttling and licensing", "url": "https://trustbutveri.fyi/mechanisms/hardware-performance-throttling/" } ] }, { "id": "S-0072", "slug": "rescorla-rfc-3552-security-considerations", "status": "published", "risk_flags": [], "title": "Guidelines for Writing RFC Text on Security Considerations (RFC 3552, BCP 72)", "authors": [ "E. Rescorla", "B. Korver", "Internet Architecture Board" ], "year": 2003, "venue": "Internet Engineering Task Force", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/rescorla-rfc-3552-security-considerations/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.17487/RFC3552", "arxiv": null, "version_note": "IETF Best Current Practice (BCP 72), July 2003. Section 3, \"The Internet Threat Model\", defines a threat model and states its purpose. The RFC Editor lists it as updated by RFC 8996 (deprecating TLS 1.0 and 1.1) and RFC 9416 (transient numeric identifiers); neither changes Section 3.", "organizations": [], "source_type": "standard", "source_url": "https://www.rfc-editor.org/rfc/rfc3552.html", "source_file": "content/sources/rescorla-rfc-3552-security-considerations.yaml", "flags_all": [], "cited_by": [ { "id": "K-0018", "title": "Threat model", "url": "https://trustbutveri.fyi/concepts/threat-model/" } ], "referenced_by": [ { "id": "K-0018", "title": "Threat model", "url": "https://trustbutveri.fyi/concepts/threat-model/" } ] }, { "id": "S-0001", "slug": "reuel-open-problems-technical-ai-governance", "status": "published", "risk_flags": [], "title": "Open Problems in Technical AI Governance", "authors": [ "A. Reuel", "B. Bucknall", "S. Casper", "T. Fist", "L. Soder", "O. Aarne", "L. Hammond", "L. Ibrahim", "A. Chan", "P. Wills", "M. Anderljung", "B. Garfinkel", "L. Heim", "A. Trask", "G. Mukobi", "R. Schaeffer", "M. Baker", "S. Hooker", "I. Solaiman", "A. S. Luccioni", "N. Rajkumar", "N. Moës", "J. Ladish", "D. Bau", "P.-A. Bricman", "N. Guha", "J. Newman", "Y. Bengio", "T. South", "A. Pentland", "S. Koyejo", "M. J. Kochenderfer", "R. Trager" ], "year": 2025, "venue": "Transactions on Machine Learning Research", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/reuel-open-problems-technical-ai-governance/", "archive_url": null, "accessed": "2026-09-23", "doi": null, "arxiv": "2407.14981", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Research agendas and surveys\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://openreview.net/forum?id=1nO4qFMiS0", "source_file": "content/sources/reuel-open-problems-technical-ai-governance.yaml", "flags_all": [], "cited_by": [ { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" } ], "referenced_by": [ { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" } ] }, { "id": "S-0026", "slug": "reuter-proofs-compartmentalization", "status": "published", "risk_flags": [], "title": "Verifiable constraints on frontier training via proofs of compartmentalization", "authors": [ "D. Reuter", "L. Marks", "A. Carlucci", "J. Ng", "J. Petrie", "J. Hausenloy", "A. Karvonen", "M. Baker" ], "year": 2026, "venue": "ICML 2026 Workshop on Technical AI Governance Research", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/reuter-proofs-compartmentalization/", "archive_url": "https://icml.cc/virtual/2026/78588", "accessed": "2026-09-23", "doi": null, "arxiv": null, "version_note": "Author list and abstract verified on the official ICML 2026 TAIGR workshop poster page on 2026-09-25. The OpenReview full text was inaccessible; only abstract-level claims are used.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Zero-knowledge proofs\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "preprint", "source_url": "https://openreview.net/forum?id=ajrtHuFw2S", "source_file": "content/sources/reuter-proofs-compartmentalization.yaml", "flags_all": [], "cited_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" } ], "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" } ] }, { "id": "S-3020", "slug": "ribeiro-alves-eigenai", "status": "published", "risk_flags": [], "title": "EigenAI: Deterministic Inference, Verifiable Results", "authors": [ "D. Ribeiro Alves", "V. Patankar", "M. Pereira", "J. Stephens", "N. Vaziri", "S. Kannan" ], "year": 2026, "date": "2026-01-30", "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/ribeiro-alves-eigenai/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2602.00182", "version_note": "v1 (30 Jan 2026), the only version, read in HTML.", "note": "Developer's paper on its own system (all authors give eigenlabs.org addresses); statements about EigenAI are provider-reported. Describes a deterministic inference engine built on llama.cpp with custom GEMM and reduction kernels, and an optimistic re-execution protocol in which a stake-weighted committee re-executes challenged outputs inside TEEs. Table 5 reports bitwise-identical outputs on the same host and GPU and across hosts with the same GPU SKU, and a 0.0% match between A100 and H100.", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2602.00182", "source_file": "content/sources/ribeiro-alves-eigenai.yaml", "flags_all": [], "cited_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "I-0016", "title": "Batch-invariant inference kernels (Thinking Machines)", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/" } ], "referenced_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "I-0016", "title": "Batch-invariant inference kernels (Thinking Machines)", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/" } ] }, { "id": "S-1504", "slug": "rinberg-auditor-in-a-box", "status": "published", "risk_flags": [], "title": "Auditor-in-a-Box: Tools for Third-Party Auditing", "authors": [ "R. Rinberg", "B. Penchas" ], "year": 2026, "date": "2026-07-28", "venue": "LessWrong", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/rinberg-auditor-in-a-box/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Authors' own post describing a reference implementation (code linked from the post at github.com/RoyRin/auditor-in-a-TEE; its README was empty when fetched on 2026-09-23, so the code was not read).", "organizations": [], "source_type": "blog", "source_url": "https://www.lesswrong.com/posts/uWYk7MM9hAf9GEbGe/auditor-in-a-box-tools-for-third-party-auditing", "source_file": "content/sources/rinberg-auditor-in-a-box.yaml", "flags_all": [], "cited_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" } ], "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" } ] }, { "id": "S-1508", "slug": "rinberg-haiku-to-opus-compression", "status": "published", "risk_flags": [], "title": "Haiku to Opus in Just 10 bits: LLMs Unlock Large Compression Gains", "authors": [ "R. Rinberg", "A. M. Carrell", "S. Henniger", "N. Carlini", "K. Warr" ], "year": 2026, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/rinberg-haiku-to-opus-compression/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2604.02343", "version_note": "v2 or later, read via the arXiv HTML rendering; v1 was titled \"Haiku to Opus in Just 10 bits: LLMs Unlock Massive Compression Gains\". Code: https://github.com/RoyRin/model-compression-bit-limiting. No venue found as of 2026-09-25.", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2604.02343", "source_file": "content/sources/rinberg-haiku-to-opus-compression.yaml", "flags_all": [], "cited_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" } ], "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" } ] }, { "id": "S-0015", "slug": "rinberg-verifying-llm-inference-weight-exfiltration", "status": "published", "risk_flags": [], "title": "Verifying LLM Inference to Detect Model Weight Exfiltration", "authors": [ "R. Rinberg", "A. Karvonen", "A. Hoover", "D. Reuter", "K. Warr" ], "year": 2025, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/rinberg-verifying-llm-inference-weight-exfiltration/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2511.02620", "arxiv": "2511.02620", "version_note": "arXiv v1 4 Nov 2025, v2 10 Dec 2025, v3 12 Mar 2026; v3 read. No journal or conference version found as of 2026-09-25.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Inference verification\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2511.02620", "source_file": "content/sources/rinberg-verifying-llm-inference-weight-exfiltration.yaml", "flags_all": [], "cited_by": [ { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0009", "title": "Recomputation", "url": "https://trustbutveri.fyi/concepts/recomputation/" }, { "id": "K-0022", "title": "Weight exfiltration", "url": "https://trustbutveri.fyi/concepts/weight-exfiltration/" } ], "referenced_by": [ { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0001", "title": "Sampled inference recomputation", "url": "https://trustbutveri.fyi/mechanisms/sampled-inference-recomputation/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "I-0002", "title": "DiFR (Divergence From Reference)", "url": "https://trustbutveri.fyi/implementations/difr/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0009", "title": "Recomputation", "url": "https://trustbutveri.fyi/concepts/recomputation/" }, { "id": "K-0022", "title": "Weight exfiltration", "url": "https://trustbutveri.fyi/concepts/weight-exfiltration/" } ] }, { "id": "S-1501", "slug": "sahara-verifiable-clawguard-code", "status": "published", "risk_flags": [], "title": "Verifiable-ClawGuard: proof-of-guardrail reference code", "authors": [ "SaharaLabsAI" ], "year": 2026, "venue": "GitHub", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/sahara-verifiable-clawguard-code/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "main branch as viewed on 2026-09-23, README read via raw.githubusercontent.com. No release tag exists; the main branch head at that time was commit 77a4fdbcf9f7ff196bf577bd42b8817472729126 (from git ls-remote). README re-read on 2026-09-25; the repository page still showed 28 commits and no releases, and the commit list could not be read (robots.txt).", "license_note": "MIT (as stated in the repository).", "note": "Code release linked from S-1500. The README describes it as a proof of concept that is not production-ready.", "organizations": [], "source_type": "code", "source_url": "https://github.com/SaharaLabsAI/Verifiable-ClawGuard", "source_file": "content/sources/sahara-verifiable-clawguard-code.yaml", "flags_all": [], "cited_by": [ { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" } ], "referenced_by": [ { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" } ] }, { "id": "S-3606", "slug": "sash-about", "status": "published", "risk_flags": [], "title": "Singapore AI Safety Hub: About", "authors": [], "year": 2026, "venue": "Singapore AI Safety Hub", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/sash-about/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Official organization page, read 2026-09-25; the year is the access year because the page is undated. Supports the organization's attributed self-description: research and field-building organisation rooted in Singapore.", "organizations": [ "O-0160" ], "source_type": "docs", "source_url": "https://www.aisafety.sg/about", "source_file": "content/sources/sash-about.yaml", "flags_all": [], "cited_by": [ { "id": "O-0160", "title": "Singapore AI Safety Hub (SASH)", "url": "https://trustbutveri.fyi/organizations/singapore-ai-safety-hub/" } ], "referenced_by": [ { "id": "O-0160", "title": "Singapore AI Safety Hub (SASH)", "url": "https://trustbutveri.fyi/organizations/singapore-ai-safety-hub/" } ] }, { "id": "S-3200", "slug": "sash-field-notes-international-ai-verification", "status": "published", "risk_flags": [], "title": "Field Notes on International AI Verification from Shanghai, Seoul, and Sydney", "authors": [ "Singapore AI Safety Hub (SASH)" ], "year": 2026, "venue": "SASH blog", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/sash-field-notes-international-ai-verification/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Dated July 2026 on the page; no individual byline.", "organizations": [ "O-0160" ], "source_type": "blog", "source_url": "https://www.aisafety.sg/blog/field-notes-on-international-ai-verification-from-shanghai-seoul-and-sydney", "source_file": "content/sources/sash-field-notes-international-ai-verification.yaml", "flags_all": [], "cited_by": [ { "id": "I-0008", "title": "SASH confidential network logger", "url": "https://trustbutveri.fyi/implementations/sash-confidential-network-logger/" } ], "referenced_by": [ { "id": "I-0008", "title": "SASH confidential network logger", "url": "https://trustbutveri.fyi/implementations/sash-confidential-network-logger/" } ] }, { "id": "S-1319", "slug": "sash-inference-verification-repo", "status": "published", "risk_flags": [], "title": "inference-verification: Inference Verification Prototype", "authors": [ "Singapore AI Safety Hub (SASH)" ], "year": 2026, "venue": "GitHub", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/sash-inference-verification-repo/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Pinned to commit d7353b55fea6f36a6faa801011b203df7f0945e3 (head of main on 2026-09-23, committed 2026-09-02; 35 commits, no tagged releases).", "license_note": "MIT (per the repository's LICENSE.txt, \"Copyright (c) 2026 Project Contributors\").", "organizations": [ "O-0160" ], "source_type": "code", "source_url": "https://github.com/sg-ai-safety-hub/inference-verification", "source_file": "content/sources/sash-inference-verification-repo.yaml", "flags_all": [], "cited_by": [ { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "I-0008", "title": "SASH confidential network logger", "url": "https://trustbutveri.fyi/implementations/sash-confidential-network-logger/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "O-0160", "title": "Singapore AI Safety Hub (SASH)", "url": "https://trustbutveri.fyi/organizations/singapore-ai-safety-hub/" } ], "referenced_by": [ { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "I-0008", "title": "SASH confidential network logger", "url": "https://trustbutveri.fyi/implementations/sash-confidential-network-logger/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "O-0160", "title": "Singapore AI Safety Hub (SASH)", "url": "https://trustbutveri.fyi/organizations/singapore-ai-safety-hub/" } ] }, { "id": "S-1320", "slug": "sash-internationalising-ai-verification", "status": "published", "risk_flags": [], "title": "Internationalising AI Verification", "authors": [ "Singapore AI Safety Hub (SASH)" ], "year": 2026, "venue": "SASH blog", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/sash-internationalising-ai-verification/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Dated June 2026 on the page; no individual byline.", "organizations": [ "O-0160" ], "source_type": "blog", "source_url": "https://www.aisafety.sg/blog/making-ai-verification-international", "source_file": "content/sources/sash-internationalising-ai-verification.yaml", "flags_all": [], "cited_by": [ { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "I-0008", "title": "SASH confidential network logger", "url": "https://trustbutveri.fyi/implementations/sash-confidential-network-logger/" }, { "id": "O-0206", "title": "Future of Life Institute", "url": "https://trustbutveri.fyi/organizations/future-of-life-institute/" }, { "id": "O-0205", "title": "Oxford Martin AI Governance Initiative", "url": "https://trustbutveri.fyi/organizations/oxford-martin-ai-governance-initiative/" }, { "id": "O-0160", "title": "Singapore AI Safety Hub (SASH)", "url": "https://trustbutveri.fyi/organizations/singapore-ai-safety-hub/" } ], "referenced_by": [ { "id": "M-0013", "title": "Network taps and certifiers", "url": "https://trustbutveri.fyi/mechanisms/network-taps-and-certifiers/" }, { "id": "I-0008", "title": "SASH confidential network logger", "url": "https://trustbutveri.fyi/implementations/sash-confidential-network-logger/" }, { "id": "O-0206", "title": "Future of Life Institute", "url": "https://trustbutveri.fyi/organizations/future-of-life-institute/" }, { "id": "O-0205", "title": "Oxford Martin AI Governance Initiative", "url": "https://trustbutveri.fyi/organizations/oxford-martin-ai-governance-initiative/" }, { "id": "O-0160", "title": "Singapore AI Safety Hub (SASH)", "url": "https://trustbutveri.fyi/organizations/singapore-ai-safety-hub/" } ] }, { "id": "S-0053", "slug": "sastry-computing-power-governance-ai", "status": "published", "risk_flags": [], "title": "Computing Power and the Governance of Artificial Intelligence", "authors": [ "G. Sastry", "L. Heim", "H. Belfield", "M. Anderljung", "M. Brundage", "J. Hazell", "C. O'Keefe", "G. K. Hadfield", "R. Ngo", "K. Pilz", "G. Gor", "E. Bluemke", "S. Shoker", "J. Egan", "R. F. Trager", "S. Avin", "A. Weller", "Y. Bengio", "D. Coyle" ], "year": 2024, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/sastry-computing-power-governance-ai/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2402.08797", "arxiv": "2402.08797", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Motivations and policy proposals\" in the Hodgkins bibliography (CC BY 4.0). Also listed on the Oxford Martin AI Governance Initiative's publications page (https://aigi.ox.ac.uk/publications/computing-power-and-the-governance-of-artificial-intelligence/, read 2026-09-24). Also presented as a GovAI research paper (https://www.governance.ai/research-paper/computing-power-and-the-governance-of-artificial-intelligence, read 2026-09-24).", "organizations": [ "O-0205", "O-0211" ], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2402.08797", "source_file": "content/sources/sastry-computing-power-governance-ai.yaml", "flags_all": [], "cited_by": [ { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0001", "title": "Compute stock is at most a declared amount", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0017", "title": "Compartmentalization", "url": "https://trustbutveri.fyi/concepts/compartmentalization/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" }, { "id": "K-0025", "title": "Inference and training workloads", "url": "https://trustbutveri.fyi/concepts/inference-and-training-workloads/" }, { "id": "K-0021", "title": "Interconnect bandwidth", "url": "https://trustbutveri.fyi/concepts/interconnect-bandwidth/" }, { "id": "K-0016", "title": "Undeclared compute", "url": "https://trustbutveri.fyi/concepts/undeclared-compute/" }, { "id": "O-0211", "title": "Centre for the Governance of AI", "url": "https://trustbutveri.fyi/organizations/centre-for-the-governance-of-ai/" } ], "referenced_by": [ { "id": "M-0019", "title": "Chip registries and manufacturing records", "url": "https://trustbutveri.fyi/mechanisms/chip-registries-and-manufacturing-records/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0001", "title": "Compute stock is at most a declared amount", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0017", "title": "Compartmentalization", "url": "https://trustbutveri.fyi/concepts/compartmentalization/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" }, { "id": "K-0025", "title": "Inference and training workloads", "url": "https://trustbutveri.fyi/concepts/inference-and-training-workloads/" }, { "id": "K-0021", "title": "Interconnect bandwidth", "url": "https://trustbutveri.fyi/concepts/interconnect-bandwidth/" }, { "id": "K-0016", "title": "Undeclared compute", "url": "https://trustbutveri.fyi/concepts/undeclared-compute/" }, { "id": "O-0211", "title": "Centre for the Governance of AI", "url": "https://trustbutveri.fyi/organizations/centre-for-the-governance-of-ai/" } ] }, { "id": "S-0063", "slug": "scher-agreement-prevent-premature-asi", "status": "published", "risk_flags": [], "title": "An International Agreement to Prevent the Premature Creation of Artificial Superintelligence", "authors": [ "A. Scher", "D. Abecassis", "P. Barnett", "B. Abeyta" ], "year": 2025, "venue": "Machine Intelligence Research Institute", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/scher-agreement-prevent-premature-asi/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2511.10783", "arxiv": "2511.10783", "version_note": "v3 (2026-05-08) read on 2026-09-25; the Strict and Monitored Thresholds are in §4.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Motivations and policy proposals\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [ "O-0202" ], "source_type": "tech-report", "source_url": "https://arxiv.org/abs/2511.10783", "source_file": "content/sources/scher-agreement-prevent-premature-asi.yaml", "flags_all": [], "cited_by": [ { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "C-0001", "title": "Compute stock is at most a declared amount", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" } ], "referenced_by": [ { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "C-0001", "title": "Compute stock is at most a declared amount", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" } ] }, { "id": "S-3220", "slug": "scher-derisking-interconnect-limits", "status": "published", "risk_flags": [], "title": "De-risking Interconnect Limits for AI Verification", "authors": [ "A. Scher", "D. Sarbakysh", "A. Moskvin" ], "year": 2026, "date": "2026-07-30", "venue": "MIRI Technical Governance Team", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/scher-derisking-interconnect-limits/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Prototype of inter-node bandwidth monitoring against a threshold on a two-node Azure cluster (four A100 PCIe GPUs, one 100 GbE link), with violating training, compliant inference and a DiLoCo evasion workload. Carried out by Sarbakysh and Moskvin through SPAR under Scher's supervision. Code and reproduction instructions are linked at https://github.com/Yayka/ml-infra-profiler (MIT licence); no commit could be pinned, because the commit list could not be read on 2026-09-25.", "organizations": [ "O-0202" ], "source_type": "blog", "source_url": "https://techgov.intelligence.org/blog/de-risking-interconnect-limits-for-ai-verification", "source_file": "content/sources/scher-derisking-interconnect-limits.yaml", "flags_all": [], "cited_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" } ], "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" } ] }, { "id": "S-0005", "slug": "scher-mechanisms-verify-ai-agreements", "status": "published", "risk_flags": [], "title": "Mechanisms to Verify International Agreements About AI Development", "authors": [ "A. Scher", "L. Thiergart" ], "year": 2025, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/scher-mechanisms-verify-ai-agreements/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2506.15867", "arxiv": "2506.15867", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Research agendas and surveys\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [ "O-0202" ], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2506.15867", "source_file": "content/sources/scher-mechanisms-verify-ai-agreements.yaml", "flags_all": [], "cited_by": [ { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "C-0001", "title": "Compute stock is at most a declared amount", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0017", "title": "Compartmentalization", "url": "https://trustbutveri.fyi/concepts/compartmentalization/" }, { "id": "K-0019", "title": "Evidence binding", "url": "https://trustbutveri.fyi/concepts/evidence-binding/" }, { "id": "K-0025", "title": "Inference and training workloads", "url": "https://trustbutveri.fyi/concepts/inference-and-training-workloads/" }, { "id": "K-0021", "title": "Interconnect bandwidth", "url": "https://trustbutveri.fyi/concepts/interconnect-bandwidth/" }, { "id": "K-0003", "title": "Positive and negative claims", "url": "https://trustbutveri.fyi/concepts/positive-and-negative-claims/" }, { "id": "K-0011", "title": "Proof of (useful) work", "url": "https://trustbutveri.fyi/concepts/proof-of-useful-work/" }, { "id": "K-0004", "title": "Remote attestation", "url": "https://trustbutveri.fyi/concepts/remote-attestation/" }, { "id": "K-0013", "title": "Side channel", "url": "https://trustbutveri.fyi/concepts/side-channel/" }, { "id": "K-0016", "title": "Undeclared compute", "url": "https://trustbutveri.fyi/concepts/undeclared-compute/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" } ], "referenced_by": [ { "id": "M-0007", "title": "Proofs of useful work for capacity accounting", "url": "https://trustbutveri.fyi/mechanisms/proofs-of-useful-work/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "C-0001", "title": "Compute stock is at most a declared amount", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "K-0017", "title": "Compartmentalization", "url": "https://trustbutveri.fyi/concepts/compartmentalization/" }, { "id": "K-0019", "title": "Evidence binding", "url": "https://trustbutveri.fyi/concepts/evidence-binding/" }, { "id": "K-0025", "title": "Inference and training workloads", "url": "https://trustbutveri.fyi/concepts/inference-and-training-workloads/" }, { "id": "K-0021", "title": "Interconnect bandwidth", "url": "https://trustbutveri.fyi/concepts/interconnect-bandwidth/" }, { "id": "K-0003", "title": "Positive and negative claims", "url": "https://trustbutveri.fyi/concepts/positive-and-negative-claims/" }, { "id": "K-0011", "title": "Proof of (useful) work", "url": "https://trustbutveri.fyi/concepts/proof-of-useful-work/" }, { "id": "K-0004", "title": "Remote attestation", "url": "https://trustbutveri.fyi/concepts/remote-attestation/" }, { "id": "K-0013", "title": "Side channel", "url": "https://trustbutveri.fyi/concepts/side-channel/" }, { "id": "K-0016", "title": "Undeclared compute", "url": "https://trustbutveri.fyi/concepts/undeclared-compute/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" } ] }, { "id": "S-3127", "slug": "schluter-fabricked", "status": "published", "risk_flags": [], "title": "Fabricked: Misconfiguring Infinity Fabric to Break AMD SEV-SNP", "authors": [ "B. Schlüter", "C. Wech", "S. Shinde" ], "year": 2026, "venue": "35th USENIX Security Symposium (USENIX Security '26)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/schluter-fabricked/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "USENIX Security '26 presentation page (pp. 5613–5632, August 2026) and the paper PDF (https://www.shwetashinde.org/publications/fabricked_usenix26.pdf), read on 2026-09-25. The authors' site (https://fabricked-attack.github.io/) could not be read by the fetch tool.", "note": "Independent attack paper (ETH Zurich). A software adversary that controls the hypervisor and UEFI firmware misconfigures Infinity Fabric routing so that writes by the AMD Secure Processor during SEV-SNP initialisation go astray, leaving the system falsely initialised. Demonstrated on a Zen 5 EPYC 9135: arbitrary read and write in the victim CVM, debug mode enabled after attestation, and forged attestation reports. The authors could not test Zen 3 or Zen 4. AMD's bulletin is S-3128.", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://www.usenix.org/conference/usenixsecurity26/presentation/schlueter-1", "source_file": "content/sources/schluter-fabricked.yaml", "flags_all": [], "cited_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ], "referenced_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ] }, { "id": "S-1212", "slug": "schluter-rmpocalypse", "status": "published", "risk_flags": [], "title": "RMPocalypse: How a Catch-22 Breaks AMD SEV-SNP", "authors": [ "B. Schlüter", "S. Shinde" ], "year": 2025, "venue": "2025 ACM SIGSAC Conference on Computer and Communications Security (CCS '25)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/schluter-rmpocalypse/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.1145/3719027.3765233", "arxiv": null, "version_note": "Read the authors' site and the paper PDF linked from it (https://rmpocalypse.github.io/rmpocalypse-CCS2025.pdf) on 2026-09-24. The DOI and venue are from the ACM reference block printed in the PDF.", "note": "Independent attack paper (ETH Zurich). A malicious hypervisor corrupts AMD's Reverse Map Table during SEV-SNP initialisation, with no physical access. Confirmed on Zen 3, Zen 4 and Zen 5 processors. Demonstrates enabling debug on production confidential VMs, faking attestation, register-state replay and code injection. AMD assigned CVE-2025-0033; AMD's bulletin is S-1213.", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://rmpocalypse.github.io/", "source_file": "content/sources/schluter-rmpocalypse.yaml", "flags_all": [], "cited_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" } ], "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" } ] }, { "id": "S-0009", "slug": "schnabl-attestable-audits", "status": "published", "risk_flags": [], "title": "Attestable Audits: Verifiable AI Safety Benchmarks Using Trusted Execution Environments", "authors": [ "C. Schnabl", "D. Hugenroth", "B. Marino", "A. R. Beresford" ], "year": 2025, "venue": "ICML 2025 Workshop on Technical AI Governance", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/schnabl-attestable-audits/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2506.23706", "arxiv": "2506.23706", "version_note": "arXiv v1 (2025-06-30) read on 2026-09-23 and 2026-09-25; no later version or archival publication found. The arXiv comment reads \"ICML 2024 Workshop TAIG\", and the PDF and HTML carry the ICML 2025 template header (\"Proceedings of the 42nd International Conference on Machine Learning ... PMLR 267\"). The ICML 2025 virtual site (https://icml.cc/virtual/2025/48334) lists it as a workshop poster at the Workshop on Technical AI Governance, and it was not found in the PMLR 267 volume, so it is recorded as a workshop paper, not a main-conference publication.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Trusted execution and attestation\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [ "O-0142" ], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2506.23706", "source_file": "content/sources/schnabl-attestable-audits.yaml", "flags_all": [], "cited_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0007", "title": "Attestable Audits", "url": "https://trustbutveri.fyi/implementations/attestable-audits/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" }, { "id": "K-0006", "title": "Trusted execution environment (TEE)", "url": "https://trustbutveri.fyi/concepts/trusted-execution-environment/" }, { "id": "O-0142", "title": "University of Cambridge", "url": "https://trustbutveri.fyi/organizations/university-of-cambridge/" } ], "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0007", "title": "Attestable Audits", "url": "https://trustbutveri.fyi/implementations/attestable-audits/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" }, { "id": "K-0006", "title": "Trusted execution environment (TEE)", "url": "https://trustbutveri.fyi/concepts/trusted-execution-environment/" }, { "id": "O-0142", "title": "University of Cambridge", "url": "https://trustbutveri.fyi/organizations/university-of-cambridge/" } ] }, { "id": "S-0059", "slug": "seferis-detecting-compute-structuring", "status": "published", "risk_flags": [], "title": "Detecting Compute Structuring in AI Governance Is Likely Feasible", "authors": [ "E. Seferis", "T. Fist" ], "year": 2026, "venue": "Proceedings of the AAAI Conference on Artificial Intelligence 40(44), pp. 37904–37912 (AAAI-26, Special Track on AI Alignment)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/seferis-detecting-compute-structuring/", "archive_url": null, "accessed": "2026-09-23", "doi": "10.1609/aaai.v40i44.41127", "arxiv": null, "version_note": "An earlier version appeared at the ICML 2025 Workshop on Technical AI Governance (https://openreview.net/forum?id=qseqw1sWzz). AAAI proceedings page checked 2026-09-24 for venue, pages and DOI.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Motivations and policy proposals\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://ojs.aaai.org/index.php/AAAI/article/view/41127", "source_file": "content/sources/seferis-detecting-compute-structuring.yaml", "flags_all": [], "cited_by": [ { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" } ], "referenced_by": [ { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" } ] }, { "id": "S-1804", "slug": "selmanaj-hacking-apple-pcc", "status": "published", "risk_flags": [], "title": "Beyond Prompt Injection: Hacking Apple's Private Cloud Compute", "authors": [ "D. Selmanaj" ], "year": 2026, "date": "2026-07-31", "venue": "Sentry blog", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/selmanaj-hacking-apple-pcc/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Independent researcher's write-up of CVE-2026-20685, found in Apple's Virtual Research Environment: a path traversal in darwin-init's cryptex extraction gave root file writes that survived the node's userspace reboot and redirected splunkloggingd telemetry, exposing per-request metadata such as token counts and timings (the post does not say whether prompt or response content was exposed). All work was done in the VRE. States that the tampered node was indistinguishable from a clean one under `pccvre attestation verify`, and that Apple paid a $150,000 bounty.", "organizations": [], "source_type": "blog", "source_url": "https://blog.sentry.security/beyond-prompt-injection-hacking-apples-private-cloud-compute/", "source_file": "content/sources/selmanaj-hacking-apple-pcc.yaml", "flags_all": [], "cited_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/" } ], "referenced_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0013", "title": "Apple Private Cloud Compute", "url": "https://trustbutveri.fyi/implementations/apple-private-cloud-compute/" } ] }, { "id": "S-1307", "slug": "seshadri-swatt", "status": "published", "risk_flags": [], "title": "SWATT: SoftWare-based ATTestation for Embedded Devices", "authors": [ "A. Seshadri", "A. Perrig", "L. van Doorn", "P. Khosla" ], "year": 2004, "venue": "IEEE Symposium on Security and Privacy 2004, pp. 272–282", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/seshadri-swatt/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.1109/SECPRI.2004.1301329", "arxiv": null, "note": "Venue confirmed from the ETH Zurich Network Security Group publication list; pages and DOI from Crossref, checked 2026-09-25.", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://netsec.ethz.ch/publications/papers/swatt.pdf", "source_file": "content/sources/seshadri-swatt.yaml", "flags_all": [], "cited_by": [ { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" } ], "referenced_by": [ { "id": "M-0016", "title": "Timed challenge-response and memory-occupation challenges", "url": "https://trustbutveri.fyi/mechanisms/timed-challenge-response/" } ] }, { "id": "S-1211", "slug": "seto-wiretap", "status": "published", "risk_flags": [], "title": "WireTap: Breaking Server SGX via DRAM Bus Interposition", "authors": [ "A. Seto", "O. K. Duran", "S. Amer", "J. Chuang", "S. van Schaik", "D. Genkin", "C. Garman" ], "year": 2025, "venue": "2025 ACM SIGSAC Conference on Computer and Communications Security (CCS '25)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/seto-wiretap/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.1145/3719027.3765204", "arxiv": null, "version_note": "Read the authors' site and the paper PDF linked from it (https://wiretap.fail/files/wiretap.pdf) on 2026-09-24. The DOI and venue are from the ACM reference block printed in the PDF; the ACM Digital Library page could not be fetched.", "note": "Independent attack paper (Purdue University, Georgia Tech). DDR4 memory-bus interposer built for under $1000 (bill of materials $857) that records bus traffic. Recovers the ECDSA attestation key of Intel SGX's Quoting Enclave from a Xeon Scalable server in fully trusted status and forges SGX DCAP quotes. The site states that Intel considers the attack outside the SGX threat model and that there are no mitigations besides physical security. Several authors also wrote TEE.fail (S-1202).", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://wiretap.fail/", "source_file": "content/sources/seto-wiretap.yaml", "flags_all": [], "cited_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ], "referenced_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ] }, { "id": "S-1012", "slug": "sglang-deterministic-inference", "status": "published", "risk_flags": [], "title": "Towards Deterministic Inference in SGLang and Reproducible RL Training", "authors": [ "The SGLang Team" ], "year": 2025, "date": "2025-09-22", "venue": "LMSYS Org blog", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/sglang-deterministic-inference/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Developers' post about their own inference engine; performance figures are provider-reported. The post states it was updated on 24 Sep 2025.", "organizations": [], "source_type": "blog", "source_url": "https://www.lmsys.org/blog/2025-09-22-sglang-deterministic/", "source_file": "content/sources/sglang-deterministic-inference.yaml", "flags_all": [], "cited_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "I-0016", "title": "Batch-invariant inference kernels (Thinking Machines)", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/" }, { "id": "O-0214", "title": "Thinking Machines Lab", "url": "https://trustbutveri.fyi/organizations/thinking-machines-lab/" } ], "referenced_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "I-0016", "title": "Batch-invariant inference kernels (Thinking Machines)", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/" }, { "id": "O-0214", "title": "Thinking Machines Lab", "url": "https://trustbutveri.fyi/organizations/thinking-machines-lab/" } ] }, { "id": "S-0029", "slug": "shavit-catch-a-chinchilla", "status": "published", "risk_flags": [], "title": "What does it take to catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring", "authors": [ "Y. Shavit" ], "year": 2023, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/shavit-catch-a-chinchilla/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2303.11341", "arxiv": "2303.11341", "version_note": "Read arXiv v2 (30 May 2023).", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Proof of learning and training\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2303.11341", "source_file": "content/sources/shavit-catch-a-chinchilla.yaml", "flags_all": [], "cited_by": [ { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "C-0001", "title": "Compute stock is at most a declared amount", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0019", "title": "Evidence binding", "url": "https://trustbutveri.fyi/concepts/evidence-binding/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" }, { "id": "K-0025", "title": "Inference and training workloads", "url": "https://trustbutveri.fyi/concepts/inference-and-training-workloads/" }, { "id": "K-0008", "title": "Numerical nondeterminism", "url": "https://trustbutveri.fyi/concepts/numerical-nondeterminism/" }, { "id": "K-0003", "title": "Positive and negative claims", "url": "https://trustbutveri.fyi/concepts/positive-and-negative-claims/" }, { "id": "K-0001", "title": "Prover", "url": "https://trustbutveri.fyi/concepts/prover/" }, { "id": "K-0009", "title": "Recomputation", "url": "https://trustbutveri.fyi/concepts/recomputation/" }, { "id": "K-0005", "title": "Root of trust", "url": "https://trustbutveri.fyi/concepts/root-of-trust/" }, { "id": "K-0020", "title": "Sampling and assurance", "url": "https://trustbutveri.fyi/concepts/sampling-and-assurance/" }, { "id": "K-0015", "title": "Tamper evidence and tamper resistance", "url": "https://trustbutveri.fyi/concepts/tamper-evidence/" }, { "id": "K-0018", "title": "Threat model", "url": "https://trustbutveri.fyi/concepts/threat-model/" }, { "id": "K-0016", "title": "Undeclared compute", "url": "https://trustbutveri.fyi/concepts/undeclared-compute/" }, { "id": "K-0002", "title": "Verifier", "url": "https://trustbutveri.fyi/concepts/verifier/" } ], "referenced_by": [ { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" }, { "id": "C-0008", "title": "Communication between compute groups is bounded", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "C-0001", "title": "Compute stock is at most a declared amount", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/" }, { "id": "C-0004", "title": "This compute runs inference, not training", "url": "https://trustbutveri.fyi/claims/inference-not-training/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0019", "title": "Evidence binding", "url": "https://trustbutveri.fyi/concepts/evidence-binding/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" }, { "id": "K-0025", "title": "Inference and training workloads", "url": "https://trustbutveri.fyi/concepts/inference-and-training-workloads/" }, { "id": "K-0008", "title": "Numerical nondeterminism", "url": "https://trustbutveri.fyi/concepts/numerical-nondeterminism/" }, { "id": "K-0003", "title": "Positive and negative claims", "url": "https://trustbutveri.fyi/concepts/positive-and-negative-claims/" }, { "id": "K-0001", "title": "Prover", "url": "https://trustbutveri.fyi/concepts/prover/" }, { "id": "K-0009", "title": "Recomputation", "url": "https://trustbutveri.fyi/concepts/recomputation/" }, { "id": "K-0005", "title": "Root of trust", "url": "https://trustbutveri.fyi/concepts/root-of-trust/" }, { "id": "K-0020", "title": "Sampling and assurance", "url": "https://trustbutveri.fyi/concepts/sampling-and-assurance/" }, { "id": "K-0015", "title": "Tamper evidence and tamper resistance", "url": "https://trustbutveri.fyi/concepts/tamper-evidence/" }, { "id": "K-0018", "title": "Threat model", "url": "https://trustbutveri.fyi/concepts/threat-model/" }, { "id": "K-0016", "title": "Undeclared compute", "url": "https://trustbutveri.fyi/concepts/undeclared-compute/" }, { "id": "K-0002", "title": "Verifier", "url": "https://trustbutveri.fyi/concepts/verifier/" } ] }, { "id": "S-3130", "slug": "shen-vcek-seed-milan", "status": "published", "risk_flags": [], "title": "Insecure Despite Proven Updated: Extracting the Root VCEK Seed on EPYC Milan via a Software-Only Attack", "authors": [ "M. Shen", "Y. Qin" ], "year": 2026, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/shen-vcek-seed-milan/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2605.12990", "arxiv": "2605.12990", "version_note": "Read the arXiv HTML on 2026-09-25; the abstract page could not be read by the fetch tool, so the version and submission date were not confirmed. No venue is given.", "note": "Attack preprint (Institute of Software, Chinese Academy of Sciences). With root control of the host and the ability to rewrite the SPI flash, but no physical access, the attacker downgrades an EPYC Milan (Zen 3) platform to legacy AMD Secure Processor firmware, gains code execution there (MilanLaunchy) and extracts the full hardware root seed from which SEV-SNP's VCEK attestation keys are derived (BadFuse). The authors state this lets them forge attestation reports for any firmware version. Disclosed to AMD in January and April 2026; AMD's bulletin on MilanLaunchy is S-3131.", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2605.12990", "source_file": "content/sources/shen-vcek-seed-milan.yaml", "flags_all": [], "cited_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ], "referenced_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ] }, { "id": "S-1316", "slug": "smartt-tamper-indicating-enclosures-survey", "status": "published", "risk_flags": [], "title": "Tamper-Indicating Enclosures, A Current Survey", "authors": [ "H. A. Smartt", "Z. N. Gastelum" ], "year": 2015, "venue": "Sandia National Laboratories, SAND2015-4251C", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/smartt-tamper-indicating-enclosures-survey/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "organizations": [], "source_type": "tech-report", "source_url": "https://www.osti.gov/servlets/purl/1256541", "source_file": "content/sources/smartt-tamper-indicating-enclosures-survey.yaml", "flags_all": [], "cited_by": [ { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" } ], "referenced_by": [ { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" } ] }, { "id": "S-0024", "slug": "south-verifiable-evaluations-zksnarks", "status": "published", "risk_flags": [], "title": "Verifiable evaluations of machine learning models using zkSNARKs", "authors": [ "T. South", "A. Camuto", "S. Jain", "S. Nguyen", "R. Mahari", "C. Paquin", "J. Morton", "A. Pentland" ], "year": 2024, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/south-verifiable-evaluations-zksnarks/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2402.02675", "arxiv": "2402.02675", "version_note": "v2 (22 May 2024) read; no journal or conference version is listed on arXiv as of 2026-09-25.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Zero-knowledge proofs\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [ "O-0213" ], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2402.02675", "source_file": "content/sources/south-verifiable-evaluations-zksnarks.yaml", "flags_all": [], "cited_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0014", "title": "EZKL", "url": "https://trustbutveri.fyi/implementations/ezkl/" }, { "id": "O-0213", "title": "Zkonduit", "url": "https://trustbutveri.fyi/organizations/zkonduit/" } ], "referenced_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0014", "title": "EZKL", "url": "https://trustbutveri.fyi/implementations/ezkl/" }, { "id": "O-0213", "title": "Zkonduit", "url": "https://trustbutveri.fyi/organizations/zkonduit/" } ] }, { "id": "S-1404", "slug": "sovereignty-certificates-specification", "status": "published", "risk_flags": [], "title": "Sovereignty Certificates: draft specification, version 0.1.0", "authors": [ "Sovereignty Certificates Working Group" ], "year": 2025, "date": "2025-10-21", "venue": "GitHub (Lucid-Computing/sovereignty-certificate-specification)", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/sovereignty-certificates-specification/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Status \"Draft\", version 0.1.0, dated 2025-10-21, \"Prepared by: Sovereignty Certificates Working Group\". Read from spec/sovereignty-certificates.md on the main branch on 2026-09-23, at commit e65f5c1ba19094430bf586917b4760151759e990 (commit dated 2026-02-09; the repository's v0.1.0 tag points to the earlier commit 1d4bd34a6bff342db4fa4a43d143fe95eaa46b7d). Re-read on 2026-09-25: the repository still shows three commits and no code, and the title block, threat model (§8.1), §7.1.2-7.1.4, §8.3.3-8.3.4 and Annex B read as before.", "license_note": "The README states the specification is licensed CC BY 4.0.", "note": "Hosted in Lucid Computing's GitHub organization. Documentation about the specifying party's own system, so it supports \"the specification states / Lucid reports\" statements only.", "organizations": [ "O-0180" ], "source_type": "docs", "source_url": "https://github.com/Lucid-Computing/sovereignty-certificate-specification", "source_file": "content/sources/sovereignty-certificates-specification.yaml", "flags_all": [], "cited_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "I-0009", "title": "Lucid sovereignty (location) certificates", "url": "https://trustbutveri.fyi/implementations/lucid-location-certificates/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "O-0180", "title": "Lucid Computing", "url": "https://trustbutveri.fyi/organizations/lucid-computing/" } ], "referenced_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "I-0009", "title": "Lucid sovereignty (location) certificates", "url": "https://trustbutveri.fyi/implementations/lucid-location-certificates/" }, { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "O-0180", "title": "Lucid Computing", "url": "https://trustbutveri.fyi/organizations/lucid-computing/" } ] }, { "id": "S-1405", "slug": "sovereignty-certificates-working-group-site", "status": "published", "risk_flags": [], "title": "Sovereignty Certificates Working Group", "authors": [], "year": 2026, "venue": "sovcert.org", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/sovereignty-certificates-working-group-site/", "archive_url": null, "accessed": "2026-09-23", "doi": null, "arxiv": null, "note": "Working-group website; content copyright 2025-2026, no page date. Content as read on 2026-09-23; the site could not be reached on 2026-09-25 (connection error). Self-description of the initiative, so it supports \"the working group states\" statements only.", "organizations": [], "source_type": "docs", "source_url": "https://sovcert.org/", "source_file": "content/sources/sovereignty-certificates-working-group-site.yaml", "flags_all": [], "cited_by": [ { "id": "I-0009", "title": "Lucid sovereignty (location) certificates", "url": "https://trustbutveri.fyi/implementations/lucid-location-certificates/" } ], "referenced_by": [ { "id": "I-0009", "title": "Lucid sovereignty (location) certificates", "url": "https://trustbutveri.fyi/implementations/lucid-location-certificates/" } ] }, { "id": "S-3081", "slug": "srivastava-optimistic-verifiable-training", "status": "published", "risk_flags": [], "title": "Optimistic Verifiable Training by Controlling Hardware Nondeterminism", "authors": [ "M. Srivastava", "S. Arora", "D. Boneh" ], "year": 2024, "venue": "Advances in Neural Information Processing Systems 37 (NeurIPS 2024)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/srivastava-optimistic-verifiable-training/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2403.09603", "version_note": "arXiv v3 (25 November 2024) read in HTML; v1 posted 14 March 2024. Venue from the arXiv comments field and the code README. The NeurIPS proceedings page was not read.", "note": "Code at https://github.com/meghabyte/verifiable-training (not pinned). Exact FP32 replication of ResNet-50 (CIFAR-10) training and GPT-2 fine-tuning across NVIDIA A40, Titan XP and RTX 2080 Ti; trainer overhead 1.2–1.4×, auditor 1.3–1.7×; assumes one of n auditors is honest.", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://arxiv.org/abs/2403.09603", "source_file": "content/sources/srivastava-optimistic-verifiable-training.yaml", "flags_all": [], "cited_by": [ { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" } ], "referenced_by": [ { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" } ] }, { "id": "S-0052", "slug": "staat-anti-tamper-radio", "status": "published", "risk_flags": [], "title": "Anti-Tamper Radio: System-Level Tamper Detection for Computing Systems", "authors": [ "P. Staat", "J. Tobisch", "C. Zenger", "C. Paar" ], "year": 2022, "venue": "2022 IEEE Symposium on Security and Privacy", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/staat-anti-tamper-radio/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.1109/SP46214.2022.00067", "arxiv": "2112.09014", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Tamper resistance and detection\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://ieeexplore.ieee.org/document/9833631/", "source_file": "content/sources/staat-anti-tamper-radio.yaml", "flags_all": [], "cited_by": [ { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" } ], "referenced_by": [ { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" } ] }, { "id": "S-1108", "slug": "sun-zkllm-code", "status": "published", "risk_flags": [], "title": "zkllm-ccs2024: code for zkLLM: Zero Knowledge Proofs for Large Language Models", "authors": [ "H. Sun" ], "year": 2024, "venue": "GitHub; archived on Zenodo", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/sun-zkllm-code/", "archive_url": "https://zenodo.org/records/13621754", "accessed": "2026-09-25", "doi": "10.5281/zenodo.13621754", "arxiv": null, "version_note": "Tag ae-alpha (Zenodo release of 3 September 2024, after CCS 2024 artifact evaluation). The GitHub repository was archived read-only on 10 July 2025; the README read at access is from main.", "license_note": "MIT (GitHub repository); the Zenodo record is CC BY 4.0.", "organizations": [ "O-0122" ], "source_type": "code", "source_url": "https://github.com/jvhs0706/zkllm-ccs2024", "source_file": "content/sources/sun-zkllm-code.yaml", "flags_all": [], "cited_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0003", "title": "zkLLM", "url": "https://trustbutveri.fyi/implementations/zkllm/" }, { "id": "O-0122", "title": "University of Waterloo", "url": "https://trustbutveri.fyi/organizations/university-of-waterloo/" } ], "referenced_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0003", "title": "zkLLM", "url": "https://trustbutveri.fyi/implementations/zkllm/" }, { "id": "O-0122", "title": "University of Waterloo", "url": "https://trustbutveri.fyi/organizations/university-of-waterloo/" } ] }, { "id": "S-0023", "slug": "sun-zkllm", "status": "published", "risk_flags": [], "title": "zkLLM: Zero Knowledge Proofs for Large Language Models", "authors": [ "H. Sun", "J. Li", "H. Zhang" ], "year": 2024, "venue": "2024 ACM SIGSAC Conference on Computer and Communications Security (CCS 2024)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/sun-zkllm/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.1145/3658644.3670334", "arxiv": "2404.16109", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Zero-knowledge proofs\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [ "O-0122" ], "source_type": "peer-reviewed", "source_url": "https://doi.org/10.1145/3658644.3670334", "source_file": "content/sources/sun-zkllm.yaml", "flags_all": [], "cited_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "M-0005", "title": "Zero-knowledge proofs of training constraints", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-training-constraints/" }, { "id": "I-0003", "title": "zkLLM", "url": "https://trustbutveri.fyi/implementations/zkllm/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "K-0024", "title": "Cryptographic commitment", "url": "https://trustbutveri.fyi/concepts/cryptographic-commitment/" }, { "id": "K-0001", "title": "Prover", "url": "https://trustbutveri.fyi/concepts/prover/" }, { "id": "K-0010", "title": "Zero-knowledge proof", "url": "https://trustbutveri.fyi/concepts/zero-knowledge-proof/" }, { "id": "O-0122", "title": "University of Waterloo", "url": "https://trustbutveri.fyi/organizations/university-of-waterloo/" } ], "referenced_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "M-0005", "title": "Zero-knowledge proofs of training constraints", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-training-constraints/" }, { "id": "I-0003", "title": "zkLLM", "url": "https://trustbutveri.fyi/implementations/zkllm/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "K-0024", "title": "Cryptographic commitment", "url": "https://trustbutveri.fyi/concepts/cryptographic-commitment/" }, { "id": "K-0001", "title": "Prover", "url": "https://trustbutveri.fyi/concepts/prover/" }, { "id": "K-0010", "title": "Zero-knowledge proof", "url": "https://trustbutveri.fyi/concepts/zero-knowledge-proof/" }, { "id": "O-0122", "title": "University of Waterloo", "url": "https://trustbutveri.fyi/organizations/university-of-waterloo/" } ] }, { "id": "S-3261", "slug": "tabar-anti-tamper-radio-ris", "status": "published", "risk_flags": [], "title": "Anti-Tamper Radio Meets Reconfigurable Intelligent Surface for System-Level Tamper Detection", "authors": [ "M. S. Tabar", "J. Kortz", "P. Staat", "H. Elders-Boll", "C. Paar", "C. Zenger" ], "year": 2025, "venue": "18th ACM Conference on Security and Privacy in Wireless and Mobile Networks (WiSec 2025)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/tabar-anti-tamper-radio-ris/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.1145/3734477.3734702", "arxiv": "2503.14279", "version_note": "Read via the arXiv HTML (v1) and PDF; the ACM DOI page returned HTTP 403. Venue and author order from the dblp record (conf/wisec/TabarKSEPZ25) and P. Staat's publication list.", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://arxiv.org/abs/2503.14279", "source_file": "content/sources/tabar-anti-tamper-radio-ris.yaml", "flags_all": [], "cited_by": [ { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" } ], "referenced_by": [ { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" } ] }, { "id": "S-1602", "slug": "tcg-glossary", "status": "published", "risk_flags": [], "title": "TCG Glossary", "authors": [ "Trusted Computing Group" ], "year": 2017, "date": "2017-05-11", "venue": "Trusted Computing Group", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/tcg-glossary/", "archive_url": "https://web.archive.org/web/20260902144328/https://trustedcomputinggroup.org/resource/tcg-glossary/", "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Version 1.1, Revision 1.00 (11 May 2017). Full text read at https://trustedcomputinggroup.org/wp-content/uploads/TCG-Glossary-V1.1-Rev-1.0.pdf", "organizations": [], "source_type": "docs", "source_url": "https://trustedcomputinggroup.org/resource/tcg-glossary/", "source_file": "content/sources/tcg-glossary.yaml", "flags_all": [], "cited_by": [ { "id": "K-0004", "title": "Remote attestation", "url": "https://trustbutveri.fyi/concepts/remote-attestation/" }, { "id": "K-0005", "title": "Root of trust", "url": "https://trustbutveri.fyi/concepts/root-of-trust/" } ], "referenced_by": [ { "id": "K-0004", "title": "Remote attestation", "url": "https://trustbutveri.fyi/concepts/remote-attestation/" }, { "id": "K-0005", "title": "Root of trust", "url": "https://trustbutveri.fyi/concepts/root-of-trust/" } ] }, { "id": "S-1403", "slug": "tee-gpu-fingerprinting-location-verification", "status": "published", "risk_flags": [], "title": "GPU Fingerprinting for Location Verification", "authors": [ "W. Tee", "J. Happel" ], "year": 2026, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/tee-gpu-fingerprinting-location-verification/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2605.01930", "version_note": "v1 (2026-05-03) read on 2026-09-23; v2 (2026-06-01) read on 2026-09-25 via arxiv.org/html/2605.01930v2. The figures cited (24 H200 GPUs, 480 runs, 98.8% single-run and 100% paired-run accuracy, about 2.9 s per run) and the stated limitations are unchanged in v2. Affiliations: W. Tee (Pivotal Research), J. Happel (TamperSec).", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2605.01930", "source_file": "content/sources/tee-gpu-fingerprinting-location-verification.yaml", "flags_all": [], "cited_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "I-0009", "title": "Lucid sovereignty (location) certificates", "url": "https://trustbutveri.fyi/implementations/lucid-location-certificates/" } ], "referenced_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" }, { "id": "I-0009", "title": "Lucid sovereignty (location) certificates", "url": "https://trustbutveri.fyi/implementations/lucid-location-certificates/" } ] }, { "id": "S-1813", "slug": "thinking-machines-batch-invariant-ops-code", "status": "published", "risk_flags": [], "title": "thinking-machines-lab/batch_invariant_ops (GitHub repository)", "authors": [ "Thinking Machines Lab" ], "year": 2025, "venue": "GitHub", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/thinking-machines-batch-invariant-ops-code/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Main branch as viewed on 2026-09-24 (7 commits). MIT licence.", "note": "Companion library to S-1009. Replaces torch.mm, torch.addmm, torch.log_softmax and torch.mean with batch-invariant kernels through torch.Library, and includes a deterministic vLLM example: 18 unique samples out of 1000 completions without the upstream vLLM change, one unique sample with it.", "organizations": [ "O-0214" ], "source_type": "code", "source_url": "https://github.com/thinking-machines-lab/batch_invariant_ops", "source_file": "content/sources/thinking-machines-batch-invariant-ops-code.yaml", "flags_all": [], "cited_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "I-0016", "title": "Batch-invariant inference kernels (Thinking Machines)", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/" }, { "id": "O-0214", "title": "Thinking Machines Lab", "url": "https://trustbutveri.fyi/organizations/thinking-machines-lab/" } ], "referenced_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "I-0016", "title": "Batch-invariant inference kernels (Thinking Machines)", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/" }, { "id": "O-0214", "title": "Thinking Machines Lab", "url": "https://trustbutveri.fyi/organizations/thinking-machines-lab/" } ] }, { "id": "S-3562", "slug": "thinking-machines-homepage", "status": "published", "risk_flags": [], "title": "Thinking Machines Lab", "authors": [], "year": 2026, "venue": "Thinking Machines Lab", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/thinking-machines-homepage/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Company homepage, read 2026-09-25. States: \"Thinking Machines Lab is an artificial intelligence research and product company.\" Lists its models, the Tinker platform and its research blog, Connectionism. Supports \"Thinking Machines reports\" statements only.", "organizations": [ "O-0214" ], "source_type": "docs", "source_url": "https://thinkingmachines.ai/", "source_file": "content/sources/thinking-machines-homepage.yaml", "flags_all": [], "cited_by": [ { "id": "O-0214", "title": "Thinking Machines Lab", "url": "https://trustbutveri.fyi/organizations/thinking-machines-lab/" } ], "referenced_by": [ { "id": "O-0214", "title": "Thinking Machines Lab", "url": "https://trustbutveri.fyi/organizations/thinking-machines-lab/" } ] }, { "id": "S-3360", "slug": "tinfoil-auditing-frontier-model-without-weights", "status": "published", "risk_flags": [], "title": "Auditing a Frontier Model Without Seeing its Weights", "authors": [ "D. McCann-Sayles", "T. Verma" ], "year": 2026, "date": "2026-06-09", "venue": "Tinfoil blog", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/tinfoil-auditing-frontier-model-without-weights/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Tinfoil's account of Pour Demain running interpretability evaluations (vllm-lens) on GLM-5.1, about 744 billion parameters, on eight H200 GPUs in Tinfoil's confidential-computing platform, with the lab supplying the model, the auditor its evaluation code, and the weights kept inside the enclave. Reports 33-38% overhead with interpretability hooks enabled. Supports only \"Tinfoil reports\" statements.", "organizations": [ "O-0141" ], "source_type": "blog", "source_url": "https://tinfoil.sh/blog/2026-06-09-auditing-frontier-model-without-weights", "source_file": "content/sources/tinfoil-auditing-frontier-model-without-weights.yaml", "flags_all": [], "cited_by": [ { "id": "O-0141", "title": "Tinfoil", "url": "https://trustbutveri.fyi/organizations/tinfoil/" } ], "referenced_by": [ { "id": "O-0141", "title": "Tinfoil", "url": "https://trustbutveri.fyi/organizations/tinfoil/" } ] }, { "id": "S-1207", "slug": "tinfoil-docs-attestation-architecture", "status": "published", "risk_flags": [], "title": "Backend infrastructure", "authors": [ "Tinfoil" ], "year": 2026, "venue": "Tinfoil documentation", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/tinfoil-docs-attestation-architecture/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Living documentation with no publication date shown. The year is the year the page was accessed.", "note": "Tinfoil's description of its measured boot chain, reproducible builds, Sigstore publication of expected measurements, and client-side verification. Supports \"Tinfoil reports\" statements.", "organizations": [ "O-0141" ], "source_type": "docs", "source_url": "https://docs.tinfoil.sh/verification/attestation-architecture", "source_file": "content/sources/tinfoil-docs-attestation-architecture.yaml", "flags_all": [], "cited_by": [ { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" }, { "id": "O-0141", "title": "Tinfoil", "url": "https://trustbutveri.fyi/organizations/tinfoil/" } ], "referenced_by": [ { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" }, { "id": "O-0141", "title": "Tinfoil", "url": "https://trustbutveri.fyi/organizations/tinfoil/" } ] }, { "id": "S-1206", "slug": "tinfoil-docs-secure-enclave-primer", "status": "published", "risk_flags": [], "title": "A primer on secure enclaves", "authors": [ "Tinfoil" ], "year": 2026, "venue": "Tinfoil documentation", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/tinfoil-docs-secure-enclave-primer/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Living documentation with no publication date shown. The year is the year the page was accessed.", "note": "Tinfoil's own documentation: the hardware it supports, its trust model, and the limitations it documents (physical attacks, side channels, I/O leakage, denial of service, supply chain, rollback). Supports \"Tinfoil reports\" statements.", "organizations": [ "O-0141" ], "source_type": "docs", "source_url": "https://docs.tinfoil.sh/verification/secure-enclave-primer", "source_file": "content/sources/tinfoil-docs-secure-enclave-primer.yaml", "flags_all": [], "cited_by": [ { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" }, { "id": "O-0140", "title": "NVIDIA", "url": "https://trustbutveri.fyi/organizations/nvidia/" }, { "id": "O-0141", "title": "Tinfoil", "url": "https://trustbutveri.fyi/organizations/tinfoil/" } ], "referenced_by": [ { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" }, { "id": "O-0140", "title": "NVIDIA", "url": "https://trustbutveri.fyi/organizations/nvidia/" }, { "id": "O-0141", "title": "Tinfoil", "url": "https://trustbutveri.fyi/organizations/tinfoil/" } ] }, { "id": "S-1208", "slug": "tinfoil-docs-verification-in-tinfoil", "status": "published", "risk_flags": [], "title": "How verification works in Tinfoil", "authors": [ "Tinfoil" ], "year": 2026, "venue": "Tinfoil documentation", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/tinfoil-docs-verification-in-tinfoil/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Living documentation with no publication date shown. The year is the year the page was accessed.", "note": "Tinfoil's overview of connection-time verification and transparency logs. Supports \"Tinfoil reports\" statements.", "organizations": [ "O-0141" ], "source_type": "docs", "source_url": "https://docs.tinfoil.sh/verification/verification-in-tinfoil", "source_file": "content/sources/tinfoil-docs-verification-in-tinfoil.yaml", "flags_all": [], "cited_by": [ { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "O-0141", "title": "Tinfoil", "url": "https://trustbutveri.fyi/organizations/tinfoil/" } ], "referenced_by": [ { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "O-0141", "title": "Tinfoil", "url": "https://trustbutveri.fyi/organizations/tinfoil/" } ] }, { "id": "S-3605", "slug": "tinfoil-homepage", "status": "published", "risk_flags": [], "title": "Tinfoil homepage", "authors": [], "year": 2026, "venue": "Tinfoil", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/tinfoil-homepage/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Official organization page, read 2026-09-25; the year is the access year because the page is undated. Supports the organization's attributed self-description: AI models running inside secure hardware enclaves.", "organizations": [ "O-0141" ], "source_type": "docs", "source_url": "https://tinfoil.sh/", "source_file": "content/sources/tinfoil-homepage.yaml", "flags_all": [], "cited_by": [ { "id": "O-0141", "title": "Tinfoil", "url": "https://trustbutveri.fyi/organizations/tinfoil/" } ], "referenced_by": [ { "id": "O-0141", "title": "Tinfoil", "url": "https://trustbutveri.fyi/organizations/tinfoil/" } ] }, { "id": "S-1209", "slug": "tinfoil-modelwrap-code", "status": "published", "risk_flags": [], "title": "modelwrap: Reproducible dm-verity read-only image of Huggingface models", "authors": [ "Tinfoil" ], "year": 2026, "venue": "GitHub", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/tinfoil-modelwrap-code/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Release v0.3.0 (commit b6850e3), still the latest release listed on 2026-09-25.", "license_note": "MIT", "note": "Open-source tool Tinfoil uses to commit model weights to a dm-verity root hash (see S-0013).", "organizations": [ "O-0141" ], "source_type": "code", "source_url": "https://github.com/tinfoilsh/modelwrap", "source_file": "content/sources/tinfoil-modelwrap-code.yaml", "flags_all": [], "cited_by": [ { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" }, { "id": "O-0141", "title": "Tinfoil", "url": "https://trustbutveri.fyi/organizations/tinfoil/" } ], "referenced_by": [ { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" }, { "id": "O-0141", "title": "Tinfoil", "url": "https://trustbutveri.fyi/organizations/tinfoil/" } ] }, { "id": "S-0013", "slug": "tinfoil-proving-model-identity", "status": "published", "risk_flags": [], "title": "How Tinfoil Proves Exactly What Model Is Running", "authors": [ "Tinfoil Team" ], "year": 2026, "venue": "Tinfoil", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/tinfoil-proving-model-identity/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Trusted execution and attestation\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [ "O-0141" ], "source_type": "blog", "source_url": "https://tinfoil.sh/blog/2026-02-03-proving-model-identity", "source_file": "content/sources/tinfoil-proving-model-identity.yaml", "flags_all": [], "cited_by": [ { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "O-0141", "title": "Tinfoil", "url": "https://trustbutveri.fyi/organizations/tinfoil/" } ], "referenced_by": [ { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "I-0006", "title": "Tinfoil model identity (Modelwrap)", "url": "https://trustbutveri.fyi/implementations/tinfoil-model-identity/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "O-0141", "title": "Tinfoil", "url": "https://trustbutveri.fyi/organizations/tinfoil/" } ] }, { "id": "S-3362", "slug": "tinfoil-safety-without-compromising-privacy", "status": "published", "risk_flags": [], "title": "Safety Without Compromising on Privacy", "authors": [ "D. McCann-Sayles", "S. Servan-Schreiber", "T. Verma" ], "year": 2026, "date": "2026-09-14", "venue": "Tinfoil blog", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/tinfoil-safety-without-compromising-privacy/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "Tinfoil's description of its own safeguard pipeline: safeguard models that \"run exclusively inside secure enclaves\" and output only a flag, with the pipeline code public and its enforcement \"verifiable through attestation\". Updated 16 September 2026. The post does not say explicitly which safeguard components are in the attested measurement. Supports only \"Tinfoil reports\" statements.", "organizations": [ "O-0141" ], "source_type": "blog", "source_url": "https://tinfoil.sh/blog/2026-09-14-safety-without-compromising-privacy", "source_file": "content/sources/tinfoil-safety-without-compromising-privacy.yaml", "flags_all": [], "cited_by": [ { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "O-0141", "title": "Tinfoil", "url": "https://trustbutveri.fyi/organizations/tinfoil/" } ], "referenced_by": [ { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "O-0141", "title": "Tinfoil", "url": "https://trustbutveri.fyi/organizations/tinfoil/" } ] }, { "id": "S-3361", "slug": "tlaie-confidential-computing-frontier-ai-auditing", "status": "published", "risk_flags": [], "title": "Confidential computing can enable better frontier AI auditing", "authors": [ "A. Tlaie Boria" ], "year": 2026, "date": "2026-07-22", "venue": "Pour Demain", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/tlaie-confidential-computing-frontier-ai-auditing/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "The auditor's own account. States that all measurements were collected on a production confidential-computing platform (Tinfoil Containers, Intel TDX, 8x NVIDIA H200) running GLM-5.1, a 744-billion-parameter open-weights mixture-of-experts model; five interpretability workloads; \"raw tensors never cross the enclave boundary\". Governance features were shown at single-session level only. Links a PDF white paper that was not read.", "organizations": [], "source_type": "blog", "source_url": "https://www.pourdemain.ngo/en/post/confidential-computing-can-enable-better-frontier-ai-auditing", "source_file": "content/sources/tlaie-confidential-computing-frontier-ai-auditing.yaml", "flags_all": [], "cited_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "O-0141", "title": "Tinfoil", "url": "https://trustbutveri.fyi/organizations/tinfoil/" } ], "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "O-0141", "title": "Tinfoil", "url": "https://trustbutveri.fyi/organizations/tinfoil/" } ] }, { "id": "S-0061", "slug": "trager-international-governance-civilian-ai", "status": "published", "risk_flags": [], "title": "International Governance of Civilian AI: A Jurisdictional Certification Approach", "authors": [ "R. Trager", "B. Harack", "A. Reuel", "A. Carnegie", "L. Heim", "L. Ho", "S. Kreps", "R. Lall", "O. Larter", "S. Ó hÉigeartaigh", "S. Staffell", "J. J. Villalobos" ], "year": 2023, "venue": "Centre for the Governance of AI", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/trager-international-governance-civilian-ai/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2308.15514", "arxiv": "2308.15514", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Motivations and policy proposals\" in the Hodgkins bibliography (CC BY 4.0). Also listed on the Oxford Martin AI Governance Initiative's publications page (https://aigi.ox.ac.uk/publications/international-governance-of-civilian-ai-a-jurisdictional-certification-approach/, read 2026-09-24). Also presented as a GovAI research paper (https://www.governance.ai/research-paper/international-governance-of-civilian-ai, read 2026-09-24).", "organizations": [ "O-0205", "O-0211" ], "source_type": "tech-report", "source_url": "https://arxiv.org/abs/2308.15514", "source_file": "content/sources/trager-international-governance-civilian-ai.yaml", "flags_all": [], "cited_by": [ { "id": "O-0211", "title": "Centre for the Governance of AI", "url": "https://trustbutveri.fyi/organizations/centre-for-the-governance-of-ai/" } ], "referenced_by": [ { "id": "O-0211", "title": "Centre for the Governance of AI", "url": "https://trustbutveri.fyi/organizations/centre-for-the-governance-of-ai/" } ] }, { "id": "S-0070", "slug": "trailofbits-ezkl-security-assessment", "status": "published", "risk_flags": [], "title": "Zkonduit EZKL Security Assessment", "authors": [ "F. Casal", "T. Hess", "L. Bourtoule", "S. Hussain", "G. Larregay" ], "year": 2025, "date": "2025-03-12", "venue": "Trail of Bits (prepared for Zkonduit Inc.)", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/trailofbits-ezkl-security-assessment/", "archive_url": null, "accessed": "2026-09-23", "doi": null, "arxiv": null, "version_note": "Final report of 12 March 2025. Review of 6–27 January 2025 (11 engineer-weeks) of ezkl at commit bdcba5ca61ada24f17dd754e6e3c71d0a1ef72d9 and two halo2-solidity-verifier pull requests; fix review 3–4 March 2025 (Appendix D).", "note": "Independent security audit of the ezkl zero-knowledge inference library, commissioned by its developer. Reports 34 findings (8 high severity), including circuit soundness issues and quantization-activated model backdoors. Appendix D's limited fix review marks 29 resolved (including all 8 high-severity issues), 3 partially resolved and 2 unresolved. It says some contract fixes were in private repositories and had not been merged into public repositories at review time.", "organizations": [], "source_type": "tech-report", "source_url": "https://github.com/trailofbits/publications/blob/master/reviews/2025-03-zkonduit-ezkl-securityreview.pdf", "source_file": "content/sources/trailofbits-ezkl-security-assessment.yaml", "flags_all": [], "cited_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0014", "title": "EZKL", "url": "https://trustbutveri.fyi/implementations/ezkl/" }, { "id": "O-0213", "title": "Zkonduit", "url": "https://trustbutveri.fyi/organizations/zkonduit/" } ], "referenced_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" }, { "id": "I-0014", "title": "EZKL", "url": "https://trustbutveri.fyi/implementations/ezkl/" }, { "id": "O-0213", "title": "Zkonduit", "url": "https://trustbutveri.fyi/organizations/zkonduit/" } ] }, { "id": "S-3124", "slug": "trailofbits-meta-whatsapp-private-processing-review", "status": "published", "risk_flags": [], "title": "Meta WhatsApp Private Processing (security review)", "authors": [ "Trail of Bits" ], "year": 2025, "venue": "Trail of Bits publications library", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/trailofbits-meta-whatsapp-private-processing-review/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Library entry for the review dated August 2025. The full report (https://github.com/trailofbits/publications/blob/master/reviews/2025-08-meta-whatsapp-privateprocessing-securityreview.pdf) could not be read by the fetch tool on 2026-09-25; the library entry gives the finding titles and severities.", "note": "Independent auditors' summary of their review of WhatsApp Private Processing. Lists 28 issues (8 high, 4 medium, 4 low, 12 informational). The eight high-severity findings include attestation without freshness, the SEV-SNP TCB version not checked against the VCEK certificate, SEV-SNP attestation not bound to Meta-specific machines, CVMs compromised through environment-variable injection, ACPI SSDT injection by a malicious hypervisor, CVM images that cannot be reproduced, and GPU-hosted models that do not verify NVIDIA GPU attestation. The client is not named on the page.", "organizations": [], "source_type": "docs", "source_url": "https://trailofbits.com/library/meta-whatsapp-private-processing/", "source_file": "content/sources/trailofbits-meta-whatsapp-private-processing-review.yaml", "flags_all": [], "cited_by": [ { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ], "referenced_by": [ { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ] }, { "id": "S-3121", "slug": "trailofbits-whatsapp-private-inference-audit", "status": "published", "risk_flags": [], "title": "What we learned about TEE security from auditing WhatsApp's Private Inference", "authors": [ "Trail of Bits" ], "year": 2026, "date": "2026-04-07", "venue": "Trail of Bits blog", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/trailofbits-whatsapp-private-inference-audit/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "The auditors' own summary of their pre-launch security review of WhatsApp Private Processing (AMD SEV-SNP with NVIDIA confidential GPUs); the post does not say who commissioned it. Reports 28 issues, eight of high severity, and describes environment variables and ACPI tables loaded outside the attestation measurement, patch levels trusted without checking AMD's certificates, and attestations with no freshness guarantee. Meta resolved 16 issues and partly addressed four before launch; the remaining eight are low or informational. States that the SEV-SNP threat model does not fully protect against advanced physical attacks, so Meta added controls on which CPUs are trusted (TOB-WAPI-10). The full report (https://github.com/trailofbits/publications/blob/master/reviews/2025-08-meta-whatsapp-privateprocessing-securityreview.pdf, dated August 2025) could not be read by the fetch tool on 2026-09-25; finding severities come from the library entry, S-3124.", "organizations": [], "source_type": "blog", "source_url": "https://blog.trailofbits.com/2026/04/07/what-we-learned-about-tee-security-from-auditing-whatsapps-private-inference/", "source_file": "content/sources/trailofbits-whatsapp-private-inference-audit.yaml", "flags_all": [], "cited_by": [ { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ], "referenced_by": [ { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ] }, { "id": "S-3320", "slug": "trask-double-blind-evals", "status": "published", "risk_flags": [], "title": "Double Blind Evals: Resolving the Dual Confidentiality Dilemma in AI Safety Auditing", "authors": [ "A. Trask", "S. Messing", "V. Pahwa", "P. Maham", "R. Kolga", "A. Frantz", "A. Tash", "K. Thomas", "S. McGregor", "G. Balston", "P. Paskov", "M. Brundage", "A. Vij", "B. Hillenbrand", "A. Karargyris", "T. Acosta", "J. Fenster", "M. Eilish", "R. Elasmar", "M. Khan", "K. van der Veen", "R. S", "S. Wagh", "S. Gabriel", "P. Werneck", "L. Strahm", "K. McDonough", "R. Falcon", "K. Lum", "W. Isaac" ], "year": 2026, "venue": "Google DeepMind", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/trask-double-blind-evals/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Technical report linked from Google DeepMind's blog post \"Piloting the world's first double-blind AI evaluations\" (https://deepmind.google/blog/piloting-the-worlds-first-double-blind-ai-evaluations/, dated 2026-08-27). The PDF prints no date. Author affiliations as printed: Google, AVERI, Singapore AISI, OpenMined and ML Commons. No arXiv version was found on 2026-09-25.", "note": "Report by the pilot's participants on their own system; supports \"the participants report\" statements. Gemini 2.5 Flash Lite was evaluated against private AILuminate prompts in a GCP Confidential Space enclave (a3-highgpu-1g, one NVIDIA H100 with Intel TDX), using OpenMined's PySyft; both parties verified the attestation before uploading assets.", "organizations": [ "O-0215" ], "source_type": "tech-report", "source_url": "https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/piloting-the-worlds-first-double-blind-ai-evaluations/double-blind-evaluations-technical-report.pdf", "source_file": "content/sources/trask-double-blind-evals.yaml", "flags_all": [], "cited_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "I-0017", "title": "PySyft double-blind evaluations", "url": "https://trustbutveri.fyi/implementations/pysyft-double-blind-evaluations/" }, { "id": "O-0215", "title": "OpenMined", "url": "https://trustbutveri.fyi/organizations/openmined/" } ], "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "I-0017", "title": "PySyft double-blind evaluations", "url": "https://trustbutveri.fyi/implementations/pysyft-double-blind-evaluations/" }, { "id": "O-0215", "title": "OpenMined", "url": "https://trustbutveri.fyi/organizations/openmined/" } ] }, { "id": "S-3570", "slug": "ulyssean-ping-based-location-demo", "status": "published", "risk_flags": [], "title": "Ping-based Location", "authors": [ "Ulyssean" ], "year": 2025, "venue": "Ulyssean demonstration site", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/ulyssean-ping-based-location-demo/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "The site gives no publication date. Its text was read from the public JavaScript asset https://ping-location.info/assets/index-sg1kynaJ.js linked by the home page on 2026-09-25. The 2025 IAPS brief (S-1401) links to the demonstration, so 2025 is the latest possible year for its initial publication.", "note": "Ulyssean's account of its H100 and AMD SEV-SNP demonstration, using 38 Google Cloud Run landmarks. It describes the setup and timing method and says the code will be open-sourced after cleanup. It gives no systematic end-to-end location results or error rates. Developer documentation supports attributed claims about this demo.", "organizations": [], "source_type": "docs", "source_url": "https://ping-location.info/", "source_file": "content/sources/ulyssean-ping-based-location-demo.yaml", "flags_all": [], "cited_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" } ], "referenced_by": [ { "id": "M-0018", "title": "Chip location verification", "url": "https://trustbutveri.fyi/mechanisms/chip-location-verification/" } ] }, { "id": "S-1013", "slug": "vllm-batch-invariance-docs", "status": "published", "risk_flags": [], "title": "Batch Invariance (vLLM documentation)", "authors": [ "vLLM project" ], "year": 2026, "venue": "vLLM documentation (GitHub, docs/features/batch_invariance.md)", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/vllm-batch-invariance-docs/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Main branch as viewed on 2026-09-25 (GitHub source file; the rendered page at docs.vllm.ai did not return body text to the fetch tool on 2026-09-23). The feature is described as beta, supported on NVIDIA GPUs of compute capability 8.0 or higher and on Intel XPUs with Triton, and tested on dense and mixture-of-experts models.", "organizations": [], "source_type": "docs", "source_url": "https://github.com/vllm-project/vllm/blob/main/docs/features/batch_invariance.md", "source_file": "content/sources/vllm-batch-invariance-docs.yaml", "flags_all": [], "cited_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "I-0016", "title": "Batch-invariant inference kernels (Thinking Machines)", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/" } ], "referenced_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "I-0016", "title": "Batch-invariant inference kernels (Thinking Machines)", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/" } ] }, { "id": "S-1814", "slug": "vllm-batch-invariance-tracking-issue", "status": "published", "risk_flags": [], "title": "[Feature]: Batch Invariant Feature and Performance Optimization (vLLM issue #27433)", "authors": [ "vLLM project contributors" ], "year": 2025, "venue": "GitHub (vllm-project/vllm issues)", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/vllm-batch-invariance-tracking-issue/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "note": "vLLM developers' tracking issue for batch invariance. Its body states \"We have basically support Batch Invariant based on\" the Thinking Machines post (S-1009), and lists open work such as performance, NVFP4, AMD testing and speculative decoding. Still open on 2026-09-25.", "organizations": [], "source_type": "forum", "source_url": "https://github.com/vllm-project/vllm/issues/27433", "source_file": "content/sources/vllm-batch-invariance-tracking-issue.yaml", "flags_all": [], "cited_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "I-0016", "title": "Batch-invariant inference kernels (Thinking Machines)", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/" }, { "id": "O-0214", "title": "Thinking Machines Lab", "url": "https://trustbutveri.fyi/organizations/thinking-machines-lab/" } ], "referenced_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" }, { "id": "I-0016", "title": "Batch-invariant inference kernels (Thinking Machines)", "url": "https://trustbutveri.fyi/implementations/batch-invariant-inference-kernels/" }, { "id": "O-0214", "title": "Thinking Machines Lab", "url": "https://trustbutveri.fyi/organizations/thinking-machines-lab/" } ] }, { "id": "S-0022", "slug": "waiwitlikhit-trustless-audits", "status": "published", "risk_flags": [], "title": "Trustless Audits without Revealing Data or Models", "authors": [ "S. Waiwitlikhit", "I. Stoica", "Y. Sun", "T. Hashimoto", "D. Kang" ], "year": 2024, "venue": "41st International Conference on Machine Learning (ICML 2024)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/waiwitlikhit-trustless-audits/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": "2404.04500", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Zero-knowledge proofs\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://proceedings.mlr.press/v235/waiwitlikhit24a.html", "source_file": "content/sources/waiwitlikhit-trustless-audits.yaml", "flags_all": [], "cited_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" }, { "id": "M-0005", "title": "Zero-knowledge proofs of training constraints", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-training-constraints/" } ], "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" }, { "id": "M-0005", "title": "Zero-knowledge proofs of training constraints", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-training-constraints/" } ] }, { "id": "S-0068", "slug": "wang-nanozk", "status": "published", "risk_flags": [], "title": "NanoZK: Privacy-Preserving Verifiable Inference for Large Language Models via Layerwise Zero-Knowledge Proofs", "authors": [ "Z. Wang" ], "year": 2026, "venue": "International Conference on Information and Communications Security (ICICS 2026)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/wang-nanozk/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2603.18046", "arxiv": "2603.18046", "version_note": "Current arXiv HTML read on 2026-09-25; its abstract matches v2, which states acceptance at the 28th ICICS (Springer LNCS, Fukui, 27–30 October 2026) and is an extended version with appendices not in the proceedings. v1 (March 2026) reports different timings. The abstract gives 3.5–3.7 KB per sub-circuit proof, while §3.1 and §6.1 give 3.2–3.7 KB.", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Zero-knowledge proofs\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://arxiv.org/abs/2603.18046", "source_file": "content/sources/wang-nanozk.yaml", "flags_all": [], "cited_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" } ], "referenced_by": [ { "id": "M-0004", "title": "Zero-knowledge proofs of inference", "url": "https://trustbutveri.fyi/mechanisms/zk-proofs-of-inference/" } ] }, { "id": "S-0062", "slug": "wasil-verification-methods-international-ai-agreements", "status": "published", "risk_flags": [], "title": "Verification methods for international AI agreements", "authors": [ "A. R. Wasil", "T. Reed", "J. W. Miller", "P. Barnett" ], "year": 2024, "venue": "arXiv", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/wasil-verification-methods-international-ai-agreements/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.48550/arXiv.2408.16074", "arxiv": "2408.16074", "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Motivations and policy proposals\" in the Hodgkins bibliography (CC BY 4.0).", "organizations": [], "source_type": "preprint", "source_url": "https://arxiv.org/abs/2408.16074", "source_file": "content/sources/wasil-verification-methods-international-ai-agreements.yaml", "flags_all": [], "cited_by": [ { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "C-0001", "title": "Compute stock is at most a declared amount", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" }, { "id": "K-0016", "title": "Undeclared compute", "url": "https://trustbutveri.fyi/concepts/undeclared-compute/" } ], "referenced_by": [ { "id": "C-0002", "title": "Chips are where they are declared to be", "url": "https://trustbutveri.fyi/claims/chips-are-where-declared/" }, { "id": "C-0001", "title": "Compute stock is at most a declared amount", "url": "https://trustbutveri.fyi/claims/compute-stock-is-bounded/" }, { "id": "C-0003", "title": "Declared hardware is idle or shut down", "url": "https://trustbutveri.fyi/claims/declared-hardware-is-idle/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" }, { "id": "C-0007", "title": "A training run stayed within declared limits", "url": "https://trustbutveri.fyi/claims/training-within-declared-limits/" }, { "id": "K-0023", "title": "FLOP accounting", "url": "https://trustbutveri.fyi/concepts/flop-accounting/" }, { "id": "K-0016", "title": "Undeclared compute", "url": "https://trustbutveri.fyi/concepts/undeclared-compute/" } ] }, { "id": "S-0073", "slug": "yang-part-time-power-measurements", "status": "published", "risk_flags": [], "title": "Accurate and Convenient Energy Measurements for GPUs: A Detailed Study of NVIDIA GPU's Built-In Power Sensor", "authors": [ "Z. Yang", "K. Adamek", "W. Armour" ], "year": 2024, "venue": "SC24: International Conference for High Performance Computing, Networking, Storage and Analysis", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/yang-part-time-power-measurements/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.1109/SC41406.2024.00028", "arxiv": "2312.02741", "version_note": "Published in the SC24 proceedings (presented 2024-11-19; DOI 10.1109/SC41406.2024.00028). The arXiv preprint is titled \"Part-time Power Measurements: nvidia-smi's Lack of Attention\" (v1 2023-12-05, v3 2024-12-12). The arXiv v3 abstract and HTML were read on 2026-09-25; the SC24 programme abstract was read the same day and states the 25% sampling finding. The IEEE and ACM pages could not be read.", "note": "Micro-benchmark study of nvidia-smi power readings on over 70 NVIDIA GPUs across 12 architecture generations. States that nvidia-smi uses NVML (§2.4), that on A100 and H100 GPUs only 25% of runtime is sampled for power, and that the reading's error is ±5% (within ±5% in most cases, §4.2) rather than the ±5 W NVIDIA claims.", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://arxiv.org/abs/2312.02741", "source_file": "content/sources/yang-part-time-power-measurements.yaml", "flags_all": [], "cited_by": [ { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" } ], "referenced_by": [ { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0021", "title": "Workload classification from telemetry and side channels", "url": "https://trustbutveri.fyi/mechanisms/workload-classification-from-telemetry/" } ] }, { "id": "S-3500", "slug": "yao-tao-tolerance-aware-verification", "status": "published", "risk_flags": [], "title": "TAO: Tolerance-Aware Optimistic Verification for Floating-Point Neural Networks", "authors": [ "J. Yao", "H. Su", "T. Liao", "Z. Cheng", "H. Zhang", "X. Wang", "P. Viswanath" ], "year": 2026, "venue": "Proceedings of the 21st European Conference on Computer Systems (EuroSys 2026), pp. 1515-1532", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/yao-tao-tolerance-aware-verification/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.1145/3767295.3803612", "arxiv": "2510.16028", "version_note": "Full text read from arXiv HTML v4 (6 Jun 2026; v1 15 Oct 2025). Venue, pages and DOI are from the arXiv journal reference. The ACM DOI page could not be read by the fetch tool (HTTP 403).", "note": "Yao, Cheng and Viswanath are at Princeton University, Su and Wang at HKUST (Guangzhou), Liao and Zhang at the University of Illinois Urbana-Champaign. TAO accepts operator-level outputs within bounds that combine IEEE-754 worst-case error bounds with empirical percentile profiles, instead of requiring bitwise equality, and keeps hardware heterogeneity. Disputes are settled by a Merkle-anchored, threshold-guided dispute game whose coordinator is a smart-contract deployment on the Ethereum Holesky testnet. Evaluated on RTX 4090, A100, H100 and RTX 6000 GPUs; reports 0.3% overhead on Qwen3-8B.", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://arxiv.org/abs/2510.16028", "source_file": "content/sources/yao-tao-tolerance-aware-verification.yaml", "flags_all": [], "cited_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" } ], "referenced_by": [ { "id": "M-0002", "title": "Deterministic and bit-exact inference", "url": "https://trustbutveri.fyi/mechanisms/deterministic-inference/" } ] }, { "id": "S-1109", "slug": "zhang-adversarial-examples-proof-of-learning", "status": "published", "risk_flags": [], "title": "\"Adversarial Examples\" for Proof-of-Learning", "authors": [ "R. Zhang", "J. Liu", "Y. Ding", "Z. Wang", "Q. Wu", "K. Ren" ], "year": 2022, "venue": "2022 IEEE Symposium on Security and Privacy (SP), pp. 1408-1422", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/zhang-adversarial-examples-proof-of-learning/", "archive_url": null, "accessed": "2026-09-25", "doi": "10.1109/SP46214.2022.9833596", "arxiv": "2108.09454", "note": "Read the arXiv version. Venue per the authors' code repository README (github.com/ZhangRui98/Adversarial-examples-for-Proof-of-Learning). The DOI resolves to IEEE Xplore document 9833596; OpenAlex gives the title, authors, venue and pages for it (checked 2026-09-25).", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://arxiv.org/abs/2108.09454", "source_file": "content/sources/zhang-adversarial-examples-proof-of-learning.yaml", "flags_all": [], "cited_by": [ { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" } ], "referenced_by": [ { "id": "M-0006", "title": "Training-transcript verification (proof-of-learning)", "url": "https://trustbutveri.fyi/mechanisms/proof-of-learning/" } ] }, { "id": "S-3123", "slug": "zhang-stackwarp", "status": "published", "risk_flags": [], "title": "StackWarp: Breaking AMD SEV-SNP Integrity via Deterministic Stack-Pointer Manipulation through the CPU's Stack Engine", "authors": [ "R. Zhang", "T. Hornetz", "D. Weber", "F. Thomas", "M. Schwarz" ], "year": 2026, "venue": "35th USENIX Security Symposium (USENIX Security '26)", "type": "source", "tier": "A", "url": "https://trustbutveri.fyi/sources/zhang-stackwarp/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "USENIX Security '26 presentation page (pp. 5691–5709, August 2026). Also read the authors' site (https://stackwarpattack.com/) and paper PDF (https://stackwarpattack.com/stackwarp_usenix26.pdf) on 2026-09-25.", "note": "Independent attack paper (CISPA Helmholtz Center for Information Security). A malicious hypervisor toggles an undocumented MSR bit on a sibling hyperthread to shift an SEV-SNP guest's stack pointer, breaking the guest's integrity; demonstrated on AMD Zen 1 to Zen 5 with simultaneous multithreading enabled, including OpenSSH authentication bypass, sudo privilege escalation and RSA key recovery. CVE-2025-29943; AMD released hot-loadable microcode patches (bulletin AMD-SB-3027), and disabling SMT is an interim mitigation. Proof-of-concept code at https://github.com/cispa/StackWarp.", "organizations": [], "source_type": "peer-reviewed", "source_url": "https://www.usenix.org/conference/usenixsecurity26/presentation/zhang-ruiyi", "source_file": "content/sources/zhang-stackwarp.yaml", "flags_all": [], "cited_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ], "referenced_by": [ { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" } ] }, { "id": "S-0014", "slug": "zhao-tees-privacy-preserving-monitoring", "status": "published", "risk_flags": [], "title": "On TEEs for Privacy-Preserving Monitoring in AI Governance", "authors": [ "Gloria Z" ], "year": 2026, "date": "2026-06-18", "venue": "MIRI Technical Governance Team", "type": "source", "tier": "C", "url": "https://trustbutveri.fyi/sources/zhao-tees-privacy-preserving-monitoring/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "imported_from": "hodgkins-ai-verification-papers@c71e59ff0e8e", "note": "Listed under \"Trusted execution and attestation\" in the Hodgkins bibliography (CC BY 4.0), which gives the author as \"Zhao\". The page itself (read 2026-09-23, dated 2026-06-18) shows the byline \"Aaron Scher\" and \"Gloria Z\", and states that the post was conducted as part of the MIRI Technical Governance Fellowship and \"authored solely by Gloria\", with thanks to Aaron Scher for guidance. The author is therefore recorded as shown on the page.", "organizations": [ "O-0202" ], "source_type": "blog", "source_url": "https://techgov.intelligence.org/blog/on-tees-for-privacy-preserving-monitoring-in-ai-governance", "source_file": "content/sources/zhao-tees-privacy-preserving-monitoring.yaml", "flags_all": [], "cited_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" }, { "id": "K-0005", "title": "Root of trust", "url": "https://trustbutveri.fyi/concepts/root-of-trust/" }, { "id": "K-0013", "title": "Side channel", "url": "https://trustbutveri.fyi/concepts/side-channel/" }, { "id": "K-0006", "title": "Trusted execution environment (TEE)", "url": "https://trustbutveri.fyi/concepts/trusted-execution-environment/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" } ], "referenced_by": [ { "id": "M-0025", "title": "Confidential multi-party verification", "url": "https://trustbutveri.fyi/mechanisms/confidential-multi-party-verification/" }, { "id": "M-0012", "title": "Model identity attestation", "url": "https://trustbutveri.fyi/mechanisms/model-identity-attestation/" }, { "id": "M-0010", "title": "On-chip telemetry from timing, memory and performance counters", "url": "https://trustbutveri.fyi/mechanisms/on-chip-telemetry/" }, { "id": "M-0023", "title": "Safeguard attestation", "url": "https://trustbutveri.fyi/mechanisms/safeguard-attestation/" }, { "id": "M-0008", "title": "TEE remote attestation for AI workloads", "url": "https://trustbutveri.fyi/mechanisms/tee-remote-attestation/" }, { "id": "C-0005", "title": "The declared model is the one being served", "url": "https://trustbutveri.fyi/claims/declared-model-is-served/" }, { "id": "C-0010", "title": "There is no undeclared relevant compute", "url": "https://trustbutveri.fyi/claims/no-undeclared-compute/" }, { "id": "C-0006", "title": "Declared safeguards were applied during inference", "url": "https://trustbutveri.fyi/claims/safeguards-were-applied/" }, { "id": "K-0005", "title": "Root of trust", "url": "https://trustbutveri.fyi/concepts/root-of-trust/" }, { "id": "K-0013", "title": "Side channel", "url": "https://trustbutveri.fyi/concepts/side-channel/" }, { "id": "K-0006", "title": "Trusted execution environment (TEE)", "url": "https://trustbutveri.fyi/concepts/trusted-execution-environment/" }, { "id": "O-0202", "title": "Machine Intelligence Research Institute", "url": "https://trustbutveri.fyi/organizations/machine-intelligence-research-institute/" } ] }, { "id": "S-1806", "slug": "zkonduit-ezkl-code", "status": "published", "risk_flags": [], "title": "zkonduit/ezkl (GitHub repository)", "authors": [ "Zkonduit Inc." ], "year": 2026, "venue": "GitHub", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/zkonduit-ezkl-code/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Main branch as viewed on 2026-09-25 (860 commits); the releases page lists v23.0.5 as the latest release. The root listing showed no licence file; the README carries a no-warranty notice and a contributor licence agreement.", "note": "Developer's own repository; supports \"Zkonduit reports\" statements about the tool. README: ezkl proves inference of ONNX models with halo2, supports public or private models and data, notes that quantization can make outputs differ slightly from Python, and links the Trail of Bits audit (S-0070).", "organizations": [ "O-0213" ], "source_type": "code", "source_url": "https://github.com/zkonduit/ezkl", "source_file": "content/sources/zkonduit-ezkl-code.yaml", "flags_all": [], "cited_by": [ { "id": "I-0014", "title": "EZKL", "url": "https://trustbutveri.fyi/implementations/ezkl/" }, { "id": "O-0213", "title": "Zkonduit", "url": "https://trustbutveri.fyi/organizations/zkonduit/" } ], "referenced_by": [ { "id": "I-0014", "title": "EZKL", "url": "https://trustbutveri.fyi/implementations/ezkl/" }, { "id": "O-0213", "title": "Zkonduit", "url": "https://trustbutveri.fyi/organizations/zkonduit/" } ] }, { "id": "S-1606", "slug": "zkproof-community-reference", "status": "published", "risk_flags": [], "title": "ZKProof Community Reference", "authors": [ "D. Benarroch", "L. Brandão", "M. Maller", "E. Tromer" ], "year": 2022, "date": "2022-07-17", "venue": "ZKProof", "type": "source", "tier": "B", "url": "https://trustbutveri.fyi/sources/zkproof-community-reference/", "archive_url": null, "accessed": "2026-09-25", "doi": null, "arxiv": null, "version_note": "Version 0.3 (17 July 2022). The listed persons are the editors of a community document.", "organizations": [], "source_type": "tech-report", "source_url": "https://docs.zkproof.org/reference.pdf", "source_file": "content/sources/zkproof-community-reference.yaml", "flags_all": [], "cited_by": [ { "id": "K-0024", "title": "Cryptographic commitment", "url": "https://trustbutveri.fyi/concepts/cryptographic-commitment/" }, { "id": "K-0001", "title": "Prover", "url": "https://trustbutveri.fyi/concepts/prover/" }, { "id": "K-0002", "title": "Verifier", "url": "https://trustbutveri.fyi/concepts/verifier/" }, { "id": "K-0010", "title": "Zero-knowledge proof", "url": "https://trustbutveri.fyi/concepts/zero-knowledge-proof/" } ], "referenced_by": [ { "id": "K-0024", "title": "Cryptographic commitment", "url": "https://trustbutveri.fyi/concepts/cryptographic-commitment/" }, { "id": "K-0001", "title": "Prover", "url": "https://trustbutveri.fyi/concepts/prover/" }, { "id": "K-0002", "title": "Verifier", "url": "https://trustbutveri.fyi/concepts/verifier/" }, { "id": "K-0010", "title": "Zero-knowledge proof", "url": "https://trustbutveri.fyi/concepts/zero-knowledge-proof/" } ] } ] }