{ "schema_version": "1.2.0", "rubric_version": "1.1", "license": "CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)", "record": { "id": "C-0008", "slug": "bandwidth-is-bounded", "title": "Communication between compute groups is bounded", "aliases": [ "bandwidth limits", "interconnect limits" ], "status": "published", "last_reviewed": "2026-09-23", "review_interval_days": 90, "steward": null, "provenance": { "drafted_by": "ai", "reviewed_by": [ "codex-review" ] }, "risk_flags": [], "flags": [], "one_liner": "Data flowing between specified groups of chips, or out of a facility, stays below a declared rate, so the groups cannot jointly run large workloads.", "summary": "The claim is that data flowing between specified groups of chips, or out of a facility, stays below a declared rate. A proposed pod design aims to allow inference tokens while restricting training gradients if the served model and routing fit within each pod. A verified bound on communication between compute groups could therefore restrict training while allowing inference. A bound on a facility's external links also limits how much weights or data could leave it. The claim is negative: no channel exceeds the bound, including channels outside the monitored network. Proposed designs cap the external bandwidth of small pods of chips or restrict chip networking in hardware, and use network taps to observe the links. The hard parts are finding every path, including physical side channels; choosing bounds that stay meaningful as distributed training methods improve; and monitoring high-speed accelerator fabrics that are difficult to tap.", "claim_class": "negative", "editors_synthesis": { "assessment": true, "markdown": "A software bandwidth monitor has been demonstrated (R2). It runs on the operator's nodes, and its authors say the measurements are trivially spoofable [[S-3220]]. No bandwidth cap that a verifier can check has been demonstrated publicly.\n\n[[M-0014|Bandwidth limits and compartmentalization]] (R2 for software monitoring) proposes to cap or remove links between accelerator groups. Its pod-cap design aims to allow inference tokens while restricting training gradients, provided the served model and routing fit within each pod [[S-0005]] [[S-3565]]. It needs no access to a facility's code [[S-0005]]. Hardware versions include RAND's fixed-set design [[S-0057]] and [[M-0009|guarantee processors]] (R1), and the [[I-0011|AI 2040 stack]] (R1) removes back-end networking [[S-0067]]. [[M-0022|Side-channel suppression]] (R1) addresses paths outside the network [[S-0038]]. [[M-0017|Tamper evidence]] (R2) would protect the enforcing devices, and [[M-0024|bounding unexplained information]] (R2) limits what the permitted bandwidth can carry.\n\nLucid Computing's pod-cap design is \"not yet implemented or red-teamed\" [[S-1301]], and Amodo's rate limiting on 400G links was for weight security with a cooperating operator [[S-1313]].\n\nCopper scale-up links and encrypted interconnects are hard to observe [[S-0031]], and the designs aim to cut side channels to low rates, not to eliminate them [[S-0018]] [[S-0038]]. Sastry and colleagues note that decentralised training, if it becomes more viable, might undermine the detectability of training runs [[S-0053]], and training methods that need less communication could likewise erode a bound that separates training from inference today.", "text": "A software bandwidth monitor has been demonstrated (R2). It runs on the operator's nodes, and its authors say the measurements are trivially spoofable [S-3220]. No bandwidth cap that a verifier can check has been demonstrated publicly. Bandwidth limits and compartmentalization (R2 for software monitoring) proposes to cap or remove links between accelerator groups. Its pod-cap design aims to allow inference tokens while restricting training gradients, provided the served model and routing fit within each pod [S-0005] [S-3565]. It needs no access to a facility's code [S-0005]. Hardware versions include RAND's fixed-set design [S-0057] and guarantee processors (R1), and the AI 2040 stack (R1) removes back-end networking [S-0067]. Side-channel suppression (R1) addresses paths outside the network [S-0038]. Tamper evidence (R2) would protect the enforcing devices, and bounding unexplained information (R2) limits what the permitted bandwidth can carry. Lucid Computing's pod-cap design is \"not yet implemented or red-teamed\" [S-1301], and Amodo's rate limiting on 400G links was for weight security with a cooperating operator [S-1313]. Copper scale-up links and encrypted interconnects are hard to observe [S-0031], and the designs aim to cut side channels to low rates, not to eliminate them [S-0018] [S-0038]. Sastry and colleagues note that decentralised training, if it becomes more viable, might undermine the detectability of training runs [S-0053], and training methods that need less communication could likewise erode a bound that separates training from inference today." }, "sources": [ { "source": "S-0005", "supports": "pods with high internal and very low external bandwidth; enough for inference tokens not training gradients; implementable with modest changes, physical access and monitoring, without code access", "locator": "Interconnect bandwidth limits" }, { "source": "S-0053", "supports": "compute caps via physical limits on chip-to-chip networking; decentralised training could undermine detectability", "locator": "enforcement; §3.B.1 Detectability" }, { "source": "S-1314", "supports": "DiLoCo on 8 workers matched fully synchronous training while communicating 500 times less", "locator": "abstract" }, { "source": "S-0057", "supports": "fixed-set HEM restricting networking of small GPU sets", "locator": "p. viii" }, { "source": "S-0018", "supports": "front-end vs back-end bandwidth; inference bandwidth assumption; covert side-channel target in kilobits per second; egress explainable by ingress", "locator": "inference vs training; open problems" }, { "source": "S-0031", "supports": "front-end vs back-end tapping costs and feasibility; copper links and scale-up domains; encrypted interconnect", "locator": "frontend vs backend; open problems" }, { "source": "S-0038", "supports": "physical side channels can bypass network monitoring; defences; tolerable low rate", "locator": "channels of concern; defences" }, { "source": "S-0029", "supports": "retroactively confirming data-centre topology is beyond scope", "locator": "open problems" }, { "source": "S-0002", "supports": "network taps intercepting inter-chip data", "locator": "§4.2" }, { "source": "S-0067", "supports": "removing back-end networking to create isolated inference units", "locator": "verification mechanisms" }, { "source": "S-1301", "supports": "pod-level traffic cap design; not yet implemented or red-teamed", "locator": "summary; status" }, { "source": "S-1313", "supports": "DPU-enforced rate limiting on 400G links for weight security, with the limits set by a trusted operator's controller", "locator": "whole note" }, { "source": "S-3220", "supports": "software bandwidth-monitor prototype on AI GPUs; authors say operator-controlled measurements are trivially spoofable", "locator": "Abstract; Pros and Cons analysis" }, { "source": "S-1508", "supports": "egress limits cap what can be stolen", "locator": "§5.1" }, { "source": "S-3565", "supports": "expert-parallel MoE inference can require cross-device all-to-all communication", "locator": "abstract" } ], "concepts": [ "K-0021", "K-0017", "K-0014", "K-0013" ], "order": 8, "type": "claim", "url": "https://trustbutveri.fyi/claims/bandwidth-is-bounded/", "source_file": "content/claims/bandwidth-is-bounded.md", "flags_all": [], "body_markdown": "## Why it matters\nDistributed training depends on fast communication between chips, so limiting communication limits training [[S-0005]].\n\n- Scher and Thiergart describe pods of chips with high-bandwidth interconnect inside each pod but very low external bandwidth: enough for inference tokens, not for training gradients [[S-0005]]. They suggest this could be implemented with modest changes to existing technology, physical access and monitoring, without access to code [[S-0005]].\n- That token-only description applies when the served model and its expert routing fit within a pod. Expert-parallel mixture-of-experts inference can require substantial all-to-all traffic between devices [[S-3565]]. A pod boundary must therefore be sized and tested for the declared serving workload.\n- Compute-governance analyses list physical limits on chip-to-chip networking as a way to enforce compute caps [[S-0053]]. RAND's \"fixed set\" design restricts networking so that small, fixed sets of GPUs cannot be aggregated into large clusters [[S-0057]].\n- A low-trust system overview asks how far a facility can be required to make all egress traffic explainable by its ingress [[S-0018]]. It assumes that the information bandwidth of inference is not more than two orders of magnitude above expected API traffic at full utilisation [[S-0018]]. Rinberg and colleagues note that egress limits cap theft directly: if only 10 GB leaves a data centre, an adversary cannot steal more than 10 GB [[S-1508]]. This links the claim to [[C-0009]].\n\n## Why it is hard\nThe claim requires knowing every path.\n\n- **Taps see only the links they are on.** RAND's network-tap layer observes data exchanged between chips [[S-0002]]. The cost and feasibility of tapping differ by link: Cankaya describes front-end links to the outside world as the most viable to monitor, while the back-end compute fabric has far higher bandwidth, is latency-critical, and may require sampling rather than full capture [[S-0031]].\n- **Some links are hard to observe at all.** As scale-up domains grow, more traffic runs over copper links that passive optical taps cannot see, and encrypted interconnects complicate inspection [[S-0031]]. Shavit notes that methods for verifiers to confirm, after the fact, the topology in which chips were configured in a data centre were beyond the scope of his work [[S-0029]].\n- **Physical side channels.** Data could bypass network monitoring altogether. A MIRI Technical Governance Team post surveys electromagnetic, acoustic, conducted and fibre-optic channels, and proposes shielding, active jamming with sensing, inspection and filtering [[S-0038]]. The system overview's target is to suppress covert side-channel bandwidth to kilobits per second, not to zero [[S-0018]].\n- **Choosing the bound.** A threshold that separates inference from training today may not do so later. On the widely used C4 dataset, DiLoCo on 8 workers matched fully synchronous training while communicating 500 times less [[S-1314]]. Sastry and colleagues note that decentralised training across many data centres, or with lower-quality compute, could undermine compute detectability if it becomes more viable [[S-0053]].", "body_text": "Why it matters Distributed training depends on fast communication between chips, so limiting communication limits training [S-0005]. - Scher and Thiergart describe pods of chips with high-bandwidth interconnect inside each pod but very low external bandwidth: enough for inference tokens, not for training gradients [S-0005]. They suggest this could be implemented with modest changes to existing technology, physical access and monitoring, without access to code [S-0005]. - That token-only description applies when the served model and its expert routing fit within a pod. Expert-parallel mixture-of-experts inference can require substantial all-to-all traffic between devices [S-3565]. A pod boundary must therefore be sized and tested for the declared serving workload. - Compute-governance analyses list physical limits on chip-to-chip networking as a way to enforce compute caps [S-0053]. RAND's \"fixed set\" design restricts networking so that small, fixed sets of GPUs cannot be aggregated into large clusters [S-0057]. - A low-trust system overview asks how far a facility can be required to make all egress traffic explainable by its ingress [S-0018]. It assumes that the information bandwidth of inference is not more than two orders of magnitude above expected API traffic at full utilisation [S-0018]. Rinberg and colleagues note that egress limits cap theft directly: if only 10 GB leaves a data centre, an adversary cannot steal more than 10 GB [S-1508]. This links the claim to Model weights have not left the facility. Why it is hard The claim requires knowing every path. - Taps see only the links they are on. RAND's network-tap layer observes data exchanged between chips [S-0002]. The cost and feasibility of tapping differ by link: Cankaya describes front-end links to the outside world as the most viable to monitor, while the back-end compute fabric has far higher bandwidth, is latency-critical, and may require sampling rather than full capture [S-0031]. - Some links are hard to observe at all. As scale-up domains grow, more traffic runs over copper links that passive optical taps cannot see, and encrypted interconnects complicate inspection [S-0031]. Shavit notes that methods for verifiers to confirm, after the fact, the topology in which chips were configured in a data centre were beyond the scope of his work [S-0029]. - Physical side channels. Data could bypass network monitoring altogether. A MIRI Technical Governance Team post surveys electromagnetic, acoustic, conducted and fibre-optic channels, and proposes shielding, active jamming with sensing, inspection and filtering [S-0038]. The system overview's target is to suppress covert side-channel bandwidth to kilobits per second, not to zero [S-0018]. - Choosing the bound. A threshold that separates inference from training today may not do so later. On the widely used C4 dataset, DiLoCo on 8 workers matched fully synchronous training while communicating 500 times less [S-1314]. Sastry and colleagues note that decentralised training across many data centres, or with lower-quality compute, could undermine compute detectability if it becomes more viable [S-0053].", "addressed_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/", "role": "primary", "note": "Caps or removes communication between declared groups of accelerators (S-0067, S-1301)." }, { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/", "role": "supporting", "note": "Bounds effective unexplained output bandwidth rather than raw link bandwidth." }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/", "role": "supporting", "note": "Interlocks on NVLink or NICs, and RAND's fixed-set pods, would bound communication (S-1204, S-0057)." }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/", "role": "primary", "note": "Bounds the capacity of physical covert channels out of an enclosure, so that monitored links carry all significant traffic." }, { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/", "role": "supporting", "note": "Protects network devices that enforce or monitor bandwidth boundaries (S-0018)." }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/", "role": "supporting", "note": "Removing back-end networking limits communication between inference units." } ], "referenced_by": [ { "id": "M-0014", "title": "Bandwidth limits and compartmentalization", "url": "https://trustbutveri.fyi/mechanisms/bandwidth-limits-and-compartmentalization/" }, { "id": "M-0024", "title": "Bounding unexplained information in outputs", "url": "https://trustbutveri.fyi/mechanisms/bounding-unexplained-information/" }, { "id": "M-0009", "title": "Hardware-enabled guarantees (flexHEG) and guarantee processors", "url": "https://trustbutveri.fyi/mechanisms/flexheg-guarantee-processors/" }, { "id": "M-0022", "title": "Side-channel suppression for isolated facilities", "url": "https://trustbutveri.fyi/mechanisms/side-channel-suppression/" }, { "id": "M-0017", "title": "Tamper evidence for verifier devices", "url": "https://trustbutveri.fyi/mechanisms/tamper-evidence-for-verifier-devices/" }, { "id": "I-0011", "title": "AI 2040 inference-only verification stack", "url": "https://trustbutveri.fyi/implementations/ai-2040-inference-only-verification-plan/" }, { "id": "C-0009", "title": "Model weights have not left the facility", "url": "https://trustbutveri.fyi/claims/weights-have-not-left/" }, { "id": "O-0211", "title": "Centre for the Governance of AI", "url": "https://trustbutveri.fyi/organizations/centre-for-the-governance-of-ai/" } ] } }