Bandwidth limits and compartmentalization

How it works

Training and serving use the network differently ^{1 2}. The AI 2040 plan notes that AI servers "have a lot of high bandwidth communication between them, but this is (mostly) only necessary for training" ¹. Lucid Computing gives the same rationale: inference sends little external traffic, while training moves gradients, model state and activations among many machines ².

Two families of design use this asymmetry ^{1 2}:

• The AI 2040 plan would remove a significant amount of back-end networking to "convert datacenters into groups of isolated inference-units" ¹. Its illustration retrofits 40% of US compute into inference units of four server racks ¹.
• Lucid's Traffic Shaping design caps links with a network shaper at the north-south uplink of each pod, modelled as one GB200 NVL72 rack ². It recommends a cap of 1 Gbps per direction per pod, averaged over 60 seconds ². The parties would configure and agree on the shaping devices, built on commodity hardware or open standards through a mediated co-design process ².

The MIRI system overview frames the same boundary as a perimeter: untrusted devices are grouped so that only monitored links communicate in and out ⁴. For inference, that perimeter can be as small as pods of a few hundred accelerators ⁴. Monitored links are covered in Network taps and certifiers.

Amodo reports building a related enforcement point for weight security ⁷. It replaced a node's NICs with NVIDIA BlueField-3 DPUs, reasoning that "all the traffic in and out of a node goes through a NIC", which creates "a clean bandwidth boundary" ⁷.

Reuter and colleagues propose a separate "proofs of compartmentalization" approach. Their ICML 2026 workshop abstract argues that inference workloads can be compartmentalized while frontier training cannot, and proposes checking that property without revealing confidential workloads ¹⁰. The abstract gives no protocol or performance results; it does not establish that their approach has been implemented.

What it establishes

Lucid aims to let a third party verify that frontier training either is not happening in a declared facility or would cost so much more that training a model ten times the current frontier becomes economically infeasible ². It estimates a lower bound of about 350x on the inefficiency of covert frontier training, with central estimates of 1,000x to 10,000x, if the auditor controls pod routing and per-pod storage is capped ². For decentralized pipeline training, the floor falls to about 140x if two unproven techniques both succeed. Lucid calls its figures "engineering estimates, not proofs" ². This addresses Communication between compute groups is bounded, and indirectly This compute runs inference, not training and A training run stayed within declared limits.

It does not establish:

• A cap alone does not show what the pods compute, so the AI 2040 plan pairs inference units with taps and recomputation (Sampled inference recomputation, Whole-workload recomputation (reproducible packets)) ¹.
• Lucid places training that fits within one pod outside its modelled threat, along with fine-tuning and reinforcement-learning post-training within one pod ².
• The caps apply only inside declared facilities. Outside them, Rahman judges it "infeasible to prevent evaders from getting fast internet connections" ⁹.

Threat model

• Favourable adversary. Lucid's analysis grants the adversary perfect overlap of communication and computation, zero pipeline bubbles and "the best published communication-reduction techniques", among other advantages ².
• Auditor-controlled routing. The bound holds only if the auditor randomizes the assignment of pods to routers and periodically re-randomizes it ².
• Declared storage. Per-pod storage must be declared, capped and inspected ².
• Complete perimeter. The shaper sits at each pod's uplink ². The verifier must also know that no other path leaves the pod, which the MIRI overview handles with a perimeter where only monitored links communicate in and out ⁴; see Network taps and certifiers and Tamper evidence for verifier devices.

Limitations

• Low-communication training. DiLoCo matched fully synchronous training on 8 workers while communicating 500 times less ⁸. Rahman writes that this family theoretically allows large-scale training at under 100 Mbps ⁹. Lucid's bounds include DiLoCo- and SWARM-family methods, but it notes that extreme activation compression or modular paradigms could erode the margin ².
• Inference that crosses pod boundaries. Expert-parallel mixture-of-experts inference uses all-to-all communication between devices holding different experts, which can be a bottleneck ¹¹. A cap calibrated for token-only ingress and egress may disrupt that serving architecture.
• Low-traffic training. LoRA fine-tuning and reinforcement learning involve less inter-node traffic than ordinary training, and the MIRI authors leave whether a limit would stop them to future work ⁵. They judge that training for longer at lower throughput is not a major concern, because a reasonable limit would slow training by a prohibitive factor, such as 100x ⁵.
• Routing control. If the operator can group pods freely behind routers, the bound falls to about 90–220x uncompressed and as low as about 25x with compression ².
• Hidden storage. Undeclared per-pod storage weakens the bound ².
• Scale-up bypass. In GB200 topologies, GPUs reach other nodes through NVSwitches without a NIC on the path, and compromising one or two parallel switches would bypass a limit ⁷.
• Durability. Lucid positions the design as "a short-to-medium-term deterrent and Phase-1 milestone" ².