Training-transcript verification (proof-of-learning)

How it works

Three variants are covered here: tolerance-based proof-of-learning, heuristic tests on the training data ⁶, and exact replication with a dispute protocol ^{7 8}. Published spoofing attacks defeat the first ⁴.

The transcript. Proof-of-learning (PoL) asks a trainer to record a transcript of training ¹. It contains:

• weight checkpoints saved every k steps;
• the indices of the data batches used at each step;
• signatures of the training data;
• metadata such as hyperparameters, optimiser and architecture ¹.

Checking it. A verifier checks the transcript by re-running training for selected segments. For each epoch, it takes the Q largest weight updates and recomputes the k steps from the earlier checkpoint. It accepts if the result lies within a distance threshold δ of the logged checkpoint ¹. The threshold absorbs the numerical noise from hardware and software nondeterminism ¹.

Why forging should be costly. The PoL code repository explains that gradient descent "accumulates secret information due to its stochasticity" ². On this basis, Jia et al. claim a forger must spend "at least as much work than is needed for gradient descent itself" ¹.

Use for rule verification. Shavit adapts the idea to verifying rules on large training runs ⁵:

1. Chips log weight snapshots.
2. Inspectors sample chips.
3. The prover must show a training transcript that explains each logged snapshot.
4. The verifier re-runs only the segments concerned ⁵.

Tests on the training data. Choi et al. add checks aimed at training data ⁶:

• the random seed, initialisation and data order are derived from a hash of the dataset;
• memorisation tests check that training points are fitted best just after their segment;
• suspicious segments are flagged for retraining ⁶.

Exact replication. Srivastava et al. make training bit-for-bit reproducible on different GPU types, so that checkpoints can be compared by hash rather than within a tolerance ⁷:

• the trainer trains at higher precision than the target model, rounds intermediate results, and logs the rounding direction for values close to a rounding boundary;
• an auditor replicates the whole run, following the logged rounding directions, and stores hashes of checkpoints in a Merkle tree;
• if the two trees differ, a binary search finds the first checkpoint at which training diverged ⁷.

What it establishes

What it can establish.

• If verification holds, the transcript is consistent with training that actually took place. This supports claims about ownership and the compute spent ¹.
• With data-focused tests, it also supports claims about "the amount and kind of data and compute used to train the model" ⁶. This bears on A training run stayed within declared limits.

What it cannot establish.

• A link to derived models. PoL cannot connect a model to a surrogate derived from it by extraction ¹.
• Anything without reproducible training. The approach relies on training being approximately reproducible from the hyperparameters, data order and seed ⁵.
• Online or reinforcement learning. Where the data is not known in advance, these are not covered ^{5 6}.

Threat model

The adversary tries to produce a valid-looking transcript for weights it did not train, or for a run that broke the rules, while spending less than honest training ^{1 5}.

The scheme assumes several things:

• the verifier can re-run training segments on comparable hardware ⁶;
• honest noise is small compared with genuine updates ¹;
• the verifier receives the data and weights, since Jia et al. require that the training data "be shared with the verifier" ¹.

Srivastava et al.'s scheme assumes that at least one of several auditors is honest ⁷.

In Shavit's framework, three further assumptions tie transcripts to real chips ⁵:

• the chips support firmware attestation and logging;
• they have tamper-evident packaging;
• a trusted chip-owner directory exists.

Limitations

Independent attack. Zhang et al. of Zhejiang University showed that adversarial-example-style perturbations let an attacker generate a passing proof "with significantly less cost" than honest training ³. Fang et al. dispute this. They state that the attack assumes the adversary chooses the checkpoint interval, which the verifier should set, and that a small interval prevents it ⁴.

Attack by the original authors. Fang et al., whose authors include all seven original PoL authors, present cheaper spoofs that work across PoL configurations. They find that "current PoL verification is not robust to adversaries", and show that the assumptions needed for robust verification reduce to open problems in learning theory ⁴.

Gaps in later defences. Choi et al. defend against several known attacks, but note that small-scale data changes and masked hyperparameters remain open ⁶. In Srivastava et al.'s scheme, the auditor re-runs the whole run, and all parties must trust it. A trainer that knows the auditor's GPU can choose among models that differ only in rounding decisions at the few steps where the auditor is close to a rounding boundary. Distributed training is left to future work ⁷.

Cost. Storage is a burden: checkpoints may each take terabytes ⁵. Shavit identifies cheaper alternatives to retraining-based verification as future work ⁵.

Alternatives. Zhang et al. suggest proofs built on verifiable computation as a countermeasure ³. Zero-knowledge proofs of training take that route, proving each training step cryptographically ¹⁰.