Decision Rule API¶

Each panel's per-unit scores and its decision statistic (D10).

Scores are long: one row per panel, statistic and item, with the item's resampling unit.

• Outcome panel. An item is a validation query and its unit is its true code. Every Req 3 metric is a statistic; the decision statistic is mrr, the per-query reciprocal rank that D6 already selects checkpoints on.
• Regressor regimes. An item is a level-6 code in a feature year and its unit is its four-digit group. Every Req 2 comparator is a statistic, valued at the row's squared error averaged over the read's repeats, so a row counts once however many repeats scored it. The decision statistic is the mean squared error of covariates+embedding.

The paired two-stage bootstrap (Req 5: "both arms are scored on the same resample of the evaluation unit, with seeds nested within the resample").

Each replicate draws the panel's units with replacement, one draw shared by every arm, and then, inside it, each arm's seeds with replacement, a draw of the arm's own. The replicate's statistic is the mean over the drawn seeds of the item-weighted mean over the drawn units:

sum_s m_s (C . S_s) / (n_seeds (C . N))

with C the unit counts, m the seed counts, S_s seed s's per-unit sums and N the items per unit. Unit counts come from the generator seeded [bootstrap_seed, stream(panel)] and an arm's seed counts from [bootstrap_seed, stream(panel), stream(arm)], so an arm's replicates are the same in every comparison it enters.

Req 5's rule on paired resamples, over a decision's panels.

• Non-inferior on a panel: the lower bound of the 95 % interval on Δ exceeds −δ.
• Superior on a panel: the interval at 1 − 0.05/n lies above zero, for a decision over n panels, each of which gets an even share of the 5 % error rate (Req 24). D8's three panels read 98⅓ %, which supersedes the 97.5 % that Req 5 names for two; the economic decision's two growth panels read 97.5 %.
• Adopted: non-inferior on every panel and superior on at least one.
• Several arms: the survivors are the arms no other arm is adopted over; when every arm is, dominance cycles and the tie order picks among all of them. The pairwise comparisons are not corrected for multiplicity: the tie order toward the simpler arm is the guard (Req 5).
• Tie order: fewer components, then lower dimension, then non-hyperbolic geometry, then the higher held-out gain over covariates+ancestors (D11): on regressor_heldout for D8's decision, on growth_heldout for the economic decision, where it is the reference (Req 24).

The records of Req 5's decisions (Verification "Decision records"; roadmap Stage 4).

Three kinds, each one JSON file:

• Arm record. One configuration's seed sweep: its spec, the text-only table it was paired with and that table's provenance (D9), the panels it read, and per seed the encoder checkpoint, the 2,125-code table, the stored scores and the selection-log records of the run's reads (the log itself is gitignored and dies with its worktree or Lambda instance). A trained seed also carries the monitor reads that selected its checkpoint (spec 4.4).
• Margin record. Each panel's δ, a stated multiple of a reference arm's across-seed standard deviation, fixed before any other arm's first read.
• Decision record. Its arms and margins, every paired comparison with its 95 % non-inferiority and 98⅓ % superiority intervals (D8), the non-dominated set, the tie order, the chosen arm, and the reported statistics.

The economic decision (Req 24) has kinds of its own:

• Relative margin record. Each growth panel's δ, r times the reference comparator's validation MSE, from a margin read that reads no candidate.
• Economic arm record. A deterministic candidate's single replicate: its table and that table's provenance (Req 23), read once on each growth panel, with no checkpoint or monitor reads.
• Economic decision record. Its candidates and margins, each candidate's comparison with the reference covariates+ancestors with its 95 % non-inferiority and 97.5 % superiority intervals, the candidates adopted over the reference, the non-dominated set and tie order among them, and the chosen candidate, or none when the reference stands.

Artifact references are paths relative to an ArtifactStore root with their sha256, so a record names exactly the bytes it was computed from.

The content-addressed store behind decision records' artifact references (roadmap Stage 4).

A file is copied to objects/<sha[:2]>/<sha>/<name> under the store root, and its reference is that relative path with its sha256 and size. The store never deletes or overwrites: an object is written once and verified again on every read. Records need these files until Stage 12, so the root belongs outside any worktree (removing a worktree deletes its ignored files), and a Lambda instance's store must be copied off before the instance terminates (specs/lambda-remote-workflow.md).

Margins from a reference arm, and a decision over arms (Req 5; D8, D10, D11).

Before any number is computed, every arm is checked:

• it has at least min_seeds seeds, each read once on each of the three panels;
• every stored artifact still hashes to its reference;
• its text-only table's stored provenance matches the arm's backbone, revision, descriptions, summaries and window (D9);
• each run's log records are validation reads that name the run, its table and its text-only table by the fingerprints the store recorded; its outcome read decodes by the distance of the arm's geometry, and its regressor reads name the arm's dimension (Req 12);
• a trained run's monitor records are its training run's reads of the outcome panel's validation split, under the distance of the arm's geometry, no epoch twice, and its checkpoint is from the earliest epoch with the highest MRR (spec 4.4); a run with no training run has neither monitor records nor a checkpoint epoch;
• all arms read the same panels, with the same data on them and the same fit settings, so Δ pairs item for item.

A decision also requires the margins to hold one δ per panel, for its decision statistic, and every run other than the margin record's own reference runs to have read nothing, its monitor reads included, before the margins were fixed. Every read's time, and the margins', must name its timezone.

The seed-sweep driver (roadmap Stage 4): run a configuration for N seeds, read each seed once on each of D8's three panels, and keep everything a decision record references.

A runner trains (or loads) one seed of a configuration and returns its encoder checkpoint, its 2,125-code table in the export form and a QueryCodeEncoder; for a trained seed, also its training run, its checkpoint's epoch and the monitor reads that selected it, which the seed's record carries (spec 4.4).

Every read goes through the panels, so it is logged, and it carries the run's id, which is how the arm record picks its runs' records out of the selection log. The checkpoint, the table, the text-only table with its provenance, the scores, the decoding and the predictions go to the artifact store.

The economic artifact's decision (Req 24; Req 23; Req 18): the margin read, the candidates' reads and the decision over the two growth panels.

• Panels. growth_seen and growth_heldout, the growth outcome's regimes at level 6 (Req 21), and no other: no artifact's decision reads another artifact's panel (Req 18).
• Statistic. D10's, applied to growth: each row's squared error averaged over its repeats (decision.scores.regressor_scores); a candidate's is covariates+embedding's MSE.
• Reference. The comparator covariates+ancestors (U9). It reads no candidate, and the folds depend on the regime, level, repeat and group alone, so the margin read's row errors are those every candidate's read computes, and they pair exactly.
• Margins. δ = r × the reference's validation MSE on each panel (U13), from a margin read that reads no candidate, fixed before any candidate is read.
• Candidates. A deterministic candidate, such as the text-only table or a tree embedding, is one replicate: its table, with the provenance its builder wrote, read once on each growth panel the margins read, with no checkpoint or monitor reads (Req 24; Req 23). Its reads pair with the margin read only where the reference predicts bit for bit as it did there.
• Decision. Each candidate against the reference on both panels, at 95 % non-inferiority and 97.5 % superiority. Only a candidate adopted over the reference is eligible; among the eligible, Req 5's non-dominated set and tie order pick one, the higher held-out gain over the reference breaking a last tie (D11). With none eligible, the reference stands. The decision scores each read's stored predictions itself, so the values it resamples are those the pairing compared.