Outcome Panel API¶

Index-entry roles for the outcome panel (Req 3; roadmap D4).

Every entry of the Census NAICS index file holds exactly one role: examples-channel text, or a training, validation or test query. The assignment is made once, frozen in a committed table (conf/data/index_roles.csv), and consumed by hash afterwards; regeneration is never how the sealed splits are preserved.

• Eligibility. Validation and test queries are drawn only from entries that no training text can leak: an entry is withheld from both held-out splits when it matches, exactly or as a near-duplicate (leakage), any title, description, exclusion text, fallback examples text or any other index entry. Checking against every other entry, not only those that end up as training text, makes the held-out splits leak-free whatever the assignment.
• Quotas. Per code, role counts are largest-remainder quotas of the configured fractions, with remainder ties broken by a seeded draw. Every code keeps at least examples_floor examples-channel entries, and held-out quotas beyond the code's eligible entries move to training, one at a time from the larger of the two held-out quotas (test on a tie).
• Seeds. Each code draws from its own generator, seeded by (seed, code), so a code's roles do not depend on any other code.

Draw the frozen index-entry role table (roadmap Stage 2; Req 3; D4).

naics-embedder data roles runs this once. The table it writes (conf/data/index_roles.csv) is committed and data preprocess applies it from then on, so the sealed validation and test splits never move. Redrawing reassigns every entry and unseals both splits, so an existing table is replaced only with force.

Leakage between held-out index-entry queries and training text (Req 3, "Leakage").

Texts are compared after normalize_text: lowercase ASCII letters and digits, with every other run of characters collapsed to one space. A query leaks into a training segment in two ways:

• exact: the query occurs in the segment as whole words (equality included);
• near-duplicate: the character-trigram Jaccard similarity of query and segment reaches the threshold, 9/10 by default. Trigrams are taken inside word boundaries, as scikit-learn's char_wb analyzer builds them, so reordered words score as the same text.

Jaccard is compared in integers (denominator * shared >= numerator * union), so no pair sits on a floating-point boundary. The check compares whole queries with whole segments, so a query whose words appear reordered inside a longer segment is not caught.

Text-to-code decoding scores for the outcome panel (Req 3).

Each query decodes to the nearest candidate code under the arm's own distance. Ties are broken against the truth: the true code's rank counts every candidate at a distance no greater than its own, so an encoder that places every point alike ranks the truth last, never first. Distances are computed in float64 on the CPU whatever the device or dtype of the points.

Metrics per query: exact top-1 accuracy, reciprocal rank, Hit@k for k in HIT_KS, and the level of the lowest common ancestor of the top-1 code and the truth (6 for the same six-digit code, 2 for a shared sector, 1 for the virtual root above the sectors).

The outcome panel (Req 3; Req 4): text-to-code decoding over sealed query splits.

Validation queries may be read at any time, and every read is logged. The test split is sealed: reading it needs a logged opening by the same panel object, and opening the same split (the same role assignment, by fingerprint) a second time needs a stated reason. Training queries are training data, not a selection, so reading them is not logged.

The selection log (Req 4; Verification "Selection hygiene").

An append-only JSON-lines file with one record per event: every read of a panel's validation or test split, and every opening of a test split. It is the evidence that selections read validation splits only and that each test split was opened once, for the final configuration. A second opening needs a stated reason and is recorded as reopen.

A training-free lexical encoder for the outcome panel.

Texts are hashed character trigrams of their normalized form (leakage.normalize_text), L2-normalized, so cosine distance decodes a query to the code whose text shares most of its trigrams. It is the stub arm that exercises the panel end to end on the real validation split until a trained encoder can embed a query (roadmap Stage 6), and a floor for trained arms. It is never a candidate configuration.