Code Cache and Outcome Monitor API

The cache supplies detached candidates, with current live-anchor replacements. OutcomeMonitor reads only outcome validation and records float64 MRR with training-run, seed and epoch identity.

D6's monitor (spec 4.4): the outcome panel's validation split, read on the live model each epoch.

Training holds a cache of every code's point, refreshed at fit start and after each epoch's last step (spec 4.3). After the end-of-epoch refresh the monitor scores the validation split through OutcomePanel.score_logged, the path every read takes, so the selection log records the read. Its LiveEncoder encodes queries through the live model and decodes codes from the cache, and both go through the head's float64 read map, as ArmEncoder's do, under the head's distance (Req 12). The cache is encoded as the export encodes the code table, in the same batches and order, so on the CPU a read of the live model and a read of the table exported from its checkpoint agree exactly.

Each read's record, as logged, goes with its MRR to monitor_reads.jsonl in the run's checkpoint directory: the durable carrier that takes a run's validation reads into its decision records (Req 4). An exact resume keeps the records through the restored checkpoint's epoch, so the file holds each epoch of the surviving run exactly once.

CodeCache dataclass

Every code's point on the live model at its last refresh, in codebook order (spec 4.3).

Attributes:

Name Type Description
codes Tuple[str, ...]

The codes, in codebook order.
radius Tensor

Each code's radius r, (N,) float32 on the model's device.
direction Tensor

Each code's direction û, (N, d) float32 on the model's device.
tangent Tensor

Each code's coordinates in its arm's export form, the head's tangent, (N, d) float64 on the CPU: r · û in the hyperbolic arm, v in the Euclidean and û in the spherical (Req 12). It is what the monitor decodes against and what the export writes.

with_live(ids, radius, direction)

Every code's radius and direction, with the anchors' rows replaced by their live points.

The rows are replaced out of place, so the cache is unchanged. Every other row is a constant of the last refresh, so gradient flows through the anchors' rows only, whether a term reads them as anchors or as candidates (spec 4.3).

Parameters:

Name Type Description Default
ids Tensor

The anchors' code ids (A,), distinct: their rows in codes.

 required
radius Tensor

The anchors' live radii (A,).

 required
direction Tensor

The anchors' live directions (A, d).

 required

Returns:

Type Description
Tuple[Tensor, Tensor]

The radii (N,) and the directions (N, d).

Raises:

Type Description
ValueError

If the ids, radii and directions do not fit the cache.

LiveEncoder

QueryCodeEncoder for the live model (spec 4.4): queries through the model, codes from the code cache, and both through the head's read map (read_points).

A query is encoded as ArmEncoder encodes one (encode_query_texts), in eval mode and in float32 with autocast off, and the model's training flags are put back afterwards. A code is decoded from its cached tangent, as ArmEncoder decodes one from the exported table.

Parameters:

Name Type Description Default
model Module

The live model.

 required
cache CodeCache

The code cache of its last refresh.

 required
tokenizer Any

The token cache's tokenizer.

 required
max_length int

The token cache's window.

 required
batch_size int

Queries per forward pass.

 ENCODE_BATCH_SIZE

Attributes:

Name Type Description
head

The model's geometry head.
distance

The head's decoding distance, as ArmEncoder's: 'lorentz' in the hyperbolic arm, 'euclidean' or 'cosine' in the flat arms (Req 12).

encode_codes(codes)

The codes' cached coordinates through the head's read map: float64.

Raises:

Type Description
ValueError

If a code has no row in the cache.

encode_queries(texts)

Marked query: texts through the live model, then the head's read map: float64.

MonitorRead dataclass

One epoch's monitor read.

Attributes:

Name Type Description
epoch int

The epoch the read scored.
mrr float

Its validation MRR: the one value training logs as val/outcome_mrr, steps the learning rate on and records.
record Dict[str, Any]

The read's selection-log record, as the log appended it.

OutcomeMonitor

D6's monitor: one logged read of the outcome panel's validation split per epoch, kept in the run's monitor_reads.jsonl (spec 4.4).

Parameters:

Name Type Description Default
panel OutcomePanel

The outcome panel of the run's bundle (OutcomePanel.from_bundle), whose log is conf/data/outcome_panel.yaml's selection_log.

 required
tokenizer Any

The token cache's tokenizer.

 required
max_length int

The token cache's window.

 required
records_path Union[str, Path]

The run's monitor_reads.jsonl, in its checkpoint directory.

 required
purpose str

Why the reads happen; the selection log records it with each read.

 required

Raises:

Type Description
ValueError

If purpose is blank, which the log would refuse only at the first read, an epoch into the run.

append(read)

Append one read to the records file, as {"mrr": …, "read": <its record>}.

Raises:

Type Description
ValueError

If the file already records the read's epoch or a later one: each epoch of the surviving run appears once, in order.

read(model, cache, *, training_run, seed, epoch)

Score the validation split on the live model under the encoder's distance, logging the read.

The distance is the decoding distance of the arm's geometry (Req 12): 'lorentz', 'euclidean' or 'cosine'. Every refusal comes before the read is logged. The read's detail names the training run, the seed, the epoch and the epoch's code table: the matrix_fingerprint of the cache's tangents, the name a read of the exported table logs it by.

Parameters:

Name Type Description Default
model Module

The live model, just after its end-of-epoch refresh.

 required
cache CodeCache

The code cache of that refresh.

 required
training_run str

The training run's id, which every checkpoint saves.

 required
seed int

The run's seed.

 required
epoch int

The epoch the read scores.

 required

Returns:

Type Description
MonitorRead

The read: its epoch, its MRR, and its record as logged.

Raises:

Type Description
ValueError

If the training run is blank, the epoch is negative, or a candidate has no row in the cache.
TypeError

If the seed or the epoch is not an integer.

start(*, resumed_epoch)

Ready the records file when a fit starts.

A fresh run starts a new file, which its first read writes. An exact resume continues the file it resumed with: it keeps the records through the restored checkpoint's epoch and drops any later one, which an interrupted segment left (that read stays in the segment's selection log). The kept lines are rewritten as they were, and the file is replaced in one step.

Parameters:

Name Type Description Default
resumed_epoch Optional[int]

The restored checkpoint's epoch, the last epoch it completed; None for a fresh run.

 required

Raises:

Type Description
ValueError

If a fresh run finds a records file; or if a resume finds none, which would start a file without the run's earlier reads (spec 4.6), or its epoch is negative.

preserve_training_flags(module)

Record every submodule's training flag, and put each back on exit, an error's included.

Lightning never calls train() in its fit loop, so an encode in a training hook that leaves the model in eval mode, as encode_token_rows does, would turn dropout and gradient checkpointing off for the rest of the run, with no error. Each flag is put back as it was, so a submodule that was in eval mode stays there.

Parameters:

Name Type Description Default
module Module

The module whose flags, and its submodules', are kept.

 required

read_monitor_records(path)

The records of a monitor_reads.jsonl, oldest first, each {'mrr': …, 'read': …}.

Blank lines are skipped.

Raises:

Type Description
ValueError

If a line is not a monitor record: a JSON object with exactly mrr, a number, and read, a record whose detail names a non-negative integer epoch.
FileNotFoundError

If the file does not exist.

refresh_code_cache(model, code_rows, codes, batch_size=ENCODE_BATCH_SIZE)

Encode every code into a new cache: in eval mode, without gradient, in float32 with autocast off, in batches in codebook order (spec 4.3).

The codes go through encode_token_rows as the export's do, in the same batches and order, so on the CPU the cache's tangents are the exported table's, bit for bit. The model's training flags are put back afterwards.

Parameters:

Name Type Description Default
model Module

The live model: the Lightning module, or its shared encoder.

 required
code_rows Sequence[Mapping[str, Mapping[str, Any]]]

Every code's cached token rows, in codebook order.

 required
codes Sequence[str]

The codes, in the same order.

 required
batch_size int

Codes per forward pass.

 ENCODE_BATCH_SIZE

Returns:

Type Description
CodeCache

The cache.

Raises:

Type Description
ValueError

If the rows are not one per code, or a code repeats.