Diagnostics API

Req 6's structural diagnostics: reported, never selected on and never a headline (roadmap Stage 4).

Req 6's structural diagnostics over every code of a table: reported, never selected on and never a headline (Req 1; Req 6; Verification "Diagnostics").

  • Sector separation. The AUC of distance between same-sector and cross-sector pairs: the probability that a cross-sector pair lies farther apart than a same-sector pair, ties counting half.
  • Within-sector rank correlation. Per query, Spearman's correlation of distance with D* over the other codes of its sector; averaged over queries, and over sectors (each sector's mean over its queries).
  • MAP over ancestors. Per non-sector query, the average precision of its ancestors among all other codes ranked by distance.
  • NDCG@k. Gains are integer depths of the lowest common ancestor (0 across sectors, 1 for a shared sector, up to 4), with linear gain and a 1/log2(rank + 1) discount.
  • Distance Pearson. The Pearson correlation of distance with D* over all pairs: the statistic formerly named "cophenetic", which no dendrogram underlies.
  • Parent retrieval@k. Per non-sector query, whether its parent is among its k nearest codes. The 522 unary pairs, five-digit industries whose only child is their six-digit code (Req 9), are not scored.

D* is the tree metric through a virtual root (Req 7): depth_i + depth_j − 2 depth_LCA, where a sector has depth 1. It comes from the codes themselves, through utils.naics_hierarchy.tree_distance_matrix, the function the supervision bundle's distances use (combined sectors as one). Distances are the arm's own: Euclidean, cosine for a spherical arm, and for a hyperbolic arm the geodesic distance between the exponential maps of its tangent coordinates at the origin. A tie in distance is broken against relevance, as the decoding scorer breaks it against the truth, except in the AUC and Spearman's ranks, which average ties. No statistic has a threshold, and the report has no pass or fail.

DiagnosticsReport

Bases: _Report

Req 6's statistics, stratified as it lists them; nothing else.

DistancePearson

Bases: _Report

None when every distance is equal.

Tree dataclass

The codes' tree.

Attributes:

Name Type Description
lineage ndarray

(codes, 5) row numbers of each code's ancestors from its sector down to itself, −1 below its depth.
depth ndarray

Each code's depth, 1 for a sector.

parent property

Each code's parent row, −1 for a sector.

from_codes(codes) classmethod

Raises:

Type Description
ValueError

If a code repeats or an ancestor of a code is not among the codes.

lca_depth()

(codes, codes) depth of each pair's lowest common ancestor, 0 across sectors.

unary_children()

Whether each code is the six-digit half of a unary pair (Req 9).

WithinSectorRankCorrelation

Bases: _Report

The means are None when no query's correlation is defined (a collapsed embedding).

average_precision(distances, relevant)

One query's average precision of its relevant candidates, ranked by distance.

diagnostics_report(table, geometry, *, codebook_codes=None, curvature=1.0)

Req 6's statistics for an arm's code table in the export form.

Parameters:

Name Type Description Default
table DataFrame

code plus coordinates (panels.regressor.coordinate_matrix).

 required
geometry Geometry

The arm's geometry, which sets its distance.

 required
codebook_codes Optional[Sequence[str]]

If given, the table must cover exactly these codes.

 None
curvature float

A hyperbolic arm's curvature magnitude.

 1.0

Raises:

Type Description
ValueError

If the table's codes differ from codebook_codes, or an ancestor of a code is missing.

distance_pearson(distances, target)

The Pearson correlation of distance with D* over all unordered pairs.

map_over_ancestors(distances, tree)

Per non-sector query, the average precision of its ancestors among all other codes.

ndcg(distances, grades, depth, k)

Per query, NDCG@k over all other codes with integer LCA-depth gains; a query that shares a sector with no other code has no gain to rank and is not scored.

ndcg_at(distances, gains, k)

One query's NDCG@k with linear gains; None when no candidate has a gain.

pairwise_distances(matrix, geometry, curvature=1.0)

The arm's own distances between every pair of rows.

A hyperbolic arm's rows are tangent coordinates at the origin (the export form); each maps to the hyperboloid of curvature −curvature by the exponential map at the origin.

parent_retrieval(distances, tree)

Per non-sector query outside the unary pairs, whether its parent is among its k nearest.

sector_separation(distances, sector)

The Mann–Whitney AUC of cross-sector over same-sector distances, over unordered pairs.

within_sector_rank_correlation(distances, target, sector, codes)

Per query, Spearman's correlation of distance with D* over its sector's other codes.