Core Metrics API

Core metrics classes for embedding evaluation.

Contains: - lorentz_distance_matrix: Float64 pairwise Lorentz distances for evaluation - EmbeddingEvaluator: Float32 pairwise Lorentz distances on the evaluation device - HierarchyMetrics: Cophenetic/Spearman correlation, NDCG ranking, distortion

EmbeddingEvaluator

__init__()

Evaluator for embedding quality metrics.

Device is automatically detected via get_device().

compute_pairwise_distances(embeddings, metric='lorentz', curvature=1.0)

Compute pairwise Lorentz distances between embeddings.

HGCN's hyperbolic points are the only input; a flat arm's distances come from panels.decoding.

Parameters:

Name Type Description Default
embeddings Tensor

Lorentz embeddings of shape (N, D+1)

 required
metric str

Distance metric; only 'lorentz' is accepted

 'lorentz'
curvature float

Curvature parameter for the Lorentz metric (default: 1.0)

 1.0

Returns:

Type Description
Tensor

Float32 distance matrix of shape (N, N) on self.device

Raises:

Type Description
ValueError

If metric is not 'lorentz'.

HierarchyMetrics

__init__()

Metrics for evaluating hierarchy preservation.

Device is automatically detected via get_device().

cophenetic_correlation(embedding_distances, tree_distances, min_distance=0.1)

Compute cophenetic correlation coefficient with better handling.

Measures how well embedding distances preserve hierarchical tree distances. Filters out pairs with very small tree distances to avoid noise.

Parameters:

Name Type Description Default
embedding_distances Tensor

Distance matrix from embeddings (N, N)

 required
tree_distances Tensor

Ground truth tree distances (N, N)

 required
min_distance float

Minimum tree distance to include (filters out same-level codes)

 0.1

Returns:

Type Description
Dict[str, Union[Tensor, int]]

Dictionary with correlation and metadata

distortion(embedding_distances, tree_distances)

Compute distortion metrics (how much distances are stretched/compressed).

Parameters:

Name Type Description Default
embedding_distances Tensor

Distance matrix from embeddings (N, N)

 required
tree_distances Tensor

Ground truth tree distances (N, N)

 required

Returns:

Type Description
Dict[str, Tensor]

Dictionary with distortion metrics

ndcg_ranking(embedding_distances, tree_distances, k_values=[5, 10, 20], min_distance=0.1)

Compute NDCG@k for ranking evaluation.

For each anchor code, ranks all other codes by embedding distance and evaluates using NDCG based on tree distance relevance.

Parameters:

Name Type Description Default
embedding_distances Tensor

Distance matrix from embeddings (N, N)

 required
tree_distances Tensor

Ground truth tree distances (N, N)

 required
k_values List[int]

List of k values for NDCG@k

 [5, 10, 20]
min_distance float

Minimum tree distance to consider

 0.1

Returns:

Type Description
Dict[str, Tensor]

Dictionary with NDCG@k for each k value

spearman_correlation(embedding_distances, tree_distances, min_distance=0.1)

Compute structural-spearman-v1 over unique unordered non-self pairs.

Inputs must be same-shaped square, real numeric tensors. The calculation validates both off-diagonal orientations, averages mirrored values on CPU in float64, filters canonical targets >= min_distance, and uses SciPy's average ranks for exact ties. Diagonal values are ignored.

Returns:

Type Description
StructuralSpearmanResult

A detached float32 scalar on self.device, pair counts, definition,
StructuralSpearmanResult

status, and reason. Undefined correlations are NaN with an explicit
StructuralSpearmanResult

reason; malformed input raises StructuralMetricInputError.

lorentz_distance_matrix(embeddings, curvature=1.0)

Pairwise Lorentz distances in float64 on the CPU, for evaluation (no gradients flow).

Far from the origin, -_L = x0 * y0 - cancels catastrophically for nearby points, and acosh near 1 amplifies the error. Float64 arithmetic alone is not enough: float32 rounding of the stored x0 already moves -_L off 1/c by up to x0 * ulp(x0), about 5e-5 at radius 4, i.e. distances of about 1e-2. So x0 is re-derived in float64 from the spatial coordinates, which assumes the points lie on the curvature-c hyperboloid.

Like every distance formula here, sqrt(c) * acosh(-_L) is only correct for c = 1; see "One curvature everywhere" in specs/naics-embedding-methodology.md.

Parameters:

Name Type Description Default
embeddings Tensor

Hyperbolic embeddings of shape (N, D+1), on any device

 required
curvature float

Curvature parameter c

 1.0

Returns:

Type Description
Tensor

Float64 CPU distance matrix of shape (N, N), exactly symmetric with a zero diagonal