metric-decomposition

---
name: metric-decomposition
description: Decompose composite metrics into constituent signals, analyze polarity and ceiling effects
execution: subagent
prompt: ./prompt.md
input: metric_name, metric_definition, score_distribution
used-by: benchmark-archaeology
---

# Metric Decomposition SOP

Decompose a benchmark's evaluation metric into its constituent signals to understand what is actually being measured, identify ceiling effects, and detect polarity issues.

## Input

- **metric_name**: Name of the metric (e.g., "F1", "BLEU", "Pass@k", "ELO")
- **metric_definition**: Mathematical definition or description of how the metric is computed
- **score_distribution**: Known score distribution data (top scores, median, floor)

## Procedure

1. Parse metric formula into component signals
2. Identify what each component rewards/penalizes
3. Analyze ceiling and floor effects
4. Check for known pathologies (metric gaming, Goodhart's law instances)
5. Assess granularity and discriminative power at current SOTA level

## Output

Structured decomposition showing constituent signals, their weights, ceiling effects, and known pathologies.