Validate metric meaning

Checks whether a dashboard number answers the intended question with correct populations, units and time boundaries.

data-semantics-metric-validation

Team members using this skill

See how it helps

When is it useful?

Use when totals or conversion rates differ between reports.

What does it help you do?

Defines numerator, denominator, observation unit and relevant time window.
Avoids averaged averages and checks changes in population composition.
Distinguishes missing values, actual zero and not-applicable data.

Try asking this

Verify monthly conversion rates across unequal sales teams using underlying counts and explicit timezone boundaries. Combine a small high-conversion team with a large low-conversion team. Recompute the total from underlying counts and explain why averaging percentages gives a different answer.

Detailed documentation

Read the skill’s original instructions, examples and checks in their authored language.

Details…Show less

Data Semantics & Metric Validation

Use this skill when defining, querying, transforming, or aggregating data for charts, KPIs, and dashboards.

Core Metric Integrity Principles

Clarify the Question, Numerator, Denominator, and Grain First:
- Before choosing a chart, define the exact metric formula, unit of measure, currency/scale, entity grain (per-user, per-session, per-order, per-event), and population inclusion rules.
- Never modify an approved business metric definition autonomously to make a chart look smoother.
Never Average Averages or Unweighted Percentages:
- Computing the mean of daily conversion rates or regional margins without weighting by the underlying denominator ($\sum \text{numerator}_i / \sum \text{denominator}_i$) produces mathematically false totals whenever bucket sizes differ.
- Always carry both numerator and denominator through rollups and compute ratios at the final display grain.
Guard Against Simpson's Paradox & Hidden Mix Shifts:
- Aggregate trends can reverse or mask opposing subgroup movements when cohort composition shifts. Provide cohort/category breakdowns or drill-downs alongside top-level KPIs when mix effects can mislead decisions.

Missing Data vs. Zero vs. Not Applicable

Never Coerce null / Missing Measurements to 0:
- A sensor outage, uncollected day, or future period is null (missing), whereas $0 revenue or 0 errors is an observed measurement.
- Render missing intervals as explicit gaps, dashed segments, or "no data" markers—never plot them as drops to zero or silently interpolate without marking the estimate.
Separate Measured Facts from Estimates & Forecasts:
- Visually and textually distinguish historical actuals, incomplete current-period data ("Month-to-date, partial"), and projections/confidence intervals.

Temporal & Data Quality Hygiene

Distinguish Event Time (occurred_at) from Ingestion Time (processed_at):
- Know which timestamp governs the metric and display data freshness ("Last updated at…") and late-arriving data windows.
Inspect Anomalies Before Plotting:
- Check for duplicates, negative adjustments/refunds, extreme outliers, and currency/unit mismatches so raw ingestion artifacts are not shown as genuine spikes.

Checklist

Are the metric formula, unit, grain, and denominator explicitly defined and traced to approved rules?
Are rates, percentages, and averages aggregated using weighted numerators and denominators ($\sum N_i / \sum D_i$)?
Are missing values (null), observed zeros (0), and partial current periods distinguished visually and mathematically?
Are event timestamps, timezones, and data freshness timestamps explicitly handled?

Included files 1

SKILL.md