Performance regression checks

Compares changes under equivalent workloads and uses focused counters or benchmarks to detect meaningful regressions.

performance-regression-checks

Team members using this skill

AgentPulse

See how it helps

When is it useful?

Use when verifying an optimization or introducing a repeatable performance check.

What does it help you do?

Keeps dataset, environment, load and warm-up comparable.
Uses multiple trials and variability rather than a selected fast run.
Adds appropriate query-count, size or targeted runtime checks.

Try asking this

Verify a dashboard optimization on the same dataset and load, comparing response tails and query counts before and after. Repeat both versions several times with identical warm-up. Report slower tail responses and variance as well as improvements, so one unusually fast run cannot decide.

Detailed documentation

Read the skill’s original instructions, examples and checks in their authored language.

Details…Show less

Performance Regression Checks

When to Use

Use when verifying whether a code, query, or infrastructure change improved performance or introduced a regression, or when embedding repeatable performance gates into development and CI workflows.

Rules for Apples-to-Apples Comparison ("A/B Benchmark Discipline")

Never compare two measurements taken under different conditions without explicitly stating the differences. For a valid before-vs-after comparison:

Identical Workload & Dataset:
- Same arrival rate ($\lambda$), concurrency, request mix, payload sizes, and seeded database volume/distribution.
Identical Environment & Warmup:
- Same hardware/container CPU & memory limits, runtime version, configuration flags, and cache warmup phase before recording steady-state metrics.
Multiple Trials & Variance Control:
- Single short runs on shared machines suffer from noisy-neighbor CPU spikes, OS background tasks, and GC timing.
- Run multiple iterations (or interleaved A/B/A/B runs on the same host) and compare medians/percentiles alongside dispersion (IQR / min–max across runs), not a single cherry-picked run.

Lightweight CI & Local Performance Regression Gates

Full multi-node production load tests are too heavy for every commit, so combine deterministic proxy metrics with targeted micro-benchmarks:

1. Deterministic Algorithmic & Resource Counters (Zero-Flake CI Gates)

Even on noisy shared CI runners where wall-clock milliseconds fluctuate by $\pm 30%$, work-unit counters are 100% deterministic:

SQL Query Count per Endpoint/Operation: Assert that GET /api/projects/:id/board executes <= K queries regardless of whether there are 10 or 500 items (catches $N+1$ regressions immediately).
Payload & Bundle Size Budgets: Assert response byte size, serialized JSON size, and frontend bundle/chunk byte limits.
Memory Allocation & Round-Trip Counts: Track external API calls, cache lookups, and peak heap allocations per operation.

2. Targeted Component & Endpoint Benchmarks

Maintain a version-controlled suite of lightweight, self-contained benchmark scenarios (e.g., k6 smoke/baseline scripts or Vitest/Criterion/pytest-benchmark suites) with reproducible fixture seeders.
Define explicit SLO Thresholds in Code (e.g., k6 options.thresholds: http_req_duration: ['p(95)<200', 'p(99)<450'], http_req_failed: ['rate<0.001']) so regressions fail with a clear, machine-readable exit code.

3. Reporting Before vs. After Evidence

When validating a performance fix or reporting a regression, always present a structured comparison table:

Baseline (Before) vs. Candidate (After) across RPS / Goodput, p50, p95, p99, Error %, and key resource metrics (CPU %, Memory MB, DB queries/req).
Clearly state the relative delta (e.g., p95: 420ms -> 85ms (-79.8%)) and confirm that functional correctness checks passed under load.

Included files 1

SKILL.md