Leistungsregressionen prüfen

Performance Regression Checks

When to Use

Use when verifying whether a code, query, or infrastructure change improved performance or introduced a regression, or when embedding repeatable performance gates into development and CI workflows.

Rules for Apples-to-Apples Comparison ("A/B Benchmark Discipline")

Never compare two measurements taken under different conditions without explicitly stating the differences. For a valid before-vs-after comparison:

1. Identical Workload & Dataset:
   • Same arrival rate ($\lambda$), concurrency, request mix, payload sizes, and seeded database volume/distribution.
2. Identical Environment & Warmup:
   • Same hardware/container CPU & memory limits, runtime version, configuration flags, and cache warmup phase before recording steady-state metrics.
3. Multiple Trials & Variance Control:
   • Single short runs on shared machines suffer from noisy-neighbor CPU spikes, OS background tasks, and GC timing.
   • Run multiple iterations (or interleaved A/B/A/B runs on the same host) and compare medians/percentiles alongside dispersion (IQR / min–max across runs), not a single cherry-picked run.

Lightweight CI & Local Performance Regression Gates

Full multi-node production load tests are too heavy for every commit, so combine deterministic proxy metrics with targeted micro-benchmarks:

1. Deterministic Algorithmic & Resource Counters (Zero-Flake CI Gates)

Even on noisy shared CI runners where wall-clock milliseconds fluctuate by $\pm 30%$, work-unit counters are 100% deterministic:

• SQL Query Count per Endpoint/Operation: Assert that GET /api/projects/:id/board executes <= K queries regardless of whether there are 10 or 500 items (catches $N+1$ regressions immediately).
• Payload & Bundle Size Budgets: Assert response byte size, serialized JSON size, and frontend bundle/chunk byte limits.
• Memory Allocation & Round-Trip Counts: Track external API calls, cache lookups, and peak heap allocations per operation.

2. Targeted Component & Endpoint Benchmarks

• Maintain a version-controlled suite of lightweight, self-contained benchmark scenarios (e.g., k6 smoke/baseline scripts or Vitest/Criterion/pytest-benchmark suites) with reproducible fixture seeders.
• Define explicit SLO Thresholds in Code (e.g., k6 options.thresholds: http_req_duration: ['p(95)<200', 'p(99)<450'], http_req_failed: ['rate<0.001']) so regressions fail with a clear, machine-readable exit code.

3. Reporting Before vs. After Evidence

When validating a performance fix or reporting a regression, always present a structured comparison table:

• Baseline (Before) vs. Candidate (After) across RPS / Goodput, p50, p95, p99, Error %, and key resource metrics (CPU %, Memory MB, DB queries/req).
• Clearly state the relative delta (e.g., p95: 420ms -> 85ms (-79.8%)) and confirm that functional correctness checks passed under load.