Realistic load-test design

Workload Modeling & Test Design

When to Use

Use before writing or executing any load, stress, spike, or soak test. Never generate synthetic traffic by hammering a single endpoint in an infinite zero-think-time loop and calling the result "system capacity."

Open vs. Closed Workload Models (Avoiding Coordinated Omission)

Choosing the wrong arrival model invalidates latency and throughput measurements:

1. Open System Model (Arrival-Rate Driven — Constant/Ramping RPS):
   • New requests arrive independently of whether previous requests have completed (e.g., public APIs, webhooks, checkout events, search traffic).
   • Why it matters (Gil Tene's Coordinated Omission): In a closed VU loop, when the server slows down from 50ms to 5s, each VU sends 100x fewer requests, hiding the queue buildup and making p95 look artificially low. Use arrival-rate executors (k6 constant-arrival-rate / ramping-arrival-rate, Locust/Gatling open arrival profiles) to measure true queueing delay and dropped iterations.
2. Closed System Model (Concurrency / Virtual-User Driven):
   • A fixed pool of active workers or logged-in operators waits for each response before issuing the next action after a realistic think time.
   • Use only when the real caller population is bounded and strictly sequential per session.

Little's Law & Sanity Checking the Model

Relate concurrency $L$, arrival rate $\lambda$, and mean time in system $W$ across steady-state intervals: $$L = \lambda \times W$$

• Example: To sustain $\lambda = 200\text{ req/s}$ when average response time is $W = 0.25\text{ s}$, the system holds $L = 50$ concurrent in-flight requests. If p99 climbs to $2.5\text{ s}$ under contention, in-flight concurrency spikes toward $500$, exhausting connection pools or worker threads.
• Size preAllocatedVUs / maxVUs in k6 (or async worker pools in Locust) using $\lambda_{\text{peak}} \times W_{\text{timeout}}$ so the load generator itself never starves for VUs when latency degrades.

Representative Traffic Mix & Data Cardinality

1. Journey Mix Ratios:
   • Model realistic read/write/search/background proportions (e.g., 70% catalog browse/read, 20% search/filter, 8% cart/mutation, 2% checkout/heavy write) rather than 100% uniform hits.
2. Data Cardinality & Cache Realism:
   • Never query the same user_id=1 or single cached row across 10,000 requests—that benchmarks the L1 memory cache, not the database or disk I/O.
   • Parameterize payloads from realistic, pre-seeded datasets (following Zipfian/Pareto hot-key + cold-tail distributions) and ensure database row counts match target scale.
3. Think Time & Pacing:
   • Include randomized think time (jittered intervals, never synchronized fixed timers that create artificial thundering herds) when modeling human user sessions.

Test Type Selection & Explicit SLO Thresholds

Match the test profile to the engineering question and define pass/fail SLIs upfront:

• Smoke / Calibration Test: 1–5 VUs for 30–60s to verify scripts, auth, telemetry, and zero functional errors.
• Baseline / Target Load Test: Expected normal and peak traffic ($\lambda_{\text{target}}$) with strict SLO gates (e.g., http_req_failed < 0.1%, p(95) < 300ms, p(99) < 800ms, dropped_iterations == 0).
• Breakpoint / Stress Test: Stepwise ramp to locate the knee of the curve where latency or error rate violates SLOs.
• Spike / Soak Test: Sudden burst resilience or multi-hour memory/connection leak detection.