Property-based testing

Property-Based Testing

Use this skill when implementing property-based tests, custom generators (Arbitrary / Strategy / Gen), input distributions, and property assertions.

Ecosystem & Tooling Alignment

• Adapt to the repository's language and existing test runner before proposing dependencies:
  • TypeScript / JavaScript: fast-check with Vitest or Jest.
  • Python: Hypothesis with pytest.
  • PHP / Laravel: PHPUnit data providers / Eris / QuickCheck-style generators integrated with Pest or PHPUnit.
  • Rust / Go / JVM: proptest or quickcheck (Rust), rapid or pgregory.net/rapid (Go), jqwik or ScalaCheck (JVM).
• Do not introduce a new testing framework without clear justification when the project already has one.

Generator Design Principles (Hughes & Claessen)

1. Construct Valid States by Design, Not by Filtering:
   • Build domain objects compositionally (fc.record, st.composite, prop_compose!) so generated values satisfy structural constraints by construction.
   • Avoid heavy .filter() / assume() chains that discard most samples and cause health-check failures or starvation of deep paths.
2. Generate Dependent Entities Coherently:
   • Use monadic chaining (chain / flatMap / @st.composite) when one input depends on another: e.g., generate a payment first, then generate a partial refund amount in $[1, \text{payment.capturedAmount}]$, or generate a workspace first, then draw actors and resources from that workspace's membership pool.
3. Weight Boundary and Adversarial Values Explicitly:
   • Pure uniform random numbers rarely hit exact thresholds. Bias generators toward domain boundaries:
     • Zero, $1$ minor unit, $\text{threshold} - 1$, $\text{threshold}$, $\text{threshold} + 1$, maximum allowed limit, and exact expiration timestamps.
     • Empty collections, single-element collections, duplicate items, and multi-item batches with identical keys.
4. Preserve Shrinkability:
   • Build custom generators by mapping and composing library primitives rather than drawing raw random bytes or calling Math.random() / random.randint() outside the PBT engine, which breaks integrated shrinking.

Execution Budgets: CI vs. Exploratory Profiles

• Fast CI Profile: Configure a deterministic or bounded run budget (e.g., numRuns: 100–250, explicit per-test timeout, zero external network calls) so the suite runs reliably in seconds on every pull request.
• Deep Exploratory Profile: Expose an environment-driven override (e.g., PBT_RUNS=5000 or HYPOTHESIS_PROFILE=deep) for stress-testing complex state spaces and rare boundary combinations during investigation.
• Strict Test Isolation: Reset in-memory stores, database transactions, fake clocks, and queues between every generated sample (beforeEach / per-iteration setup) so sample $N$ never leaks state into sample $N+1$.

Checklist

• Do generators construct valid domain structures directly instead of relying on high-rejection .filter() / assume() predicates?
• Are dependent values (e.g., refund vs. payment, line items vs. invoice currency) chained from their parent entities?
• Are boundary values ($0$, $1$, $\text{limit}-1$, $\text{limit}$, $\text{limit}+1$) explicitly represented in the generator distribution?
• Is state completely isolated between individual generated iterations, with bounded runtime suitable for CI?