Executable bug reproductions

Turns a bug report into a self-contained test that fails for the real defect and can protect its fix afterward.

executable-reproduction-tests

See how it helps

When is it useful?

Use it when a reproduction exists but still depends on manual steps, leftover data or unpredictable timing.

What does it help you do?

Ensures the test reaches and fails on the actual defect.
Creates minimal independent fixtures with controlled time and randomness.
Cleans owned resources and reports bounded intermittent failure rates.

Try asking this

Turn this duplicate-submit report into a regression test that fails before the fix. Control the two submissions with a synchronization point instead of a sleep. The failing assertion should identify duplicate persisted records, not a setup error.

Detailed documentation

Read the skill’s original instructions, examples and checks in their authored language.

Details…Show less

Executable Reproduction Tests

When to Use

Use whenever a reproduced bug can be expressed as an automated test case (unit, integration, API, or Playwright/E2E) or a self-contained reproduction script using the project's existing test runner.

Properties of a High-Signal Reproduction Test

1. Fails for the Right Reason (Target-Specific Assertion)

A reproduction test is useless if it fails because of a missing mock, a syntax error, an unseeded database table, or a generic timeout instead of the actual product defect:

Verify Code-Path Execution: Confirm the test actually reaches the defective function, SQL query, or UI handler before failing.
Assert the Specific Expected Contract:
- Do not write weak assertions like expect(() => run()).not.toThrow() when the bug is a wrong return value, or expect(res.status).toBe(500) (which cements the buggy behavior!).
- Write the assertion for the correct expected behavior (expect(res.status).toBe(200) and expect(body.total).toBe(150)), so the test fails today with the exact bug signature and will pass unmodified once the fix is applied.

2. Self-Contained Setup & Teardown

Seed only the minimal fixture data required by the reproduction inside the test (beforeEach / factory helper / temp directory).
Never depend on external network availability, live third-party credentials, or leftover state from previous test files.
Clean up all temporary files, timers, child processes, and database records in afterEach / try ... finally.

3. Deterministic Control Over Time, Randomness & Concurrency

Eliminate flakiness in reproduction tests:

Time-Dependent Bugs: Use fake timers or injected clocks (vi.setSystemTime(...), sinon.useFakeTimers, frozen Clock interface) rather than sleep(2000).
Race Conditions: Use explicit synchronization barriers (Deferred promises, deterministic request interceptors, controlled stream chunks) rather than hoping setTimeout(..., 10) hits the race window.
Intermittent Bugs: If a bug remains probabilistic (p < 1), run a bounded loop or stress harness with fixed seeds and report the exact failure rate (k failures in N runs).

4. Non-Intrusive Instrumentation Rule

Never silently mutate production application code to force a test to fail.
If temporary diagnostic logging or a test hook is needed to prove a timing window, keep it strictly isolated and clearly marked as diagnostic instrumentation to be removed.

Included files 1

SKILL.md