Isolated prototypes and benchmarks

Tests a risky technical assumption in a small controlled experiment before committing to an implementation approach.

empirical-micro-benchmark-prototyping

See how it helps

When is it useful?

Use when documentation is ambiguous or competing libraries claim incompatible behavior or performance.

What does it help you do?

Builds isolated probes without production credentials or live data connections.
Tests representative inputs, compatibility, cancellation or resource use directly.
Compares equivalent outputs and warmed-up distributions rather than one timing sample.

Try asking this

Compare two parsers on our realistic file shapes, measuring correctness, memory and latency in an isolated prototype. Include a large valid file and malformed input. Compare equivalent parsed results, repeated timings and peak memory; record correctness failures before ranking speed.

Detailed documentation

Read the skill’s original instructions, examples and checks in their authored language.

Details…Show less

Empirical Micro-Benchmark & Isolated Prototyping

When documentation is ambiguous or two candidate libraries make competing performance/behavior claims, a 15-minute isolated experiment settles what hours of speculation cannot.

Workflow

Build in a clean, isolated scratch environment:
- Create prototypes in a temporary scratch directory outside production source paths.
- Never pass production credentials, personal API tokens, or live database URLs into an experimental script.
- Inspect package scripts before installing (npm install --ignore-scripts when probing unknown packages).
Design a minimal, falsifiable experiment:
- Test the exact riskiest assumption first:
  - Functional probe: Does library X actually parse our real-world edge-case input, preserve types, and handle cancellation?
  - Compatibility probe: Does it bundle cleanly with our TypeScript/ESM/bundler configuration without polyfill hacks?
  - Resource probe: What is its cold-start time, peak heap memory (process.memoryUsage()), and behavior under backpressure?
Run fair, controlled micro-benchmarks:
- When comparing performance between Candidate A and Candidate B:
  - Use representative data shapes and sizes (e.g., 100, 10,000, and 100,000 items), not trivial 3-element toy arrays.
  - Include a warm-up phase so JIT compilation and module loading are separated from steady-state throughput.
  - Measure distribution metrics (median/p50, p95, p99 latency, throughput, and memory allocation), not just a single Date.now() run.
  - Verify that both candidates produce the exact same valid output (a fast function that skips validation or drops data is not a valid winner).
Preserve artifacts and clean up:
- Record the exact script, Node/runtime version, hardware context, and raw numbers in the research summary so findings remain auditable.
- Remove temporary packages and scratch files so the repository working tree stays completely clean.

Included files 1

SKILL.md