Metamorphic testing

Metamorphic Testing

Use this skill when an exact expected output for a single arbitrary input is difficult to compute without re-implementing the production algorithm (e.g., complex search/filtering, reporting aggregations, multi-rule pricing/tax engines, scheduling optimizers, or data pipelines).

How Metamorphic Testing Works

Instead of asserting $f(x) = y_{\text{exact}}$, generate a source input $x$, compute $y_1 = f(x)$, apply a controlled transformation $T$ to produce a follow-up input $x' = T(x)$, compute $y_2 = f(x')$, and assert a Metamorphic Relation (MR) $R(x, x', y_1, y_2)$.

Core Metamorphic Relation Patterns

1. Subset / Restriction Relations (Filtering & Search):
   • Adding a conjunctive filter (AND status = 'active', narrower date range, stricter permission scope) must produce a result set $y_2 \subseteq y_1$, with $\text{count}(y_2) \le \text{count}(y_1)$.
   • Expanding a filter (OR, wider date window) must produce $y_2 \supseteq y_1$.
2. Permutation & Partition Invariance (Order Independence):
   • When an aggregation or batch processor is specified to be order-independent, permuting the input rows or splitting a batch into sub-batches must yield the exact same aggregate totals (verify first whether rounding or stateful caps make the operation order-dependent!).
3. Monotonicity & Scaling Relations (Pricing, Quotas, Tax):
   • Increasing quantity or usage on a non-promotional monotonic tariff must never decrease the total charge: $q_2 > q_1 \implies \text{price}(q_2) \ge \text{price}(q_1)$.
   • Applying an additional valid discount code must satisfy $0 \le \text{price}{\text{discounted}} \le \text{price}{\text{base}}$.
4. Additive Decomposition (Reporting & Rollups):
   • A report over interval $[t_0, t_2)$ partitioned at $t_1$ must satisfy $\text{total}([t_0, t_1)) + \text{total}([t_1, t_2)) = \text{total}([t_0, t_2))$, and summing per-status or per-department breakdowns must equal the grand total.
5. Equivalence Under Representation / Pagination:
   • Fetching 100 items in pages of size $k \in {1, 7, 25, 100}$ with stable sorting must yield the exact same ordered sequence of items without duplicates or gaps.

Guarding Against Invalid Relations

• Check Boundary Preconditions: Document where a relation intentionally breaks (e.g., tiered volume pricing where buying 1 more unit crosses a whole-order discount cliff, or per-line rounding vs. invoice-level rounding).
• Verify Differential Execution: Ensure $x$ and $x'$ actually exercise non-empty result sets so subset/equality relations do not pass trivially on $\emptyset \subseteq \emptyset$.

Checklist

• Is the metamorphic relation grounded in an approved domain rule rather than an assumed mathematical identity?
• Have rounding modes, volume cliffs, and cap thresholds been checked before asserting commutativity or monotonicity?
• Do source and follow-up inputs produce non-empty results often enough to prevent trivial $\emptyset = \emptyset$ passes?
• Are pagination, filter composition, and rollup consistency verified across generated parameters?