← Back to Business Systems Specialist Collection
See how it helps
Retries and compensation
Recovers failed workflow steps using durable results, bounded retries and explicit handling of uncertain external effects.
retries-compensation
Team members using this skill
When is it useful?
Use when a multi-step process can fail after earlier actions have already completed.
What does it help you do?
- Retries the failed step with stable identities instead of replaying successful work.
- Distinguishes transient failure, rejection and unknown remote outcome.
- Plans compensating business actions and visible manual recovery where reversal is incomplete.
Try asking this
Design recovery for reserved stock followed by failed provisioning, checking remote status before repeating any uncertain charged action. Fail provisioning after a confirmed reservation and an uncertain charge. Release only safely reversible reservations and reconcile the charge before retrying or compensating.
Detailed documentation
Read the skill’s original instructions, examples and checks in their authored language.
Show less
Retries & Compensation
Use this skill when designing error handling, backoff retry policies, idempotency guarantees, manual recovery, and Saga compensations in workflows.
Step-Scoped Retries & Idempotency (Helland & Hohpe)
- Retry the Failing Step, Never the Whole Workflow from Scratch:
- When step $S_k$ fails after steps $S_1 \dots S_{k-1}$ succeeded, checkpointed upstream results remain intact. Retry only $S_k$ according to its step-level policy (
maxAttempts, initial delay, exponential backoff, jitter).
- When step $S_k$ fails after steps $S_1 \dots S_{k-1}$ succeeded, checkpointed upstream results remain intact. Retry only $S_k$ according to its step-level policy (
- Deterministic Idempotency Keys for External Side Effects:
- Never promise "exactly-once" external execution without an idempotency mechanism.
- Derive a deterministic idempotency key (
hash(tenantId, runId, stepExecutionId)) that stays identical across retry attempts of the same step execution, and pass it to downstream APIs (payments, emails, ticket creation, order placement).
- Separate Retryable vs. Non-Retryable Failures:
- Retryable: Transient network timeouts,
429 Too Many Requests,502/503/504gateway errors. - Non-Retryable (Fail Immediately): Validation errors (
400/422), permission denials (401/403), missing resources (404), or business rule rejections.
- Retryable: Transient network timeouts,
- Surface Uncertain External Outcomes Explicitly:
- If a non-idempotent external call times out after the request was sent, do not blindly re-send a second charge or message. Mark the attempt as
uncertain_side_effector query the remote provider's status before retrying or escalating to operator review.
- If a non-idempotent external call times out after the request was sent, do not blindly re-send a second charge or message. Mark the attempt as
Saga Compensation & Partial Failure Recovery
- Compensating Steps Are Forward Business Actions, Not Magic Time Travel:
- When a multi-step workflow fails at step 4 after steps 1–3 committed external changes (e.g., reserved stock, created invoice draft, provisioned user), execute explicit compensating steps in reverse topological order (
ReleaseStock,VoidDraftInvoice,DeprovisionUser). - Record compensation status (
compensating$\to$compensated|compensation_failed) distinctly in the run history, and make compensating handlers idempotent and retryable themselves.
- When a multi-step workflow fails at step 4 after steps 1–3 committed external changes (e.g., reserved stock, created invoice draft, provisioned user), execute explicit compensating steps in reverse topological order (
- Controlled Operator Replay (
Retry from Failed Step):- Allow authorized operators to retry a failed step or resume a paused run from its last durable checkpoint with full audit logging.
Checklist
- Are retries scoped to the individual failing step using a stable per-step
idempotencyKeyrather than re-running completed prior steps? - Are non-retryable
4xx/validation errors separated from transient5xx/429errors? - Are post-dispatch timeouts on non-idempotent external calls flagged or reconciled before retrying?
- Are Saga compensations modeled as explicit, idempotent reverse steps with their own tracked execution status?
Included files 1
SKILL.md
