Orqestria Download
← Back to Echo
AgentEcho

Skill · Version 1.0.0

Durable agent execution

Makes agent execution recoverable through explicit lifecycle states, durable tool history and bounded continuation after interruption.

agent-runtime-state

See how it helps

When is it useful?

Use it when runs pause for approval, lose a connection or crash while a tool may already have changed something.

What does it help you do?

  • Defines queued, running, approval and terminal states explicitly.
  • Persists tool intent before execution and reuses recorded results.
  • Bounds loops and reconciles uncertain writes before retrying.

Try asking this

Recover an interrupted agent run without sending its completed message twice. Simulate a crash after delivery but before the final checkpoint. Reconcile the recorded message identifier and resume only the unfinished work.

Detailed documentation

Read the skill’s original instructions, examples and checks in their authored language.

Details…Show less

Agent Runtime & State Machines

Treat an agent run as a durable, event-sourced state machine rather than an in-memory while loop around an LLM call. Every model turn, tool invocation, human approval pause, and cancellation must transition through explicit states and persist enough history to resume cleanly after a process restart or crash.

Explicit Run State Machine

Model the lifecycle of an agent run and each turn with unambiguous states and legal transitions:

State Meaning Allowed Next States Persistence Requirement
queued Run created, awaiting worker slot running, cancelled Initial input, configuration, correlation ID
running Model inference or turn orchestration active awaiting_tool, awaiting_approval, completed, failed, cancelled Append turn_started and streamed deltas or final assistant message
awaiting_tool One or more tool calls dispatched running, awaiting_approval, failed, cancelled Persist tool_call_requested with tool_call_id, name, and canonical arguments before executing
awaiting_approval Paused on human or policy gate awaiting_tool, running, cancelled, timed_out Persist pending tool_call_id, approval payload, and resume token
completed Terminal goal or stop condition reached (terminal) Final structured output, stop reason, aggregated usage
failed Unrecoverable error or budget exhausted queued (explicit retry/resume) Structured error code, failure phase, last checkpoint ID
cancelled / timed_out Aborted by caller, signal, or deadline queued (explicit resume) Cancellation reason, any partial tool outputs completed before abort

Append-Only Event Log & Checkpointing

  1. Write-ahead tool intent: Persist the assistant message containing tool_use blocks to the run store before invoking any tool handler. If the worker crashes mid-tool, recovery knows exactly which tool call was in flight.
  2. Idempotent tool result recording: Record each tool_result keyed by (run_id, turn_index, tool_call_id). When resuming an interrupted turn, replay already-recorded tool_result entries from storage instead of re-executing completed tools.
  3. Side-effect classification:
    • read_only: Safe to automatically re-run on crash recovery if no result was recorded.
    • idempotent_write: Safe to re-run only when called with the persisted deterministic idempotency_key (sha256(run_id + tool_call_id)).
    • non_idempotent: Never auto-retry after an ambiguous crash (in_flight without result); transition to awaiting_approval or emit a reconciliation status check tool first.

Loop Guards, Budgets & Termination

Never rely solely on the model deciding to stop:

  • Hard turn & step ceilings: Enforce max_turns and max_tool_calls_per_turn. When turn_count == max_turns - 1, inject a final system notice instructing the model to synthesize its best answer or return a structured partial-progress handoff without calling further tools.
  • Token and cost budgets: Track cumulative input_tokens, output_tokens, cache_creation_input_tokens, and cache_read_input_tokens after every response. Halt with budget_exceeded before starting a turn that would breach the run budget.
  • Cycle and repetition detection: Hash (tool_name, canonical_json(args)) across a sliding window. If the agent invokes the identical failing or no-op tool call $N$ times (e.g., 3 consecutive identical calls), intercept the call and return a synthetic isError: true tool result explaining the loop and requiring a different strategy.
  • Cooperative cancellation (AbortSignal): Thread an AbortSignal into both the LLM stream and every tool execution context. On abort, cancel in-flight HTTP streams, terminate child processes with SIGTERM followed by SIGKILL after grace period, and close the turn with a synthetic cancelled tool_result so conversation history remains valid for future resumption.

Included files 1

  • SKILL.md