Költséganomáliák felismerése

Cost Anomaly Detection

Sudden cost spikes rarely look like normal growth. They usually stem from a specific pathology: an infinite retry loop, a broken prompt cache prefix, a runaway subagent tree, or an unauthenticated endpoint leak.

Workflow

1. Monitor structural efficiency ratios alongside raw spend:
   • Track per-model, per-agent, and per-tenant health indicators over rolling windows (5m, 1h, 24h):
     • Prompt cache hit ratio: $\text{cache_read_input_tokens} / \text{total_input_tokens}$. A sudden drop from 85% to 5% after a deploy indicates a dynamic timestamp or UUID was injected near the start of the system prompt.
     • Cost per completed task: $\text{Total Run Cost} / \text{Completed Tasks}$. Spikes indicate excessive turns, tool-call loops, or high failure/retry rates.
     • Retry amplification factor: $\text{Total Provider Calls} / \text{Logical Steps}$.
     • Input-to-output token ratio: Unusually large input growth across consecutive turns within a single runId signals unbounded context accumulation or huge tool outputs being dumped into context.
2. Baseline comparison and velocity checks:
   • Compare current window spend velocity (microsPerMinute) against:
     • Rolling historical baseline for the same tenant/workflow (seasonality-aware day-of-week / hour-of-day median and MAD/IQR).
     • Absolute velocity circuit breakers (e.g., a single task run exceeding $X in 60 seconds or N consecutive identical tool calls).
3. Produce actionable diagnostic payloads:
   • When flagging an anomaly, never emit a vague "Cost is high" alert. Include the top contributing dimensions:
     • Top tenantId, taskId / runId, agentRole, model, and operationType responsible for the delta.
     • Decomposition of the spike: volume change (more requests), mix shift (switching from a small model to a frontier model), size inflation (larger prompts/outputs), or cache collapse (lost cache hits).
4. Coordinate automated containment:
   • Feed runaway loop signals (maxTurnsExceeded, repeated identical tool errors, velocity breach) into the budget reservation circuit breaker to pause the offending runId or apiKeyId without taking down healthy tenants.

Anti-Patterns to Avoid

• Alerting only on daily/monthly totals: Waiting 24 hours to detect a runaway loop that spends $50/hour.
• Global-only thresholds: A $10/hour spike on a small workspace gets masked by aggregate platform traffic unless baselines are evaluated per tenant and per workflow.