AI token, cache and cost control
Reduces avoidable model context and paid usage while keeping task requirements, accounting and budget limits explicit.
token-cache-cost
Team members using this skill
When is it useful?
Use when agent runs repeatedly resend large context or tool outputs.
What does it help you do?
- Keeps stable prompt prefixes reusable and volatile content separate.
- Bounds tool results and loads only needed tool definitions.
- Compares model routing with recorded usage and enforceable budgets.
Try asking this
Analyze an expensive coding run and reduce repeated log output while preserving facts required to complete the task. Compare the same investigation with full logs and bounded excerpts. Preserve error identifiers and unresolved decisions, then report actual cost and completion-quality differences.
Detailed documentation
Read the skill’s original instructions, examples and checks in their authored language.
Show less
Token, Cache & Cost Optimization
In multi-turn agent loops, input tokens grow quadratically ($O(T^2)$) with turn count $T$ because the full conversation history and tool definitions are re-sent on every step. Controlling agent cost and latency requires high prompt-cache hit rates, disciplined tool-output shaping, and right-sized model routing.
Maximizing Prompt Cache Hit Rates
Provider KV-cache reuse (Anthropic cache_control, OpenAI automatic prefix caching, Gemini context caching) cuts input token cost by 75–90% and drastically reduces Time-To-First-Token (TTFT), but a single mutated token near the top of the prompt invalidates the entire downstream cache prefix:
| Common Cache-Busting Anti-Pattern | Fix |
|---|---|
Injecting Current time: 2026-10-20T04:01:12Z or per-turn counters in the system prompt |
Keep the system prompt 100% static; pass dynamic timestamps or turn metadata in the final user/tool turn (or round dates to the day if needed globally). |
Non-deterministic Object.keys() or dynamic registration order of tools |
Sort tool definitions and JSON Schema properties deterministically by name before serialization. |
Toggling tools on/off inside the tools array on every turn |
Keep the core tools array stable across a phase, or place volatile deferred tools after explicit cache breakpoints. |
| Rewriting or re-summarizing the entire message history on every turn | Append-only during normal turns; compact only when crossing a high-water threshold (e.g., 75% window) so the prefix stays cached for many consecutive turns. |
Tool Output & Schema Token Discipline
- Bound tool outputs at the source: A single unfiltered
grep, DOM dump, or log file read can inject 50,000 tokens that get re-billed on every subsequent turn of the run. Enforce strictmaxLines/maxBytes, return structured summaries with line numbers, and let the agent fetch targeted slices on demand. - Lazy / deferred tool schemas: When an agent has access to dozens of MCP or domain tools, avoid injecting 20,000 tokens of JSON Schema on turn 1 if only 5 core tools are routinely used. Provide a compact catalog or search/discovery tool (
defer_loading: true/ schema lookup) and load full schemas only when needed. - Concise structured intermediate turns: Instruct subagents and internal classification steps to emit compact JSON or terse bulleted findings rather than verbose conversational boilerplate.
Tiered Model Routing & Reasoning Effort
Match model capability and reasoning effort (low | medium | high) to the cognitive complexity of each step:
- Frontier / High-Effort Model: Architecture design, multi-file root-cause diagnosis, subtle concurrency/security review, and complex orchestration planning.
- Mid-Tier / Standard-Effort Model: Routine code implementation following an approved plan, standard tool-calling workflows, and structured test generation.
- Fast / Lightweight Model: Context compaction summaries, query expansion, log/file triage, classification, and schema formatting.
Cost Accounting & Budget Circuit Breakers
Compute exact cost per turn from the provider's normalized usage object:
$$\text{Cost}{\text{turn}} = I{\text{uncached}} \cdot P_{\text{in}} + I_{\text{cache_write}} \cdot P_{\text{cw}} + I_{\text{cache_read}} \cdot P_{\text{cr}} + O_{\text{total}} \cdot P_{\text{out}}$$
Track cache_hit_ratio = cache_read_input_tokens / (input_tokens + cache_read_input_tokens + cache_creation_input_tokens) in telemetry. Alert when cache hit ratio drops below expected baselines (typically $> 75%$ on multi-turn agent loops) and enforce hard per-run token and USD ceilings.
Included files 1
SKILL.md