Orqestria Download
← Back to Noel
AgentNoel

Skill · Version 1.0.0

Token and cache accounting

Turns differing provider usage reports into comparable records while preserving cache, reasoning and incomplete-stream distinctions.

llm-token-cache-accounting

See how it helps

When is it useful?

Use it when costs disagree across providers or cached and streamed requests produce confusing token totals.

What does it help you do?

  • Normalizes provider counters into non-overlapping usage buckets.
  • Separates cache reads, writes, reasoning and ordinary input.
  • Marks incomplete streams and subscription-mode accounting limitations.

Try asking this

Reconcile cached streaming usage without billing the same input tokens twice. Compare the provider's final counters with accumulated chunks. Store cache reads and ordinary input separately, and flag a missing final usage record for reconciliation.

Detailed documentation

Read the skill’s original instructions, examples and checks in their authored language.

Details…Show less

LLM Token & Cache Accounting

LLM providers report token usage with incompatible semantics. Naive summation across providers routinely double-counts cached tokens or misses hidden reasoning and tool-use tokens by 2x–10x.

Workflow

  1. Normalize provider token semantics into disjoint canonical buckets: Map every provider response into disjoint quantities so total cost is a clean dot product of (bucket_quantity * bucket_rate):
    • uncached_input_tokens: Fresh input tokens billed at standard input rate.
    • cache_write_input_tokens: Tokens written into prompt cache (1.25x or 2x base input rate, or TTL-specific write rate).
    • cache_read_input_tokens: Cached input tokens read at discounted rate (0.1x or 0.25x base input rate).
    • standard_output_tokens: Visible completion tokens billed at output rate.
    • reasoning_output_tokens: Internal chain-of-thought / thinking tokens billed at output rate.
  2. Handle each provider's exact schema:
    • Anthropic (messages API):
      • usage.input_tokens, usage.cache_creation_input_tokens, and usage.cache_read_input_tokens are mutually exclusive (disjoint).
      • Total prompt context = input_tokens + cache_creation_input_tokens + cache_read_input_tokens.
      • If usage.cache_creation splits ephemeral_5m_input_tokens vs. ephemeral_1h_input_tokens, rate each at its respective write multiplier.
    • OpenAI (chat.completions & responses API):
      • prompt_tokens (input_tokens) already includes prompt_tokens_details.cached_tokens.
      • Compute uncached_input_tokens = prompt_tokens - cached_tokens and cache_read_input_tokens = cached_tokens. Never add cached_tokens on top of prompt_tokens!
      • completion_tokens (output_tokens) already includes completion_tokens_details.reasoning_tokens. Compute standard_output_tokens = completion_tokens - reasoning_tokens.
    • Google Gemini (generateContent usageMetadata):
      • promptTokenCount already includes cachedContentTokenCount. Compute uncached_input_tokens = promptTokenCount - cachedContentTokenCount.
      • thoughtsTokenCount is billed at the output rate; store standard_output_tokens (candidatesTokenCount) and reasoning_output_tokens (thoughtsTokenCount) distinctly.
  3. Streaming responses and aborted connections:
    • Enable final usage chunks on streams (stream_options: { include_usage: true } for OpenAI; message_delta / message_stop for Anthropic).
    • If a stream aborts before the final usage frame, record a fallback event with certainty: 'estimated', counting prompt tokens plus tokenized output chunks emitted before abort.
  4. Multi-turn agent loops and CLI / subscription modes:
    • Record each turn in an agent loop as its own event linked to runId and turnIndex, tracking contextGrowthTokens across turns to spot quadratic prompt growth or cache-busting prefixes.
    • For subscription CLI runs with no per-call charge, record tokens, duration, turns, and imputedListCostMicros (billingMode: 'subscription_cli' | 'api_metered' | 'byok').

Included files 1

  • SKILL.md