Orqestria Download
← Back to AI automation and cost control

Skill · Version 1.0.0

Verify usage metering

Tests usage accounting for conservation, precise accumulation, duplicate delivery and historical pricing consistency.

metering-verification

Team members using this skill

See how it helps

When is it useful?

Use when metered costs or quotas determine customer balances and reporting.

What does it help you do?

  • Checks that every reporting breakdown sums to the original ledger total.
  • Verifies repeated and upgraded usage evidence cannot charge the same event twice.
  • Tests price boundaries and provider-specific usage normalization.

Try asking this

Verify tiny token charges, duplicate stream events and estimated-to-final adjustments without changing historical amounts after a new price version. Apply a new price only to later usage, then reconcile the report. Old settled amounts remain unchanged and repeated evidence contributes no extra charge.

Detailed documentation

Read the skill’s original instructions, examples and checks in their authored language.

Details…Show less

Metering Verification

A metering bug silently corrupts financial visibility, unit economics, or customer quota enforcement. Every metering pipeline, price calculator, and attribution query must be verified with deterministic tests covering edge cases and failure modes.

Workflow

  1. Verify accounting invariants and conservation laws:
    • Assert that the sum of drill-down slices across any dimension (SUM(cost) GROUP BY tenant_id, GROUP BY agent_role, GROUP BY model) equals the grand total ledger cost (100% conservation, zero dropped or duplicated rows from join fan-out).
    • Assert sub-cent precision: verify that 100,000 micro-events of 10 cached tokens each accumulate to the exact expected microdollar total with zero floating-point drift.
  2. Test deduplication and out-of-order ingestion:
    • Replay the exact same usage event payload 5 times (and under concurrent workers) and assert that ledger count and total quantity increase only once.
    • Test upgrade transitions: ingesting an 'estimated' event followed by a 'provider_reported' event with the same idempotencyKey must supersede or adjust the estimate without double-counting.
  3. Test rate card boundaries and version transitions:
    • Test events with occurredAt 1 ms before effectiveFrom, at effectiveFrom, and 1 ms after effectiveTo.
    • Verify that inserting a new price version does not alter the computed cost of historical events unless an explicit repricing command is run.
  4. Test provider-specific LLM usage normalization:
    • Include fixtures for OpenAI (cached_tokens inside prompt_tokens, reasoning_tokens inside completion_tokens), Anthropic (input_tokens disjoint from cache_creation_input_tokens and cache_read_input_tokens), and Gemini (cachedContentTokenCount inside promptTokenCount, thoughtsTokenCount).
    • Verify that normalized billable buckets never double-charge cached or reasoning tokens.
  5. Test failure, timeout, and reservation lifecycle:
    • Verify that failed upstream calls that consumed tokens still emit usage events.
    • Verify that pre-execution budget reservations (reserve -> settle -> release or expire) never leak locked quota under concurrency.

Verification Checklist

  • Idempotent replay of identical events produces zero duplicate ledger entries.
  • Sub-cent microdollar / nanodollar arithmetic matches exact reference calculations.
  • Provider token fixtures (OpenAI, Anthropic, Gemini) normalize without double-counting.
  • Historical queries use occurredAt rate versions and remain invariant when today's prices change.
  • Sum of tenant + system buckets equals total ingested cost with no join fan-out.

Included files 1

  • SKILL.md