> ## Documentation Index > Fetch the complete documentation index at: https://docs.caveman.so/llms.txt > Use this file to discover all available pages before exploring further. # Caveman accounting, evidence bases, and savings reporting > Understand how Caveman labels token counts, cost estimates, and savings by evidence basis so numbers stay honest and comparable across reports. Caveman labels every number by how it was produced. Token estimates, provider usage records, benchmark comparisons, and verified savings records each answer different questions, and they must not share one label. This page explains the evidence bases, how token counts and costs are calculated, and how to read the `caveman stats` report. ## Evidence bases | Basis | What it means | What it does not mean | | - | - | - | | `measured` | Directly counted by named local or provider mechanism | Automatically billable or causal | | `inferred` | Estimated by local model, tokenizer, or assumption | Provider-confirmed usage | | `provider_reported` | Returned by provider usage fields | Independent invoice reconciliation | | `benchmark_counterfactual` | Difference between controlled fixture variants | Production savings | | `observed` | Before/after correlation in live observations | Proof that change caused difference | | `verified` | Meets a named enforced verification method | Universal quality or future savings | | `unpriced` | No supported public price is known | Zero real-world cost | UI and reports should show the basis next to the value. A value cannot silently change basis while being aggregated. ## Token counts The Engine uses offline `o200k_base` counting where available and character estimate fallback. Those counts are `inferred` for provider billing purposes. Providers may tokenize the same text differently and may count cache or image inputs under separate units. Provider-reported token fields keep the provider basis. Do not replace them with a local estimate when the field is missing. ## Cost calculation The provider catalog contains dated public list prices. Local cost display is: ```text theme={"dark"} provider-reported or explicitly selected token units × dated public catalog rate = list-price subtotal ``` This is not an invoice. It can differ because of negotiated terms, subscriptions, regional prices, batch rates, caching rules, credits, taxes, or provider billing adjustments. Unknown provider or model prices produce zero plus `unpriced`. Zero prevents invented cost from entering totals; `unpriced` prevents zero from being mistaken for free use. ## Inferred headroom Local compression and learning reports can estimate avoidable context, reported as headroom under stated assumptions. Keep its time basis unchanged: a per-day estimate must not become a monthly claim unless the method measures each day and states that the result is a projection. ## Benchmark evidence A useful benchmark report states: * exact fixtures and their hashes or versions * command and code revision * counter or provider usage source * baseline and variant definitions * quality or invariant checks * failures and excluded cases * scope of conclusion One fixture result supports only that fixture and method. Average reduction does not prove equal task quality, and recovery checks establish source availability rather than model comprehension. ## Verified values `verified` is reserved for methods whose prerequisites are enforced and whose records identify the method. Public local runtime should not mark Engine estimates, skill output, pixel conversion, TOON output, cache plans, or merged code as verified by themselves. When evidence is incomplete, use `inferred`, `observed`, `provider_reported`, or honest zero. ## Negative and failed results Keep negative deltas and failed transformations. Dropping regressions biases a result. A rejected optimization can still incur provider usage; account for that usage even though compact output was not served. ## Publication checklist Before publishing a number: 1. Name the exact evidence basis. 2. Link the committed fixture or source record. 3. Disclose the counter and pricing date. 4. State the quality test and failure count. 5. Distinguish list price from invoice. 6. Avoid extrapolating across model, provider, task, or time. 7. Publish zero or `unpriced` when support is absent. See [Honest Numbers](https://github.com/JuliusBrussee/caveman/blob/main/docs/HONEST-NUMBERS.md) for currently supported public claims. ## Local stats and savings accounting `caveman stats` reports the requests routed through this machine's Caveman proxy. It prints a terminal summary and writes an offline dashboard. In an interactive terminal the dashboard opens automatically; `--plain` keeps the command in the terminal. `--json` produces the complete report without opening a browser or writing HTML. ```bash theme={"dark"} caveman stats caveman stats --days 7 caveman stats --all-time --provider anthropic caveman stats --model gpt-5.6 --auth subscription caveman stats --json caveman stats --out ./savings.html --open ``` The default window is 30 UTC calendar days. Provider, model, agent, and auth filters use the exact values captured on each request. A current model selection never changes the price or identity of older requests. The report does not scan prompts from other agents or assume their traffic used Caveman. ### What a token delta means For an eligible request, the proxy normalizes original and final request JSON and counts both with the same offline tokenizer. Key order and equivalent escaping cannot manufacture a reduction. The difference includes replacement text, recovery markers, and request overhead. Negative differences remain negative. Original-request fallbacks earn no compression savings. Failed and incomplete requests do not earn savings merely because a compressor produced output. These are local estimates, not exact provider token counts. The provider can use a different tokenizer, add hidden framing, or bill image and audio content separately. The report keeps provider-reported usage apart from the local before/after pair. Multimodal requests, payloads over 4 MiB, and unsupported comparisons remain unavailable instead of counting base64 image bytes as text tokens. The baseline is the request at the proxy boundary. It answers what this request would contain without the proxy's transforms. It does not reconstruct an entire alternate agent run. Native masking or MCP compression performed before that boundary, extra agent turns, recovery behavior, output verbosity, and task quality need their own paired evidence. A file is only represented when its read output reaches this boundary; a file's size on disk alone earns no savings. Repeated requests are repeated observations, not unique files. ### Three separate money questions * **API spend:** provider-counted billing buckets at the captured published rates. Complete reported usage remains eligible when a stream later fails; that failure still earns no counterfactual savings. It is a catalog estimate, not an invoice reconciliation. * **Estimated API input reduction:** the signed input delta valued using the request's observed mix of uncached input, cache reads, and cache writes. The original request's cache behavior is unknown, so this remains an estimate. * **Subscription API equivalent:** the same arithmetic for subscription or non-billable OAuth traffic. It illustrates the API value of that context; it does not reduce a subscription bill or prove additional quota. OAuth is an authentication mechanism, not a billing plan. Paid cloud OAuth routes such as Vertex stay in the API category when their billing provenance supports it. Unknown billing modes stay separate. Unknown models never borrow a sibling model's rate or a blended provider price. Rates and catalog provenance are captured with new requests. Prices are not recomputed from the model selected when the report is opened. Unsupported prices and absent measurements are `null`/unavailable; a displayed subtotal only covers the priced rows. Legacy compressed-segment measurements remain separate from the stronger request comparison. The report never promotes local evidence to `verified` savings or projects a window into a monthly claim. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.