Caveman Learn: Audit Your Agent's Token Spend

caveman learn shows where your coding agent’s tokens go, and what to fix first. It reads the setup files and session history already on your computer. Nothing is sent anywhere. You get:

a Setup Score from 0 to 100 (higher is leaner)
a list of findings: places your tokens go, biggest first, each with a suggested fix
a local HTML report and a JSON file

It never edits your files. Fixes happen inside Claude Code or Codex, through the caveman-learn skill, one edit at a time, and only after you say yes. Every number is an estimate from your own history (inferred in JSON). No local number is promoted to verified; see Accounting and evidence.

Quick start

caveman learn             # read your sessions, score them, show the top findings
caveman learn implement   # open Claude Code or Codex to review and fix, with consent
caveman learn savings     # later: what the applied fixes returned

Privacy: what it reads and writes

Analysis runs locally. export writes a file for you to inspect and sends nothing.

Reads

Agent	Session transcripts	Config and memory files	Root override
Claude Code	`~/.claude/projects/*/.jsonl`	`~/.claude/CLAUDE.md`, `~/.claude/skills/*`, hook count from `~/.claude/settings.json`, plugin count from `~/.claude/plugins/installed_plugins.json`, MCP servers from `~/.claude.json` and `.mcp.json`, every `CLAUDE.md` from the current directory up to `/`	`CAVEMAN_CLAUDE_ROOT`, then `CLAUDE_CONFIG_DIR`
Codex	`~/.codex/sessions/*/rollout-.jsonl`, `~/.codex/archived_sessions/`	`~/.codex/AGENTS.md`, `./AGENTS.md`	`CAVEMAN_CODEX_ROOT`, then `CODEX_HOME`
Gemini CLI	`~/.gemini/tmp//chats/session-.json`	`~/.gemini/GEMINI.md`, `GEMINI.md` from repo root to cwd	`CAVEMAN_GEMINI_ROOT`
opencode	`~/.local/share/opencode/storage/{session,message,part}`	none	`CAVEMAN_OPENCODE_ROOT`
aider	`.aider.chat.history.md` files under the root	none	`CAVEMAN_AIDER_ROOT` (required; off without it)

The memory and rules doctor also reads .claude/CLAUDE.md, CLAUDE.local.md, .claude/rules/*.md and ~/.claude/rules/*.md (path-scoped rules with a paths: frontmatter are skipped because they load on demand), and Claude Code auto memory at <claude root>/projects/<repo-key>/memory/MEMORY.md plus its topic files, honoring CLAUDE_CODE_PROJECT_DIR_NAME. Two other inputs: the session_outcomes finding reads commit timestamps with git in up to 12 repositories (3 seconds per call, budgeted overall), and the wrap_measured block reads rows the Caveman proxy already recorded in caveman.db.

Writes

All paths are under $CAVEMAN_HOME (default ~/.caveman) unless noted. CAVEMAN_DB overrides the database path.

When	What is written
Any command that builds a plan	`caveman.db` tables `config_snapshots`, `config_snapshot_history`, `learn_sinks`
`caveman learn` and `learn scan` (not with `--no-remember`)	Also: one cavemem memory per `reducible` sink in `mem/mem.db`, and a matching row in the `learnings` table
`caveman learn`, `learn scan --write-report`, `learn report`	`reports/caveman-learn.html`, `reports/caveman-learn.json`, `reports/caveman-learn.YYYY-MM-DD.json` (last 8 dated snapshots kept)
`learn apply <sink>` without `--dry-run`	`candidates/learn-<sink>.json`
`learn applied`	row in `applied_fixes`
`learn experiment start\|arm\|stop`	rows in `experiments`, `experiment_arms`
`learn export`	`reports/caveman-learn-digest.json` (or `--out <path>`)
Autopilot	`runtime/learn-autopilot.json`, `.lock`, `-nudge.json`, `-nudge.inflight.json`, `-announced.json`, and its own report under `runtime/learn-autopilot/reports/`
`learn autopilot on/off`	key `learnAutopilot` in `~/.caveman-cloud/config.json`
`learn implement`	the skill file, only if missing: `./.claude/skills/caveman-learn/SKILL.md` (Claude Code) or `$CODEX_HOME/skills/caveman-learn/SKILL.md` (Codex)

Report files are created with mode 0600.

Commands and flags

The default window is --since 30d. The scan times out after 120 seconds (CAVE_LEARN_TIMEOUT, seconds).

Command	What it does
`caveman learn`	In an interactive terminal: progress, Setup Score, top findings, then a menu (implement, show all findings, open report, done). Otherwise compact text.
`--plain`	Compact text; no animation or menu. `CAVEMAN_PLAIN=1` or `TERM=dumb` has the same effect.
`--all`	Every finding with internal id, basis, evidence and suggestion; confirmed outcomes; per-repository rows.
`--json`	The `caveman.learn.v1` plan on stdout.
`--md`	Detailed Markdown.
`--since <Nd>`	Window, e.g. `7d`.
`--sources <list>`	Any of `claude,codex,gemini,opencode,aider`. Default: all.
`--repo <substring>`	Keep only sessions whose repository matches, before any detector runs. Config is still read from the current directory.
`--retro`	Adds the “would have saved” replay block. Budgets: `--behavior-budget-ms`, `--retro-budget-ms`. Omitted when `--repo` is set.
`implement [claude\|codex] [--prompt "<focus>"]`	Opens the agent with the `caveman-learn` skill and the current report.
`apply <sink_id> [--dry-run]`	Materializes a candidate for a `reducible` or `recurring_context` sink. Edits nothing.
`applied <sink_id> [--fix-kind <k>] [--note <t>]`	Records an approved, re-measured fix in the outcome ledger.
`simulate <sink_id...>`	Token counterfactual over scanned history only.
`savings`	Attributed savings ledger, grouped by measurement method.
`experiment start\|arm\|report\|list\|stop`	On/off holdout over your own sessions.
`export [--out <path>]`	Privacy-reduced digest file.
`reconcile --usage-export <csv>`	Measured tokens vs a provider usage export.
`autopilot [status\|on\|off]`	Background refresh after sessions end.
`scan`, `report`	Lower-level passthroughs to the proxy. `report --json` prints the plan and rewrites the report files.

Examples:

caveman learn --plain --since 7d --sources claude,codex
caveman learn --repo my-service --all
caveman learn --json > plan.json
caveman learn implement codex --prompt "only config trims"
caveman learn apply claude_md_weight:project --dry-run
caveman learn applied claude_md_weight:project --note "trimmed setup section"
caveman learn simulate claude_md_weight:project recurring_context:repaste:<fingerprint>
caveman learn savings --json
caveman learn experiment start deploy-skill --sink procedure_repeat:<key> --fix-kind skill_distillation
caveman learn experiment arm deploy-skill off
caveman learn experiment report deploy-skill
caveman learn experiment list
caveman learn experiment stop deploy-skill
caveman learn export --out ./learn-digest.json
caveman learn reconcile --usage-export ~/Downloads/usage.csv
caveman learn autopilot off

Notes per command:

apply on a behavioral or load_bearing sink returns applied: false with a reason; there is no mechanical fix. A recurring_context candidate carries locators (file, JSONL line, block index, SHA-256), never the block text.
applied needs a fix kind. It is taken from the sink’s evidence or a default (claude_md_weight, claude_md_sections, config_tax, dead_load, dumbzone_advice); otherwise pass --fix-kind. The “before” value comes from the newest report snapshot older than the current scan, so a scan run after your edit is not used as the baseline. The edited file is fingerprinted at record time.
simulate accepts only claude_md_weight:*, claude_md_sections:*, dead_load:skills and recurring_context:repaste:*. Config sinks are tokens per turn times scanned provider-counted turns; recurring blocks are occurrence sums minus the largest occurrence and one pointer per occurrence. Overlapping selections on the same file are counted once. No dollars, no forward projection.
experiment report defaults its window to the experiment’s start.
export carries sink ids, classes, magnitudes, the score, spend totals and per-model rows. It carries no paths, locators, evidence maps, repository names or session text.
reconcile matches CSV columns by header name only and fails on any ambiguous header. Coverage is a token comparison, not a saving.

Setup Score

The Setup Score (cave_score in JSON) grades how lean your local setup is. It starts at 100 and subtracts four capped penalties. The caps sum to 100.

Component	Penalty	Cap	Measured when
`config_tax`	50 x (config tokens/turn + recurring re-paste tokens/turn) / median turn context	35	config tax > 0 and transcripts with usage exist
`dumbzone`	50 x share of turns over 50% of the model window	25	any usage-bearing turn
`dead_load`	40 x skill-description tokens with no detected use / config tokens	20	sessions scanned and config tax > 0
`subagent_pressure`	20 x min(1, subagent spawns per session / 5)	20	any session scanned

Config tokens per turn = user CLAUDE.md + every CLAUDE.md from cwd to / + Claude skill descriptions. AGENTS.md has its own finding and is not in this sum. A component that cannot be measured has penalty 0, measured: false, and a detail string saying why. What it is not:

Not money. It carries no currency and is not derived from spend.
Not billed usage. It is transcript and file arithmetic.
Not the Caveman Cloud Cave Score, which scores organization traffic on a different scale. The two will not match.

The terminal and TUI show the score only once at least one block has recurred across three or more sessions (a recurring_context sink exists). Before that, learn prints the session count and top findings without a score. --json always includes it.

Sink classes

Sinks are ranked: forward-rate sinks first by tokens per day, then historical sinks by observed tokens. Classes (the report’s label in brackets):

reducible (“safe fix”): a mechanical fix exists and must pass the net-token-negative gate.
recurring_context (“repeated text”): content re-established across sessions; fix is a cavemem offload.
behavioral (“habit”): a measured habit. Numbers are stated; suggestions are soft.
load_bearing (“needed”): measured config you need. Listed so the score stays honest; never edited.

Fixing

caveman learn implement (or the TUI’s “Implement with agent”) opens Claude Code or Codex with the caveman-learn skill, installing it if missing. The skill:

Reads caveman learn report --json and shows the score, components and ranked sinks.
For each sink you choose, proposes one concrete diff with before and after tokens per turn, and asks yes or no.
Applies only approved edits, with the agent’s own file tools.
Re-measures. Net-token-negative gate: if tokens per turn did not drop, the edit is reverted.
For a cavemem offload, also checks that caveman mem recall returns the content and a pointer is in place; otherwise it reverts (caveman mem forget <id> and restores the source).
After an edit passes its gate, runs caveman learn applied <sink_id>.

load_bearing sinks are never edited. procedure_repeat candidates skip the gate and go to an experiment instead, because a skill adds prefix tokens every session and pays back only on some. You can install the skill directly: caveman tools skills install caveman-learn (add --user for all repos, --agent codex for Codex).

Applied ledger and verdicts

Each later scan compares sessions after applied_at with the recorded before-value and reports a row in confirmed:

Fix	After-value	Verdict rule (needs 3+ sessions after)	Method
`claude_md_weight`, `claude_md_sections`, `dead_load`, `config_tax`	Current token count of the same file, sections or skills	lower = `improved`, higher = `regressed`, equal = `unchanged`	`deterministic_remeasure`
`cavemem_offload`	Whether the block’s fingerprint recurs after the fix	absent = `improved`, present = `unchanged`	`interrupted_time_series`
`dumbzone_advice`	Share of turns over 50% of the window	5+ points lower = `improved`, 5+ higher = `regressed`	`interrupted_time_series`
Anything else	none	always `insufficient_data`	`unattributed`

Provenance compares the file’s SHA-256 with the fingerprint taken at applied time: intact, changed_since (someone edited past the fix), target_missing, or not_fingerprinted. changed_since and target_missing force confidence to low.

Savings

caveman learn savings lists every recorded fix, including regressions, grouped by measurement method (rung), strongest first:

Rung	Method	Confidence
4	`deterministic_remeasure`: the edited file was re-counted	high
3	`controlled_holdout`: change on vs off on this machine	high
2	`counterfactual_replay`: scanned history re-run with the change	medium
1	`interrupted_time_series`: before vs after sessions, no control	low

Rungs are never summed together, and there is no blended headline. Each row carries standing confounders for its method. Only positive savings are priced, as a per-day rate over the scanned window at your measured effective input rate, when the plan has a spend block. total_saved_usd_by_rung sums within one rung only. Nothing is projected to a month or labeled verified.

Experiments

A holdout compares arms over your own sessions:

experiment start <label> [--sink <id>] [--fix-kind <kind>] [--note <t>] opens the on arm.
experiment arm <label> off (later on again) switches arms. Switching to the running arm is a no-op.
experiment report <label> assigns each session to the arm active at its start and compares median tokens per session.
experiment stop <label> closes it.

Verdict	Condition
`insufficient_data`	Either arm has fewer than 5 sessions (method drops to `unattributed`)
`improved`	on-arm median 10% below off-arm
`regressed`	on-arm median 10% above off-arm
`unchanged`	otherwise

When the on-arm has more than 1.25x the off-arm’s tool errors per turn, the report says so. Arms run at different times on different work, so the result is still inferred.

Reverting

Every edit is made by the skill and reported exactly, so it can be undone with your own tools or version control. An offload reverts with caveman mem forget <id> plus restoring the trimmed text. A regressed verdict is shown with no dollar figure; the skill offers the revert path.

Autopilot

Autopilot rescans in the background and tells you about new findings without you running learn. It keeps its own report under runtime/learn-autopilot/reports/: its scans run with whatever working directory and environment the last session-end hook had, so they never overwrite the canonical report, dated snapshots or trend history your own caveman learn writes.

Aspect	Behavior
Trigger	The Caveman native hook’s `SessionEnd` event (Claude Code, Codex, Gemini CLI; opencode and Hermes bridges forward it). The hook spawns a detached, idle-priority `caveman learn autopilot run` and returns immediately.
Throttle	At most one scan per 6 hours. `CAVEMAN_LEARN_AUTOPILOT_HOURS` changes it. One scan at a time (lock file; stale after timeout + 60 s).
Scan	`learn scan --write-report --no-remember --reports-home $CAVEMAN_HOME/runtime/learn-autopilot` with the default window and sources, killed at `CAVE_LEARN_TIMEOUT`.
Proxy check	Before scanning, the detached child runs `caveman-proxy learn capabilities` and needs `no_remember`, `reports_home` and `memory_health`. An older proxy would ignore those flags and write cavemem and the canonical report, so it never scans: `last_error` becomes “proxy too old for autopilot (needs learn capabilities)” and `learn autopilot status` shows it. The result is cached per proxy path, mtime and size, so an upgrade re-probes. The `SessionEnd` hook itself never probes.
Writes	Its own report files, the `caveman.db` plan tables every scan updates, and its own `runtime/learn-autopilot*` state. Never cavemem memories or `learnings` rows: `--no-remember` skips them, because sink titles carry changing counts and each unattended run would add a near-duplicate.
Enable precedence	`CAVEMAN_LEARN_AUTOPILOT` env (`0`, `false`, `off`, `no` disable; anything else enables) then `learnAutopilot` in `~/.caveman-cloud/config.json` then off when `CI` is set (not `0`/`false`) or under `NODE_TEST_CONTEXT` then on.
Opt out	`caveman learn autopilot off`, or `CAVEMAN_LEARN_AUTOPILOT=0`.
State	`$CAVEMAN_HOME/runtime/learn-autopilot*.json`. Writes are temp-file + rename and refuse symlinked parents.
Status	`caveman learn autopilot status`: on/off and why, last scan, next scan, last error, last line shown, and a line waiting to show.

Session-start nudge

After a background scan, autopilot may show one line at the next session start, for example:

caveman learn: new finding — <title> (~2.4k tokens in every message, estimate). Run `caveman learn` to review.

Rules:

Sinks of class reducible or recurring_context with tokens_per_turn >= 2000 qualify. This includes memory_health:duplicate_rules and memory_health:memory_orphans when they are reducible and above the threshold.
memory_health:broken_imports and memory_health:memory_truncation qualify with no token threshold: they break what the agent loads. When they are all that is new they lead the line (caveman learn: memory files (<repo>) — <title>. Run \caveman learn —all` to review.`); otherwise they are appended as ”, plus N memory-file findings”.
Only sinks not seen by an earlier autopilot scan. The first scan records a baseline and announces nothing.
While a nudge is still unclaimed, newer sinks stay unseen and are announced after it.
Shown only on Claude Code, Codex and Gemini CLI SessionStart with source startup or clear; never on resume, compact or fork. It is delivered as a user-visible systemMessage, not model context.
Claimed with one atomic rename, so concurrent sessions announce it once. The claim parks it in runtime/learn-autopilot-nudge.inflight.json with the claimer’s token; it counts as announced only when that claimer confirms, after the line was written to the host (the fast hook owns the token and confirms after relaying the delegated output). At most one nudge is in flight: a newer nudge waits behind an unconfirmed one. An unconfirmed nudge, for example after the delegate hit its 3 s timeout, is shown once more after 10 minutes, never twice.
Suppressed when autopilot is disabled.

Trends

The trends block compares your own sessions week over week.

Aspect	Rule
Buckets	UTC ISO weeks (`2026-W39`), up to 8, inside the window. Sessions are bucketed by start time; undated sessions are counted in `undated_sessions` and left out.
Compared week	The last complete week (`current_week`). The week in progress is shown (`in_progress: true`) but never drives the comparison. No trends block when the window holds no complete week.
Prior	Sessions of up to 4 weeks before the compared week, pooled.
Minimum	A bucket with fewer than 5 sessions has a null value and `insufficient_data`.
Direction	`improved`, `worse`, `flat` or `insufficient_data`. Flat when the relative change is under 10%, or, for percent and per-100 metrics, the absolute change is at most 1 point.
Partial weeks	`partial: true` marks a week cut by the window start or still in progress.

Metrics:

Key	Statistic	Better
`tokens_per_session`	median	lower
`peak_context_pct`	median	lower
`dumbzone_turn_pct`	pooled share of turns	lower
`first_turn_tokens`	median	lower
`cache_read_pct`	pooled share of context	higher
`tool_errors_per_100_turns`	pooled rate	lower
`score_session_penalty`	recomputed dumbzone + subagent penalty	lower

Only the session-derived score components are recomputed per week; config_tax and dead_load depend on today’s config and are not replayed. Full score history comes from the dated report snapshots (score.history). movers lists up to three sinks that grew and three that shrank by at least 10 tokens/turn versus the newest snapshot at least 7 days old (else the newest one). A trend is not a saving and does not show what caused the change.

Output files and JSON

caveman learn --json and caveman learn report --json print one document with schema: "caveman.learn.v1" and basis: "inferred". The same plan, plus moves, generated_at and generation, is saved as reports/caveman-learn.json. The Go types are in proxy/internal/store/learn_types.go.

Block	Present	Content
`window`	always	`from`, `to`, `since`
`cave_score`	always	`score`, `scope: local_setup`, four `components`
`sessions_scanned`, `sessions_by_source`	always	counts
`sinks`	always	ranked sinks: `sink_id`, `class`, `basis`, `framing` (`forward_rate` or `historical`), token fields, `evidence`, `suggestion`
`caveats`	always	plain-language limits for this run
`spend`	usage for a cataloged model exists	window cost at dated catalog rates, per-component and per-model rows, `unpriced` models, effective input rate and multiplier
`confirmed`	a fix was recorded	ledger rows with verdict and attribution
`trends`	a complete week with dated sessions exists	see Trends
`context_depth`	usage-bearing sessions exist	peak-depth histogram in 10% buckets
`wrap_measured`	the proxy recorded savings in the window	tokens only, proxy-counted
`retro`	`--retro` and something to report	replay totals by family
`portfolio`	sinks share a fix family	grouping of existing sinks, no new estimate
`repos`	2+ repositories with 2+ sessions each	per-repository summary

Other subcommands print their own schemas: caveman.learn.savings.v1, caveman.learn.simulate.v1, caveman.learn.experiment.v1, caveman.learn.digest.v1, caveman.learn.reconcile.v1. spend is what the scanned window cost at published rates. It is a floor when unpriced is non-empty, it is not an invoice, and on a subscription plan the marginal cost is zero. The compact plain view folds several unpriced models into one line; --all, --md, JSON and the HTML report list each model.

Limits and known gaps

Headless sessions can consume the nudge. A scripted claude -p or codex exec fires SessionStart with source startup, so it can claim and confirm the nudge where no one sees it.
Unsupported agents. Only the five sources above are scanned. Cursor, Windsurf, Hermes, Pi and other agents are not read.
aider is off unless CAVEMAN_AIDER_ROOT points at a directory to walk. aider history has no provider usage, so it adds nothing to depth, dumbzone, spend or trends.
Context windows. A window comes from the shared model catalog. For un-cataloged models learn assumes 1M if the model id contains 1m, 400k for OpenAI, 1,048,576 for Gemini, and 200k otherwise. If a session’s context grows past the assumed window, that whole session is re-measured against 1M (then 2M). A 1M-window session that never exceeds 200k is still measured against 200k. Assumed windows add a caveat and are excluded from the excess token floor and from cross_provider:depth.
Turn-1 prefix includes the first user prompt, so the measured prefix is an upper bound.
Plugins are counted but their tokens are not measured.
claude_md_sections sees only visible transcript text. Claude Code injects CLAUDE.md into the system prompt, so the finding is emitted only when at least one section echoes (a positive control).
--repo drops Gemini sessions (their transcripts have no working directory) and omits retro.
Auto memory location ignores a custom autoMemoryDirectory setting.
Time budget. With --retro, a scan that hits its budget covers partial history; repository, section-echo, trend and applied-fix blocks are omitted rather than zero-filled.
Ledger coverage. memory_health fixes can be recorded with applied but have no after-value measurement yet, so they stay insufficient_data. simulate does not accept memory_health sinks.
Savings rungs. Confirmed ledger rows are currently produced only at the deterministic_remeasure and interrupted_time_series rungs. controlled_holdout results appear in experiment report, not in savings, and no command yet produces counterfactual_replay rows.

⌘I

Get Started

Skills & Agents

Proxy & Engine

Build on Caveman

Reference

Caveman Learn: Audit Your Agent's Token Spend

Quick start

Privacy: what it reads and writes

Reads

Writes

Commands and flags

Setup Score

Sink classes

Fixing

Applied ledger and verdicts

Savings

Experiments

Reverting

Autopilot

Session-start nudge

Trends

Output files and JSON

Limits and known gaps

​Quick start

​Privacy: what it reads and writes

​Reads

​Writes

​Commands and flags

​Setup Score

​Sink classes

​Fixing

​The consent loop

​Applied ledger and verdicts

​Savings

​Experiments

​Reverting

​Autopilot

​Session-start nudge

​Trends

​Output files and JSON

​Limits and known gaps

Quick start

Privacy: what it reads and writes

Reads

Writes

Commands and flags

Setup Score

Sink classes

Fixing

The consent loop

Applied ledger and verdicts

Savings

Experiments

Reverting

Autopilot

Session-start nudge

Trends

Output files and JSON

Limits and known gaps