# Caveman documentation > Documentation for the open Caveman stack: the agent skill, the compression engine, and cavemem. What each layer does, what it costs, and how every number is counted. Every page of https://docs.caveman.so in one file, in reading order. Each page is also available on its own at its canonical URL with `.md` appended, or by sending `Accept: text/markdown` to that URL. A machine index with headings, anchors and last-changed dates is at https://docs.caveman.so/llms-index.json Measurement vocabulary, which the pages below assume: - inferred: A local, per-run estimate computed on your own machine from an offline token counter. Every public Caveman tool emits this and nothing else. Not a saving, not a dollar figure, and never valid to multiply into a monthly or annual total. - measured: Traffic that was actually observed, rather than estimated from bytes. Not proof that a dollar was saved. Observing a request is not comparing it to its alternative. - verified: A saving confirmed against a real bill by a hosted rollout system that ran both arms of the comparison. No tool documented on this site can emit it. There is no configuration, flag or account tier that upgrades a local number to this. ## Contents - Overview (https://docs.caveman.so/docs): What Caveman is, which parts are open, and where to start reading. - Quickstart (https://docs.caveman.so/docs/quickstart): Install the skill and compress a real payload in about five minutes. - Architecture (https://docs.caveman.so/docs/architecture): The path a request takes through the stack, and where the open-core line falls. - How numbers are counted (https://docs.caveman.so/docs/counting): Inferred against verified, per run against per month, and the four rules the code follows. - Skill (https://docs.caveman.so/docs/skill): A set of instructions that changes how your agent spends tokens. No runtime, no proxy. - caveman learn (https://docs.caveman.so/docs/skill/learn): Profile a local agent session and rank what is actually eating the budget. - caveman explore (https://docs.caveman.so/docs/skill/explore): Read a repository without dragging every file into context. - Agent profiles (https://docs.caveman.so/docs/skill/profiles): The registry of coding agents the skill knows how to configure. - Engine (https://docs.caveman.so/docs/engine): Content-aware compression that keeps the parts an answer depends on. - Compressors (https://docs.caveman.so/docs/engine/compressors): One router per content shape: JSON, logs, code, diffs, search results, prose. - Recoverable compression (https://docs.caveman.so/docs/engine/recoverable): Why lossy stays honest: every removed byte can be fetched back. - Token counting (https://docs.caveman.so/docs/engine/tokens): The offline counter behind every ratio the engine reports. - caveman-shrink (https://docs.caveman.so/docs/engine/shrink): Shrink command output before it reaches the model. - Memory (https://docs.caveman.so/docs/memory): cavemem holds the context you would otherwise paste into every session. - Recall and offload (https://docs.caveman.so/docs/memory/offload): Moving a heavy instruction file out of the prompt and into recall. - Cloud (https://docs.caveman.so/docs/cloud): The managed plane: fleet visibility, spend attribution, and verified numbers. - CLI (https://docs.caveman.so/docs/cli): The command surface: compress, detect, learn, and agent setup. - TypeScript SDK (https://docs.caveman.so/docs/sdk/typescript): Compress payloads and read spend from Node. - Python SDK (https://docs.caveman.so/docs/sdk/python): The same surface for Python agents. - Agent SDK (https://docs.caveman.so/docs/agent-sdk): Build an agent that is efficient by construction. - MCP server (https://docs.caveman.so/docs/mcp): Expose compression and recall as tools any MCP client can call. - caveman-browse (https://docs.caveman.so/docs/browse): Read web pages as compressed accessibility trees instead of raw HTML. - Licensing (https://docs.caveman.so/docs/licensing): Which surfaces are MIT, which are BSL 1.1, and what that means for you. - Telemetry (https://docs.caveman.so/docs/telemetry): What the CLI sends, what it never sends, and the three ways to turn it off. - Eval graders (https://docs.caveman.so/docs/evals): The grader set used to check that compression did not change an answer. - Provider catalog (https://docs.caveman.so/docs/provider-catalog): The model price table every cost figure is read from. === --- title: Overview summary: What Caveman is, which parts are open, and where to start reading. canonical: https://docs.caveman.so/docs updated: 2026-08-26T03:57:26+02:00 basis: inferred --- # Overview > What Caveman is, which parts are open, and where to start reading. Caveman is an efficiency stack for AI agents. It sits between an agent and the model API it calls, and it removes cost at four different points. Each point is a separate piece of software with its own licence, and you can adopt any one of them without the others. [Diagram: the four layers of the stack, bottom to top: Skill, Engine, Memory, Cloud.] The layer at the bottom is a set of instructions. It has no runtime, no daemon, and no account. The layer at the top runs on our machines and is the only paid part. This site documents everything below the network boundary in full. For the managed plane it documents the shape, not the internals. ## What each layer does ### 00 Skill A skill is a persistent instruction set your coding agent reads before it answers. It cuts filler, narration and hedging out of the agent's own output, and it is explicit about what it must never touch: code blocks, function names, CLI commands and exact error strings stay byte for byte. It costs nothing to run because it is not software. It is text your agent already reads. ### 01 Engine The engine compresses payloads on their way to a model. It detects what a payload is, routes it to a compressor built for that shape, and stores the original bytes under a content-addressed handle before it removes anything. If the agent needs what was dropped, it asks for the handle and gets the original back exactly. ### 02 Memory cavemem holds the context you would otherwise paste into every session. It stores text once, recalls the parts a question actually needs, and compresses each hit through the engine on the way out. ### 03 Cloud Caveman Cloud is the managed plane. It is where a fleet's numbers stop being local estimates and become something a finance team can read. It is in private development and the waitlist is open. ## What the numbers mean Every local tool on this site reports `inferred`. That word is load bearing. It means a per-run estimate computed on your machine from an offline token counter, and it is never turned into a monthly figure or a dollar amount. No tool in this repository can emit `verified`. That label is reserved for hosted rollout systems that can compare against a real bill. If a number here has no basis label next to it, treat it as an example rather than a promise. [How numbers are counted](/docs/counting) explains the four rules the code follows. ## Where to go next Install the skill, then compress something real and read the report. The path one request takes, and where the open-core line falls. One command, thirty agents, no runtime. Content-aware compression that keeps what an answer depends on. --- --- title: Quickstart summary: Install the skill and compress a real payload in about five minutes. canonical: https://docs.caveman.so/docs/quickstart updated: 2026-08-26T03:57:26+02:00 basis: inferred --- # Quickstart > Install the skill and compress a real payload in about five minutes. There are two things worth doing first, and they are independent. The skill changes how your agent writes and takes about thirty seconds. The CLI installs the local tools and gives you a report on where your tokens actually go. ## Install the skill
This works for most agents: ```bash npx skills add JuliusBrussee/caveman ``` Without `-g` this writes into `./.agents/skills` under the directory you are standing in, not into a global config. That is usually what you want for a project, and surprising if you expected otherwise.
Some agents have a native path that wires up more than the skill file: ```bash # Claude Code claude plugin marketplace add JuliusBrussee/caveman claude plugin install caveman@caveman # Gemini CLI gemini extensions install https://github.com/JuliusBrussee/caveman # Codex CLI npx skills add JuliusBrussee/caveman -a codex # Cursor, which needs the global flag npx skills add JuliusBrussee/caveman -a cursor -g ```
Check it took. In your agent, run: ```text /caveman ``` You should get a confirmation that the mode is active. `/caveman off` turns it back off, and so does saying "normal mode".
The skill drops articles, filler and narration from your agent's replies. It is explicit about what stays byte for byte: code blocks, function and API names, CLI commands, and exact error strings. It also steps aside on its own for security warnings, irreversible actions, and any moment where being terse would make an instruction ambiguous. ## Install the local tools The CLI is published on npm under a scoped name. ```bash npm i -g @caveman-ai/cli caveman setup --install ``` The bare `caveman` package on npm is an unrelated JavaScript templating library. Install `@caveman-ai/cli`. On PyPI the same applies: our SDK is `caveman-sdk`, imported as `caveman_cloud`. `caveman setup --install` downloads the companion binaries into `~/.caveman/bin`. It checks a key-signed manifest first, then verifies each artefact against its own SHA-256 before installing it. Command names, version, platform, duration, exit class, and aggregate token counts from local sessions. Never prompts, code, file paths, arguments, model names or dollar figures. It is how we see which commands people use and which ones break, and it is a large part of why the local tools can stay free. Turn it off with `caveman telemetry off`, `CAVEMAN_TELEMETRY=0`, or `DO_NOT_TRACK=1`. The full payload is listed on [Telemetry](/docs/telemetry). Then start your agent through the CLI so it picks up the local tools: ```bash caveman claude ``` The same works for `codex`, `gemini`, `opencode`, `aider`, `hermes` and `openclaw`. Seven agent profiles ship today, and each is one JSON file in the registry. ## Find out where your tokens go ```bash caveman learn ``` This reads your local agent sessions and ranks what is actually consuming the budget. It measures and does nothing else. Applying a fix is a separate, consent-gated step: ```bash caveman learn report --json caveman learn apply --dry-run caveman learn apply ``` Every applied fix is checked for being net token negative. If an edit does not make the thing smaller, it is reverted. ## What you should expect to see On the committed benchmark suite, the skill cuts output tokens by about 65 percent on average, with a range from 22 to 87 percent across ten prompts. That is a per-run inferred estimate measured on that suite, not a promise about your bill. Nothing you run locally will ever report a dollar figure or a monthly total. That is deliberate and [the reasoning is here](/docs/counting). --- --- title: Architecture summary: "The path a request takes through the stack, and where the open-core line falls." canonical: https://docs.caveman.so/docs/architecture updated: 2026-08-26T03:57:26+02:00 basis: inferred --- # Architecture > The path a request takes through the stack, and where the open-core line falls. Caveman is not one program. It is four pieces that compose, and most people only ever install one of them. This page shows how a single request moves through them and why the pieces are separable. ## One request [Diagram: one request, from your agent through the skill and the engine to the provider, with a recovery path back.] The agent decides what to say. The skill has already shaped how it says it. The engine reduces whatever payload is about to leave. The provider bills for what arrives. The dashed line is the part that makes the rest defensible. Compression here is not a one-way door. Before the engine emits a smaller payload it writes the original bytes to a local store keyed by a hash of those bytes, and the compressed output carries that handle. An agent that needs the part that was removed asks for the handle and gets the original back, byte for byte, in the same turn. The rule the engine follows has three possible outcomes and no fourth: the payload is compressed and recoverable, or it passes through untouched, or it is recovered. If the recovery store is unavailable, the engine does not compress. It passes the original through. ## Why the pieces are separable Each layer answers a different question, and they fail independently. - Skill: Changes what the agent writes. No process, no config, no network. - Engine: Changes what a payload weighs. A binary you run, or a library you link. - Memory: Changes what you have to say twice. A local SQLite store. - Cloud: Changes what you can prove to someone else. An account, across a network. Nothing above a layer is required by the layer below it. The skill does not know the engine exists. The engine does not need an account. This is why the licence split follows the same lines rather than cutting across them. ## Where the open-core line falls [Diagram: the licence split across MIT, BSL 1.1 and Commercial surfaces.] The rule that governs new code is one sentence: a module that imports, links, embeds or ships as part of the engine-linked runtime is BSL 1.1 unless a decision record explicitly classifies it as an adoption surface. The practical effect for you is in the additional use grant. BSL 1.1 here permits internal evaluation, local development, CI, integration, and self-hosted use for your own first-party traffic, production included. What it does not permit is offering Caveman's functionality to third parties as a hosted or embedded service. That boundary is the commercial line, and it is the only one. Each BSL release converts to Apache 2.0 on the earlier of 21 June 2030 or four years after that version was first distributed. ## Content detection The engine does not ask you what a payload is. It reads the bytes and decides, then routes to a compressor built for that shape. The types it recognises: `json` · `log` · `code` · `diff` · `search-result` · `text` · `toon` · `html` · `a11y` · `terminal` · `tabular` · `config` Two compressors are never chosen automatically and have to be asked for by name: the tool-schema compressor and the TOON encoder. Automatic selection of either would surprise a caller who did not opt in. When detection is not confident, it answers `text`, which is the compressor that removes the least. Every unknown case in this system resolves toward doing less, not more. ## Failure behaviour This is worth stating on its own, because it is the part that decides whether you can leave the thing switched on. ```text unknown mode -> record, which never transforms transform error -> forward the original bytes result not smaller -> keep the original low-confidence type -> text unknown grader -> passed: false unknown model price -> zero, tagged unpriced ``` Every one of those resolves to the conservative answer. A compression system that is willing to guess is a compression system you have to babysit. --- --- title: How numbers are counted summary: Inferred against verified, per run against per month, and the four rules the code follows. canonical: https://docs.caveman.so/docs/counting updated: 2026-08-26T03:57:26+02:00 basis: inferred --- # How numbers are counted > Inferred against verified, per run against per month, and the four rules the code follows. Every number Caveman shows you carries a word that says how it was obtained. The words are not interchangeable and the code does not let them blur. ## Three words, kept apart - inferred: A local estimate from bytes, token counters and local records. Every public offline tool emits this. - measured: Observed traffic. Real, but not proof that a dollar was saved. - verified: Reserved for hosted rollout systems that can compare against a bill. No tool on this site emits it. If you install anything documented here and it shows you a percentage, that percentage is `inferred`. There is no configuration that upgrades it. ## The four rules ### 1. No fake savings Headline figures in these docs are local examples or target ranges. They stay labelled `inferred`, and they are never multiplied out into a monthly saving. Where the code only supports a range, the docs show the range rather than picking a number from inside it. ### 2. Byte safe `record` mode is pass-through. On a parse problem, an unsupported input, a missing recovery store, or an output that is not actually smaller, the engine keeps the original bytes. ```text record mode -> never transforms transform error -> forward the original bytes result not smaller -> keep the original ``` ### 3. No placeholders, fail closed No stub responses on shippable paths, and unknown cases resolve toward the conservative answer. ```text unknown engine mode -> record, which is pass-through low-confidence content -> text unknown grader -> passed: false unknown route -> 404 unknown model price -> zero, plus an "unpriced:" tag ``` An unpriced model contributes zero to a cost figure and says so. It does not get a guessed price. ### 4. Recoverable, so lossy stays honest Compressors drop bytes from the model-visible payload. A lossy result is emitted only after the original has been stored under a content-addressed handle, and `retrieve(handle)` returns the original byte for byte. If a tool lies about savings, every report and every rollout decision downstream inherits that lie. Public Caveman tools report what they can support locally, and stop there. ## Per run, never per month An inferred ratio is a property of one payload on one run. Projecting it forward assumes your next month looks like that payload, which nobody can know. So the engine reports a ratio and does not offer to annualise it. There is no setting for this. ## Tokens are not money On the local path, savings are counted in tokens and have no dollar field anywhere in the data structure. There is no token-to-dollar multiplication at any layer. This matters because token prices differ per model, per provider, per cache state and per contract. A local tool that has not seen your invoice cannot convert one into the other, so it does not try. ## What the counter actually is The default token counter is a real BPE tokenizer using the `o200k_base` encoding, with the vocabulary compiled into the binary. It is deterministic, so the same bytes always produce the same count, and it is offline, so it works air-gapped and never sends your payload anywhere. If the embedded codec fails to load, it degrades to a characters-divided-by-four approximation. That approximation errs low. It never guesses high. Provider-reported usage from your actual API response stays authoritative for spend. The local counter is an estimate of that, and the two are kept in separate fields. ## Numbers you will see quoted Both of these are real measurements with real limits, and the limits travel with the number. **The skill, output tokens.** About 65 percent fewer on average across ten prompts, ranging from 22 to 87 percent. Measured on the committed benchmark suite. Output tokens only. A per-run inferred estimate, not a bill-savings promise. **The wrap path, input tokens.** 33.2 percent fewer provider-reported input tokens across 18 paired runs of six cases, with a case-clustered 95 percent interval of 14.6 to 48.5 percent. All 18 exact-answer checks passed. One of the six cases regressed by 9.9 percent and stays in the aggregate, because dropping a negative case would make the average a fiction. That second result is controlled benchmark evidence on a pinned agent build. It is not production traffic, not a provider invoice, and not `verified`. --- --- title: "Skill: Overview" summary: A set of instructions that changes how your agent spends tokens. No runtime, no proxy. canonical: https://docs.caveman.so/docs/skill layer: skill license: MIT capability: skill updated: 2026-08-26T03:57:26+02:00 basis: inferred --- # Skill: Overview > A set of instructions that changes how your agent spends tokens. No runtime, no proxy. The skill is the cheapest layer in the stack because it is not software. It is a file your agent reads, and it changes how the agent writes for the rest of the session. - What it is: One Markdown instruction file, plus optional commands and subagent presets. - Runtime: None. No process, no port, no config file. - Account: Not required. - Licence: MIT. - Install: `npx skills add JuliusBrussee/caveman` ## Install ```bash npx skills add JuliusBrussee/caveman ``` Without `-g` this writes into `./.agents/skills` under your current directory rather than a global location. Several agents have their own native path: ```bash # Claude Code claude plugin marketplace add JuliusBrussee/caveman claude plugin install caveman@caveman # Gemini CLI gemini extensions install https://github.com/JuliusBrussee/caveman # Codex CLI npx skills add JuliusBrussee/caveman -a codex # Cursor, which needs -g npx skills add JuliusBrussee/caveman -a cursor -g # opencode npx -y github:JuliusBrussee/caveman -- --only opencode ``` There is also a full installer that wires up hooks and a status line for Claude Code. It is pinned to a tag, it is safe to re-run, and it supports `--dry-run`. ```bash curl -fsSL https://raw.githubusercontent.com/JuliusBrussee/caveman/v2.3.1/install.sh | bash ``` ```powershell irm https://raw.githubusercontent.com/JuliusBrussee/caveman/v2.3.1/install.ps1 | iex ``` ## What it removes Articles, filler, pleasantries, hedging and tool-call narration. Sentence fragments are allowed. Decorative tables and emoji are not. ## What it will not touch This list is the reason the skill is safe to leave on. - Code blocks, verbatim - Function, method and API names - CLI commands and flags - Commit type keywords - Exact error strings - The language you are writing in It also refuses to invent abbreviations. `cfg`, `impl`, `req`, `res` and `fn` are banned, and so are arrow characters standing in for words. The reason is measurable rather than stylistic: the tokenizer splits those the same way it splits the full word, so the abbreviation saves nothing and costs clarity. ## When it steps aside The skill drops out of terse mode on its own for: - Security warnings - Confirmations of anything irreversible - Ambiguous multi-step sequences - Any case where compressing the answer would create the ambiguity ## Levels ```text /caveman lite /caveman full # default /caveman ultra /caveman wenyan-lite /caveman wenyan-full /caveman wenyan-ultra /caveman off ``` `off`, "stop caveman" and "normal mode" all do the same thing. ## What else comes with it One install also brings a set of commands and work-pattern skills. - Commands: `/caveman-commit · /caveman-review · /caveman-compress · /caveman-stats · /caveman-help` - Subagent presets: cavecrew-investigator, cavecrew-builder, cavecrew-reviewer - Work patterns: investigate-first, lean-build, surgical-patch, safe-refactor, migration, verify-and-stop ## What it saves On the committed benchmark suite of ten prompts, output tokens drop by about 65 percent on average, with a range of 22 to 87 percent. That is a per-run inferred estimate measured on that suite. It is not a promise about your bill, and it covers output tokens only. The range matters more than the average here. A tightly scoped refactor prompt saved 22 percent. A long explanatory answer saved 87 percent. Which end you land on depends on how much of your traffic is prose. --- --- title: caveman learn summary: Profile a local agent session and rank what is actually eating the budget. canonical: https://docs.caveman.so/docs/skill/learn layer: skill license: MIT capability: skill-caveman-learn updated: 2026-08-26T04:05:35+02:00 basis: inferred --- # caveman learn > Profile a local agent session and rank what is actually eating the budget. `caveman learn` reads local agent sessions and shows where tokens are being spent. The command measures and prepares candidates. The `caveman-learn` skill is the separate, consent-gated part that can edit a file. - Input: Local Claude Code, Codex, and Caveman session records. - Default window: The last 30 days. - Report basis: `inferred` - Edits: One approved change at a time. - Protected class: `load_bearing` ## Run the profiler ```bash caveman learn ``` In an interactive terminal this shows a local Setup Score, grouped top moves, and a menu for the full report or the editing flow. Stable output modes are available for scripts and pipes: ```bash caveman learn --plain caveman learn --all caveman learn --json caveman learn --md caveman learn --since 7d --sources codex,claude,caveman ``` `--plain` has no animation or keyboard menu. `--all` includes every finding, its internal id, its basis, and its suggestion. JSON uses the `caveman.learn.v1` schema. If no matching sessions exist, the command returns no score. Repeated-context findings need the same block in three or more sessions, so a new installation may need several sessions before that class appears. ## Read the sink classes | Class | Meaning | Automatic edit | | --- | --- | --- | | `reducible` | A heavy instruction file or an installed skill that local evidence says can be smaller. | Candidate only. The skill still asks first. | | `recurring_context` | A block re-established across sessions that may fit local memory better. | Candidate only. Recall must work before source text is removed. | | `behavioral` | An observation about how the agent or user works. | None. Repetition does not prove the behaviour is wrong. | | `load_bearing` | Context that must remain available on every turn. | Never. It stays in the score so the score remains honest. | Rates such as `tokens_per_turn` and `tokens_per_day_rate` describe the observed local pattern. The daily field is a rate, not a historical total and not a monthly projection. ## Review fixes with an agent ```bash caveman learn implement caveman learn implement claude caveman learn implement codex --prompt "focus on project instructions" ``` This installs the safety guide when it is missing, opens Claude Code or Codex, and asks the agent to read the current JSON report. It does not grant permission to apply every finding. You can also install the guide directly: ```bash caveman tools skills install caveman-learn --agent claude caveman tools skills install caveman-learn --agent codex ``` ## Preview one candidate ```bash caveman learn apply --dry-run caveman learn apply ``` Both commands prepare information for review. The second writes a candidate under `~/.caveman/candidates`; it does not edit your repository. Only the installed skill performs an edit, after showing the proposed change and asking for consent. For a reducible item, the skill compares inferred tokens per turn before and after. If the result is not smaller, it restores the original. For recurring context, the skill re-reads the source block, verifies its SHA-256 locator, stores the raw block in cavemem, and confirms that recall returns it. The source is trimmed only after the pointer and recall path both work. A failed recall removes the new memory and leaves the source in place. `caveman learn apply` materializes a candidate. It does not edit a file, move a block, or prove a reduction. The proof happens after an approved edit is measured again. ## What it will not do - Apply all fixes behind one confirmation - Edit a `load_bearing` finding - Treat a behavioral finding as an instruction - Remove recurring context without a working pointer and recall path - Keep an edit whose measured result is not net token negative - Attach currency, monthly savings, or `verified` to a local result Reports are also written under `~/.caveman/reports`, including HTML and JSON forms. They contain local analysis, so review them before sharing them outside your machine. --- --- title: caveman explore summary: Read a repository without dragging every file into context. canonical: https://docs.caveman.so/docs/skill/explore layer: skill license: MIT capability: skill-caveman-explore updated: 2026-08-26T04:05:35+02:00 basis: inferred --- # caveman explore > Read a repository without dragging every file into context. `caveman-explore` is a read-only repository scout for Claude Code. It answers one localization question with small `path:line` citations, so the main agent can inspect the right code without carrying every exploratory read in its conversation. - Host: Claude Code only. - Tools: `Read, Glob, Grep` - Model: `haiku` - Output: `path/to/file.ext:START-END reason` - Licence: MIT. ## Install ```bash caveman explore install --agent claude ``` The command is a compatibility alias for installing the `caveman-explore` skill. It writes the canonical `SKILL.md` unchanged and disables pixel conversion for this file. You can choose a destination explicitly: ```bash caveman explore install --agent claude --dir ./path/to/caveman-explore ``` `caveman explore install --agent codex` exits non-zero and writes no skill. Transcript isolation for this pattern has not been verified on Codex, so the installer does not claim support. ## When to use it Use the explorer when you need to locate code before you can solve the task: - A cold start in an unfamiliar repository - A question that spans several files or packages - A direct search that did not reveal the owning code - A request that names behaviour but no file or symbol Skip it when the task already names the exact file or symbol, or when a previous turn already returned usable line citations. The explorer is a localization step, not a second opinion on a known location. ## What it returns The response contains citations only: ```text src/router/pick.go:42-71 route selection lives here src/router/pick_test.go:18-40 table test covering selection ``` Every range must have been read. The range may not extend beyond the file. If no relevant code exists, the only valid answer is: ```text no relevant locations found ``` This narrow output contract matters. The solving agent gets evidence it can open directly, without inheriting a summary, a proposed fix, or a large search transcript. ## How it searches The explorer starts with several complementary reads in parallel: likely path patterns, symbol or string matches, and the most promising file contents. It follows the evidence for one or two short rounds, then stops as soon as it can name the locations. The fixed tool list enforces the boundary. It cannot run a command, edit a file, write a patch, or test a proposed solution. ## What it cannot prove A citation proves that code exists at a location the explorer read. It does not prove that the code is correct, that a test passes, or that the cited path is the only implementation. The solving agent still owns diagnosis, changes, and verification. --- --- title: Agent profiles summary: The registry of coding agents the skill knows how to configure. canonical: https://docs.caveman.so/docs/skill/profiles layer: skill license: MIT capability: agent-profiles updated: 2026-08-26T04:05:35+02:00 basis: inferred --- # Agent profiles > The registry of coding agents the skill knows how to configure. Agent profiles are the registry entries that tell the CLI how a coding agent is installed, launched, and given local Caveman tools. Seven profiles ship in the registry. The JSON files are compiled into the CLI, so source and distributed behaviour stay tied together. - Schema: `schema_version: "1"` - Profiles: Aider, Claude Code, Codex CLI, Gemini CLI, Hermes, OpenClaw, and opencode. - Fallback: `generic-env` - Validation: Unknown hook or injection methods fail compilation. - Licence: MIT. ## Registered agents | Profile id | Wire protocol | Configuration shape | Command-output hook | | --- | --- | --- | --- | | `aider` | OpenAI Chat | Environment | Manual | | `claude` | Anthropic Messages | Environment | Claude Code hook | | `codex` | OpenAI Responses | Host-specific code path | Codex hook | | `gemini` | Gemini generateContent | Environment | Gemini hook | | `hermes` | OpenAI Chat | Environment plus builder | Hermes plugin | | `openclaw` | OpenAI Chat | Config file overlay | OpenClaw plugin | | `opencode` | OpenAI Chat | Config content overlay | opencode plugin | This table describes declared integration shape. It does not say that every host version or provider credential has been tested. Each profile carries a tested version, and profiles that need code beyond their JSON declaration say so with `builder-assisted` or `code-only`. ## What one profile contains Every profile has an id, display name, binary names, install hint, wire protocol, injection description, attribution header, tested agent version, completeness label, and fallback. Optional fields describe command hooks, memory hooks, on-disk skill directories, and config overlays. The completeness labels have exact meanings: | Label | Meaning | | --- | --- | | `declarative` | Profile data alone describes routing setup. | | `builder-assisted` | Profile data is the base, and CLI code adds host-specific setup. | | `code-only` | Declared injection is inert; host setup lives in reviewed CLI code. | The registry compiler cross-checks these labels against the real builders. A profile cannot call itself declarative when it depends on code. ## Inspect what is installed ```bash caveman status caveman doctor claude caveman doctor codex caveman setup ``` `status` shows local state. `doctor ` checks the selected host integration. `setup` reports companion binary availability and repair commands. Launch shortcuts use profile ids: ```bash caveman claude caveman codex caveman gemini caveman aider caveman hermes caveman openclaw caveman opencode ``` Each shortcut is equivalent to `caveman wrap `. ## Hooks are capabilities, not assumptions Command hooks rewrite noisy shell output through `caveman shrink` before the model reads it. A profile without a verified hard hook surface stays manual and receives command guidance instead. Memory hooks are stricter. They are off by default, and a profile may declare one only when the host exposes a verified live user-prompt hook. Today only Claude Code declares that capability. Skill directories are also explicit. Claude Code declares user and project skill roots. Codex declares its user skill root. A profile with no verified convention omits the field, and conversion skips it. ## Failure behaviour Registry compilation rejects duplicate ids, unknown methods, false completeness labels, invalid tested-version metadata, and schema drift. Runtime detection falls back to `generic-env` when a declared setup path cannot be used. A profile means the CLI knows the host's integration shape. Live support still depends on installed host version, credential path, protocol, and available recovery surface. Use `doctor` on the machine that will run it. ## Adding a profile New profiles begin as JSON that passes the shared schema and compiler. A profile also needs runtime tests for its launch path, configuration injection, fallback, and any hook it claims. Adding a name to the registry without those paths does not make an agent supported. --- --- title: "Engine: Overview" summary: "Content-aware compression that keeps the parts an answer depends on." canonical: https://docs.caveman.so/docs/engine layer: engine license: BSL-1.1 capability: engine updated: 2026-08-26T04:05:35+02:00 basis: inferred --- # Engine: Overview > Content-aware compression that keeps the parts an answer depends on. The engine reduces a payload before it reaches a model. It is written in Go, it links into other tools as a library, and it also runs as a standalone binary that reads stdin and writes stdout. - Licence: BSL 1.1, converting to Apache 2.0 after four years. - Distribution: Source only. There is no npm, pip, Docker or Homebrew package. - Binary: `caveman-engine` - Input limit: 64 MiB on stdin, refused with cave_input_too_large. - Reports: Always `inferred`. It cannot emit `verified`. ## Build it ```bash git clone https://github.com/JuliusBrussee/caveman go build -o ./bin/caveman-engine ./public/engine/cmd/caveman-engine ``` ## Use it The engine reads stdin and writes the result to stdout. The accounting report goes to stderr, so you can pipe the output without stripping the numbers out of it. ```bash cat large-payload.json | caveman-engine compress > compressed.txt ``` ```bash caveman-engine detect < payload.txt caveman-engine retrieve > original.txt caveman-engine retrieve "connection pool" > narrowed.txt caveman-engine stats caveman-engine registry ``` `retrieve` with no query returns the original bytes exactly. With a query it returns the sections of the original that match, ranked by BM25, which is usually what an agent actually wanted. ## How it picks a compressor Detection reads the bytes and answers with a content type. The type selects the compressor. There is no model call and no configuration step in that path. `json` · `log` · `code` · `diff` · `search-result` · `text` · `html` · `terminal` · `tabular` · `config` Five routes require an explicit type: `toolschema`, `toolschema-annotations`, `toon`, `a11y`, and `repetition`. None is selected automatically because each needs caller intent or an input contract that byte detection cannot prove. See [Compressors](/docs/engine/compressors) for route-specific behaviour. Built with cgo, the code compressor uses tree-sitter and handles Go, Python and JavaScript or TypeScript. Built without cgo, or as WASM, it falls back to the Go standard library parser and handles Go only. Other languages pass through untouched rather than being mangled. ## Recovery Before any lossy result is emitted, the original bytes are written to a local store. Its public handle contains the first 16 bytes of the payload's SHA-256 digest as 32 hexadecimal characters after `ccr_`. Compressing the same payload twice produces the same handle and stores it once. - Store: `~/.caveman/ccr.db` - Backend: SQLite on host platforms, an in-memory map under WASM. Same contract. - Budget: 512 MiB of payloads by default, tunable with CAVEMAN_CCR_MAX_BYTES. - At the cap: New lossy transforms pass through instead. Existing handles are never evicted. The last row is the one to read twice. When the store fills, the engine stops compressing rather than dropping old originals to make room. Losing a recovery handle would turn an earlier honest compression into an unrecoverable one after the fact, so it does not happen. ## Token counting Ratios come from a real BPE tokenizer using the `o200k_base` encoding, with the vocabulary compiled into the binary. It is deterministic and offline. Every ratio the engine produces is `inferred`. Provider-reported usage from your actual response stays authoritative for spend, and the two live in different fields. ## Targets by content type These are honest reduction targets, not measured guarantees. What you get depends on your payload. | Content type | Target reduction | | --- | --- | | `search-result` | 80 to 95 percent | | `log` | 85 to 95 percent | | `json` | 70 to 90 percent | | `diff` | 60 to 80 percent | | `text` and HTML | 50 to 80 percent | | `code` | 40 to 70 percent | For one concrete measurement: on a committed tool-output fixture, 1,366 tokens became 246, a reduction of 82 percent. That figure is `inferred` and the original is recoverable. It is one fixture, and it is not a `verified` saving. ## Other subcommands ```bash caveman-engine toon encode | decode caveman-engine pixel render --density balanced caveman-engine pixel simulate --model anthropic caveman-engine evals run --fixtures ./fixtures ``` Pixel mode renders text as an image, which is cheaper than text for some models and much more expensive for others. `pixel simulate` exists so you can check which case you are in before switching anything on. --- --- title: Compressors summary: "One router per content shape: JSON, logs, code, diffs, search results, prose." canonical: https://docs.caveman.so/docs/engine/compressors layer: engine license: BSL-1.1 capability: engine updated: 2026-08-26T04:05:35+02:00 basis: inferred --- # Compressors > One router per content shape: JSON, logs, code, diffs, search results, prose. The engine routes each payload to a compressor built for its shape. Detection and compression are deterministic local code. Neither step calls a model. ## Automatic routes Detection checks stronger structural signals first and falls back to `text` when confidence is low. | Content type | What the compressor keeps | | --- | --- | | `json` | Object keys, error subtrees, array edges, error items, anomalies, change points, and query-relevant items. | | `terminal` | Final progress state, head and tail context, warnings, errors, and failure lines. ANSI control bytes are removed. | | `diff` | File and hunk headers, every changed line, and nearby unchanged context. | | `html` | Main readable article content rather than scripts, styles, navigation, and repeated chrome. | | `tabular` | Table structure and signal-bearing rows from CSV, TSV, or Markdown tables. | | `code` | Imports, declarations, signatures, and type structure. Function bodies may be elided. | | `log` | Head and tail context, warnings, errors, stack frames, and query-relevant lines. | | `search-result` | Top and bottom hits, diagnostic lines, and query-relevant results. | | `config` | Structure and selected values from YAML, TOML, and INI. | | `text` | Headings, opening and closing sections, marked important sections, and query-relevant prose. | The order matters. Raw terminal escape sequences are conclusive, for example, so terminal output is recognized before code or logs. Plain prose containing one code keyword does not become code. Unknown input takes the conservative `text` path. ## Forced routes Some transforms are available only when a caller names them: | Type | Why it is explicit | | --- | --- | | `toolschema` | Changes model-visible tool descriptions and annotations. The caller must identify a tool catalog. | | `toolschema-annotations` | Strips a reviewed allowlist of schema annotations without treating arbitrary JSON as schema. | | `toon` | Re-encodes JSON into another wire format. Automatic selection would surprise a JSON caller. | | `a11y` | Expects a Chrome accessibility tree and emits a compact UID view. Ordinary JSON must not enter this path. | | `repetition` | Collapses consecutive identical lines. Repetition alone does not identify the payload's meaning. | Do not depend on a published compressor count. The registry changes as content shapes and safety contracts change. Depend on the named type you need, or ask the engine for its current registry. ```bash caveman-engine registry caveman-engine detect < payload.txt ``` ## Elision markers Lossy compressors replace dropped runs with explicit markers. A marker states how much was removed, and some compressors attach invariants calculated from those exact units. ```text … 42 lines elided (caveman) … … 18 context lines elided (caveman) … ``` Markers are part of the idempotence contract. Compressing an already compressed view does not keep collapsing its own markers or append a second contract line. ## Query-aware selection JSON, logs, search results, and text can receive a query. Query-aware compressors use deterministic BM25 to keep matching units alongside their fixed safety anchors. They do not use embeddings or a model call. A query may change which records remain visible. It may not make the output larger than the queryless view. Exact original bytes remain behind the same recovery handle. ## Code builds With cgo enabled, the code route uses tree-sitter for Go, Python, JavaScript, and TypeScript. A build without cgo, including WASM, uses Go's standard parser and compresses Go only. Unsupported languages pass through. Comment removal is opt-in. Default code compression keeps comments and Python docstrings, elides bodies, then parses the result again. A result that no longer parses is rejected. ## Shared safety rules Every compressor returns either a candidate or `ok: false`. The engine applies the remaining gates around it: ```text parse or shape mismatch -> original bytes recovery write failure -> original bytes candidate not smaller -> original bytes unknown forced type -> original bytes ``` Keeping errors, names, keys, signatures, or tool parameters reduces risk. It does not prove that every model will answer identically from every compressed view. Use task graders for that claim, and keep recovery available. ## Reduction figures Content-type ranges shown on the engine overview are targets, not guarantees. Payload size, repetition, query, and important-line density decide whether a result is smaller. The engine reports the result of each run as `inferred`; it passes through when no reduction survives the gates. --- --- title: Recoverable compression summary: "Why lossy stays honest: every removed byte can be fetched back." canonical: https://docs.caveman.so/docs/engine/recoverable layer: engine license: BSL-1.1 capability: engine updated: 2026-08-26T04:05:35+02:00 basis: inferred --- # Recoverable compression > Why lossy stays honest: every removed byte can be fetched back. Recoverable compression lets the model see a smaller view without destroying source bytes. Caveman calls the local store CCR, short for Context Recovery. [Diagram: one request, from your agent through the skill and the engine to the provider, with a recovery path back.] ## Store before showing less A lossy transform follows this order:
The compressor creates a smaller candidate and the token counter confirms that it is smaller.
The engine writes the original bytes and accounting metadata to CCR.
Only after that write commits does the engine return the smaller view and its handle.
If step two fails, step three never publishes a lossy view. The caller receives the original bytes, a zero ratio, and no recovery handle. ## Handle format CCR hashes the original with SHA-256 and uses the first 16 digest bytes in the public handle: ```text ccr_<32 lowercase hexadecimal characters> ``` The same input produces the same handle and is stored once. A handle identifies content, not a session or a filename. ## Persistent store - Default path: `~/.caveman/ccr.db` - Override: `CAVEMAN_CCR_DB` - Default budget: 512 MiB of retained payload and metadata bytes. - Budget override: `CAVEMAN_CCR_MAX_BYTES` - Host backend: SQLite with WAL and a busy timeout. `CAVEMAN_HOME` changes the parent directory when `CAVEMAN_CCR_DB` is absent. WASM uses an in-memory store because the host SQLite implementation is unavailable there. A WASM handle lasts only as long as that in-memory store. The budget accepts a positive byte count. At the limit, new lossy transforms pass through. Existing handles are not evicted to make room, because eviction would invalidate a recovery promise already shown to an agent. ## Retrieve ```bash caveman-engine retrieve ccr_xxxxxxxx > original.bin caveman-engine retrieve ccr_xxxxxxxx "connection pool" > relevant.txt ``` An empty query returns the stored original byte for byte. A query asks for a deterministic BM25-selected view of the original and is useful when full recovery would refill the context window. The MCP surface exposes the same distinction: ```text caveman_retrieve({ recovery_handle: "ccr_..." }) caveman_retrieve({ recovery_handle: "ccr_...", query: "connection pool" }) ``` Unknown handles fail with an explicit error. Retrieval never guesses, returns a nearby payload, or treats a missing handle as empty content. Only empty-query retrieval is byte exact. Query recovery selects complete relevant sections from stored original bytes. Use full retrieval when exact order, omitted records, or a byte-for-byte comparison matters. ## Shared and separate stores The engine CLI, MCP server, and caveman-shrink use the shared CCR path by default, so a handle minted by one can resolve in a later process through another. cavemem keeps its recovery data under `~/.caveman/mem/ccr.db`. Recover a memory hit with `cavemem recover` or `caveman mem recover`; do not send that handle to the engine store. ## What recovery does not prove Recovery proves that omitted bytes remain available. It does not prove that the model will ask for them, that a compressed answer has equal quality, or that the local token count matches a provider bill. Those are separate grader, behaviour, and accounting questions. --- --- title: Token counting summary: The offline counter behind every ratio the engine reports. canonical: https://docs.caveman.so/docs/engine/tokens layer: engine license: BSL-1.1 capability: engine updated: 2026-08-26T04:05:35+02:00 basis: inferred --- # Token counting > The offline counter behind every ratio the engine reports. The engine needs a stable local token count to decide whether a transform is smaller. Its default counter uses an offline `o200k_base` tokenizer and records enough identity to say exactly which counter produced a result. - Backend: `tiktoken-go/o200k_base` - Pinned revision: `github.com/tiktoken-go/tokenizer@v0.8.0` - Input surface: Raw text, with no normalization. - Network: None. - Product basis: `inferred` ## What exact means here For valid UTF-8, the counter returns the exact result of the pinned `o200k_base` codec on the raw string. That is exact for this local backend and this input surface. It is not an exact provider bill. Providers may count a request envelope, messages, tools, cached input, or model- specific tokens differently. Provider-reported usage from the real response remains authoritative for spend. The code keeps these two claims separate: ```text local tokenizer result -> deterministic local evidence provider response usage -> observed provider evidence ``` Public compression reports still use `basis: "inferred"` because a local counter did not observe the bill. ## Fallback counter If the codec cannot be constructed, or if a count fails, the engine uses a deterministic approximation: ```text max(1, floor(UTF-8 rune count / 4)) ``` Empty input counts as zero. Invalid UTF-8 also takes the approximation path. Detailed count evidence labels this backend `approx/chars-per-four`, marks it modeled rather than exact, and records the fallback reason. The fallback is not presented as an `o200k_base` result. Backend, revision, semantic digest, exactness, and reason travel with detailed evidence so callers can reject modeled counts when their gate needs a pinned tokenizer. ## Provider-specific registry The token package also supports exact provider, model, and surface bindings. A binding must name a reviewed local backend. Unknown or unsupported tuples return `unmeasured`. They do not silently fall back to a nearby model or the default counter. A caller that wants a modeled fallback must choose it explicitly. Supported surfaces are: - Raw text - Messages - Tools - Full request envelope A backend may support one surface without supporting the others. Exactness never expands beyond the surface in its descriptor. ## How compression uses it The engine counts input and candidate output with the same counter. A candidate is used only when output count is strictly lower. Equal counts pass through. ```bash cat payload.json | caveman-engine compress > compact.txt caveman-engine stats ``` Reports include tokens before, tokens after, ratio, and basis. The ratio is a property of that payload and run. It is not annualized, converted to currency, or relabeled `verified`. ## Determinism and cache identity The `o200k_base` descriptor pins backend name, module revision, semantic contract digest, artifact digest, and surface. That identity lets caches and build locks distinguish a real counter change from the same name backed by different code. No Unicode normalization runs before counting. The same byte sequence and counter identity produce the same result. ## What the counter cannot tell you - What a provider charged for a request - How cached input was billed - Which model answered - Whether compression preserved answer quality - A monthly or dollar saving Use provider usage for observed token accounting and graders for quality. Keep local token estimates labeled `inferred`. --- --- title: "caveman-shrink" summary: Shrink command output before it reaches the model. canonical: https://docs.caveman.so/docs/engine/shrink layer: engine license: MIT capability: shrink updated: 2026-08-26T04:05:35+02:00 basis: inferred --- # caveman-shrink > Shrink command output before it reaches the model. `caveman-shrink` reduces MCP and OpenAI tool catalogs before a model reads them. It keeps the structural surface used to form a tool call and shortens model-visible annotation text. - Input: MCP or OpenAI tool-catalog JSON. - Stdin limit: 32 MiB. - Core licence: BSL 1.1. - npm launcher: MIT. - Reports: `inferred` ## Build the documented source ```bash git clone https://github.com/JuliusBrussee/caveman go build -o ./bin/caveman-shrink ./public/shrink/cmd/caveman-shrink ``` A published npm launcher also downloads a matching binary and verifies a key-signed manifest plus artifact SHA-256. The registry release may trail the source documented here. ```bash npx -y caveman-shrink lint tools.json ``` ## Compress a catalog ```bash cat tools.json | caveman-shrink > tools.min.json ``` Compressed JSON goes to stdout. The JSON accounting report goes to stderr, so a pipeline receives catalog bytes only. The report includes tokens before, tokens after, ratio, basis, content type, and a recovery handle when a lossy result was emitted. Accepted shapes include an MCP `{ "tools": [...] }` object, an OpenAI tool array, a `{ "functions": [...] }` object, and a single named tool object. ## What survives The structural selection profile preserves: - Tool and parameter names - Parameter types - Enum values - Required fields - Default and constant values - Internal reference targets Short descriptions remain whole. Long descriptions keep their lead and recognized constraint sentences. Examples, titles, comments, and other annotation bloat may be removed. ```bash caveman-shrink lint tools.json ``` `lint` prints inferred before and after counts per tool and for the complete catalog. It does not commit a recovery record or replace the file. Tests prove that names, parameters, enums, and required fields survive. Description text remains model-visible, so only a model eval can show that a given model selects the same tool for your cases. ## Recover ```bash caveman-shrink recover ccr_xxxxxxxx > tools.original.json ``` Before returning a lossy view, shrink stores exact original bytes in the shared CCR database. The default is `~/.caveman/ccr.db`; `CAVEMAN_CCR_DB` selects another path. Recovery works from a later process. Malformed input, an unavailable recovery store, or a candidate that is not smaller returns the original catalog unchanged with ratio zero and no handle. Input beyond 32 MiB fails with `cave_input_too_large`. ## CLI wrapper The main CLI exposes two different shrink paths: ```bash caveman tools shrink -- npm test caveman tools compress catalog < tools.json caveman tools compress catalog lint tools.json caveman tools compress catalog recover ccr_xxxxxxxx ``` The first command compresses command output. The catalog commands delegate to `caveman-shrink`. They are separate because terminal output and tool definitions have different structures and safety contracts. ## Licence boundary The npm launcher is MIT. Go source and downloaded binary use BSL 1.1 with the first-party self-hosting grant. Offering the core functionality to third parties as a hosted, managed, or embedded service needs a commercial licence. --- --- title: "Memory: Overview" summary: cavemem holds the context you would otherwise paste into every session. canonical: https://docs.caveman.so/docs/memory layer: memory license: MIT capability: mem updated: 2026-08-26T03:57:26+02:00 basis: inferred --- # Memory: Overview > cavemem holds the context you would otherwise paste into every session. cavemem is a local store for the context you keep re-explaining. You write something once, and later ask a question. It returns only the parts that answer the question, compressed through the engine on the way out. - Licence: Go core BSL 1.1. The JavaScript and Python clients are MIT. - Storage: `~/.caveman/mem` - Ranking: BM25 behind a conservative threshold. - Basis: Always `inferred`. - Binary: `cavemem` ## Build it ```bash go build -o cavemem ./public/mem/cmd/cavemem ``` ## Use it ```bash cavemem remember "the deploy key lives in vault under ops/deploy" cavemem recall "where is the deploy key" cavemem recall "full migration context" 5 0 cavemem supersede mem_xxxxxxxx "deploy key moved to vault ops/deploy-v2" cavemem history mem_yyyyyyyy cavemem forget mem_xxxxxxxx cavemem recover ccr_xxxxxxxx > original.txt ``` `recall` returns JSON with the hits and a `basis` field. The two trailing numbers on the second recall are the result limit and the token budget. A budget of `0` means unlimited, and you have to write it out, because unbounded recall should be a decision rather than a default. Run `cavemem` with no subcommand and it speaks MCP over stdio: ```json { "mcpServers": { "cavemem": { "command": "cavemem" } } } ``` ## Four guarantees **Byte safe on write.** Raw text is stored first. Compression happens on the way out, never on the way in, so a compressor bug can never cost you the original. **Fails toward nothing.** An off-topic query recalls nothing rather than returning the closest thing it found. A memory system that always answers is worse than one that admits it has nothing. **Bounded by default.** Recall packs at most 2,000 inferred tokens unless you explicitly opt out. **Reversible.** Every compressed hit carries a recovery handle, and `cavemem recover` returns the original bytes exactly. ## Superseding rather than editing `supersede` writes a new version and links it to the old one. `recall` returns only current entries. `history` shows the chain from oldest to current. Facts about a codebase go stale, and a store that silently overwrites gives you no way to notice when the thing you remembered stopped being true. ## Clients Both clients shell text over stdin rather than passing it as an argument, so a large block does not hit an operating system argv limit. ```js await remember("the deploy key lives in vault under ops/deploy"); await recall("where is the deploy key", 5, 2000); ``` ```python cavemem.remember("the deploy key lives in vault under ops/deploy") cavemem.recall("where is the deploy key", limit=5, token_budget=2000) ``` Set `CAVEMEM_BIN` if the binary is not on `PATH`. An oversized `remember` exits with code 65. The npm client `cavemem` is published but currently behind the source in this repository, and its registry description describes older behaviour. There is no PyPI package yet. Until both catch up, build the core from source and use the Python client from the repository. ## One thing to know about handles cavemem keeps its own recovery store at `~/.caveman/mem/ccr.db`, separate from the engine's store at `~/.caveman/ccr.db`. Handles are not interchangeable between the two. A handle from a cavemem recall is recovered with `cavemem recover`, not with the engine. --- --- title: Recall and offload summary: Moving a heavy instruction file out of the prompt and into recall. canonical: https://docs.caveman.so/docs/memory/offload layer: memory license: MIT capability: mem updated: 2026-08-26T04:05:35+02:00 basis: inferred --- # Recall and offload > Moving a heavy instruction file out of the prompt and into recall. Offloading moves recurring context out of an always-read instruction file and into local cavemem. A short pointer stays where the block used to be, so an agent knows how to recall the compact form and recover exact source bytes. ## Good candidates Offload context when all of these are true: - The same block appears across several sessions - It is useful for some tasks, not every turn - A specific recall query can find it - Pointer plus recall costs fewer inferred tokens than keeping the block resident - Removing it does not weaken a load-bearing instruction Repetition is evidence of recurring cost, not proof that content is unnecessary. Security rules, required build commands, repository boundaries, and other load-bearing instructions stay in place. ## Measure first ```bash caveman learn --json caveman learn apply recurring_context: --dry-run ``` The learn report identifies repeated blocks by local session evidence. A candidate contains a locator, expected token figures, and proposed pointer text. It does not contain a trusted copy of the block body. Use the consent-gated editing flow for the full procedure: ```bash caveman learn implement ``` ## Safe offload sequence
Re-read the block from its real source file. Verify its file, session line, block index, and SHA-256 against the candidate locator. Stop if source changed after the scan.
Store exact block text. `--` ends option parsing, so a block beginning with a rule such as `---` stays literal. ```bash caveman mem remember -- "" ```
Recall it with a topic query and record `tokens_added`. ```bash caveman mem recall "" ```
Compare resident cost against pointer plus recall cost. If the new path is not smaller, forget the new memory and leave source untouched. ```bash caveman mem forget mem_xxxxxxxx ```
After approval, replace source block with pointer. Confirm recall still returns a hit before finishing.
## Pointer shape A useful pointer names the topic and both recovery paths: ```text Recurring migration context lives in cavemem. Recall: caveman mem recall "migration context" Exact source: caveman mem recover ``` Do not put the removed block into the pointer. That would preserve the recurring cost under a new heading. ## Recall remains bounded Normal cavemem recall returns up to five hits and packs at most 2,000 inferred tokens. The direct `cavemem` binary allows an explicit token budget of zero for unlimited recall: ```bash cavemem recall "migration context" 5 0 ``` Unbounded recall is opt-in. The `caveman mem recall` wrapper keeps the safe default and exposes only the result limit. ## Exact recovery Each compressed recall hit carries a `recovery_handle` from cavemem's own store: ```bash caveman mem recover ccr_xxxxxxxx > original.txt ``` That store lives under `~/.caveman/mem`, separate from the engine's shared CCR database. Use the memory recovery command for a memory handle. ## Auto-recall is optional Pointer-driven recall is the default. A verified host hook can inject relevant memory on each prompt, but it is off until you enable it: ```bash caveman mem hook install claude caveman mem hook uninstall claude ``` Only hosts with a declared live prompt hook are eligible. Injection fails open, so a recall problem does not block the user's prompt. Every injected hit discloses its inferred token cost. Safe offload has two required end states: a pointer remains in source and recall returns the stored content. If either is missing, restore source and delete the new memory. ## Undo Undo requires both parts: restore original block to its source location, then remove stored memory with `caveman mem forget `. Report both changes so the user can verify that recurring context did not disappear. --- --- title: "Cloud: What Cloud adds" summary: "The managed plane: fleet visibility, spend attribution, and verified numbers." canonical: https://docs.caveman.so/docs/cloud layer: cloud license: Commercial updated: 2026-08-26T03:57:26+02:00 basis: inferred --- # Cloud: What Cloud adds > The managed plane: fleet visibility, spend attribution, and verified numbers. Everything else on this site runs on your machine and reports `inferred`. Caveman Cloud is the layer across the network boundary, and it exists to answer one question the local tools structurally cannot. ## The question local tools cannot answer A local tool sees your bytes. It does not see your invoice. It can tell you a payload got 82 percent smaller. It cannot tell you what that was worth, because the price depends on the model, the provider, the cache state at the time, and your contract. It also cannot tell you whether the smaller payload produced the same answer, because it only saw one arm of the comparison. That gap is the whole reason the managed plane exists. It runs both arms, holds the price table, and compares against traffic that actually happened. Caveman Cloud is in private development and the waitlist is open. This page describes what it is for, not how it is built. When it opens, its own surfaces get documented here in full. ## What it adds - Fleet visibility: One view across every agent, developer and workflow, instead of one local database per laptop. - Spend attribution: Which person, project, agent or branch produced which part of the bill. - Eval gates: An optimisation is only applied to traffic that passed a grader on that traffic. - Verified numbers: The only place the word verified is allowed, because it is the only place with both arms of the comparison. ## The vocabulary does not change The same rules that bind the local tools bind the managed plane. `inferred` stays `inferred` when it arrives from a laptop. A number does not get promoted by crossing a network. The four money figures are kept separate and are never added together: measured spend, inferred headroom, verified savings, and observed outcome. There is no fifth bucket, and in particular there is no "realised savings" figure that quietly blends the others. ## Getting access The waitlist and contact details are on [caveman.so](https://caveman.so). Self-hosting the open layers needs no account and no waitlist. If all you want is compression on your own machine, you already have everything you need in the three layers below this one. --- --- title: CLI summary: "The command surface: compress, detect, learn, and agent setup." canonical: https://docs.caveman.so/docs/cli license: MIT capability: cli updated: 2026-08-26T04:05:35+02:00 basis: inferred --- # CLI > The command surface: compress, detect, learn, and agent setup. `caveman` is the command surface for local tools, supported coding agents, and connected account operations. The same binary is also available as `cave`. - npm package: `@caveman-ai/cli` - Runtime: Node.js 22.13 or newer. - Licence: MIT. - JavaScript dependencies: None at runtime. - Heavy work: Companion Go binaries. ## Install ```bash npm install -g @caveman-ai/cli caveman setup ``` The bare `caveman` npm package is unrelated. Use the scoped package name. `setup` shows which companion binaries are available. Install or repair the signed local bundle with: ```bash caveman setup --install ``` Artifacts are checked against a key-signed manifest and their own SHA-256 before atomic installation under `~/.caveman/bin`. Resolution checks an explicit `CAVEMAN_*_BIN` override, `PATH`, then that directory. ## Command groups ```bash caveman help caveman tools caveman cloud ``` `tools` contains local commands and needs no account. `cloud` contains connected commands and requires login. Compatibility aliases keep older top-level commands working, but grouped discovery is the clearest way to see the current surface. Common local commands: ```bash caveman tools compress < payload.json caveman tools shrink -- npm test caveman tools toon encode < payload.json caveman tools mem recall "migration context" caveman tools retrieve ccr_xxxxxxxx caveman tools browse snapshot https://example.com caveman tools evals run caveman tools stats --json caveman tools config get ``` Some commands also keep their top-level form, such as `caveman learn`, `caveman stats`, `caveman mem`, and `caveman retrieve`. ## Launch an agent ```bash caveman claude caveman codex caveman gemini caveman aider caveman hermes caveman openclaw caveman opencode ``` Each shortcut calls `caveman wrap `. The profile registry decides binary name, protocol, setup method, hooks, and fallback. On the first interactive local wrap, the CLI can install the signed runtime bundle and continue the same command. Non-interactive runs do not change installation state without an explicit setup command. ## Learn from local sessions ```bash caveman learn caveman learn --plain caveman learn --json caveman learn implement claude ``` The profiler is read-only. Applying a proposed file change belongs to the consent-gated `caveman-learn` skill. See [caveman learn](/docs/skill/learn) for sink classes and edit gates. ## Stable pipelines Compression writes payload bytes to stdout and accounting to stderr: ```bash cat large.json | caveman tools compress > compact.json 2> report.json ``` When the engine binary is missing, this path emits original bytes and a structured zero-ratio warning. It does not break the pipe or claim compression. Use machine modes when output feeds another process: ```bash caveman learn --json caveman learn --plain caveman stats --json caveman tools skills list --json ``` They do not open interactive menus or prompt for input. ## Missing runtime behaviour Most affected local commands degrade to an explicit pass-through when a companion binary is missing. Output stays unchanged, reduction is zero, and the warning names the repair command. `wrap` needs extra care. It may point an agent at a local listener. If nothing is listening, requests cannot route. An interactive invocation offers to launch the agent directly. A script must ensure the listener is running or use `--no-proxy` when direct provider traffic is intended. The npm package does not contain compression, memory, browsing, or token-counting implementations. It resolves and drives the public binaries. `caveman setup` is the source of truth for what this machine can run. ## Numbers and telemetry Local reduction figures stay `inferred`. Commands do not convert them into monthly savings or `verified` claims. Anonymous CLI telemetry covers command name, version, platform, duration, exit class, and aggregate local token counts. It excludes prompts, code, file paths, arguments, model names, credentials, and dollar fields. Disable it with any of these: ```bash caveman telemetry off CAVEMAN_TELEMETRY=0 caveman learn DO_NOT_TRACK=1 caveman learn ``` See [Telemetry](/docs/telemetry) for the complete payload and precedence. --- --- title: TypeScript SDK summary: Compress payloads and read spend from Node. canonical: https://docs.caveman.so/docs/sdk/typescript license: MIT capability: sdk-ts updated: 2026-08-26T04:05:35+02:00 basis: inferred --- # TypeScript SDK > Compress payloads and read spend from Node. `@caveman-ai/sdk` is a zero-runtime-dependency TypeScript client for provider calls, recoverable compression, tracing, deferred tools, and request policy. It talks to a Caveman service you configure; it does not embed the local engine. - Package: `@caveman-ai/sdk` - Runtime: Node.js 22.13 or newer. - Dependencies: None at runtime. - Module: ES module with bundled TypeScript declarations. - Licence: MIT. ## Install ```bash npm install @caveman-ai/sdk ``` ## Create a client ```ts const cave = new Cave({ apiKey: process.env.CAVE_API_KEY!, baseURL: "http://127.0.0.1:8787", agent: "support-agent", }); ``` `apiKey`, `baseURL`, and `agent` are required. Service URLs must be absolute HTTP or HTTPS URLs without embedded credentials, query strings, or fragments. ## Compress ```ts const result = await cave.compress("large payload"); console.log(result.output); console.log(result.tokensBefore, result.tokensAfter); console.log(result.ratio, result.basis); console.log(result.recoveryHandle); ``` The SDK sends the payload to the configured compression endpoint and maps the engine report. On a transport or parse problem it returns original payload, ratio zero, and no recovery handle. It never reimplements a compressor inside JavaScript. `basis` is `inferred`. Token fields come from the compressor's local counter, not provider usage. ## Provider clients ```ts const openai = cave.openai({ upstreamKey: process.env.OPENAI_API_KEY }); const anthropic = cave.anthropic({ upstreamKey: process.env.ANTHROPIC_API_KEY }); const gemini = cave.gemini({ upstreamKey: process.env.GEMINI_API_KEY }); const vertex = cave.vertex({ upstreamKey: process.env.GOOGLE_ACCESS_TOKEN }); const response = await openai.responses.create({ model: "gpt-5.6", input: "Summarize this incident", }); ``` Provider clients constrain raw requests to their provider prefix. The separate `bedrock()` method returns a validated descriptor for AWS SDK configuration and performs no request. ## Narrow one request Request options can switch project-enabled work off for one call: ```ts await cave.openai().responses.create(body, { cave: { optimize: "off" }, }); await cave.openai().responses.create(body, { cave: { optimize: { compress: false, cacheHints: false } }, }); ``` Boolean `true` asks for the corresponding capability. It does not grant permission or bypass project policy. Unknown option fields and styles throw before the request. ## Read disclosure headers ```ts const response = await cave.openai().raw("/v1/responses", { method: "POST", body, }); const receipt = parseReceipt(response.headers); ``` The receipt may include mode, applied optimizations, cache status, request id, inferred compression counts, and a recovery handle. A missing header remains absent. The parser does not invent a default value. ## Local helpers Several helpers run without a request: - `assemble()` orders stable, session, and volatile context slots - `retryLoopBreaker()` stops a consecutive identical tool-call loop - `parseReceipt()` decodes response headers - `gatewayHeaders()` and `gatewayConfig()` build provider client configuration - `policyUnitFraction()` produces deterministic policy assignment input `trace()` and `exporter()` create correlated spans. `tools()` can keep an initial subset of a catalog and load more through connected search. `context.pack()` is also connected and returns deferred item ids rather than silently discarding omitted context. The `jobs` surface is reserved. Its methods throw `cave_async_jobs_unavailable` locally and send no request. This package needs a configured service for provider calls and `compress()`. Accountless local engine compression ships through the CLI and Go runtime. Installing the SDK alone does not start that runtime. ## What it will not claim SDK compression counts are not provider counts. Response receipts are not invoices. No local SDK result becomes a monthly or `verified` saving. --- --- title: Python SDK summary: The same surface for Python agents. canonical: https://docs.caveman.so/docs/sdk/python license: MIT capability: sdk-python updated: 2026-08-26T04:05:35+02:00 basis: inferred --- # Python SDK > The same surface for Python agents. `caveman-sdk` is the Python client for the same connected surface as the TypeScript SDK. It uses only the Python standard library and includes type information. - Distribution: `caveman-sdk` - Import: `caveman_cloud` - Runtime: Python 3.13 or newer. - Dependencies: None at runtime. - Licence: MIT. ## Install ```bash python -m pip install caveman-sdk ``` The `caveman` package on PyPI is unrelated. Distribution and import names are deliberately different. ## Create a client ```python from caveman_cloud import Cave cave = Cave( api_key=os.environ["CAVE_API_KEY"], base_url="http://127.0.0.1:8787", agent="support-agent", ) ``` `api_key`, `base_url`, and `agent` are required. URLs must be absolute HTTP or HTTPS service URLs without embedded credentials, a query, or a fragment. ## Compress ```python result = cave.compress("large payload") print(result.output) print(result.tokens_before, result.tokens_after) print(result.ratio, result.basis) print(result.recovery_handle) ``` `compress()` delegates to the configured service. A transport or response problem returns original payload, ratio zero, and no recovery handle. Python does not carry a second compressor implementation. Counts use `basis="inferred"`. They are local compressor estimates, not provider-reported usage. ## Provider clients ```python openai = cave.openai(upstream_key=os.environ.get("OPENAI_API_KEY")) anthropic = cave.anthropic(upstream_key=os.environ.get("ANTHROPIC_API_KEY")) gemini = cave.gemini(upstream_key=os.environ.get("GEMINI_API_KEY")) vertex = cave.vertex(upstream_key=os.environ.get("GOOGLE_ACCESS_TOKEN")) response = openai.responses.create({ "model": "gpt-5.6", "input": "Summarize this incident", }) ``` The provider wrappers expose native request paths through the configured provider prefix. `bedrock()` returns a validated configuration descriptor without making a network call. ## Narrow one request ```python cave.openai().responses.create(body, optimize="off") cave.openai().responses.create( body, optimize={"compress": False, "cache_hints": False}, ) ``` Boolean `True` asks for a capability and remains subject to project policy. It does not enable anything by itself. Unknown fields and invalid styles raise `ValueError` before a request. ## Read disclosure headers ```python from caveman_cloud import parse_receipt receipt = parse_receipt(response.headers) print(receipt.mode, receipt.optimizations) print(receipt.tokens_before, receipt.tokens_after) print(receipt.recovery_handle) ``` `parse_receipt` accepts urllib headers, a dictionary, or key-value pairs. Missing headers stay `None`. ## Local helpers Python mirrors the TypeScript names in Python style: - `assemble()` orders context by stability - `retry_loop_breaker()` interrupts identical consecutive tool calls - `parse_receipt()` decodes response disclosure - `gateway_headers()` and `gateway_config()` build provider configuration - `policy_unit_fraction()` returns deterministic policy assignment input Tracing, OTLP export, deferred tool search, context packing, shared context, checkpoints, and artifacts are also available. Connected features use the configured service. The reserved jobs client raises `cave_async_jobs_unavailable` locally and performs no request. Install `caveman-sdk`, then `import caveman_cloud`. Avoid both bare `caveman` package names. They belong to other projects. ## What it cannot do alone Installing the Python package does not install or start the local engine. It does not turn inferred compression counts into provider usage, currency, monthly savings, or `verified` results. --- --- title: Agent SDK summary: Build an agent that is efficient by construction. canonical: https://docs.caveman.so/docs/agent-sdk license: MIT capability: agent-sdk updated: 2026-08-26T04:05:35+02:00 basis: inferred --- # Agent SDK > Build an agent that is efficient by construction. `@caveman-ai/agent` is a TypeScript framework for defining an agent, its tools, context, memory, output contract, and evals in one source graph. The runtime can operate directly against a provider or use local Caveman transforms when a verified local runtime is available. - Version in source: `0.2.0` - Runtime: Node.js 22.19 or newer. - Licence: MIT. - Registry status: Not published to npm yet. - CLI: `caveman-agent` ## Build from source ```bash git clone https://github.com/JuliusBrussee/caveman cd caveman pnpm install --frozen-lockfile pnpm --filter @caveman-ai/agent build ``` The initializer and Agent SDK release are still source-only. Do not use `npm create @caveman-ai/agent` until the package is published. ## Smallest definition ```ts id: "support", instructions: "Answer from policy. Never invent policy.", model: auto(), }); ``` `auto()` resolves a configured model. It does not classify tasks or route between models. Run from code: ```ts const result = await run(support, "Can I get a refund?"); console.log(result.text); console.log(result.mode); ``` Use `stream()` for typed run, context, tool, completion, and error events. ## Define tools explicitly ```ts const lookupPolicy = tool({ name: "lookup_policy", description: "Read one policy by id.", input: schema.object({ id: schema.string() }), effect: "read", result: "compress", async execute({ id }) { return loadPolicy(id); }, }); ``` Every tool declares an effect, result policy, timeout, and input schema. Result policies are `auto`, `inline`, `page`, `compress`, or `exact_ccr`. Tool names beginning with `cave_` are reserved by the framework. Repeated identical calls stop by default unless a polling tool opts into `allowRepeat`. ## Sandbox modes | Mode | Behaviour | | --- | --- | | `required` | Default. Tool closures run in isolated, network-denied Node workers from a staged source graph. | | `fixture` | Trusted tests only. Tools run in the host process, and write effects are blocked. | | `host` | Explicit opt-in. Tools run with host access and write effects may execute. | Host mode is not eligible for a locked build. A required-sandbox parent also prevents a subagent from selecting host mode. ## Context and memory `context()` labels each segment by kind, stability, safety class, priority, recovery contract, cache region, and privacy class. `memory()` currently accepts local, local-only memory with a positive `m`, `h`, or `d` TTL. Shared memory settings fail during construction rather than after a model call. Artifacts declare paging or exact recovery. Output definitions can attach a schema and token budget. ## Run modes Without a trusted local runtime, an unlocked agent calls its provider directly in `observe-only` mode. No local transform or Caveman telemetry runs, and no efficiency result is claimed. With the CLI and local runtime available: ```bash npm install -g @caveman-ai/cli caveman start ``` Eligible calls may run in `optimized` mode. Listener identity, process state, and executable ownership are checked before provider credentials can be sent to a loopback endpoint. A locked plan refuses silent downgrade. ## Check before spending a provider call ```bash caveman-agent doctor caveman-agent doctor --json caveman-agent check ``` Doctor checks Node version, sandbox containment, local runtime identity, engine registry, configuration, context, and provider selection. Missing local transforms are warnings when observe-only execution remains valid. Broken sandbox containment, invalid config, or lock drift fails. ## Evals and builds ```bash caveman-agent dev caveman-agent build caveman-agent check ``` An eval declares approved profile, development, or holdout cases plus quality graders. A build profiles work, selects on development cases, freezes a plan, then opens untouched holdout cases. Successful builds write local lock, workload-profile, and report artifacts under `.caveman/`. This source slice has exact compiled behaviour for its native Pi lane. Other adapters remain baseline-equivalent or have explicit limits, and Claude build locking is refused. A hash binds reviewed bytes and policy; it is not a signature or runtime attestation. Core authoring, runtime, sandbox, doctor, and source build paths exist. npm publication and broader locked-adapter parity do not. Use observe-only mode or the documented native lane, and do not publish a savings percentage from a local build report. --- --- title: MCP server summary: Expose compression and recall as tools any MCP client can call. canonical: https://docs.caveman.so/docs/mcp license: MIT capability: mcp updated: 2026-08-26T04:05:35+02:00 basis: inferred --- # MCP server > Expose compression and recall as tools any MCP client can call. `caveman-mcp` exposes engine compression, recovery, statistics, and TOON conversion to any MCP host. It speaks line-delimited JSON-RPC over stdin and stdout and opens no network connection. - Transport: MCP over stdio. - Protocol version: `2024-11-05` - Tools: Compression, recovery, session stats, TOON encode, and TOON decode. - Reports: `inferred` - Core licence: BSL 1.1. ## Build from source `caveman-mcp` is not published on npm today. Build the server from the public source: ```bash git clone https://github.com/JuliusBrussee/caveman go build -o ./bin/caveman-mcp ./public/mcp/cmd/caveman-mcp ``` Register the resulting binary with an MCP host: ```json { "mcpServers": { "caveman": { "command": "/absolute/path/to/bin/caveman-mcp", "args": [] } } } ``` The main CLI can install host configuration after the binary is available: ```bash caveman tools mcp install claude --server caveman caveman tools mcp uninstall claude --server caveman ``` ## Tools | Tool | Input | Result | | --- | --- | --- | | `caveman_compress` | `input`, optional `content_type` | Smaller text or original input, counts, ratio, basis, content type, method, and optional handle. | | `caveman_retrieve` | `recovery_handle`, optional `query` | Exact original for an empty query, or a query-selected view. | | `caveman_stats` | None | Session requests, tokens before and after, ratio, and `basis: "inferred"`. | | `caveman_toon_encode` | JSON string in `input` | TOON text plus input and output byte counts, or original input with a note. | | `caveman_toon_decode` | TOON string in `input` | Decoded JSON, or an explicit invalid-TOON error. | Compression detects content type unless the caller forces one. Malformed, incompressible, or non-smaller input passes through with ratio zero. A recovery persistence error also passes through only when the returned bytes and accounting prove that no lossy view escaped. ## Recovery The server opens the shared store at `~/.caveman/ccr.db`. `CAVEMAN_CCR_DB` selects another path, and `CAVEMAN_HOME` changes the default parent directory. ```text caveman_compress({ input: largePayload }) caveman_retrieve({ recovery_handle: "ccr_..." }) ``` Set `CAVEMAN_MCP_EPHEMERAL=1` for an in-memory store. Handles from that process stop resolving when the process exits. Use recovery as a last resort. Elision markers and visible invariants often answer count or field questions without adding another agent turn. When recovery is needed, one broad query is usually cheaper than many narrow calls. An unknown handle returns `cave_unknown_handle`. It never becomes an empty successful result. ## TOON conversion TOON encoding is explicit and checks JSON round-trip. Encoding may return a valid TOON result even when it is not smaller, because the caller asked for conversion and receives both sizes. Invalid or unsupported JSON returns unchanged input with a note. Decoding fails with `cave_invalid_toon` on invalid input. It never emits raw TOON while claiming the result is JSON. ## Protocol limits Inbound JSON-RPC lines and ordinary tool results are capped at 16 MiB. Oversized values return `cave_payload_too_large` and the server continues serving later requests. Exact recovery is exempt from the result cap so a valid original is never made unrecoverable by the MCP framing limit. Handler panics become `cave_tool_panicked` tool errors instead of terminating the server. Unknown tools also return an explicit tool error. stdout carries JSON-RPC only. Logs and diagnostics go to stderr. Wrapping this binary with a script that prints a banner to stdout breaks MCP framing. ## What it cannot do The server does not browse, store durable semantic memories, call a model, or connect to a hosted account. Its session stats are local inferred compression accounting, not provider usage or `verified` savings. --- --- title: "caveman-browse" summary: Read web pages as compressed accessibility trees instead of raw HTML. canonical: https://docs.caveman.so/docs/browse license: BSL-1.1 capability: browse updated: 2026-08-26T04:05:35+02:00 basis: inferred --- # caveman-browse > Read web pages as compressed accessibility trees instead of raw HTML. `caveman-browse` is a local Chrome driver for agents. It reads Chrome's accessibility tree, turns it into compact UID-addressed text, and stores raw tree bytes behind a recovery handle. - Transport: MCP over stdio, with a direct CLI helper. - Browser path: Chrome DevTools Protocol. - Snapshots: Compressed accessibility text, not raw HTML. - Basis: `inferred` - Licence: BSL 1.1. ## Build from source `caveman-browse` is not published on npm today. ```bash git clone https://github.com/JuliusBrussee/caveman go build -o ./bin/caveman-browse ./public/browse/cmd/caveman-browse ``` Run with no arguments to serve MCP over stdio: ```bash ./bin/caveman-browse ``` ## MCP tools | Tool | Purpose | | --- | --- | | `browser_snapshot` | Navigate when a URL is supplied, wait, then return a compact accessibility view. | | `browser_act` | Click, type, select, scroll, or wait using a UID from the latest snapshot. | | `browser_eval` | Evaluate a JavaScript expression in the page. | | `browser_recover` | Return raw accessibility bytes, or a query-selected view of them. | Configure an MCP host with the absolute binary path: ```json { "mcpServers": { "caveman-browse": { "command": "/absolute/path/to/bin/caveman-browse", "args": [] } } } ``` ## Take a useful snapshot Large pages should include the task as a query: ```text browser_snapshot({ url: "https://example.com/settings", query: "save notification settings" }) ``` The query keeps matching accessible nodes and their ancestors. CCR retains full raw tree. This usually costs less than taking a full snapshot and recovering later. `interactive: true` keeps UID-bearing nodes and their ancestors only. It is useful for dense control surfaces but hides most page text. Query focus remains the better default when the agent must read. Snapshot output is text: ```text [a1] button "Save" [a2] checkbox "Email alerts" caveman before=... view=... after=... ratio=... basis=inferred handle=ccr_... ``` Only actionable or unknown custom roles receive UID tokens. The trailing line counts the exact result delivered to the agent. `view` isolates compact tree cost; `after` includes the accounting line. ## Act, then resnapshot ```text browser_act({ action: "click", uid: "a1" }) browser_act({ action: "type", uid: "a3", text: "alerts@example.com" }) browser_act({ action: "select", uid: "a4", option: "Daily" }) browser_act({ action: "scroll", uid: "a5" }) browser_act({ action: "wait" }) ``` UIDs belong to the latest successful snapshot. Unknown or stale UIDs fail explicitly. Click, type, select, and scroll report that dispatch succeeded, but they do not prove application state settled. Take another focused snapshot to verify the result. `wait` is a fixed short delay, not a condition-based page assertion. ## Direct CLI ```bash caveman-browse snapshot https://example.com "save settings" caveman-browse snapshot -i https://example.com caveman-browse act click caveman-browse eval "document.title" caveman-browse recover caveman-browse close ``` Direct commands share one detached, isolated Chrome until `close`. ## Navigation and limits Navigation accepts HTTP, HTTPS, `about:blank`, and bounded `data:text/html` URLs. Local files and privileged browser schemes fail with `cave_browser_url_denied`. Snapshot wait is limited to 30 seconds. Query text is limited to 4 KiB, and a data URL to 1 MiB. A snapshot that cannot produce a smaller recovery-backed UID view fails and keeps the previous UID map rather than dumping raw AX JSON into model context. ## Browser selection Environment variables select runtime behaviour: ```text CAVEMAN_BROWSE_CHROME explicit Chrome executable CAVEMAN_BROWSE_CDP attach to an existing CDP endpoint CAVEMAN_BROWSE_USER_DATA_DIR profile directory CAVEMAN_BROWSE_HEADFUL=1 show the browser window CAVEMAN_BROWSE_EPHEMERAL=1 use in-memory recovery ``` Default mode is headless. The recovery store otherwise uses the shared CCR path. This tool exposes accessibility structure and page JavaScript. It does not prove visual layout, screenshot appearance, animation, or pointer geometry. Use a visual browser tool when those properties matter. `browser_eval` runs caller-supplied JavaScript in the active page. Treat it as code execution in that browser session and use it only on pages you intend to control. --- --- title: Licensing summary: Which surfaces are MIT, which are BSL 1.1, and what that means for you. canonical: https://docs.caveman.so/docs/licensing license: MIT updated: 2026-08-26T03:57:26+02:00 basis: inferred --- # Licensing > Which surfaces are MIT, which are BSL 1.1, and what that means for you. Caveman uses three licences, one per kind of surface. Which one applies depends on what a piece of code is, not on how much you use it. Nothing here has a usage threshold that changes your terms. [Diagram: the licence split across MIT, BSL 1.1 and Commercial surfaces.] ## MIT: adoption and interop Anything a developer installs to work with Caveman is MIT, and stays MIT inside your commercial product. The skill, the CLI, the TypeScript and Python SDKs, the Agent SDK and its initialiser, cavekit, the eval graders, the extension shell, the wire contracts, the provider catalog, and the JavaScript and Python clients for cavemem. ## BSL 1.1: the engine-linked runtime The compression engine and everything that links it: the cavemem core, the caveman-shrink core, the MCP server core, caveman-browse, and the shared platform libraries. BSL 1.1 is source available, not open source. The distinction is one paragraph long. ### What you may do Read the source, modify it, run it. Internal evaluation, local development, CI, integration work, and self-hosted use for your own first-party traffic, production included. ### What needs a commercial licence Offering Caveman or its functionality to third parties as a hosted, managed or embedded service. That is the OEM and platform boundary, and it is the only restriction. ### When it becomes Apache 2.0 Each BSL version converts on the earlier of 21 June 2030, or the fourth anniversary of that version's first public distribution under BSL. The conversion is per release, so an old version converts before a new one. ## Commercial: Caveman Cloud Multi-tenancy, verified accounting, governance, billing and the hosted dashboards. These run on our machines under an account and are not distributed. ## Mixed directories Two packages ship an MIT launcher around a BSL binary: the MCP server and caveman-shrink. In those directories the launcher terms are in `LICENSE.launcher`, and the Go source plus downloaded-binary terms are in `LICENSE` and `BINARY_LICENSE.md`. ## The default for new code One sentence, quoted because the wording is the rule: > A new module that imports, links, embeds, or ships as part of Engine-linked runtime is BSL 1.1 unless a > later ADR explicitly classifies it as MIT adoption surface. ## Contributing In MIT areas, inbound equals outbound. Your contribution is MIT. In BSL areas, contributions need a DCO sign-off and a relicense grant to Julius Brussee. That grant is what makes the eventual Apache 2.0 conversion possible, since a licence cannot be changed on code the project does not have the rights to change. ## Third-party code The pixel renderer is a Go port of pxpipe, which is MIT, and uses glyph atlases from Spleen 5x8 (BSD 2-Clause) and GNU Unifont (OFL 1.1, or GPLv2 with the font embedding exception). caveman-browse vendors chromedp, which is MIT. Notices ship next to the code in each case. ## Trademarks "Caveman" and the Caveman logos are trademarks of Julius Brussee. A code licence grants no trademark rights. Nominative use is fine when it is true: "Powered by Caveman" and "Optimized by Caveman" are both allowed if that is what your product does. The authoritative text is `LICENSING.md` and `TRADEMARKS.md` in the repository, alongside the licence files themselves. Where this page and those files disagree, those files win. --- --- title: Telemetry summary: What the CLI sends, what it never sends, and the three ways to turn it off. canonical: https://docs.caveman.so/docs/telemetry license: MIT capability: cli updated: 2026-08-26T03:57:26+02:00 basis: inferred --- # Telemetry > What the CLI sends, what it never sends, and the three ways to turn it off. The CLI sends anonymous usage counts. It is on by default, it is disclosed on the first run that would send anything, and there are three separate ways to turn it off. This page lists the exact contents. Nothing here is a summary of a longer policy: this is the payload. The local tools are free and stay free. Anonymous counts are how we see which commands people actually reach for, which ones fail, and whether anyone comes back after the first day. That is what tells us where to spend the work, and it is the reason the open layers can stay free instead of needing a licence to justify themselves. If you would rather not be counted, the switch is one line and we do not treat it as a loss. ## What is sent Two kinds of event, both counts. **Command events.** One per command you run. - Event name: Which of the known command names ran. - CLI version: The version of @caveman-ai/cli. - Platform: Operating system, architecture, and Node major version. - Duration: How long the command took. - Exit class: Whether it succeeded, and if not, which class of failure. **Engine session events.** One aggregate after a local wrapped session that was eligible for compression. - Counts: Request count and token counts. - Eligibility: How much of the session was eligible for compression. - Before and after: Measured totals and the amount cut. - Cache counters: Hit and miss counts. - Coverage flags: Which parts of the session could be measured at all. - Mode: observe or compress. There is also a one-time `first_run` event carrying scan aggregates. ## What is never sent This list is enforced server side by an allowlist, not by convention on the client. - Prompts, completions, or any model output - Your code, file paths, or URLs - Command arguments - Provider or model names - Request, session or account identifiers - Payload hashes - Any dollar figure The token figures in a session event are local `inferred` measurements. They cannot become a verified saving, because there is no account attached to them and no bill to compare against. ## The anonymous id The first interactive run of a real command writes a stable anonymous id to your config, so a returning user is not counted as a new one. It is not linked to an account, and there is nothing on our side that can turn it back into a person. Automation never mints one. CI and non-interactive runs default to off and do not persist a decision, because generating an anonymous identity for a build machine would corrupt the counts it is meant to inform. ## Turning it off Any one of these is enough. ```bash caveman telemetry off ``` ```bash ``` ```bash ``` `DO_NOT_TRACK` is the cross-tool convention and it is honoured here without needing to know that Caveman exists. Setting `CAVEMAN_TELEMETRY=1` turns it on for a single automated run without persisting a decision. ## If you already answered the old prompt Earlier versions asked a yes or no question on first run. Whichever way you answered, that answer is kept and is never rewritten by the current default. Opting out once means opting out permanently. ## The disclosure line On the first interactive run that would send anything, the CLI prints this and does not hide it behind a flag: ```text anonymous usage stats on — command + aggregate engine counts only, never prompts, code, or file paths · caveman telemetry off ``` ## What is deliberately outside this Agent SDK economic receipts contain a per-call spend tree. They are never sent to this endpoint. Moving them anywhere would need an authenticated, account-scoped transport with explicit consent, and that is a different decision from this one. Download counts come from the npm registry. They are not collected by the client. --- --- title: Eval graders summary: The grader set used to check that compression did not change an answer. canonical: https://docs.caveman.so/docs/evals license: MIT capability: evals updated: 2026-08-26T04:05:35+02:00 basis: inferred --- # Eval graders > The grader set used to check that compression did not change an answer. `@caveman/evals` is the public fail-closed grader package. Each grader takes one candidate and returns `{ passed, reason }`. Unknown types, invalid options, missing measurements, and unsafe execution paths fail. - Package: `@caveman/evals` - Runtime: Node.js with no production package dependencies. - Result: `{'{ passed: boolean, reason: string }'}` - Unknown grader: `passed: false` - Licence: MIT. ## Build from source No public registry install is documented for this package. Build the workspace package from source: ```bash git clone https://github.com/JuliusBrussee/caveman cd caveman pnpm install --frozen-lockfile pnpm --filter @caveman/evals build ``` ## Grade a value ```ts const result = await grade( { type: "exact_match", expected: "ready" }, "READY", ); if (!result.passed) { throw new Error(result.reason); } ``` Exact match is case-insensitive by default and compares structured values with sorted object keys. Optional `case_sensitive` and string-only `remove_punctuation` knobs change that behaviour. Invalid knob types fail. ## Deterministic graders Text and structure: - `exact_match`, `contains`, `not_contains` - `regex`, `not_regex`, `blocklist` - `json_schema`, `json_path_assertion` - `bleu_score`, `rouge_score`, `context_f1` - `localization_f1`, `no_pii` Tool and request evidence: - `tool_called`, `tool_not_called`, `tool_sequence` - `tool_argument_assertion` - `http_status`, `latency_threshold`, `cost_threshold`, `token_threshold` Threshold graders require a present, finite, non-negative measurement. An absent cost, token count, or latency cannot pass a ceiling by being treated as zero. The JSON Schema grader implements a documented subset: primitive type, enum, required keys, properties, and items. An unknown schema type fails rather than being ignored. ## Network graders `custom_webhook` posts `{ candidate }` and requires a JSON response with `passed`. Judge graders are `llm_judge`, `llm_score`, `llm_category`, `llm_pairwise`, and `llm_answer_match`. Network calls share a default ten-second deadline and refuse redirects. Private, loopback, link-local, and documentation IP ranges are blocked. A hostname requires an injected transport that pins the checked DNS result through connection setup; a normal fetch wrapper cannot claim that property. Model judges also fail when configured judge and subject model share a known family. A model grading its own family is a bias risk, not independent evidence. ## Regex limits Patterns are capped at 1,024 characters and candidates at 64 KiB. Backreferences and risky quantified groups are rejected. Accepted patterns run in a worker with a 250 ms execution budget. These limits choose a failed grade over a process hang. They do not truncate the candidate and pretend the partial result is complete. ## Localization evidence `localization_f1` accepts compact `path:start-end` lines or a structured file-to-ranges map. It scores both cited files and line overlap. Empty, unparseable, or zero-quality candidates fail even when a threshold is explicitly zero. This makes it suitable for checking repository explorer output without treating a correct filename and wrong lines as a full match. ## Run engine fixtures The engine bundles a local fixture harness around the grader contract: ```bash caveman tools evals run caveman tools evals run --fixtures ./fixtures ``` Caller-supplied fixture paths are confined to the supplied directory. A failing grader makes the command exit non-zero. Exact match proves normalized equality. A token threshold proves a reported token field stayed under a ceiling. Neither proves general answer quality. Choose graders that cover the failure modes of the task and keep holdout cases separate from development cases. --- --- title: Provider catalog summary: The model price table every cost figure is read from. canonical: https://docs.caveman.so/docs/provider-catalog license: MIT capability: provider-catalog updated: 2026-08-26T04:05:35+02:00 basis: inferred --- # Provider catalog > The model price table every cost figure is read from. The provider catalog is the public, source-backed table of model prices and operational facts. Cost calculations read a specific catalog version instead of scattering model prices through application code. - Package: `@caveman/provider-catalog` - Source: `catalog/current.yaml` - Snapshots: Immutable files named by price verification date. - Currency: Declared per row, currently USD rows. - Licence: MIT. ## What a row says One row identifies provider, model, region, currency, prices, capabilities, source links, and verification times. Typed fields may also describe lifecycle, aliases, historical price intervals, tokenizer support, count surface, and cache behaviour. Support facts use three states: | State | Meaning | | --- | --- | | `supported` | Reviewed source says the capability exists. | | `unsupported` | Reviewed source says it does not exist. | | `unknown` | No reviewed fact is available. | Unknown is not false and not permission to guess. ## Read generated data Build consumers use generated JSON rather than parsing YAML on a request path: ```text generated/catalog.json generated/manifest.json ``` The catalog carries a semantic digest, price-provenance digest, source digest, and content digest per entry. The manifest records snapshot digests and entry counts. These hashes detect drift; they do not prove that a provider served a request or charged a particular invoice. Use the generated manifest to read current counts. Do not copy a fixed model or provider count into dependent code, because rows change as providers publish and retire models. ## Unknown prices fail closed An unknown provider, model, or region combination is zero-priced with `unpriced:` provenance. Callers must not borrow a nearby model's price or assume two regions match. Zero in this case means no supported price entered the calculation. It does not mean the model is free. Any total containing an unpriced row is incomplete. Surface the `unpriced:` tag next to the total so a reader can distinguish missing price data from a real zero rate. ## Price provenance `verified_at` is an RFC3339 timestamp tied to price sources. Current rows may not be future-dated or older than 120 days. Every current row must match an immutable `catalog/YYYY-MM-DD.yaml` snapshot selected by its verification date. A price change adds a new dated file. It never rewrites an old snapshot. Sources use provider-owned HTTPS pricing or model documentation. Automated proposals carry a review marker and cannot pass default validation until a person confirms each changed row against its sources and removes the marker. Capability provenance has its own timestamp. Updating a non-price capability does not pretend that price was re-verified on that day. ## Validate a change From the catalog package directory: ```bash python3 -m pip install -r requirements-dev.txt python3 validate_catalog.py python3 -m unittest discover -s tests -p 'test_*.py' ``` From repository root: ```bash pnpm --filter @caveman/provider-catalog build pnpm --filter @caveman/provider-catalog lint pnpm --filter @caveman/provider-catalog test ``` Validation rejects duplicate YAML keys, duplicate provider/model/region identities, unsafe identifiers, invalid rates, stale provenance, broken aliases, unavailable replacements, unknown fields, and price changes without a matching snapshot. ## Lifecycle and routing Lifecycle distinguishes discovery, review, routing eligibility, and retirement. A reviewed price row is not automatically eligible for routing. Routing needs its own capability and evidence gates outside this data file. Aliases stay provider-local, may not collide with canonical model ids, may not form cycles, and may not resolve to a retired or missing model. ## What this catalog does not prove The table can support list-price accounting from observed usage. It does not turn inferred local token reductions into saved money, prove a contract rate, or compare against an invoice. Those claims need observed traffic and the right accounting authority.