---
title: Context packing
summary: "Pack a query's context into a token budget, assemble a stable prefix, and restore checkpoints."
canonical: https://docs.caveman.so/docs/sdk/context
license: MIT
capability: sdk-ts
updated: 2026-09-16T21:32:40-07:00
basis: inferred
---

# Context packing

> Pack a query's context into a token budget, assemble a stable prefix, and restore checkpoints.
`cave.context.pack()` decides which of your context fragments fit a token budget for one query, and names the ones it left out so you can put them back later. `cave.assemble()` decides where the chosen content sits in the request: stable content first, volatile content last, so a provider's cached prefix survives the next turn. Packing calls your configured service; assembly runs entirely in your process.

### TypeScript

```ts
import { Cave } from "@caveman-ai/sdk";

const { CAVE_API_KEY = "", CAVE_BASE_URL = "" } = process.env;
const cave = new Cave({ apiKey: CAVE_API_KEY, baseURL: CAVE_BASE_URL, agent: "support-agent" });

const packed = await cave.context.pack(
  "refund window",
  [
    { id: "turn-1", text: "Customer asked about a refund.", timestamp: "2026-09-16T09:00:00Z" },
    { id: "turn-2", text: "Order 4182 shipped on 3 March.", pin: true },
  ],
  { maxTokens: 800, reserveTokens: 200 },
);

console.log(packed.items.map((i) => i.id), packed.tokensUsed, packed.deferredIds);
```

### Python

```python
import os
from caveman_cloud import Cave, ContextPackItem, ContextPackOptions

cave = Cave(api_key=os.environ["CAVE_API_KEY"], base_url=os.environ["CAVE_BASE_URL"], agent="support-agent")

packed = cave.context.pack(
    "refund window",
    [
        ContextPackItem(id="turn-1", text="Customer asked about a refund.", timestamp="2026-09-16T09:00:00Z"),
        ContextPackItem(id="turn-2", text="Order 4182 shipped on 3 March.", pin=True),
    ],
    ContextPackOptions(max_tokens=800, reserve_tokens=200),
)

print([i.id for i in packed.items], packed.tokens_used, packed.deferred_ids)
```

## An item and a budget

| Item field | Meaning |
| --- | --- |
| `id` | Your own stable id, used to report omissions exactly |
| `text` | The model-visible bytes |
| `tokens` | A count you already have; the service counts when this is absent or zero |
| `timestamp` | RFC 3339, read by the recency signal |
| `priority` | Your own relevance boost |
| `pin` | Required context. A pinned item that cannot fit makes the whole call return everything with zero savings |

`maxTokens` is the only required option and must be a positive integer. `reserveTokens` holds room back for the reply or the next tool call. `now`, `recencyHalfLifeMs`, `recencyWeight` and `errorBoost` tune the scoring; `now` exists so a test can pin the clock.

## What packing returns

`items` are your own item objects, in the order you supplied them, filtered down to the selection. `deferredIds` is the exact set difference, in request order, so re-supplying a dropped fragment is a lookup rather than a diff.

The client verifies the reply before it trusts it: every selected id must be one of yours and appear once, `deferredIds` must match the ids that were dropped exactly, and `tokensUsed + tokensSaved` must equal `tokensBefore`. Anything else, including a transport failure, returns all your items untouched:

```json
{
  "items": [
    { "id": "turn-1", "text": "Customer asked about a refund.", "timestamp": "2026-09-16T09:00:00Z" },
    { "id": "turn-2", "text": "Order 4182 shipped on 3 March.", "pin": true }
  ],
  "tokensUsed": 0,
  "tokensBefore": 0,
  "tokensSaved": 0,
  "deferredCount": 0,
  "deferredIds": [],
  "basis": "inferred"
}
```

That was captured with the client pointed at a closed port. The items come back as the objects you passed in, `timestamp` and `pin` included. Zero `tokensBefore` with your full list back is the signal that packing was a pass-through.

## Assembling a request

`assemble()` takes slots you label `stable`, `session` or `volatile` and builds the provider request body with the stable and session slots above every volatile one. The whole thing runs in your process.

```ts
const assembly = cave.assemble({
  provider: "anthropic",
  model: "claude-sonnet-4-5",
  sessionId: "session-42",
  slots: [
    { id: "policy", stability: "stable", content: "Answer from the handbook only." },
    { id: "question", stability: "volatile", content: "What is the refund window?" },
  ],
});

console.log(assembly.request, assembly.headers, assembly.stableTokens);
```

```python
from caveman_cloud import AssembleOptions, AssemblySlot

assembly = cave.assemble(AssembleOptions(
    provider="anthropic",
    model="claude-sonnet-4-5",
    session_id="session-42",
    slots=[
        AssemblySlot("policy", "stable", "Answer from the handbook only."),
        AssemblySlot("question", "volatile", "What is the refund window?"),
    ],
))

print(assembly.request, assembly.headers, assembly.stable_tokens)
```

Both languages produce the same request and the same hash:

```json
{
  "model": "claude-sonnet-4-5",
  "system": [{ "type": "text", "text": "Answer from the handbook only." }],
  "messages": [{ "role": "user", "content": [{ "type": "text", "text": "What is the refund window?" }] }]
}
```

```console
x-cave-assembly: v1;slots=2;prefix=76f29e54b826;vbb=1
prefixHash: 76f29e54b8260d88a9236254049a621a934062e9702bef143c39d32441de5782
stableTokens: 24  tokenBasis: estimated_bytes_div_4  basis: inferred
```

`provider` shapes the body. `"anthropic"` builds `system` blocks plus one user message, `"openai"` builds a system message followed by user messages, and any other value returns the slots as a labelled list for you to map yourself. A slot with the id `tools` whose content is an array becomes the request's `tools` field.

`stableTokens` counts the prefix bytes divided by four, which is what `tokenBasis: "estimated_bytes_div_4"` says. It is a size estimate for the part you are trying to keep cacheable, not a provider figure. [Token counting](/docs/proxy/tokens) covers the counter behind the engine's own numbers.

### Cache hints

`emitCacheHints` defaults to `"gateway"`, which leaves the cache decision to the service in front of the provider. `"self"` is for calling a provider directly: it marks a breakpoint in the request and names it in `breakpoints`. For OpenAI that is a `prompt_cache_key` derived from the prefix hash; for Anthropic it is `cache_control` on the last tool or system block. `"none"` places nothing.

```json
{
  "model": "gpt-5",
  "messages": [
    { "role": "system", "content": "Answer from the handbook only." },
    { "role": "user", "content": "What is the refund window?" }
  ],
  "prompt_cache_key": "40921c76756dba9da7e5f3cce95295d4"
}
```

### When a stable slot changes

Each `Cave` keeps an in-process hash ledger keyed by `sessionId` and slot id. Re-declaring a `stable` or `session` slot with different content inside the same session raises:

```console
AssemblyStabilityError: assembly slot "policy" changed after being declared stable
```

Python raises the same class with single quotes around the id. The ledger lives in one process, so keeping slot content deterministic across processes is your side of the contract.

## Checkpoints, artifacts and handoff

Three more context surfaces hang off a trace or the client. Each one is a call to your configured service.

| What | TypeScript | Python |
| --- | --- | --- |
| Store a conversation snapshot | `trace.context.checkpoint(messages, options)` | `trace.checkpoint(messages, options)` |
| Read it back | `trace.context.expand(sourceRef)` | `trace.expand(source_ref)` |
| Store a large value, get a stub | `trace.artifacts.page(value, options)` | `trace.page_artifact(value, options)` or `trace.artifacts.page(...)` |
| Fetch the stored value | `trace.artifacts.get(artifactId)` | `trace.get_artifact(artifact_id)` or `trace.artifacts.get(...)` |
| Hand context to another agent | `cave.sharedContext.put(key, content)` | `cave.shared_context.put(key, content)` |
| Read it in that agent | `cave.sharedContext.get(key)` | `cave.shared_context.get(key)` |

`expand()` is the other half of `checkpoint()`: it returns the stored `source_ref`, `version`, `messages` and `checkpoint`, and the TypeScript client throws `cave checkpoint response missing source_ref or messages` if either is absent.

`page()` takes a `source`, a `strategy` of `verbatim`, `json-index`, `text-chunks`, `table-index` or `llm-summary`, and an optional `contentType` and `maxInlineTokens`. `strategy: "verbatim"` returns your value unchanged without storing it. Any other strategy stores the value and returns a stub the model can read:

```text
[cave-artifact id=… source=order-export type=application/json]
summary: artifact stored.
retrieve: authenticated GET /sdk/v1/artifacts/… or trace.artifacts.get("…").
[/cave-artifact]
```

The id in that stub is generated per call, shown here as `…`.

`sharedContext` is keyed by a session key your agents agree on, and the project namespaces the key, so only a peer in the same project can read it.

## Limits

Packing is a connected call. With no service reachable it returns every item with zero savings, which is safe but is also silent: check `tokensBefore` before you report a saving.

Item ids must be unique and non-empty. A duplicate or blank id makes the call return everything rather than guess which fragment you meant.

`assemble()` refuses a slot whose content is not JSON-serializable, a duplicate slot id, an unknown stability, and a missing `model` or `sessionId`, with a message naming the slot.