---
title: Vercel AI SDK
summary: The one line diff for streamText, and how to see that it applied.
canonical: https://docs.caveman.so/docs/sdk/middleware/vercel-ai-sdk
license: MIT
capability: sdk-ts
updated: 2026-09-16T21:32:40-07:00
basis: inferred
---

# Vercel AI SDK

> The one line diff for streamText, and how to see that it applied.
`withCaveman` wraps the options object you already pass to `streamText`, `generateText` or a `ToolLoopAgent`. It returns a new object with the model wrapped and a `caveman_retrieve` tool added, so eligible tool results leave smaller and the model can read the original back.

```diff
-const result = streamText(options);
+const result = streamText(withCaveman(options, { runtime, scope }));
```

[Middleware](/docs/sdk/middleware) has the packages, the runtime and the `{ runtime, scope }` pair this line takes. `ai` must be 7.0.94 or newer below 8, and `@ai-sdk/provider` 4.0.11 or newer below 5. Outside that range the adapter reports `unsupported_version` and changes nothing.

## The complete example

This is the file the docs test runs against a real runtime and a local fixture provider. It is executable as written, and adds `@ai-sdk/openai` to the packages the middleware page installs.

[Download demo.mts](/examples/vercel-ai-sdk/demo.mts)

```ts
import { randomUUID } from "node:crypto";
import { createOpenAI } from "@ai-sdk/openai";
import { jsonSchema, stepCountIs, streamText, tool } from "ai";
import { createMiddlewareRuntime } from "@caveman-ai/sdk/middleware";
import { withCaveman } from "@caveman-ai/middleware/ai-sdk";

const mode = process.env.CAVE_MODE ?? "record";
if (mode !== "off" && mode !== "record" && mode !== "compress") {
  throw new Error("CAVE_MODE must be off, record, or compress");
}
if (!process.env.OPENAI_API_KEY || !process.env.CAVE_MODEL) {
  throw new Error("Set OPENAI_API_KEY and CAVE_MODEL before running");
}
const runtime = createMiddlewareRuntime({
  endpoint: process.env.CAVE_RUNTIME_URL ?? "http://127.0.0.1:8787",
  mode,
  deadlineMs: 2000,
  onReport: report => console.error("Caveman:", JSON.stringify(report)),
});
const openai = createOpenAI({ baseURL: process.env.OPENAI_BASE_URL });
const deploymentLog = Array.from({ length: 400 }, (_, i) =>
  `2026-09-16T10:00:00Z INFO worker=${i} health check passed`,
).join("\n") + "\n2026-09-16T10:01:00Z ERROR deployment failed: missing DATABASE_URL";
const scope = {
  namespace: "vercel-ai-sdk-demo",
  session_id: randomUUID(),
  branch_id: "main",
  cache_epoch: "0",
};

try {
  if (mode !== "off") await runtime.ready();
  const options = {
    model: openai.chat(process.env.CAVE_MODEL),
    prompt: "Read the deployment log. Explain the failure in one sentence. Recover the original if needed.",
    tools: {
      readDeploymentLog: tool({
        description: "Read the deployment log.",
        inputSchema: jsonSchema<Record<string, never>>({
          type: "object", properties: {}, additionalProperties: false,
        }),
        execute: async () => deploymentLog,
      }),
    },
    stopWhen: stepCountIs(5),
  };
  const result = streamText(withCaveman(options, { runtime, scope }));
  for await (const chunk of result.textStream) process.stdout.write(chunk);
  process.stdout.write("\n");
  // The native tool result remains the original, even if its outbound copy shrinks.
  const source = (await result.steps).flatMap(step => step.toolResults)
    .find(part => part.toolName === "readDeploymentLog");
  if (!source || source.output !== deploymentLog) {
    throw new Error("The example did not preserve its original tool result");
  }
  console.error("Original tool result preserved.");
  console.error("Provider usage:", JSON.stringify(await result.totalUsage));
} finally {
  runtime.close(); // The stream and any recovery calls have finished.
}
```

Keep `options.tools` as `withCaveman` returned it. Replacing that table after the wrap removes `caveman_retrieve`, and the model then has no way back to the original.

## Run it

`verify.mjs` runs `demo.mts` three times, in `off`, `record` and `compress`, against a local fixture server that speaks the OpenAI streaming wire format. It asserts that the outbound tool result shrank, that the model's `caveman_retrieve` call returned the original byte for byte, and that the application's own tool result was never rewritten. No provider call, no spend.

[Download verify.mjs](/examples/vercel-ai-sdk/verify.mjs)

Start the local proxy in compress mode.

```bash
CAVEMAN_MODE=compress caveman start
```

Run all three modes.

```bash
npm install && node verify.mjs
```

```console
PASS off: disabled; original tool result preserved; streaming completed.
PASS record: recorded; original tool result preserved; streaming completed.
PASS compress: applied; original tool result preserved; streaming completed; exact recovery passed.
Local mechanism proof only. Provider responses are fixtures; no live model quality or billing claim.
```

Point it at your own provider. This one bills you for the model calls.

```bash
OPENAI_API_KEY=sk-… CAVE_MODEL=gpt-5.5 CAVE_MODE=compress node --experimental-strip-types demo.mts
```

## What the reports say

`demo.mts` prints one report per provider call on stderr. These are the lines from the `record` and `compress` runs above, with the per-run ids cut:

```console
record:
Caveman: {"schema_version":1,"status":"skipped","reason":"no_candidate","transform_ids":[],"replacement_count":0,"reused_count":0,"adapter":"ai-sdk",…}
Caveman: {"schema_version":1,"status":"recorded","reason":"record","transform_ids":[],"replacement_count":0,"reused_count":0,"adapter":"ai-sdk",…}

compress:
Caveman: {"schema_version":1,"status":"skipped","reason":"no_candidate","transform_ids":[],"replacement_count":0,"reused_count":0,"adapter":"ai-sdk",…}
Caveman: {"schema_version":1,"status":"applied","reason":"eligible","transform_ids":["caveman.engine.log.v1"],"replacement_count":1,"reused_count":0,"adapter":"ai-sdk",…}
Caveman: {"schema_version":1,"status":"reused","reason":"eligible","transform_ids":["caveman.engine.log.v1"],"replacement_count":1,"reused_count":1,"adapter":"ai-sdk",…}
```

The first call of every run is `no_candidate`: there is no tool result yet. Compression starts on the call after the first tool returns. The third call in compress mode is `reused`, because the replacement was frozen on the previous turn and resending the identical bytes keeps the provider's cached prefix warm.

## Symptoms

| What you see | Status and reason | Fix |
|---|---|---|
| Nothing shrank, one report per call | `skipped` / `no_candidate` | The call carried no tool result. Compression starts on the turn after the first tool returns |
| Nothing shrank at all | `skipped` / `runtime_unavailable` | The runtime is not on `endpoint`. Check `caveman start` and the URL |
| Reports say `recorded` | `recorded` | The runtime is in record mode. Start it with `CAVEMAN_MODE=compress` |
| Reports stop after three failures | `skipped` / `circuit_open` | Three consecutive failures open the circuit for 30 seconds. Fix the runtime, then wait |
| Reports say the framework is wrong | `skipped` / `unsupported_version` | `ai` or `@ai-sdk/provider` is outside the accepted range. `npm ls ai @ai-sdk/provider` prints what is installed; [Supported frameworks](/docs/sdk/middleware/frameworks) has the bounds |
| The model never calls `caveman_retrieve` | `applied`, no recovery call | The tools table was replaced after the wrap, or `stopWhen` stops before the extra step. Raise `stepCountIs` |
| Nothing shrank, runtime reachable | `skipped` / `recovery_unavailable` | The runtime cannot store originals, so it refuses to shorten one. Check that its recovery storage is writable |

`runtime.ready()` fetches the runtime's capabilities before the first model call, so `mode`, `persistent` and `recovery` answer the first three rows on their own.