Terminal output

The terminal compressor handles raw command output: it strips ANSI control sequences, collapses each progress bar to the last frame it drew, and folds the repetitive middle while keeping the head, the tail and every error, traceback and warning line. It is what caveman shrink runs a wrapped command's output through.

Before and after#

The fixture is 124 lines of build output, 9,741 bytes: colour escapes on every status line, ten progress bars redrawing in place with carriage returns, one compiler error.

Four lines of it, from the start, the middle and the failure, printed through cat -v so the escapes are visible: ^[ is the escape byte and ^M is a carriage return. The progress line is one physical line, truncated here.

^[[1m$ npm run build^[[0m
[#.........] 10% bundling module-01^M[##........] 20% bundling module-01^M[###.......] 30% bundling module-01^M…
^[[32m✓^[[0m compiled src/routes/route_00.tsx in 12ms
^[[31mERROR^[[0m src/routes/route_23.tsx:14:7: TypeError: Cannot read properties of undefined (reading 'id')

curl -O https://docs.caveman.so/examples/compressors/terminal/build.log
caveman-engine compress < build.log 2> report.json

The whole output, 9 lines:

$ npm run build
[##########] 100% bundling module-01
[##########] 100% bundling module-02
… 68 lines elided (caveman) …
ERROR src/routes/route_23.tsx:14:7: TypeError: Cannot read properties of undefined (reading 'id')
… 49 lines elided (caveman) …
✓ compiled src/routes/route_109.tsx in 13ms
build finished with 1 error
exit status 1

The report on stderr, with the per-run digests and the handle trimmed to …:

{"content_type":"terminal","tokens_before":3888,"tokens_after":109,"ratio":0.9719650205761317,
 "basis":"inferred","recovery_handle":"ccr_8f3f…","method":"terminal"}

Captured from a caveman-engine build dated 2026-08-24; caveman setup --install today pins bin-v1.1.7.

3,888 tokens to 109, counted by the offline o200k_base counter behind Token counting.

How it works#

Empty input and input that is not valid UTF-8 are reported as a parse problem and the caller forwards the original bytes.

Lines are split on LF and each line keeps its own CR, so a document with mixed endings rejoins byte for byte. That split happens before anything else, which is why a CRLF break is never read as a redraw. Each line is then cleaned in two passes:

1. 1

   Carriage returns inside the line collapse to the last non-empty segment. A bare CR rewinds the cursor to the start of the line, so 10%\r20%\r100% done was only ever showing 100% done and the earlier frames are display history.

2. 2

   ANSI and VT control sequences are removed: CSI sequences (ESC [ … final byte), OSC sequences (ESC ] … BEL or ST), and the standalone two-byte Fe escapes.

A payload of fewer than 4 lines stops there, because the cleanup alone may already have made it smaller. Longer output goes on to the same head, tail and importance pass the log compressor uses.

• The first 3 and the last 3 lines are kept.
• A line is kept when it matches the importance pattern: ERROR, ERR, FATAL, PANIC, EXCEPTION, TRACEBACK, FAIL, FAILED, FAILURE, WARN or WARNING at a word boundary, case insensitive; or a camelCase type ending in Error or Exception, which is how AssertionError and NullPointerException are caught; or an indented at frame, a File " line, a .go: line number, a :line:column: compiler location, a leading --->, the words caused by, or exit status with a non-zero code.
• A marker from an earlier pass is kept, which keeps a second pass identical to the first.
• The shared redundancy guard adds one representative of every distinct kind of line, comparing each line by its set of words with digit runs masked to #, so a hundred status lines that differ only in a file name and a duration are represented by the ones already kept.

Each run of dropped lines becomes one line, … N lines elided (caveman) …, with the line ending the majority of the document uses. If cleanup and elision together changed nothing, the compressor reports that and the original bytes go through.

What is always kept#

The first 3 and last 3 lines. Every line matching the importance pattern, wherever it sits, including stack frames, compiler locations and a non-zero exit status. The final visible state of every progress bar. One representative of every distinct kind of line. Markers from an earlier pass.

When it is chosen#

Detection returns terminal on a raw ANSI escape sequence, which appears in command output and nowhere else. The secondary signal is 3 or more bare carriage returns, counting CRLF pairs out, which is a progress bar redrawing in place.

caveman-engine detect < build.log

terminal

See how detection decides for where this test sits among the others.

A caller can also force the route. caveman shrink -- <command> runs the command, captures stdout and stderr together, and compresses the result as terminal before printing it with a recovery footer. See caveman-shrink.

Detection's floor is 3 bare carriage returns where there is no escape sequence; the compressor takes any non-empty payload. It declines, and the engine forwards the original bytes, on empty or non-UTF-8 input, when cleanup and elision leave the bytes unchanged, and when the result counts no fewer tokens than the input.

Options#

None. NewTerminal fixes 3 head lines and 3 tail lines, and there is no query path.

Recovery#

The byte-exact original, escapes and progress frames included, is stored before the compressed view is emitted, and the handle in the report retrieves it: see Recovery.