--- title: Config files summary: YAML, TOML, and INI with defaults and comments folded. canonical: https://docs.caveman.so/docs/proxy/compressors/config layer: proxy license: BSL-1.1 capability: engine updated: 2026-09-16T21:32:40-07:00 basis: inferred --- # Config files > YAML, TOML, and INI with defaults and comments folded. The config compressor handles YAML, TOML, and INI. It keeps every top-level YAML key, every line naming a secret or a failure, the head and tail of the file, and the parents and children of anything it kept, so the YAML it emits still parses. Dropped lines become one comment line stating how many went. ## Before and after The fixture is a 159 line service manifest: a `service` block with a database section, fifty near-identical `feature_NN` blocks, and an `incident` block at the end. ```yaml service: name: checkout owner: payments database: host: db.internal port: 5432 feature_00: enabled: true rollout: 0 ``` ```bash curl -O https://docs.caveman.so/examples/compressors/config/service.yaml caveman-engine compress < service.yaml 2> report.json ``` Stdout is the whole compressed file, 16 lines. This is all of it. ```yaml service: name: checkout owner: payments database: host: db.internal port: 5432 feature_00: enabled: true rollout: 0 feature_01: enabled: false rollout: 1 # … 144 config lines elided (caveman) … incident: status: ERROR action: rollback ``` The report on stderr carries the counts. The handle varies per run and per machine, so it is trimmed with `…` here. ```json {"content_type":"config","tokens_before":841,"tokens_after":87,"ratio":0.896551724137931, "basis":"inferred","recovery_handle":"ccr_d83c…","method":"config","lossless_to_model":false} ``` Captured from a caveman-engine build dated 2026-08-24; `caveman setup --install` today pins bin-v1.1.7. 841 tokens to 87, counted by the offline `o200k_base` counter described on [Token counting](/docs/proxy/tokens). The `incident` block survives because `ERROR` and `ROLLBACK` are both on the important-word list, and `incident:` itself survives as their parent. TOML and INI take the same path. A 130 line file of twenty `[pool.shard_NN]` sections went from 651 tokens to 95, keeping `[server]`, the first shard section in full, and `[audit]` with its `on_write_failure = ROLLBACK` line. ```ini [server] host = 0.0.0.0 port = 8080 workers = 8 timeout_ms = 30000 [pool.shard_00] driver = postgres max_conns = 40 idle_conns = 8 weight = 1 # … 114 config lines elided (caveman) … [audit] retention_days = 365 on_write_failure = ROLLBACK ``` ## How it works `parseConfig` classifies the payload first. It walks every line, skipping blanks and lines starting with `#` or `;`, and counts four kinds: | Kind | Pattern | | --- | --- | | Section | `^\s*\[[A-Za-z0-9_.:"' -]+\]\s*(?:[#;].*)?$` | | Assignment | `^\s*[A-Za-z0-9_.-]+\s*=\s*\S` | | YAML key | `^\s*(?:-\s*)?[A-Za-z0-9_.-]+\s*:\s*` | | Prose | anything else that is not a `- ` sequence item | Those four counts added together are the file's significant lines. A blank line, a line starting with `#` or `;`, and a `- ` sequence item are none of the four and count towards none of them. The payload is TOML or INI when it has at least 2 sections, at least 4 assignments, zero prose lines, and no `"""` or `'''` anywhere. It is YAML when it has at least 6 YAML keys, those keys are at least half of all significant lines, prose is at most a quarter of them, no line ends in a `|` or `>` block scalar header, and `yaml.v3` decodes the whole payload. Anything else is not config and the compressor declines. A classified file of at least 14 lines is then marked line by line: The first 3 and the last 2 lines. Any line matching `ERROR|FAIL|FAILED|FATAL|PANIC|EXCEPTION|WARNING|SECURITY|SECRET|TOKEN|PASSWORD|DENIED|REJECTED|ROLLBACK`, case insensitive, on a word boundary. In YAML, every line at indentation 0 that is a key. A tab counts as two columns. Query relevance, when a query is supplied. Deterministic BM25 scores the lines, anything at or above 0.30 of the best score becomes a candidate, and candidates fill up to 16 extra lines in score order. One representative of every distinct kind of line, added by the shared redundancy guard, so a dropped line always resembles a line still visible. Then the structure around the kept lines is restored, which is what keeps the output parseable. In YAML, each kept line walks backwards for the nearest earlier line at a smaller indentation that is a parent (a key ending in `:`, or a `- ` item), keeps it, and repeats from that line's indentation up to column 0. A second pass then walks forward from every kept parent and keeps up to 8 following lines that are indented deeper than it, stopping at the first line indented the same or less. That second pass is why a query that matches one nested value brings its sibling values with it. In TOML and INI, every kept line pulls in its enclosing `[section]` header, and every kept section header pulls in up to 8 following lines, stopping at the next section header. Each run of dropped lines becomes one comment line, `# … N config lines elided (caveman) …`, carrying the same line ending as the file. The output is rejected if it is no shorter than the input, if it kept every line, or, for YAML, if `yaml.v3` no longer decodes it. ## What is always kept The first three and last two lines. Every top-level YAML key. Every line naming an error, a failure, a secret, a token, a password, or a rollback. The parent chain above every kept line and up to 8 lines under every kept parent or section. One line of each distinct shape. YAML output always parses; it is checked before the result is returned. ## When it is chosen `LooksConfig` runs the same classifier and returns true when it succeeds and the file has at least 12 lines. See [how detection decides](/docs/proxy/compressors#how-detection-decides) for where this test sits among the others. ```bash caveman-engine detect < service.yaml ``` ```text config ``` The compressor declines, and the engine forwards the original bytes, when the payload already contains a `config lines elided (caveman)` marker, when the classifier rejects it, when the file is shorter than 14 lines, when every line was marked to keep, when the result is no shorter, when compressed YAML no longer decodes, and when no recovery store is available. **A file of fewer than 14 lines passes through** `LooksConfig` accepts at 12 lines but the compressor's own floor is 14, so a 12 or 13 line config detects as `config` and then comes back unchanged with ratio 0. ## Options None. `NewConfig` fixes the floor at 14 lines, 3 head lines, 2 tail lines, a query fill of 16 lines, a relevance threshold of 0.30, and the 8 line context window on both expansion passes. ## Recovery The original file is stored before the compressed view is emitted, and the handle in the report retrieves it byte for byte: see [Recovery](/docs/proxy/recoverable).