In brief
CAVEWOMAN separates shrinking the question from shrinking the answer. Its reported cost results make a useful case for measuring the whole request, while its fidelity checks show what a correct final answer can conceal.
Originally published on Jack Gross.
CAVEWOMAN separates shrinking the question from shrinking the answer. Its reported cost results make a useful case for measuring the whole request, while its fidelity checks show what a correct final answer can conceal.
The two sides of the bill
Deleting small words from a question feels economical. The model can then spend the saved input tokens on a longer reply. CAVEWOMAN measures those channels separately: it filters the input while keeping the system prompt fixed, or leaves the question intact and constrains the response. Adobe lists the work on October 29; the linked arXiv paper dates to June 23, 2026. This is a fresh reading of existing research, not a claim that the method launched today.
What the authors report
The paper describes eight models, five benchmarks and five compression levels. Its abstract reports output-compression cost reductions of 1.4-2.4 times per model on most API models, with a best case of three times. Input compression raised net cost by about 1.15 times on the five-benchmark mean in that setup. These are the authors' results under their pricing and tasks, not savings I reproduced or a forecast for another model. The repository contains broader scale counts that differ from the paper summary, so I would pin a revision before using its totals.
An accurate answer can still change meaning
The protocol also checks agreement with each model's unconstrained reference answer. The authors report that roughly half of generations on non-reasoning models can be correct while their surface text no longer entails that reference. This measures agreement between texts. It does not reveal hidden reasoning or prove that the original answer was true. For a business summary, losing a condition or exception may matter even when a final numeric answer survives.
My proposed local comparison
I would keep the original question intact and compare an unconstrained answer with one concise-answer instruction on the same fixed cases. Record billed input and output tokens, model settings, retries, correctness and preservation of required conditions. Include failures in the denominator. The illustrative Python below computes cost from supplied token counts and per-million rates. It makes no API call, uses fictional numbers and is not a reproduction of the paper.
# Fictional token counts; supplied rates per million tokens.
def cost(input_tokens, output_tokens, input_rate, output_rate):
return (input_tokens * input_rate + output_tokens * output_rate) / 1_000_000
baseline = cost(100, 300, 1, 4)
short_input = cost(80, 340, 1, 4)
short_output = cost(100, 180, 1, 4)
assert short_input > baseline > short_outputWhere I would stop
A lower token count would not pass my test if a caveat, unit or required action disappeared. The repository says its main findings concern single-turn tasks with short extractable answers and direct per-token billing; long-form work behaves differently. Subscription limits, cached input, reasoning tokens and tool charges need their own accounting when applicable. A compact reply can be useful, but I would select a length only after comparing both cost and preserved information on the actual workload.