Cap explanation length by bullets and sentences, not words

[mattgodbolt]

Summary

One sentence changes in app/prompt.yaml: the length instruction goes from "under roughly 250 words" to a hard cap on bullets and sentences. Output length is nearly all of the latency (a Sonnet 5.5 call takes about 0.4s plus 7.7ms per output token), and the word limit was being ignored: explanations averaged 423 words on the hard corpus.

-Keep the whole explanation under roughly 250 words; only exceed that when the assembly is unusually long or complex, and even then stay brief.
+Hard limit: at most 6 bullets or short paragraphs in total, none longer than two sentences, and no headings. Only when the assembly is unusually long or complex may you use up to 10. Lead with the single most important insight, and omit anything the reader could infer from the instruction names alone.

Anthropic's latency guide notes that word limits work worse than sentence or paragraph limits, which matches what we saw.

Eval

Exactly the text above (nothing else changed), 2 runs of the full suite, Opus 5 adaptive reviewer, against 2 runs of current main on the same corpora. This depends on the fixture and reviewer fixes in #54 (the runs used them), but the prompt change itself does not.

	current	this PR
Mean latency	8.2s	6.5s (-21%)
p95 / max latency	12.2s / 17.7s	9.0s / 9.9s
Output tokens / words	1002 / 423	791 / 335
$/call	0.0172	0.0151
Correct, original corpus	30/32 (94%)	27/32 (84%)
Correct, hard corpus	50/62 (81%)	58/62 (94%)
Correct, both pooled	80/94 (85.1%)	85/94 (90.4%) (Fisher p=0.37)

The two corpora move in opposite directions, which is what reviewer noise looks like here (two prompts differing only in JSON formatting scored 81% and 90% on the same hard-case reviews). Adding a second run of the same prompt edit with compacted input gives 165/188 (87.8%) vs 166/188 (88.3%) for baseline, p=1.0. So: no detectable change in accuracy, a clear and low-noise latency gain. The errors on the original corpus are the same kind as baseline's and spread over different cases (an imul operand form, a clang peel count, the edge_undefined_001 UB case that fails for both), not a pattern of dropped content.

Before / after on basic_loop_001 (beginner, gcc -O2 loop)

Before (333 words, 5.7s): headed sections "Calling convention", "Setup (before the loop)", "The loop (.L3)", "Why this is efficient", then an -O0/-O3 comparison.

After (279 words, 5.3s):

• The loop counter i is gone. With -O2, GCC replaces the index with a pointer that walks through the array. rdi (the first argument, arr, by x86-64 convention) is advanced by 4 bytes each iteration, and the loop stops when it reaches an "end" address held in rdx.
• Setup computes that end address. movsx rsi, esi widens the 32-bit n ... Then lea rdx, [rdi+rsi*4] calculates arr + n*4. lea only does the arithmetic; it does not read memory.
• The loop body is just four instructions (.L3). ...
• xor eax, eax is the idiom for sum = 0. ...
• The n <= 0 case is handled up front. ...
• Comparison with -O0: unoptimised code would keep sum, i and the pointer in stack memory ... Here everything lives in registers.

Things to know

• It removes the headed sections (the beginner "Calling convention" heading, for instance) in favour of bullets; the content is still there inline. If you prefer headings for beginners, an audience-specific override is possible, at some latency cost.
• Reviewer is Opus 5, which judges correctness and completeness, not pedagogy. Worth reading a few outputs.
• It also reduces cost slightly (shorter outputs).

Testing

• uv run pytest: 113 passed. uv run pre-commit run --all-files: clean.

🤖 Generated with Claude Code