HUB-049: marginal-gain evidence packing behind a flag (default off)

[funsaized]

Two commits: a correction to the HUB-044 record, then the packer.

The finding

Raising REPORT_RETRIEVAL_CANDIDATES 120 → 600 lifts pool source reach a long way, and the packed evidence the model reads does not move at all:

Job (sources)	pool @120 → @600	packed @120 → @600
p-hacking (19)	8 → 15	6 → 7
kafka (20)	13 → 20	8 → 8
microservices (22)	15 → 21	8 → 8
kubernetes (56)	39 → 52	14 → 15

The context budget, not retrieval, decides what a report sees. Every retrieval-side lever has been invisible downstream.

Correction to HUB-044 (first commit)

HUB-044 recorded that the backlog's motivating case no longer reproduced (44 chunks from 15 of 22 vs the original 15 from 7). That was the retrieval stage. At the stage the original observation was made — packed evidence — bc3e5297 is 15 chunks from 8 of 22. The case was never stale; the coverage metric is correct but stops one stage short.

The packer

pack_by_marginal_gain selects by relevance minus redundancy against what is already packed — arXiv:2607.00725 for the framing, arXiv:2512.25052 for the training-free greedy rule. Two conservative departures, documented in the code:

• Redundancy is lexical, not embedding cosine. Packed text is a propositional span rewritten out of a chunk, so the stored chunk vector no longer describes it. Known cost: paraphrase is invisible.
• λ is budget pressure, not a tuned constant. A budget that admits every candidate makes the two packers identical — no knob, no behaviour change without scarcity.

Default rank. Flag threaded through config, both compose services, and the synthesis completion log.

Measured effect: small, reported as measured

Three of six jobs unchanged (p-hacking identical by construction — all candidates fit). Three move 1–2 chunks and +1 source (redis 13→14, postgres 10→11, kubernetes 14→15). The weight was not tuned upward to produce a better number. Either lexical overlap misses paraphrase redundancy, or redundancy isn't the binding problem — on kubernetes-56 the 50 candidates that never pack are dropped by budget, not similarity.

Not adopted. Nothing available says whether the swapped chunks are better. HUB-047 is promoted to the gate on this work.

Rejected alternatives, from the same prior-art pass (16 papers)

• Raising the context window — arXiv:2410.05983: more passages helps, then hurts.
• Packing for source coverage — arXiv:2603.22633: large coverage gains worth ~0.01 F1.

Verification

• 405 tests + 573 subtests green in-container (18 new).
• docker compose config --quiet clean.
• Default path unchanged; A/B run read-only against live Qdrant/Ollama with an immutable SQLite mount.
• No deployed container recreated. No metered judge calls.

🤖 Generated with Claude Code