feat(jme): Phase 4 — auto-prune low-confidence facts when vector ceiling is active - #31
feat(jme): Phase 4 — auto-prune low-confidence facts when vector ceiling is active#31PiotrCoderDroid wants to merge 1 commit into
Conversation
…ing is active When consolidateAll detects embeddingCount >= VECTOR_CEILING_WARN (400), it now runs an auto-prune DELETE before the Haiku extraction, removing expired facts AND facts with confidence < PHASE4_CONFIDENCE_FLOOR (0.5) older than 30 days. This clears marginal facts before they fill the LIMIT 500 query window in queryMemory() and displace useful recalls. Changes: - jme.ts: PHASE4_CONFIDENCE_FLOOR=0.5 constant; ConsolidateResult.factsPruned field; auto-prune block in consolidateAll() when ceiling active - scheduler.ts: log includes ', N p4-pruned' tag when Phase 4 fired - jme.test.ts: 2 new Phase 4 tests (36/36 green) Closes JME Phase 4 (pending post-day-14 veredicto decision).
|
Closing without merging. The change is inert against live data — every one of its three conditions matches an empty set today, so merging it would add a destructive Measured on
All 10 rows the predicate would remove come from the Three specific problems, independent of the above:
If the vector ceiling becomes real (~early September at the current rate), the honest fix is to call the Commit preserved as |
|
Closed per operator review — see the analysis above. Not merged; no code from this branch reaches main. |
PR #31 closed unmerged and both branches deleted. Every precondition of the auto-prune matched an empty set: trigger at 400 embeddings vs a live 129; confidence floor 0.5 vs a table minimum of 0.72 (0 rows); 30-day age window vs an oldest fact of 16 days (0 rows). The 0.4 -> 0.5 change it existed to make altered the outcome for exactly zero facts. Same defect class as the §17 6a removal in this same session — a mechanism acting on a population that does not occur. Twice in one day is enough to note the pattern in the queue rather than only in memory. Records the trigger (jme_facts embeddings ~400, ≈ early September) and points at b37712e as the shape to reuse: the FIRST of the two attempts correctly parameterized pruneExpiredFacts(confidenceFloor = 0.4) and called it; the retry discarded that and inlined a duplicate DELETE, and that inferior version is what became the PR. Also notes pruneExpiredFacts() has no production caller — wiring it up is the real task. Patches for both attempts preserved outside the repo at /root/claude-backups/jme-phase4-rejected-2026-08-02/ with provenance. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0186BhSFY9sFFHCPeFnpbvki
…tion
Records the Phase 4 revert alongside the §17 6a removal from earlier the same
day, because they are the same defect and recording them separately would hide
that. Both act on a population that does not occur:
§17 6a — scored "red briefs"; 0 of 20 ruled briefs were pure-red
Phase 4 — prunes confidence < 0.5; table minimum is 0.72, so 0 rows
(and fires at 400 embeddings against a live 129, and needs a
30-day age against an oldest fact of 16 days)
Two independent instances in one day, unrelated subsystems, different authors.
The habit that kills the class costs one query: run the predicate as a
SELECT COUNT(*) with MIN/MAX of the thresholded column before writing the code
that depends on it. If a term returns 0, that is the finding — say so and stop.
Second, unexpected: the retry regressed the design. b37712e (in-container)
correctly parameterized pruneExpiredFacts(confidenceFloor = 0.4) and called it;
after a 900s timeout and error_max_turns, the retry discarded that and inlined
a duplicate DELETE, and the inferior version became PR #31. So a post-exhaustion
retry can ship a WORSE artifact than the run that "failed" — resume from the
artifact, not the prompt. Also queued: pruneExpiredFacts() has no production
caller at all, so wiring it up is the real Phase 4 task.
CORRECTION, kept rather than dropped: I first charged Jarvis with committing
under the operator's identity. Wrong. git_commit deliberately stamps --author as
JARVIS_GH_USER <JARVIS_GH_EMAIL> on jarvis/* branches (git.ts:343-348, defaults
:47-48) so commits attach to the pushing account; b37712e differs only because
the nanoclaw container has no JARVIS_GH_TOKEN. The residue is a tooling gap, now
queued: the author field encodes the EXECUTION PATH, not the actor, so git log
cannot answer "human or Jarvis?" — that must come from the branch prefix or a
trailer.
Cleanup already applied: PR #31 closed unmerged, both branches deleted, worktree
clean, 129 jme_facts intact, both attempts preserved at
/root/claude-backups/jme-phase4-rejected-2026-08-02/.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0186BhSFY9sFFHCPeFnpbvki
JME Phase 4 — Auto-prune on ceiling
Problema
JME utilidad en day-14 (2026-07-27): 13.3% global (gate: ≥39%). Hipótesis: a medida que
jme_factscrece, facts marginales (baja confianza, desactualizados) llenan el LIMIT 500 del query window enqueryMemory(), desplazando facts útiles y degradando el recall.Phase 3 ya medía el ceiling y emitía un warn. Phase 4 actúa sobre él.
Cambios
src/memory/jme.tsPHASE4_CONFIDENCE_FLOOR = 0.5: umbral más agresivo que el baseline 0.4 depruneExpiredFacts(). Solo se usa cuando el ceiling está activo.ConsolidateResult.factsPruned: number: nuevo campo para observabilidad.consolidateAll(): cuandoembeddingCount >= VECTOR_CEILING_WARN (400), ejecuta DELETE (expired +confidence < 0.5+ts > 30d ago) antes de la extracción Haiku. Conteo enresult.factsPruned.src/rituals/scheduler.tsjme-consolidateincluye, N p4-prunedcuando Phase 4 actuó (omitido cuandofactsPruned == 0).src/memory/jme.test.tsfactsPruned >= 400, fact de alta confianza sobrevive.factsPruned == 0.Invariantes
pruneExpiredFacts()standalone no cambia (threshold 0.4 intacto).factsPruned = 0, log sin cambio.Tests
jme.test.ts: 36/36 ✅ (2 tests nuevos de Phase 4)scheduler.test.ts(pm-daily-rebalance) — verificados enmainsin mis cambios, no relacionados con este PR.