Skip to content

fix(knowledge): record the text hash on a deduped artifact row - #35417

Merged
RohanK6 merged 1 commit into
mainfrom
crew/conductor/artifact-html-dedup-hash
Nov 11, 2026
Merged

RohanK6 merged 1 commit into
mainfrom
crew/conductor/artifact-html-dedup-hash

Conversation

@RohanK6

@RohanK6 RohanK6 commented Nov 11, 2026

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A deduped html artifact's state row records the hash its claim's items carry, so deleting or editing the artifact releases that claim.

The pre-ingest duplicate gate matches and claims the holder's items by the extracted text's hash and passes it to on_duplicate. ingest_artifact ignored it (lambda _text_hash: ...) and stored the hash of the raw html body instead (#35415). remove_artifact → detach_source_location_by_hash, release_stale_claim and _adopt_reassigned_item all look the row up by that hash, and no item carries it.

Why it matters

Deleting a deduped html artifact leaves the Artifacts source holding the winner's item. When the winner is deleted later, the text moves to the Artifacts source and stays searchable with no artifact behind it, and no row names it, so nothing removes it.

Not a goal

What changed (motivation → approach → change)

  • knowledge/artifact_ingest.py: the on_duplicate finalizer writes the gate's text_hash into the deduped row. For html, text hash and markup hash differ. The other kinds read through _read_text, so for them the two match.
  • docs/system-specs/modules/knowledge.md: the sentence saying the aggregate tables store the text hash now says how that holds for a refused html artifact, and names the test.

Backwards compatibility

Compatible: nothing new is refused or rejected. A deduped html row written before this keeps its markup hash until its next re-ingest (every start, since a deduped row has an empty group), when the gate rewrites it with the text hash. Because that re-ingest compares the stored text hash with the new markup hash, it releases the claim first and the gate then re-claims it in its own transaction. This is the same cost a deduped row already pays: it is re-attempted on every start.

Removes nothing: no field, route or result shape is taken away.

Tests

  • test/test_knowledge_artifact_ingest.py::test_a_deduped_html_artifact_records_the_hash_its_claim_names: ingests an html page into a folder source, then creates an html artifact with the same body. It asserts the duplicate status, that the row's hash equals the holder item's hash, and that remove_artifact releases the claim.
  • Ran: test/test_knowledge_artifact_ingest.py (74 passed), black/isort/flake8/mypy, the black, comment-history and brand gates, the flake-pattern ratchet, the file-line cap, docs-lint.

Proof: not provable -- other: run by hand rather than with prove.py; with artifact_ingest.py reverted to main the new test fails on the hash assert ('f7f463e2799...' != 'e67f6978ee1...'), and it passes on head 865ac5a.

Manual verification

Not run against a live gateway; the test drives the real IngestionPipeline, FileReader and KnowledgeStore.

Related Issues

Fixes #35415

Sibling sweep

Swept: git grep -n "on_duplicate=" -- src/kiro_crew
Sibling: src/kiro_crew/knowledge/artifact_ingest.py -- changed here
Sibling: src/kiro_crew/knowledge/folder_watcher.py -- not affected -- already records the gate's text_hash as known_text_hash
Sibling: src/kiro_crew/knowledge/agent_source.py -- not affected -- always writes .md, read by _read_text, so its stored hash is the text hash

Pattern harvest

Rule candidate: review-prompt
Pattern: a callback parameter the caller names _x and ignores while it stores a different value in the same role

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.

An html artifact is hashed as markup, but the duplicate gate matches
and claims the holder's items by the extracted text's hash. The
deduped row stored the markup hash, so remove_artifact,
release_stale_claim and _adopt_reassigned_item all looked the claim
up by a hash no item carries: deleting the artifact left its claim
on the holder, and a later holder deletion handed the text to the
Artifacts source with nothing behind it.

Record the gate's text_hash, which on_duplicate already passes.

Fixes #35415
@RohanK6
RohanK6 requested review from a team and KiroCrewCaptain as code owners November 11, 2026 16:51
@RohanK6
RohanK6 requested a review from iamwhatever November 11, 2026 16:51
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 11, 2026
@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 865ac5a7088c768154d166f3d62bd5217afd8bf5 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 865ac5a7088c768154d166f3d62bd5217afd8bf5 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the deduped html row now stores the hash its claimed items carry, so remove_artifact and an edit both release the claim.

Notes

  • This churn hits every deduped html row, not only the legacy ones the description mentions. The row now stores the text hash, but the short-circuit compares it with the markup hash (prev_hash == content_hash), so they never match. Every re-ingest therefore calls release_stale_claim: it drops the claim, then the gate re-claims it. A sibling text_hash column, the way folder rows do it, would remove both the churn and the brief window with no claim.
  • In artifact_item_state.content_hash, a deduped row now holds a text hash while an active row holds a markup hash. That difference is written down only in the spec sentence. The next reader of the column should not have to infer it.

[DESIGN-REVIEWED] 865ac5a

Description read

[DESCRIPTION-READ] 357fdaacf4fd71874cee575cdd911c18c116ab0c01441f4eca23a4ba0d2f8cf9

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 865ac5a7088c768154d166f3d62bd5217afd8bf5 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. This is a one-argument fix at the mechanism level: the duplicate gate already passed the right hash to the callback, and the callback ignored it. A test that fails on base shows the defect.

What this change ships

Inventory (4 items) — 4 justified

Intent: when someone deletes or edits a deduped HTML artifact, its claim on the shared document is released, so no text stays searchable with nothing behind it. FIX.

  1. Deleting a deduped HTML artifact now releases its claim on the item the other source holds. — justified
  2. On each start, a deduped HTML artifact releases its claim and the gate claims it again. The description declares this cost. — justified
  3. The knowledge spec's sentence on the aggregate tables now says what a refused HTML artifact's row stores, and names the test. — justified
  4. A regression test covers the HTML dedupe-then-remove path. — justified

[FIRST-PRINCIPLES-REVIEWED] 865ac5a

Description read

[DESCRIPTION-READ] 5a7a5ebc9fe418bced0d2c20da669b5059c7ce41d0b30e7b35a5975234dc14da

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 865ac5a7088c768154d166f3d62bd5217afd8bf5 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 865ac5a

Verdict parsed from the review's SHA-scoped output markers for commit 865ac5a7088c768154d166f3d62bd5217afd8bf5.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 865ac5a7088c768154d166f3d62bd5217afd8bf5 false-positive: <reason, 20 to 500 characters>

@github-actions

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 865ac5a7088c768154d166f3d62bd5217afd8bf5 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 865ac5a

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 865ac5a7088c768154d166f3d62bd5217afd8bf5 false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 11, 2026
@RohanK6
RohanK6 enabled auto-merge (squash) November 11, 2026 19:42

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: fix (3 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: a one-line knowledge-ingest fix that records the gate's text hash (not the raw markup hash) on a deduped html artifact row, plus a regression test. Spec files changed as a ride-along (a minority of the diff on both file count and changed lines), not reviewed as a design decision: docs/system-specs/modules/knowledge.md.

@RohanK6
RohanK6 merged commit 27302ce into main Nov 11, 2026
132 of 145 checks passed
@RohanK6
RohanK6 deleted the crew/conductor/artifact-html-dedup-hash branch November 11, 2026 20:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

readiness: passed Eligible automated validation passed for the current revision

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(knowledge): deduped html artifact row records the markup hash, so its claim is never released

2 participants