Repository navigation
fix: keep the knowledge dedup sweep from failing on year 0 dates - #35364
Conversation
A zero-padded 8-digit counter in a filename ("Scan_00000105.pdf") reads
as an ISO date with year 0000, and datetime.date() refuses year 0, so
the date gate raised ValueError and failed the whole dedup sweep on
every run. Clamp the year to MINYEAR where the distance is computed, so
the gate still compares month and day.
Fixes #35363
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS The fix is a small clamp at the one spot where an extracted date becomes a Notes
[DESIGN-REVIEWED] b52020c Description read[DESCRIPTION-READ] ea471210963a6bdaa07a4782ca96e100d31920c7caba1e8cb4a14da9e1c32417 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks: a one-line clamp at the only What this change shipsInventory (2 items) — 2 justifiedIntent: keep the scheduled knowledge dedup sweep running when a library holds zero-padded counter filenames that read as year-0 dates. This is a FIX.
[FIRST-PRINCIPLES-REVIEWED] b52020c Description read[DESCRIPTION-READ] 2295ed0355a83f5e491c599228a0ff31afcf04f1249472a69ec8268054fce72f That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
This review verified head b52020c. The clamp in _date_distance_days (src/kiro_crew/knowledge/dedup.py:296-297) is the only place in the module that builds a date from a parsed filename. That means it covers both routes a year 0 can take: ISO 0000MMDD and a month name such as Apr 0000. Every pair that worked before still gets the same distance. With main's dedup.py restored, the new test fails with ValueError: year 0 is out of range. On head, all 79 tests in test_knowledge_dedup.py pass. Deleting either clamp line makes the test fail, and so does swapping the clamp for a bare except ValueError fallback.
Done well: the test checks both directions. Adjacent counters still match, and counters about 11 months apart are still kept apart by the date gate.
Optional:
- Clamping each year separately means 0000 and 0001 now read as the same date. For example,
Scan_00000105.pdfandScan_00010105.pdfgive a distance of 0. Main crashes on that pair, so this is not a regression. The embedding and cosine checks also make a wrong merge very unlikely. A follow-up could keep the gap between the years, or reject year 0 in_valid_iso_match.
|
@RohanK6 could you push a new commit, or edit this PR's title or body, so a Sensitive Change Review run starts? The helper check |
Problem / Motivation
Goal: The knowledge dedup sweep keeps running when a filename carries a zero-padded counter that reads as a year-0 date.
filename_near_match("Scan_00000105.pdf", "Scan_00000106.pdf")raisesValueError: year 0 is out of range(#35363). The ISO date pattern reads00000105as year 0000, month 01, day 05;_valid_iso_matchchecks only month and day, and_date_distance_dayshands year 0 todatetime.date(), which accepts 1..9999.Why it matters
The error escapes the cross-source dedup pass, so the scheduled sweep fails on every run for any library holding two such near-identical names (scanner output, numbered exports), and duplicates are never collapsed again. The per-ingest dedup call fails the same way.
Not a goal
What changed (motivation → approach → change)
_date_distance_daysis the one place an extracted (year, month, day) becomes adatetime.date, and nothing bounded the year.knowledge/dedup.py): clamp both years todatetime.MINYEARthere, so the gate still compares month and day. Every extractor path (ISO, numeric, month-name"Apr 0000") reachesdate()only through this function, so one clamp covers them all.Backwards compatibility
Compatible: a pair that raised now gets a verdict; every pair that did not raise computes the same distance as before.
Removes nothing: no rule, field or path is taken away.
Tests
test_knowledge_dedup.py::TestFilenameMatch::test_a_zero_padded_counter_read_as_year_zero_does_not_crash: counters one day apart match, and counters 11 months apart are still kept apart by the date gate.test_knowledge_dedup.py,test_knowledge_dedup_winpath.py(84 passed), black/isort/flake8/mypy on the changed files, the comment-history and brand gates, the flake-pattern ratchet.Proof: not provable -- other: prove.py reported INCONCLUSIVE (a collection-only error after it reverts the hunks); by hand, with
dedup.pyreverted to main the new test fails withValueError: year 0 is out of range, and it passes on head b52020c.Manual verification
N/A — unit coverage sufficient: pure function, no I/O.
Related Issues
Fixes #35363
Sibling sweep
Swept: git grep -nE "\bdate(" -- src/kiro_crew/knowledge/dedup.py
Siblings: none --
_date_distance_daysis the onlydate()construction in the dedup date gateOpen PR #22859 also edits
dedup.py(imports and_collapse_doc); its hunks do not overlap this change.Pattern harvest
Not generalizable: a single unbounded value fed to
datetime.date; the gate already validates month and day.Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.