Skip to content

fix: keep the knowledge dedup sweep from failing on year 0 dates - #35364

Merged
RohanK6 merged 1 commit into
mainfrom
crew/conductor/dedup-year-zero
Nov 11, 2026
Merged

RohanK6 merged 1 commit into
mainfrom
crew/conductor/dedup-year-zero

Conversation

@RohanK6

@RohanK6 RohanK6 commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: The knowledge dedup sweep keeps running when a filename carries a zero-padded counter that reads as a year-0 date.

filename_near_match("Scan_00000105.pdf", "Scan_00000106.pdf") raises ValueError: year 0 is out of range (#35363). The ISO date pattern reads 00000105 as year 0000, month 01, day 05; _valid_iso_match checks only month and day, and _date_distance_days hands year 0 to datetime.date(), which accepts 1..9999.

Why it matters

The error escapes the cross-source dedup pass, so the scheduled sweep fails on every run for any library holding two such near-identical names (scanner output, numbered exports), and duplicates are never collapsed again. The per-ingest dedup call fails the same way.

Not a goal

  • Changing which tokens count as dates, or the date-distance threshold.
  • Any other dedup rule.

What changed (motivation → approach → change)

  • Root cause: _date_distance_days is the one place an extracted (year, month, day) becomes a datetime.date, and nothing bounded the year.
  • Change (knowledge/dedup.py): clamp both years to datetime.MINYEAR there, so the gate still compares month and day. Every extractor path (ISO, numeric, month-name "Apr 0000") reaches date() only through this function, so one clamp covers them all.

Backwards compatibility

Compatible: a pair that raised now gets a verdict; every pair that did not raise computes the same distance as before.

Removes nothing: no rule, field or path is taken away.

Tests

  • test_knowledge_dedup.py::TestFilenameMatch::test_a_zero_padded_counter_read_as_year_zero_does_not_crash: counters one day apart match, and counters 11 months apart are still kept apart by the date gate.
  • Ran: test_knowledge_dedup.py, test_knowledge_dedup_winpath.py (84 passed), black/isort/flake8/mypy on the changed files, the comment-history and brand gates, the flake-pattern ratchet.

Proof: not provable -- other: prove.py reported INCONCLUSIVE (a collection-only error after it reverts the hunks); by hand, with dedup.py reverted to main the new test fails with ValueError: year 0 is out of range, and it passes on head b52020c.

Manual verification

N/A — unit coverage sufficient: pure function, no I/O.

Related Issues

Fixes #35363

Sibling sweep

Swept: git grep -nE "\bdate(" -- src/kiro_crew/knowledge/dedup.py
Siblings: none -- _date_distance_days is the only date() construction in the dedup date gate

Open PR #22859 also edits dedup.py (imports and _collapse_doc); its hunks do not overlap this change.

Pattern harvest

Not generalizable: a single unbounded value fed to datetime.date; the gate already validates month and day.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.

A zero-padded 8-digit counter in a filename ("Scan_00000105.pdf") reads
as an ISO date with year 0000, and datetime.date() refuses year 0, so
the date gate raised ValueError and failed the whole dedup sweep on
every run. Clamp the year to MINYEAR where the distance is computed, so
the gate still compares month and day.

Fixes #35363
@RohanK6
RohanK6 requested review from a team and KiroCrewCaptain as code owners November 11, 2026 16:17
@RohanK6
RohanK6 requested a review from tim-finnigan November 11, 2026 16:17
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 11, 2026
@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision b52020ca605e15b5c5febccc2edb32b8d6322bca touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of b52020ca605e15b5c5febccc2edb32b8d6322bca — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

The fix is a small clamp at the one spot where an extracted date becomes a date(), so nothing blocks.

Notes

  • The root cause is that a zero-padded counter like 00000105 gets read as a date at all. The clamp lets that pair get a verdict, but two such counters are still compared as dates rather than as plain numbers. Whether that is right is a question about which tokens count as dates, and the PR excludes that as "Not a goal". A follow-up issue would cover it.

[DESIGN-REVIEWED] b52020c

Description read

[DESCRIPTION-READ] ea471210963a6bdaa07a4782ca96e100d31920c7caba1e8cb4a14da9e1c32417

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of b52020ca605e15b5c5febccc2edb32b8d6322bca — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks: a one-line clamp at the only date() construction in dedup.py, with a test that fails on base.

What this change ships

Inventory (2 items) — 2 justified

Intent: keep the scheduled knowledge dedup sweep running when a library holds zero-padded counter filenames that read as year-0 dates. This is a FIX.

  1. A pair like Scan_00000105.pdf / Scan_00000106.pdf now gets a near-match verdict instead of raising ValueError and stopping the sweep. — justified
  2. A regression test pins both the one-day match and the 11-month separation for counter names. — justified

[FIRST-PRINCIPLES-REVIEWED] b52020c

Description read

[DESCRIPTION-READ] 2295ed0355a83f5e491c599228a0ff31afcf04f1249472a69ec8268054fce72f

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed b52020ca605e15b5c5febccc2edb32b8d6322bca — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] b52020c

Verdict parsed from the review's SHA-scoped output markers for commit b52020ca605e15b5c5febccc2edb32b8d6322bca.

False positive or not applicable? A repository writer can comment:
/ai-review override fable b52020ca605e15b5c5febccc2edb32b8d6322bca false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of b52020ca605e15b5c5febccc2edb32b8d6322bca and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] b52020c

False positive or not applicable? A repository writer can comment:
/ai-review override gpt b52020ca605e15b5c5febccc2edb32b8d6322bca false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 11, 2026
@RohanK6
RohanK6 enabled auto-merge (squash) November 11, 2026 17:33

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This review verified head b52020c. The clamp in _date_distance_days (src/kiro_crew/knowledge/dedup.py:296-297) is the only place in the module that builds a date from a parsed filename. That means it covers both routes a year 0 can take: ISO 0000MMDD and a month name such as Apr 0000. Every pair that worked before still gets the same distance. With main's dedup.py restored, the new test fails with ValueError: year 0 is out of range. On head, all 79 tests in test_knowledge_dedup.py pass. Deleting either clamp line makes the test fail, and so does swapping the clamp for a bare except ValueError fallback.

Done well: the test checks both directions. Adjacent counters still match, and counters about 11 months apart are still kept apart by the date gate.

Optional:

  • Clamping each year separately means 0000 and 0001 now read as the same date. For example, Scan_00000105.pdf and Scan_00010105.pdf give a distance of 0. Main crashes on that pair, so this is not a regression. The embedding and cosine checks also make a wrong merge very unlikely. A follow-up could keep the gap between the years, or reject year 0 in _valid_iso_match.

@RohanK6
RohanK6 merged commit 1c97d38 into main Nov 11, 2026
102 of 104 checks passed
@RohanK6
RohanK6 deleted the crew/conductor/dedup-year-zero branch November 11, 2026 18:04
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 11, 2026
@bolichen97

Copy link
Copy Markdown
Contributor

@RohanK6 could you push a new commit, or edit this PR's title or body, so a Sensitive Change Review run starts? The helper check rerun / Re-run Sensitive Change Review on b52020c fails with "No Sensitive Change Review run found for PR #35364 at b52020c ... push, or edit the PR title or body, to start one", and a rerun cannot fix that. Every other check is green, and the change itself (clamping year 0 to MINYEAR in _date_distance_days) looks good to us. We will approve once that check is green.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Knowledge dedup sweep fails with "year 0 is out of range" on zero-padded counter filenames

3 participants