Skip to content

test(memory): stamp distinct write instants in startup-lesson tie tests - #16441

Merged
bolichen97 merged 1 commit into
mainfrom
fix/lesson-ranking-windows-tie
Oct 3, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
fix/lesson-ranking-windows-tie

Conversation

@bolichen97

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Backend Tests (Windows) (8) goes green again on main and on every PR, because two startup-lesson ranking tests stop depending on how finely the wall clock ticks.

Two tests that #16293 added fail on Windows:

Why it matters

The Windows (8) shard is a required check, so this red blocks every open PR, whatever it touches. Whether it fails depends on runner speed, so it looks like a flake when it is really two fixtures.

Not a goal

  • Adding a secondary sort key for lessons that share one updated_at. Product ordering does not change.
  • Touching any other test in the file or any other suite.

What changed (motivation → approach → change)

Both tests assume that a row written later sorts as newer. One asserts that the newer of two rows tied on every score renders first. The other asserts that recency would put the newer of a capped pair first, because that is what lets its -lexical mutation check catch anything.

updated_at comes from vector_memory._now_iso(), which reads datetime.now. On Windows under Python 3.12 that clock ticks about every 15.6 ms, so two writes in a row can get the same stamp. get_lessons sorts on updated_at alone, so a tied pair comes back in write order, oldest first. Both CI failures show exactly that.

In test/test_lesson_startup_embedding.py, a new fixture, distinct_write_instants, patches vector_memory._now_iso to hand out increasing stamps. It is the same seam and the same tick pattern that test_lesson_ranking.py, test_lesson_keyword_scoring.py and test_lesson_turn_injection.py already use. Only the two tests take it, and their docstrings say why.

Why there is no product change: the spec orders exact ties "then newest-first" (docs/system-specs/modules/memory-skills-hooks.md:3149). Two rows with one stamp have no newest, so the product keeps its own contract. A rowid tie-break is not an option either. On a crew-lineage store semantic_memory is a view, and a view has no rowid.

Backwards compatibility

Compatible: test-only change; no product code, config or stored data changes.

Removes nothing: the diff adds a test fixture and applies it to two tests.

Tests

The two tests take distinct_write_instants and pin the same behaviour as before, now on any clock resolution. The proof was run on Linux (Python 3.12, -n0) with a throwaway pytest plugin. The plugin replaces datetime.now inside kiro_crew.vector_memory, which sits under _now_iso:

Clock under _now_iso Base This PR
every write at one instant the same 2 failures as Windows, same messages 22 of 22 pass
floored to a 15.625 ms tick, 10 runs 6 of 10 runs fail both 10 of 10 pass
  • With every write at one instant, dropping -lexical[index] from the vector-branch sort key still fails test_two_rows_at_the_cap_are_ordered_by_rarity_not_recency at its final assertion. Its mutation check works on any clock.
  • The other 20 cases in the file pass under both clocks, so none of them relies on recency.
  • Local gates: black, isort, flake8, comment-history, brand and mypy are clean. Semgrep, with CI's rule set in baseline mode, reports 0 findings. test_vector_memory_composition_contract.py passes. local-gate.py --dry-run selects 1 backend file, this one.
  • Windows cannot run locally; this PR's Backend Tests (Windows) (8) shard is the Windows evidence.

Manual verification

N/A — unit coverage sufficient: the change is a test fixture, proven red then green above.

Related Issues

no linked issue: main-red fix for the two #16293 tests above, with no tracking issue.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Two cases #16293 added rest on recency between rows written back to
back: one asserts the newer of two rows tied on every score renders
first, the other asserts, as its fixture precondition, that recency
puts the newer of a capped pair first. updated_at comes from
vector_memory._now_iso(), which reads datetime.now; on Windows under
Python 3.12 that clock ticks about every 15.6 ms, so consecutive writes
can share a stamp, and get_lessons orders on updated_at alone. Backend
Tests (Windows) (8) failed the capped-pair case on main and both cases
on a PR merge ref built on main.

Both cases now take distinct write stamps from a fixture that patches
vector_memory._now_iso, the seam the sibling lesson-ranking suites
already use. With every host-clock write frozen to one instant both
fail on the base and pass with this change.
@bolichen97
bolichen97 requested a review from a team as a code owner October 3, 2026 08:37
@bolichen97
bolichen97 requested a review from kyleseaman October 3, 2026 08:37
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 3, 2026
@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of c4cab318918c39f45347bd7456e3b2a527f0291a — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Check that this PR's Backend Tests (Windows) (8) shard passes. It is the only Windows evidence for a fix that was proven by simulating Windows' coarse clock on Linux.

[DESIGN-REVIEWED] c4cab31

Description read

[DESCRIPTION-READ] e2d4aefd39f545396a0c2be4bbcde33223bdd8848e6f5fc75c5f2ad4e4fb7b68

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed c4cab318918c39f45347bd7456e3b2a527f0291a — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] c4cab31

Verdict parsed from the review's SHA-scoped output markers for commit c4cab318918c39f45347bd7456e3b2a527f0291a.

False positive or not applicable? A repository writer can comment:
/ai-review override fable c4cab318918c39f45347bd7456e3b2a527f0291a: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of c4cab318918c39f45347bd7456e3b2a527f0291a and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] c4cab31

False positive or not applicable? A repository writer can comment:
/ai-review override gpt c4cab318918c39f45347bd7456e3b2a527f0291a: <one-sentence reason>

@bolichen97

Copy link
Copy Markdown
Contributor Author

Design Review's check request ("Backend Tests (Windows) (8) shard passes"), answered for c4cab318918c39f45347bd7456e3b2a527f0291a:

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Oct 3, 2026
@bolichen97
bolichen97 enabled auto-merge (squash) October 3, 2026 15:10
@bolichen97
bolichen97 merged commit 7f68914 into main Oct 3, 2026
96 checks passed
@bolichen97
bolichen97 deleted the fix/lesson-ranking-windows-tie branch October 3, 2026 15:22
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants