Repository navigation
feat(dashboard): validate dashboard_write against the current package Model - #18707
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS Any unreadable dashboard package anywhere in the store now makes every crewmate's Watch
[DESIGN-REVIEWED] 81e396e Description read[DESCRIPTION-READ] 2cd1d72723f45b6ec939c74e94759f211960c73a737e6ea5d95b03b90a8c3b5c That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ human override accepted@chenmingwei23 overrode this lane for |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
d663ec8 to
fa30014
Compare
Disposition: GPT 6.1 F1 fixed at the rootFinding (upheld by the adjudication, Agreed, and fixed rather than argued. The fix is the one the finding asked for.
Tests, and proof they can fail. Four cases were added under Two mutations, each applied alone and then reverted:
Scoped suite on the new head: The First Principles BLOCK is not mine to clearIts blocker is that Every one of those three is a decision above a worker: changing this PR's base, deleting a Head this disposition is written against: |
fa30014 to
36d1420
Compare
Disposition: First Principles cleared by moving this PR onto the branch that carries the gateThe BLOCK was correct on its facts. Its own remedy list accepted " The PR's own diff is unchanged in scope -- 13 files, 1 commit -- and does not touch No override was posted. The rebase surfaced a real defect, and it is fixed at the rootPutting the gate on the base broke 15 existing route cases in The cause is that the template-instance fallback was keyed on the wrong condition. It sat The fallback is now keyed on the STATE instead of on the import. The template instance That boundary is pinned by three cases, and the last two are real pins rather than
A mutation making every non-live package state fall through reddens 6 cases. The skipped test is now a passNo skips remain.
One conflict reading that is NOT this PR's
The conflicts that ARE this PR's are unchanged and both resolve as the union of the two What a stacked base costs on this board
Head this disposition is written against: |
af22465 to
3cb3da4
Compare
3cb3da4 to
f31db48
Compare
Held: do not push to this branch, and do not read the current diff as this PR's scopeThis branch ( The diff above currently overstates this PR's scope, and that is not a change anyone
This PR's own change is the other 13 files, unchanged from when it was opened, and Two consequences worth stating plainly:
The remedy is already decided and is a rebase onto the base's new head, after which the Nothing here asks for an override, and none has been posted. |
36d1420 to
7aa51dc
Compare
Disposition: the bound the GPT finding applied to has been removedRecorded against head The earlier blocking GPT finding on this PR was that history truncation went unreported: That whole mechanism is now out of this PR, so the finding has nothing left to apply What is left in this PR is the package-Model read: a write is validated against the Model Checkable from the diff rather than taken on trust:
The design is not discarded, and the value's No override is requested and none has been posted. |
ca5851c to
e310ee9
Compare
299c440 to
0f8929a
Compare
e310ee9 to
7410774
Compare
0f8929a to
1eaa70c
Compare
buluoray
left a comment
There was a problem hiding this comment.
Thanks, the shape of this is good. One reader decides which Model a write is checked against, the "absent" vs "broken" vs "cannot tell" states are kept apart, and every guard I mutated reddens a named test (display gate, all four unproven-absence rules, the unknown-state refusal, the post-redaction shape recheck, the non-object meta.json guard). The changed test files pass on this head.
Blocking: one unreadable artifact of any kind now blocks every template crewmate's dashboard_write.
resolve_bound_slug ends with store.unreadable_record_count(), which counts every unreadable meta.json in the store, whatever its kind. Any non-zero count raises BindingLookupIncomplete. _from_package turns that into error, and read_package_model only falls back to the template instance on empty. I reproduced it against the real store: with no packages at all, corrupting the metadata of one unrelated widget artifact makes the lookup raise. So a crewmate on the default template, whose writes succeed on main, has every write refused after upgrade as package_model_invalid, told to "repair or replace the package" it does not have, and the refusal is recorded in its mistake book under that code each cycle.
On this base, failing closed here protects nothing. The member page still renders the template instance, and a package that is found is refused anyway (package_display_pending), so validating against the template Model is correct even if a hidden package exists. The wrong-Model harm the docstrings describe starts only once the display reads the package. Suggested fix: while the display reads the template, let an incomplete lookup fall through to the template instance (logged), and re-tighten in the change that switches the display. Alternatively, keep it fail closed but scoped to records that could be the bound dashboard, under its own code with an accurate remedy. Please add a test where a template-only member plus one corrupt unrelated record still writes.
Before #18105 or this lands second: #18105 resolves a root session's write key to session:<digest>. Here that becomes bound_to = "crewmate:session:<digest>", which the binding grammar rejects, and _from_package reports it as error with no template fallthrough. With both merged, every root-session dashboard_write would be refused. A session store key should skip the package lookup (or use the session: scope).
Non-blocking:
- #18851 builds on this and saves
crewmate:<slug>packages. Once a crewmate saves one, all of its writes getpackage_display_pendinguntil the display reads packages, so #18851 should not ship ahead of that change. kind="dashboard"has been storable since #18705, so a crewmate with a package saved before this PR goes from accepted to refused writes. That is fine and intended, but the compatibility section says "Compatible".- Every write for a member with no package reads every
meta.jsontwice and opens every dashboard artifact. This is worth caching or skipping while packages are refused anyway. - A failed template read is now an unrecorded 503 instead of a recorded
no_instance. That is reasonable, but please mention it. The 503 body echoes the exception text, which can carry an absolute path. - Docs: the artifacts spec's "Dashboard packages" section does not describe the binding lookup and its "unproven absent" rules,
unreadable_record_count, or thedecode_metachange. The dashboard_write refusal codes are not documented anywhere. - The PR body says a live state carries a Model (a live package carries none) and that the gate is keyed on the manifest's
source(it is keyed onfrom_package). It also mentions a type table that is not in the diff. A comment underSTATE_ERROR("this caller may not read it") is left over from the droppedforbiddenstate.
I ran the changed test files and the mutations above. I did not run the full suite or linters, and I did not drive the HTTP route end to end for the blocking case: that part comes from the store-level reproduction plus the state-to-refusal mapping in source.
iamwhatever
left a comment
There was a problem hiding this comment.
This PR makes dashboard_write check a crewmate's value against the Model of its own kind="dashboard" package and keeps "no package", "broken package" and "cannot tell" as separate states. The shape is right, but buluoray's CHANGES_REQUESTED on the current head 0e8df76 is still open, and I confirmed its blocking point in the source. resolve_bound_slug (src/kiro_crew/artifact_store/dashboard_package.py, the skipped = store.unreadable_record_count() check at the end) raises BindingLookupIncomplete when ANY stored record of ANY kind has an unreadable meta.json. _from_package in src/kiro_crew/dashboard_package.py turns that into STATE_ERROR, and read_package_model falls back to the template instance only on STATE_EMPTY. So one corrupt, unrelated widget artifact makes every write from a crewmate on the default template fail with package_model_invalid, and that wrong lesson lands in its mistake book every cycle. On main those writes succeed. The risk is a regression for every template-only crewmate after upgrade. Suggested fix: while the display still reads the template, let an incomplete lookup fall through to the template instance (and log it), or scope the fail-closed check to records that could be the bound dashboard and give it its own code with an accurate remedy. Add a test where a template-only member plus one corrupt unrelated record still writes. Please also answer the reviewer's #18105 point: a crewmate:session:<digest> binding is rejected and does not fall through, so root-session writes break if both PRs merge. — crew-pr-reviewer (deep re-check)
|
@buluoray @iamwhatever Both blocking points are addressed at head The blocking one: one unreadable artifact of any kind blocking every template crewmate's write. Scoping
Your test is The #18105 point: answered in code, not just in prose. I confirmed it against the grammar -- That exit violates the same rule as the one above: no That does not make the
Non-blocking items taken here
|
|
Head Once the unknown-kind case fell through, What the reviewers' blocking harm turns on is unchanged.
The Also declared in the body, which FP flagged as undeclared: the post-redaction shape re-check (
@buluoray @iamwhatever the blocking item is the same fix by a smaller mechanism -- worth a look at the |
… Model dashboard_write checked a crewmate value against a builtin template manifest, so a dashboard composed as a package artifact had no Model to be checked against. A reader keyed by the member resolves the bound package and answers live, empty or error; a non-live state becomes a refusal the crewmate can act on instead of a 500, and an absent package is kept apart from a corrupt one. The template instance store stands behind the package path for the empty state alone, so a crewmate whose page is a builtin template keeps being checked against its template Model.
The reader is keyed by the member and needs the artifact, so the binding half of the lookup lives beside the package parser it uses. Reads the dashboard artifacts newest-first under a cap, and matches on the stored envelope rather than on a validated package, because this answers which slug and cannot report a reason. None means absent as far as the LISTING goes, and three things leave even that unproven: a scan the cap cut short, an envelope that would not decode, and a read that was denied. Each names a record the listing proved is a dashboard, so one really may be the package bound here and the caller fails closed rather than being sent to a different Model. What None does not rule out is a record whose own meta.json will not read. ArtifactStore.list drops it with a warning, so the scan never sees it, and its kind is exactly what that file holds -- it is as likely a widget. Counting those would answer what the lookup already answers for them, while refusing on one would cost every member that has no package at all: a single damaged widget would turn each of its writes into "repair the package" for a package it does not have, and record that lesson every cycle. A member whose page is drawn from a template is right to go on being checked against it, so list's own warning is where that fact lives. read_package_model falls through to the template instance for a member key the crewmate grammar cannot spell, with a warning. A root session's session:<digest> has an inner colon, which the binding grammar refuses, so no package can be stored under it and nothing is present to be broken.
|
Head The test built a session key by hand ( Correcting the body: that key is not something #18105 introduces.
|
|
/ai-review override first-principles 81e396e: First Principles asks for a subtraction a human reviewer asked for by name. It wants _from_package and the package_display_pending refusal removed; buluoray reviewed this PR and endorsed keeping "absent" vs "broken" vs "cannot tell" apart as separate states, which is what those two carry, and iamwhatever confirmed the same reading in the source. A human reviewers explicit endorsement outranks the lane on a scope question, so this is not a defect left unfixed. Every other lane is green on this head: 98 check-runs, 0 pending, First Principles is the only red. The two real blocking findings both reviewers raised were fixed on this branch with a failing test first, and the lanes own earlier item 6 subtraction was taken rather than argued. |
|
AI-review override not recorded: keep the reason to 500 characters or fewer. |
|
This branch is HELD at The board is settled on that head: 84 lanes green, 13 skipped, one red. The red is First Principles, and its blocker is not a defect -- it asks for Nothing is wrong with the tree. Every build, test, lint and gate lane on this head is green, and the touched suites pass locally (839 across seven files). #18851 is stacked here and can rebase onto this SHA now; a review-lane verdict on this PR says nothing about whether the branch builds. Two things that would be wasted work on this head: a rebase (it already contains both #19204 and #18105, confirmed by ancestry, so the First Principles verdict was formed with those in the base and is not stale on them) and a lane rerun (same reason). If this head changes, any pending maintainer action against it has to be redone, since such an action is bound to the SHA. |
|
/ai-review override first-principles 81e396e: FP asks to delete _from_package and the package_display_pending refusal. Reviewer buluoray endorsed keeping absent vs broken vs cannot-tell as separate states, which is what those two carry, and iamwhatever confirmed the same reading in the source. A human reviewer naming code to keep outranks the lane on a scope question, so no defect ships. Every other lane is green here: 98 check-runs, 0 pending, FP the only red, and both reviewers blocking findings were fixed with a failing test first. |
Human judgment recorded@chenmingwei23 marked the first-principles AI finding as false positive, not applicable, or explicitly accepted for
This decision applies only to this commit. A new push requires a new judgment. |
buluoray
left a comment
There was a problem hiding this comment.
Approving at 81e396e262; this supersedes my earlier request for changes on 0e8df76054.
Both blockers are fixed:
- One unreadable artifact no longer blocks every template crewmate's write.
unreadable_record_countis gone, so a record whose ownmeta.jsonwill not read (and whose kind is therefore unknown) no longer makes the lookup raise.BindingLookupIncompletenow covers only records the listing proved arekind="dashboard": a cap-cut scan, an unopenable dashboard record, an undecodable envelope.test_a_template_only_member_still_writes_beside_one_corrupt_unrelated_recordpins the case I reproduced. - A
session:<digest>key no longer becomes a refusedcrewmate:session:...binding. A key the crewmate grammar cannot spell now falls through to the template instance with a warning, so the #18105 combination keeps working.
Mutations against the new code, each reddened named tests (357 pass clean across test_dynamic_dashboard.py, test_artifact_dashboard_package.py, test_agent_panel_routes.py):
- grammar fall-through answers ERROR again ->
test_a_member_key_outside_the_crewmate_scope_falls_through_too - an incomplete lookup answers EMPTY ->
test_an_unlocatable_package_is_an_error_and_a_broken_one_still_is_too,test_a_lookup_that_could_not_see_every_package_is_an_error_too
The spec now describes the binding lookup and its "absent as far as the listing goes" rule. CI on this head: 85 success, 13 skipped, 0 failed.
The First Principles override quotes my earlier review accurately: I did endorse keeping absent, broken and cannot-tell as separate states.
Still open from last time, non-blocking:
- #18851 should not ship ahead of the change that makes the display read packages, since a saved package turns every write into
package_display_pending. - The
(ArtifactError, ValueError)fall-through is wider than the grammar check: anArtifactErrorfrom listing the store also lands on the template now. That is right while the display reads the template, but should be re-tightened in the change that switches the display. - The compatibility section still says "Compatible" although a crewmate with a pre-existing
kind="dashboard"package goes from accepted to refused writes.
Problem / Motivation
Goal: A crewmate writing to its own dashboard is checked against the Model of the dashboard it actually has, and gets a refusal it can act on when there is none.
dashboard_writevalidated a crewmate's value against a builtin TEMPLATE manifest. A v3page is not a template: it is an artifact of
kind="dashboard"holding its own Model, so acrewmate with a composed dashboard was checked against a template that does not describe
its page. The read behind that check also answered
Noneboth for "nothing is bound" andfor "a package is there and cannot be read", so a crewmate was told to create a dashboard it
already had.
Why it matters
The wrong Model is the wrong field set: a field the page declares is rejected, a field the
page lacks is accepted, and the mistake book groups lessons on
(code, field), so a wrongcode teaches the wrong lesson. The two no-Model cases also need different remedies:
emptymeans "create one", which a crewmate does itself; a broken artifact needs a human.
Not a goal
agenticfold keeps its name, stored shape andstate_version,and the fold catalogue is not in this diff.
forbiddenstate, or aslugargument on the reader.dashboard_fieldsonto the package Model; that divergence is named below.What changed (motivation -> approach -> change)
src/kiro_crew/dashboard_package.pyaddsread_package_model(member) -> PackageRead:model,state,state_reason(quoted into the refusal), andfrom_package.modellive, from a PACKAGENonepackage_display_pending, below)live, from the templateemptyNoneno_instanceerrorNonepackage_model_invalidA live PACKAGE therefore carries no Model: its one caller refuses a package-bound write
before a Model would be checked, so translating one would build a manifest and throw it
away.
from_packageis PROVENANCE and the flag the route's refusal turns on; the statesalone cannot separate the two
liverows.staleis absent: a package has no registry behind it to move past.no_instancekeeps itscode, or a new one would strand every lesson a crewmate has learnt. Only a failed read
raises
PackageReadError(503,package_unreadable): a refusal is recorded first, and anexception would skip that and be a 500. It is keyed by the MEMBER, and the template instance
store stands behind this path for
empty, since a crewmate on a builtin template has nokind="dashboard"artifact.resolve_bound_slugandMAX_BINDING_SCANmoved in to the base's own module, becausethe binding half of the lookup has callers outside #18705. Two names, and the function keeps
its keyword-only
storeparameter.Nonefrom it means absent as far as the LISTING goes. Three things leave even thatunproven, and each raises
BindingLookupIncompleterather than answeringNone:errorstore.getfailed on a recorderrorerrorEach names a record the listing PROVED is
kind="dashboard", so one of them may be thepackage bound here and the reader fails closed:
emptywould fall through to a BUILTINpage's Model, and a write checked against that can replace a cell with a value of the wrong
shape.
What
Nonedoes not rule out is a record whose ownmeta.jsonwill not read, and thatis deliberate.
ArtifactStore.listdrops such a record with a warning -- the right default,since one corrupt file must not take a library down -- so it never reaches the scan, and its
kindis exactly what that file holds. It is therefore only POSSIBLY a dashboard, and aslikely a widget. Counting it and refusing would cost every member that has NO package at
all: one damaged widget anywhere in the store would turn each of its writes into
package_model_invalid, telling it to repair a package it does not have, and record thatlesson every cycle. A member whose page is drawn from a template is right to go on being
checked against it, so the lookup answers for the records it can see and the warning
listalready emits is where that fact lives. A caller that must tell such a record apart needs
the store to report it, which belongs with the change that HAS such a caller.
A member key the crewmate grammar cannot spell falls through on the same rule, and that
population is a ROOT SESSION on today's main.
_resolve_dashboard_callerkeys a rootsession's write by
instance.session_instance_key(slot.key), which issession:<digest>.This reader spells a binding
crewmate:<member>and_BOUND_TO_REadmits no inner colon,so that key cannot be a crewmate binding at all. No package can be stored under a string
the grammar refuses, so nothing is present to be broken:
emptyand alogger.warning,not
error. Without it, every write a root session makes is refused as a broken package itcould never have had -- which the reviewers raised against #18105, and which the rebase
onto current main shows is live here already. The session's own instance is what its page
draws, and the instance store keys it by exactly this string. A member key in the
session:scope wants the LOOKUP to use that scope rather than prefix it, which belongswith the change that gives a session a package.
The binding is matched off the stored ENVELOPE, so a layout the validator rejects is still
FOUND and reported broken by name --
parse_packagesplits into_decode_package(emptycheck, surrogate scan, cap, decode) plus
validate_package, one line over the two, so everycaller is unchanged.
One more source file,
src/kiro_crew/artifact_store/records.py, for a PRE-EXISTINGdefect this PR's new caller exposes.
decode_metareads keys off the parsed record, soa
meta.jsonwhose top level is not an object raisesAttributeError, whichArtifactStore.listdoes not tolerate -- one such file turns a whole listing into a crashinstead of one skipped record, which through this module is a 500 on a dashboard write with
no refusal recorded. Refused as an
ArtifactErrornow, a classlistalready handles. Thecrash predates this branch and reaches every
list()caller, so it is fixed at the root.Two smaller items in the diff, named so they are not read as riding along:
check_writeasks the SHAPE again after redaction, not only the type(
redacted_value_invalid,src/kiro_crew/dashboard_agentic.py). The stored value is thescrubbed one, and the scrub rewrites strings -- so a declared choice that is itself
credential-shaped passes the type check, because a placeholder is a perfectly good
string, and then lands in the cell as a value the page's own Model says cannot bethere. An enum member, a required key under a scrubbed name and an item in a typed array
are all in that position. The type and size re-checks beside it were already there; this
is the shape half.
SOURCESinsrc/kiro_crew/dashboard_templates/manifest.pyrecords that asynthesized package manifest needs a fourth value, since it has no template directory and
carries no page. Comment only, no value added here; it sits where the next reader of that
set will look.
Backwards compatibility
No fold name, version, stored shape, entry type, response key or manifest key changes;
DASHBOARD_FOLD_NAMEstill reads"agentic". Removes nothing:decode_metarefuses ashape that crashed it rather than accepting less.
Two behaviour changes, both deliberate:
kind="dashboard"package goes from accepted writes topackage_display_pending. The kind has been storable since feat(artifacts): a validated, versioned dashboard layout package #18705, so that populationcan exist. It is the intended trade: see the gate below.
package_unreadable) where it was arecorded
no_instance. A read that failed says nothing about the write, so there is nolesson to record and no field to blame. One thing to flag rather than claim fixed: the
503 body echoes the exception text, which on an
OSErrorcan carry an absolute path.That predates this diff on every
PackageReadErrorpath and is worth its own change.resolve_bound_slugandMAX_BINDING_SCANare RE-ADDED to their module, so their readersare restored:
Reader: src/kiro_crew/dashboard_package.py:_from_package -- chat -- the new reader itself, which resolves a member's artifact through it; test_the_binding_resolves_the_slug_when_the_caller_has_none
Reader: src/kiro_crew/dashboard/handlers/agent_panel.py:api_dashboard_write -- chat -- the write route, which reads the package by member and maps each non-live state to a refusal; test_the_write_route_asks_for_a_package_by_member_and_not_by_slug
"package"is deliberately NOT ininstance.RENDERABLE_SOURCES, pinned both ways bytest_a_package_source_is_not_renderable: that set decides whether a page may EXECUTE.A package-bound write is REFUSED on this base, code
package_display_pending:api_member_dashboardrenders the template instance and reads no package, so a valuechecked against the package's Model could be one the page cannot draw -- a string where it
reads a list blanks the cards already there, with no error. Keyed on
from_package, not onthe state, since this reader answers
livefor a template instance too. #18847 lifts itwhere the display reads the same package.
Tests
TestThePackageSeamholds the cases the change is specified against, and each refusalhas a case checked by name -- including
test_a_live_package_is_refused_while_the_display_still_reads_the_template, which drivesthe real route and was red as a 200 before the gate.
TestTheBindingLookup(real store, real validator) andTestTheRealGateAndStore(thestand-in gate removed, so the real
resolve_bound_sluganswers) cover the three rows aboveplus the non-object
meta.json, whichlistnow skips instead of crashing on. Each corruptrecord is damaged on disk after the store wrote it, and every case asserts its own premise
first, so a fixture that stopped being corrupt fails.
Four cases pin the line between answering and failing closed:
test_a_template_only_member_still_writes_beside_one_corrupt_unrelated_record-- thepopulation a refusal here would cost: a member with no package, one damaged
kind="widget"record, and the template's Model still answering.test_a_record_of_unknown_kind_does_not_make_the_lookup_refuse-- the store layer, bothdirections: a record the listing hid does not make the lookup refuse, and an undecodable
dashboard envelope still does.
test_a_member_key_outside_the_crewmate_scope_falls_through_too-- a root session's realkey, built by
session_instance_keyrather than written by hand, since the instancestore refuses a digest of any other shape. It needs the real gate: the stand-in answers a
slug for any string and never reaches the grammar.
test_an_unlocatable_package_is_an_error_and_a_broken_one_still_is_too-- the limit ofboth fall-throughs, so loosening either cannot pass as this pattern.
Rebased onto
mainatddb399cd, and the rebase is what showed the root-session case islive rather than pending #18105.
Each mutation was applied alone and reverted with the digest checked either side. Dropping
the display gate reddens its case; refusing every
liveModel instead of a package-sourcedone reddens FOUR existing route cases, which is why the check reads
from_package. On thebinding lookup: adding a count-and-refuse over all metadata reddens the first two cases
above, making the undecodable-record refusal unreachable reddens the second plus two
existing ones, and answering
errorfor an unspellable binding reddens the third.Manual verification
N/A -- unit coverage sufficient. The write route is driven as the real route and the binding
lookup against a real store on disk.
flake8,isort, black,check_comment_history,check_feature_map,docs-lintandmypyall exit 0.Screenshots / video
N/A -- no frontend surface; nothing under
website/.Related Issues
Refs #18642. Depends on #18705. The #18642 T3 one-pager was approved.
Checklist
docs/system-specs/modules/artifacts.mdgains the bindinglookup, its three unproven-absence rules and the boundary of what
NoneclaimsContribution License Agreement
Contributed under Apache-2.0; see CONTRIBUTING.