Repository navigation
feat(cloud): create the Fargate cluster, per-crew roles and log group - #11521
Conversation
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Verify one real deploy of both stacks in a scratch account before #11328 wires them — all verification here is static (cfn-lint + contract tests), never CloudFormation itself. Suggestions
[DESIGN-REVIEWED] 6be91e3 |
First Principles Review (Fable 5) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS Two empty-default parameters and two echo outputs ship ahead of any consumer: a boundary nothing creates, a private-ECR grant the decided delivery never needs. Not justified as shipped
What this change shipsInventory (8 items) — 5 justifiedIntent: let an operator create the AWS resources (cluster, per-crew roles, log group) the merged Fargate launch engine refuses without — ADDITION, staged under
cloud.md spec rows update in the same commit is mandated by AGENTS.md and not counted. [FIRST-PRINCIPLES-REVIEWED] 6be91e3 |
b1f0c2d to
11782f6
Compare
Opus 4.8 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
11782f6 to
064663d
Compare
064663d to
1a1bf0b
Compare
|
Why it cannot be this PR:
Earlier on this head the same lane read Local state on this branch: 12 contract tests pass, the 14-mutation matrix has zero |
1a1bf0b to
ef492a4
Compare
ef492a4 to
bd1a8b7
Compare
bd1a8b7 to
abca7d3
Compare
|
Dispositions for the advisory CONCERNS on Design Review
First Principles
No blocking finding is open. Backend/E2E CI is running on the new head. |
Backend Tests reds on this head are main-owned, not this PR'sAttributed by test name on both sides, not by lane and not by re-run. On this head (
|
| Lane | Result | Failing tests |
|---|---|---|
Backend Tests (3.12, 5) |
2 failed, 14026 passed | test_member_session_control.py::TestMemberChildPrivateBinding::test_a_v1_child_writes_no_private_binding, ::test_global_caller_cannot_grant_a_child_private_authority |
Backend Tests (Windows) (5) |
2 failed, 13214 passed | the same two |
Backend Tests (Windows) (7) |
4 failed, 13151 passed | test_session_control.py::test_create_refuses_unreadable_delegation_identity[OSError], [ValueError], ::test_private_creator_cannot_change_memory[public-worker-canonical], [writer-canonical] |
Assertion in the first pair: AssertionError: assert 'member-radar' is None.
On main's own tip (0640c1e0d), CI run 35254907186
| Lane | Result | Failing tests |
|---|---|---|
Backend Tests (Windows) (5) |
2 failed, 13212 passed | the same two test names |
Backend Tests (Windows) (7) |
4 failed, 13149 passed | the same four test names |
Main is red at its own tip on the identical tests.
Mechanism
TestMemberChildPrivateBinding does not exist in this branch's checkout. It is on
origin/main at test/test_member_session_control.py:504, added by
0640c1e0d5 fix(session-control): let private members dispatch their own workers (#11438).
CI evaluates the merge ref refs/pull/11521/merge, so this board inherited main's
brand-new failing tests.
This PR's diff is docs/system-specs/modules/cloud.md, two CloudFormation templates
and one new offline contract test. It touches zero session-control files, so it
cannot influence them.
What follows
gh run rerun --failedwill not clear these. They are deterministic on main, not
flaky, so the re-run-proves-it method does not apply here. The main-tip control
above is the stronger evidence.- Rebasing would not help either: main's tip is the commit that introduced the
failures, so rebasing pulls them in explicitly rather than dropping them. - No session-control fix is being folded into this PR. Once main is green on
#11438, this branch rebases onto settled main to cut a fresh merge ref.
Unaffected
All four review lanes are success on this head, including the ARN-partition
finding. That fix is at kirocrew-fargate-crew.yaml:50 and :66 and is pinned by
test_both_arn_patterns_accept_exactly_the_partitions_identity_accepts against
_PARTITION_RE. Narrowing either pattern back to arn:aws: makes that test fail on
partition aws-cn, verified locally on a clean tree with both anchors asserted
unique so neither mutation could silently no-op.
abca7d3 to
e753fbf
Compare
e753fbf to
cb7d963
Compare
The Fargate launch engine is on main, and nothing can reach it: preflight and provision both refuse with "constructed without a launch spec ... it needs a placement (cluster, subnets, security_groups), a digest-pinned image, the crew's secret references, and a cpu architecture". That refusal text is a list of things this line only ever NAMED. runtask.py refuses with "no cluster is named", execution_role_arn and task_role_arn build a string, and identity.py checks the shape of a secret ARN -- no code path creates any of them, and templates/ held one file, for the EC2 lane. Two templates, because the roles are per crew and the cluster is not: - kirocrew-fargate-base.yaml, one stack per account and region: the ECS cluster and an egress-only security group with no ingress at all. - kirocrew-fargate-crew.yaml, one stack per crew: the execution role, the task role and the log group. Deleting one crew's stack cannot delete the cluster its siblings run on, and delete-stack becomes the teardown executor #10983 lacks. Every name is derived by cloud/fargate/identity.py rather than chosen here, so the templates mirror kirocrew-crew-<crew>-exec, kirocrew-crew-<crew>-task and /kirocrew/crew/<crew>. The task definition refuses a document whose ARNs disagree, so a rename on either side is a launch refusal rather than a mismatch that runs -- the contract test asserts the agreement by calling those functions. The task role is created with no policies at all, and must never gain secretsmanager:GetSecretValue: identity.py already states why, since the model credential is in the container's environment before that role exists, so the permission buys the crew nothing while a turn that could re-read a secret could read every crew's. The execution role's read is scoped to one crew's secret namespace, and every action is enumerated -- no prefix wildcard appears anywhere, because cloud/aws.py documents that a prefix admits secretsmanager get-secret-value and ssm get-parameter --with-decryption. No statement uses NotAction or NotResource either, which would grant everything they do not name and which a check reading only Action cannot see. The outputs are the whole point: they are the configuration home a launch spec is read from, so every field the refusal named has exactly one place to come from. The base stack answers the placement -- ClusterName, SecurityGroupId, SubnetIds -- plus CpuArchitecture, whose allowed values are compared against taskdef.CPU_ARCHITECTURES by importing that frozenset, so a stack that deploys and a launcher that then refuses the value cannot both be true. The crew stack answers identity: ExecutionRoleArn, TaskRoleArn, LogGroupName, and SecretArnPattern, the one ARN pattern the execution role may read, asserted character-for-character against the policy that grants it. Fixes #11044
cb7d963 to
6be91e3
Compare
The inherited reds cleared via #11593, with no change to this PR's codeRecording why the board changed, so the earlier red history reads correctly. Six session-control tests were failing on this PR's board and on main's own tip.
This branch is now rebased onto The lanes that carried the six
Board otherwise: 68 concluded lanes, zero failures, counts summing to the lane Local verification on the rebased treeEach as its own invocation, with
One correction to the matrix, worth statingThe harness now treats an anchor that matches never or more than once as a hole |
…kirodotdev#11521) The Fargate launch engine is on main, and nothing can reach it: preflight and provision both refuse with "constructed without a launch spec ... it needs a placement (cluster, subnets, security_groups), a digest-pinned image, the crew's secret references, and a cpu architecture". That refusal text is a list of things this line only ever NAMED. runtask.py refuses with "no cluster is named", execution_role_arn and task_role_arn build a string, and identity.py checks the shape of a secret ARN -- no code path creates any of them, and templates/ held one file, for the EC2 lane. Two templates, because the roles are per crew and the cluster is not: - kirocrew-fargate-base.yaml, one stack per account and region: the ECS cluster and an egress-only security group with no ingress at all. - kirocrew-fargate-crew.yaml, one stack per crew: the execution role, the task role and the log group. Deleting one crew's stack cannot delete the cluster its siblings run on, and delete-stack becomes the teardown executor kirodotdev#10983 lacks. Every name is derived by cloud/fargate/identity.py rather than chosen here, so the templates mirror kirocrew-crew-<crew>-exec, kirocrew-crew-<crew>-task and /kirocrew/crew/<crew>. The task definition refuses a document whose ARNs disagree, so a rename on either side is a launch refusal rather than a mismatch that runs -- the contract test asserts the agreement by calling those functions. The task role is created with no policies at all, and must never gain secretsmanager:GetSecretValue: identity.py already states why, since the model credential is in the container's environment before that role exists, so the permission buys the crew nothing while a turn that could re-read a secret could read every crew's. The execution role's read is scoped to one crew's secret namespace, and every action is enumerated -- no prefix wildcard appears anywhere, because cloud/aws.py documents that a prefix admits secretsmanager get-secret-value and ssm get-parameter --with-decryption. No statement uses NotAction or NotResource either, which would grant everything they do not name and which a check reading only Action cannot see. The outputs are the whole point: they are the configuration home a launch spec is read from, so every field the refusal named has exactly one place to come from. The base stack answers the placement -- ClusterName, SecurityGroupId, SubnetIds -- plus CpuArchitecture, whose allowed values are compared against taskdef.CPU_ARCHITECTURES by importing that frozenset, so a stack that deploys and a launcher that then refuses the value cannot both be true. The crew stack answers identity: ExecutionRoleArn, TaskRoleArn, LogGroupName, and SecretArnPattern, the one ARN pattern the execution role may read, asserted character-for-character against the policy that grants it. Fixes kirodotdev#11044
What is the problem?
The Fargate launch engine is on main, and nothing can reach it. Driven from a
clean tree,
preflight()andprovision()both refuse:That refusal text is a list of things this line only ever named.
runtask.pyrefuses withno cluster is named;execution_role_arnandtask_role_arnbuild a string;identity.pyvalidates the shape of a secretARN. No code path creates any of them, and
cloud/templates/held exactly onefile, for the EC2 lane. There was no issue recording that gap until #11044.
Why this issue matters to the user
Nothing in the product can launch a remote crew on Fargate, and the reason is
not visible from the outside: the modules are present, tested and green. A user
who set out to deploy one would have to author a cluster, two IAM roles with the
exact derived names, a log group and a resource-scoped secret grant by hand,
matching conventions that live only in
identity.py. Get a name wrong and thesymptom arrives later, as a task definition refused at launch, not as a
misconfiguration reported where it was made.
How our fix solves it
Fargate needs the AWS resources to exist before a task can start, so this adds
the CloudFormation that creates them, and names its outputs so every field the
refusal listed has exactly one place to be read from.
placement.clusterClusterName(base stack)placement.security_groupsSecurityGroupId, egress-onlyplacement.subnetsSubnetIdscpu_architectureCpuArchitecture, pinned totaskdef.CPU_ARCHITECTURESExecutionRoleArn,TaskRoleArn(crew stack)LogGroupName, retention cappedSecretArnPattern,SecretNamePrefixFour more outputs carry no launch-spec field but exist so the stacks are
addressable by other automation: the base stack also emits
ClusterArn,RegionandStackTag(the tag #11328's wiring reads a stack back by), and thecrew stack emits
Crew(the crew a stack belongs to, echoed for the samelookup). The contract test asserts the full output key set in both directions,
so a dropped or renamed output reds.
Two templates, because the roles are per crew and the cluster is not:
kirocrew-fargate-base.yaml, one stack per account and region: the ECScluster and an egress-only security group with no ingress at all.
kirocrew-fargate-crew.yaml, one stack per crew: the execution role, thetask role and the log group.
The split follows from the code rather than from taste.
execution_role_arn,task_role_arnandlog_group_nameall derive from aCrewBinding, so thosethree resources are per crew;
Placement.clusteris one value many tasks share.Splitting them means deleting one crew's stack cannot delete the cluster its
siblings run on, and it gives #10983 a teardown executor:
delete-stackon theper-crew stack.
Chaining symptom to cause, the names are derived, not chosen here. The
templates mirror
kirocrew-crew-<crew>-exec,kirocrew-crew-<crew>-taskand/kirocrew/crew/<crew>becauseidentity.pybuilds exactly those, andtaskdef.pythen refuses a document whose ARNs disagree. So a rename on eitherside does not produce a mismatch that runs -- it produces a launch refusal, at
the first launch after the rename, with nothing failing when the rename landed.
The contract test asserts the agreement by calling those functions, so the
drift reds at the commit that causes it.
CpuArchitectureworks the same way:its allowed values are compared against
taskdef.CPU_ARCHITECTURESby importingthat frozenset, so a stack that deploys and a launcher that then rejects the
value cannot both be true.
The two ARN parameters accept every partition
CrewBindingaccepts, not justaws.identity._PARTITION_REadmitsaws,aws-cnandaws-us-gov, so apattern hardcoding
arn:aws:would refuse a validaws-cnARN at CloudFormationparameter validation and the per-crew stack would fail to deploy in that
partition. The alternation is not restated from that regex, which is how two
copies of one table drift: a test reads
_PARTITION_REand asserts both templatepatterns agree with it partition by partition. Everything the template builds
itself is already partition-correct, because every ARN comes from
${AWS::Partition}.Two security properties are structural, not incidental:
secretsmanager:GetSecretValue.identity.task_role_arn's own docstringstates the reason: the model key is already in the container's environment by
the time that role exists, so the permission buys the crew nothing it does not
have, while a turn that could re-read a secret could read every crew's. The
execution role holds the read instead, scoped to one crew's secret namespace --
the trailing wildcard covers only the six-character suffix Secrets Manager
appends and which no caller can derive.
SecretArnPatternpublishes that onepattern, asserted character-for-character against the policy that grants it, so
an output promising reach the role does not have cannot ship.
cloud/aws.py's read allowlist documents why: a prefix admitssecretsmanager get-secret-valueandssm get-parameter --with-decryption, sosecretsmanager:Get*on the execution role would grant, through the wildcard,exactly what the task role is denied outright. No statement uses
NotActionorNotResourceeither, which grant every call they do not name and which a checkreading only
Actioncannot see.PermissionsBoundaryArndefaults to empty, and that is a declared degradedmode rather than an oversight. The EC2 lane caps its role with a shared,
pre-created, immutable boundary authored once by launcher code
(
cloud/iam.py ensure_instance_boundary) so a leaked launcher grant cannotauthor a permissive one. No equivalent creator exists for the crew boundary yet.
Rather than reference a policy nothing creates, the parameter accepts the exact
ARN once that creator lands, and its pattern is pinned by a test so widening it
is a visible edit.
What tests we did
test/test_fargate_scaffolding_contract.py-- 19 checks, static and offline.They read only the template text, so they need no AWS account and no deploy, and
cannot flake. No real AWS resource was created and no template was deployed:
this host has no usable identity for it, so the whole verification is static by
necessity as well as by design.
cfn-lint 1.22.3is clean on both templates.Crewpatternidentity.pysecretsmanagerappears on itsecretsmanager:Get*orecr:BatchGet*appears, including inside an!IfbranchNotActionorNotResourceaws:SourceAccountcondition is dropped_PARTITION_REacceptsarn:aws:, refusingaws-cnoraws-us-govtaskdef.CPU_ARCHITECTURESEvery assertion is mutation-verified: the template is broken in the specific way
that test exists to catch, the test is required to go red, then the file is
restored and checked byte-identical. 22 mutations, 22 reds, 0 holes, with the
clean tree confirmed green first and every anchor asserted to match exactly once --
an anchor that matches never, or more than once, is counted as a hole and fails
the run, so a mutation that silently did not apply cannot read as a pass.
The mutations imitate real edits rather than synthetic ones, which is where this
kind of test usually leaks:
arn:aws:iam::aws:policy/SecretsManagerReadWriteecr:BatchGet*hidden inside the conditional ECR branch!Ifbranches instead of only reading top-level statementsActionswapped forNotActionActionarn:aws:_PARTITION_REitself, so a partition table that drifts fromidentity.pyreds instead of surfacing as a failed deploy inaws-cnGates run before the commit:
scripts/check_black_formatting.py, isort, flake8,check_comment_history.py,check_brand_name.py,check_subprocess_encoding.py,check_testpaths_coverage.py,check_changelog_history.py,check_feature_map.py,docs-lint, and the scoped backend test runner.
Any other suggestions on the work
cannot start:
engine_for('aws_fargate')raisesKeyErrorbecause nothingregisters the lane, which is Fargate: FargateLaunchSpec has no configuration home, so the aws_fargate lane cannot be registered yet #11328, and the crew image is published nowhere,
which is The crew base image is published nowhere, so its digest has to be minted locally #10901. Registration must land together with
_tier_as_pairor alight/balanced/powerlaunch is refused.SubnetIdsis a parameter becausewhether a subnet reaches the registry through a NAT gateway or as a public
subnet is a property of the operator's VPC that a template cannot discover.
Passing it in keeps the failure at deploy time with a name attached instead of
a task that reaches RUNNING and then cannot pull.
CpuArchitectureis aparameter for the same reason: it is a property of the image, and a runtime
platform that disagrees with the image fails at task start.
cloud/iam.py's existing one rather than in a template, for the reason thatfile already gives: a boundary authored per launch by whoever holds the
launcher policy is not a ceiling.
EcrRepositoryArnis empty by default because the decided delivery for theshared crew image (Fargate: shared crew image and a task-not-service runtime, design of record #11408) is a public registry, which needs no execution-role
permission at all. It exists so a private repository is a parameter rather
than a patch.
CloudFormation Lintjob does not cover these files. It lintssrc/kiro_crew/deploy/skills/artifact-deploy/templates/only, so neither thesetwo templates nor the existing
kirocrew-ec2.yamlsibling are linted there.Both were linted locally with the same pinned
cfn-lintversion. Widening thatjob is a CI change and belongs on its own.
Fixes #11044