diff --git a/release_automation/docs/branching-model.md b/release_automation/docs/branching-model.md index c0ccd76..fe180b1 100644 --- a/release_automation/docs/branching-model.md +++ b/release_automation/docs/branching-model.md @@ -1,6 +1,6 @@ # Branching and Versioning Model -**Last Updated**: 2026-05-05 +**Last Updated**: 2026-10-24 `tooling/main` is the only long-lived branch. All development happens on fork branches that merge into `main` via pull request. @@ -60,9 +60,10 @@ When a critical fix cannot wait for a regular feature cycle: ## Appendix: development history -Two long-lived feature branches preceded the current single-branch model: +Two long-lived feature branches preceded the current single-branch model, and a third is in progress: - **`release-automation`** — release-automation development. Merged into `main` 2026-03-27 ([#135](https://github.com/camaraproject/tooling/pull/135)); the `ra-v1-rc` tag was retired with the merge. - **`validation-framework`** — validation framework and release-automation integration. Merged into `main` 2026-05-05 ([#260](https://github.com/camaraproject/tooling/pull/260)). The unified `v1-rc` tag spanned the merge unchanged. +- **`release-automation-refactor`** *(pending until the exit merge)* — `release-automation-reusable.yml` restructured into four Python-backed phase jobs (`plan`, `snapshot`, `act`, `finalize`). Consumed through the floating tag `ra-v1-rc2` and validated by the branch's own RA canary on [`camaraproject/ReleaseTest2`](https://github.com/camaraproject/ReleaseTest2). Merge date and pull request are recorded here when it merges into `main`; `v1-rc` then moves to the merge, and `ra-v1-rc2` and the branch are deleted. -Both branches are retired. +The first two branches are retired. diff --git a/release_automation/docs/technical-architecture.md b/release_automation/docs/technical-architecture.md index b715ba5..831b6e7 100644 --- a/release_automation/docs/technical-architecture.md +++ b/release_automation/docs/technical-architecture.md @@ -1,6 +1,6 @@ # Release Creation Workflow — Technical Architecture -**Last Updated**: 2026-02-10 +**Last Updated**: 2026-10-24 ## Executive Summary @@ -31,28 +31,43 @@ tooling/ │ │ │ ├── scripts/ │ │ ├── __init__.py -│ │ ├── bot_context.py # BotContext dataclass -│ │ ├── bot_responder.py # Message formatting + template rendering -│ │ ├── changelog_generator.py # CHANGELOG draft generation -│ │ ├── config.py # Shared constants and configuration -│ │ ├── context_builder.py # build_context() for unified context -│ │ ├── git_operations.py # Git operations for snapshot/review branches -│ │ ├── github_client.py # GitHub API wrapper -│ │ ├── issue_manager.py # Issue description/title updates -│ │ ├── issue_sync.py # Release Issue lifecycle management -│ │ ├── mechanical_transformer.py # Placeholder replacement -│ │ ├── metadata_generator.py # release-metadata.yaml generation -│ │ ├── readme_updater.py # README Release Information updater -│ │ ├── release_publisher.py # Draft release publication -│ │ ├── snapshot_creator.py # Snapshot + release-review branch creation -│ │ ├── state_manager.py # State derivation logic -│ │ ├── template_loader.py # Shared template loading utilities -│ │ ├── version_calculator.py # API version extension calculation -│ │ └── workflow_context.py # Workflow context assembly helpers +│ │ ├── act.py # Act job: dispatches the plan's action, writes the JSON outcome +│ │ ├── bot_context.py # BotContext dataclass +│ │ ├── bot_identity.py # Commit identity (App bot or github-actions[bot]) +│ │ ├── bot_responder.py # Message formatting + template rendering +│ │ ├── changelog_generator.py # CHANGELOG draft generation +│ │ ├── command_policy.py # Plan job: Trigger x State -> Plan or Rejection, the response +│ │ ├── common_sync.py # Common cache detection and common sync +│ │ ├── config.py # Shared constants and configuration +│ │ ├── context_builder.py # build_context() for unified context +│ │ ├── draft_release.py # Draft release create/delete, snapshot discard +│ │ ├── finalize.py # Finalize job: sync PR, common sync, Release Issue, result +│ │ ├── git_operations.py # Git operations for snapshot/review branches +│ │ ├── github_client.py # GitHub API wrapper +│ │ ├── issue_events.py # Release Issue auto-reopen +│ │ ├── issue_manager.py # Issue description/title updates +│ │ ├── issue_sync.py # Release Issue lifecycle management +│ │ ├── labels.py # Labels applied by release automation +│ │ ├── mechanical_transformer.py # Placeholder replacement +│ │ ├── metadata_generator.py # release-metadata.yaml generation +│ │ ├── readme_updater.py # README Release Information updater +│ │ ├── regression_runner.py # RA regression canary runner +│ │ ├── release_publisher.py # Draft release publication +│ │ ├── snapshot_creator.py # Snapshot + release-review branch creation +│ │ ├── snapshot_prep.py # Snapshot job glue: settings check, validation gate, bot identity +│ │ ├── state_manager.py # State derivation logic +│ │ ├── sync_pr.py # Post-release sync PR +│ │ ├── template_loader.py # Shared template loading utilities +│ │ ├── trigger.py # Plan job: payload -> Trigger +│ │ ├── version_calculator.py # API version extension calculation +│ │ ├── wip_checker.py # wip version check before a snapshot +│ │ └── workflow_context.py # Workflow context assembly helpers │ │ │ ├── templates/ │ │ ├── bot_messages/ │ │ │ ├── command_rejected.md +│ │ │ ├── common_sync_failed.md +│ │ │ ├── common_sync_pr_created.md │ │ │ ├── config_error.md │ │ │ ├── draft_created.md │ │ │ ├── draft_revoked.md @@ -355,9 +370,9 @@ All context variables are defined in a single `BotContext` dataclass (`bot_conte | Field | Type | Default | Source | |-------|------|---------|--------| | **Trigger fields** | | | | -| `command` | str | `""` | check-trigger | -| `command_args` | str | `""` | check-trigger | -| `user` | str | `""` | check-trigger | +| `command` | str | `""` | trigger | +| `command_args` | str | `""` | trigger | +| `user` | str | `""` | trigger | | **State fields** | | | | | `release_tag` | str | `""` | derive-state | | `state` | str | `""` | derive-state | @@ -391,7 +406,7 @@ All context variables are defined in a single `BotContext` dataclass (`bot_conte | `state_draft_ready` | bool | `False` | derived: `state == "draft-ready"` | | **Display fields** | | | | | `workflow_run_url` | str | `""` | constructed in workflow | -| `draft_release_url` | str | `""` | handle-pr-merge output | +| `draft_release_url` | str | `""` | `create-draft` outcome | | `reason` | str | `""` | command_args (discard/delete) | **APIs list — two data sources:** @@ -493,11 +508,11 @@ Handles publication of draft releases via the `/publish-release` command. ### 2.12 Common Cache Sync -Keeps `code/common/` in sync with the Commonalities release declared under `dependencies.commonalities_release` in `release-plan.yaml`. The handler runs on every `update-issue` pass. For repositories declaring `commonalities_release >= r4.2` it checks the cache and acts on drift; for earlier releases (which don't consume common files via `$ref`) it reports `common_cache_status = ""` (unchecked) and exits without action. +Keeps `code/common/` in sync with the Commonalities release declared under `dependencies.commonalities_release` in `release-plan.yaml`. Detection runs in the `plan` job and the common sync in the `finalize` job. For repositories declaring `commonalities_release >= r4.2` it checks the cache and acts on drift; for earlier releases (which don't consume common files via `$ref`) it reports `common_cache_status = ""` (unchecked) and exits without action. -**Handler location:** Embedded in the `update-issue` job of [release-automation-reusable.yml](../../.github/workflows/release-automation-reusable.yml) rather than a standalone job, so the cache converges on every RA invocation that reaches update-issue (push, `workflow_dispatch`, slash commands ending in a state update). +**Location:** [`common_sync.py`](../scripts/common_sync.py), called from two phase jobs of [release-automation-reusable.yml](../../.github/workflows/release-automation-reusable.yml). The `plan` job runs detection (step *Check common cache sync*) and carries the result in the plan; it blocks `/create-snapshot` while the cache is stale. The `finalize` job runs the common sync when the plan reports `stale`, before the Release Issue sync, so the cache converges on every RA invocation that reaches `finalize` (push, `workflow_dispatch`, slash commands ending in a state update). The module takes a git client and a GitHub client and uses no release-automation types, so it can move to a repository reconcile workflow; the release-state gate stays with the calling job. -**Signal model:** The `derive-state` job outputs `common_cache_status` with three values: +**Signal model:** Detection yields `common_cache_status` with three values: | Value | Meaning | |-------|---------| @@ -520,7 +535,7 @@ Keeps `code/common/` in sync with the Commonalities release declared under `depe - Standard CODEOWNERS review applies (one codeowner approval is sufficient; bot is author) **Data contracts:** -- [`code/common/.sync-manifest.yaml`](../../validation/schemas/sync-manifest-schema.yaml) — records source repository (Commonalities), declared release tag, and per-file git blob SHA-1. Written by the RA sync handler; read by both RA (`derive-state` drift detection) and the validation framework (`P-021 check-common-cache-sync`). +- [`code/common/.sync-manifest.yaml`](../../validation/schemas/sync-manifest-schema.yaml) — records source repository (Commonalities), declared release tag, and per-file git blob SHA-1. Written by the common sync; read by both RA (drift detection in `plan`) and the validation framework (`P-021 check-common-cache-sync`). - [`tooling_lib/cache_sync.py`](../../tooling_lib/cache_sync.py) — shared drift-detection logic (`SyncStatus`, `SourceStatus`, `check_sync_status`, `git_blob_sha`). Imported by both RA and VF so they produce identical verdicts. **Notifications:** @@ -529,7 +544,7 @@ Keeps `code/common/` in sync with the Commonalities release declared under `depe - The Release Issue config section shows the `stale` warning on every sync-issue update until the cache converges **Boundary with validation framework:** -- RA owns detection and repair (writes common files, creates/updates sync PR, writes manifest) +- RA owns detection and the common sync (writes common files, creates/updates sync PR, writes manifest) - VF's `P-021 check-common-cache-sync` is a read-only safety net. It fires only when codeowners bypass the sync process (e.g., direct edits to `code/common/`) — the design expects it to be silent in normal operation - The `camara-release-automation` App has `contents: write`; the `camara-validation` App deliberately does not @@ -610,55 +625,49 @@ Each API repository has a minimal caller workflow that forwards events to the re ### 3.3 Reusable Workflow (Tooling Repository) -The reusable workflow (`release-automation-reusable.yml`) orchestrates all release operations. Jobs execute in phases with conditional branching based on trigger type and command validation. +The reusable workflow (`release-automation-reusable.yml`) has four jobs. `plan` decides, `snapshot` or `act` executes the plan's action, `finalize` reports and syncs. The logic lives in Python modules under `release_automation/scripts/`; the workflow holds the job boundaries, checkouts, tokens and toolchain setup. Jobs pass data as JSON: one `plan` output from `plan`, one `outcome` output from `snapshot` or `act`. -**Workflow/script ref consistency:** Every tooling checkout reads the `job` context of the called reusable workflow: `repository: ${{ job.workflow_repository }}` and `ref: ${{ job.workflow_sha }}`. This binds checkout content to the exact repository and commit of the workflow file that defines the job, whether the caller references a branch, a tag or a SHA, and also when caller and tooling are both forks. There is no caller-side override. +**Workflow/script ref consistency:** Every tooling checkout reads the `job` context of the called reusable workflow: `repository: ${{ job.workflow_repository }}` and `ref: ${{ job.workflow_sha }}`. This binds checkout content to the exact repository and commit of the workflow file that defines the job, whether the caller references a branch, a tag or a SHA, and also when caller and tooling are both forks. There is no caller-side override. Each of the four jobs performs this checkout (path `_tooling`); `snapshot` adds a checkout of the API repository for validation and of the central validation settings. + +**Jobs:** + +| Job | Runs when | Does | +|-----|-----------|------| +| `plan` | Always | Acknowledges the command, classifies the trigger (`trigger.py`), derives the release state, checks the common cache, assembles the context, applies the command policy (`command_policy.py`), posts the one response (interim status, rejection, configuration error or publish confirmation) | +| `snapshot` | Plan action is `create-snapshot` | Pre-snapshot validation, snapshot branch, release-review branch and Release PR; own job because it needs the validation toolchain (Node) | +| `act` | Any other plan action | `act.py` dispatches `discard-snapshot`, `delete-draft`, `publish-release`, `create-draft`, `publish-on-merge` or `reopen-issue` to the matching module | +| `finalize` | Plan asks for it; also after a failed `snapshot` or `act` | Post-release sync PR, common sync, Release Issue sync and notifications, result comment, published Release Issue (`finalize.py`) | **Job execution order:** ```mermaid flowchart TD - CT[check-trigger] --> DS[derive-state] - DS --> AC[assemble-context] - DS -->|slash command| VC[validate-command] - DS -->|config error| HCE[handle-config-error] - - VC -->|rejected| REJ[post-rejection] - VC -->|allowed| CMD - - subgraph CMD [Command Handlers] - direction LR - CS[create-snapshot] - DSN[discard-snapshot] - DD[delete-draft] - PUB[publish-release] - end - - AC -->|pr_merge| HPM[handle-pr-merge
creates draft release] - AC -->|issue event| HIE[handle-issue-event
auto-reopen / transitions] - - CMD --> UI[update-issue
sections, labels, title] - HPM --> UI - HIE --> UI - - CMD --> PR2[post-result
slash commands only] - PUB -->|publish only| CSP[create-sync-pr] - CSP --> PR2 + PLAN["plan
trigger, state, common cache,
context, command policy, response"] + SNAP["snapshot
/create-snapshot"] + ACT["act
discard, delete-draft, publish,
draft on merge, auto-reopen"] + FIN["finalize
sync PR, common sync, Release Issue,
result comment"] + + PLAN -->|action: create-snapshot| SNAP + PLAN -->|other action| ACT + PLAN -->|no action: push, dispatch,
sync-issue, close| FIN + PLAN -->|rejected, config error,
publish confirmation| STOP([responds and stops]) + SNAP --> FIN + ACT --> FIN ``` **3-stage feedback model** (slash commands only): | Stage | Job | Timing | Content | |-------|-----|--------|---------| -| 1. Ack | check-trigger (ack step) | ~8 seconds | Brief acknowledgment that command was received | -| 2. Interim | post-interim | After validate-command passes | Processing status with workflow run link | -| 3. Result | post-result | After command completes | Final outcome (success/failure details) | +| 1. Ack | `plan` (*Post Acknowledgment*) | ~8 seconds | Brief acknowledgment that command was received | +| 2. Interim | `plan` (*Post Response*, `interim_processing`) | After the command policy allows the command | Processing status with workflow run link | +| 3. Result | `finalize` | After `snapshot` or `act` completes | Final outcome (success/failure details) | -All three stages edit the same comment (using `comment_id`). For rejected commands, the ack comment is directly updated with the rejection message (skipping interim). The interim stage is gated on `validate-command.allowed == 'true'`. +All three stages edit the same comment: `plan` passes the acknowledgment's comment ID to `finalize` as `result_comment_id`. For rejected commands, the acknowledgment is updated directly with the rejection message (skipping interim) and `finalize` does not run. -**Central context assembly** (`assemble-context` job): +**Context assembly** (`plan` job): -The `assemble-context` job runs early — right after `derive-state` — and constructs the base BotContext from `check-trigger` and `derive-state` outputs using `build_context()`. All downstream jobs (command handlers, `update-issue`, `post-result`) depend on it and consume context from this single source. Command handler outputs (e.g., `release_pr_url`) are passed as delta context to `post-result` for the final message. +The *Assemble Context* step of `plan` runs after state derivation and the common cache check. It builds the base BotContext with `build_context()` from the trigger outputs and the derived state, and the command policy stores it in the plan as `base_context`. `snapshot`, `act` and `finalize` take the context from the plan, so there is a single source. A command's outcome fields (for example `release_pr_url`) are passed to `finalize` as delta context for the result comment. --- @@ -821,9 +830,9 @@ release-metadata.yaml (snapshot branch, if exists) │ ▼ ┌──────────────────────┐ - │ assemble-context │◄── check-trigger outputs (command, user, trigger_type) + │ Assemble Context │◄── trigger outputs (command, user, trigger_type) │ └─ build_context() │◄── derive-state outputs (all above) - │ │◄── command action outputs (release_pr_url, draft_release_url, ...) + │ (plan job) │◄── command action outcome (release_pr_url, draft_release_url, ...) │ │◄── workflow env (workflow_run_url) └────────┬─────────────┘ │ @@ -924,10 +933,12 @@ Rule P-033 (`check-release-plan-active-release-state`) blocks the PR with an err | Operation | Target | Approach | |-----------|--------|----------| -| Ack comment | < 5 seconds | Lightweight job, no checkout, no Python | +| Ack comment | < 5 seconds | First steps of `plan`, before any checkout or Python setup | | State derivation | < 10 seconds | Direct GitHub API calls | -| Full `/create-snapshot` | < 3 minutes | Sequential but optimized | -| Result comment | < 5 seconds | Template rendering + API call | +| Full `/create-snapshot` | < 3 minutes | Three runners (`plan`, `snapshot`, `finalize`); only `snapshot` sets up Node and the validation toolchain | +| Result comment | < 5 seconds | Template rendering + API call in `finalize` | + +Indicative timings from single runs on a test repository, measured from the command to the result comment and including the caller run: `/create-snapshot` about 110 seconds, `/discard-snapshot` about 85 seconds (`plan` 25, `act` 15, `finalize` 22). Differences of about 15 seconds between runs are noise. ---