[MrBot] Add pipeline-safe dependency propagation mode by parth21999 · Pull Request #473 · Azure/c-build-tools

Parth Aggarwal (parth21999) · 2026-10-21T00:42:51Z

Summary

update_deps/propagate_updates.ps1 gains a -Pipeline mode for unattended Azure DevOps runs, merging the Azure-MessagingStore implementation, while keeping the interactive workflow. Both modes share one helper set.

One entry point, one helper set. propagate_updates.ps1 loads helper_scripts/ through load_helpers.ps1, and the code branches on the mode only where a workstation and a build agent need different behavior. The earlier pipeline_helper_scripts/ copy and propagate_updates_pipeline.ps1 are gone.
Interactive mode (default) keeps WAM/PAT sign-in, known_graph.json reuse, -Resume, -AutoFix, existing-PR recovery, and the live PR watchers.
-Pipeline keeps the Store behavior. It never prompts, uses SYSTEM_ACCESSTOKEN for Azure DevOps REST and git and GH_TOKEN for GitHub, and runs preflight checks before opening any PR. It also redacts secrets, time-bounds native calls, discovers the graph live, and opens each PR as soon as its dependencies have merged. It supports -azure_work_item, -root_list, -poll_interval, and -NoCloseFailedPr.
Shared by both modes:
- Azure DevOps calls go through REST, so the azure-devops CLI extension is no longer needed. Reads retry transient failures, writes retry only on 429/503, and a rejected WAM token is refreshed once.
- Git uses a mirror cache, and network git commands are retried and abort on stalled transfers.
- Only submodule paths and build/ are staged.
- Each PR gets the most detailed description that fits the service limit.
Fixes:
- Helpers no longer declare param() blocks; dot-sourcing them overwrote the entry script's parameters, such as -poll_interval.
- Ctrl+C during an Azure PR watch now stops watching.
- A failure while closing a failed PR no longer recurses.
- Pipeline git credentials replace an Azure DevOps extraheader the caller already set (the EBS YAML sets one), so git sends a single Authorization header.
Docs: update_deps/dependency_updates_propagation.md covers the modes, the pipeline environment contract, a YAML example, migration, and tests.

Pipeline migration

$script = Join-Path $env:BUILD_SOURCESDIRECTORY 'deps\c-build-tools\update_deps\propagate_updates.ps1'
& $script -Pipeline -azure_work_item $workItemId -root_list $roots -NoCloseFailedPr
exit $LASTEXITCODE

Map SYSTEM_ACCESSTOKEN: $(System.AccessToken) and GH_TOKEN into the task environment.

Validation on `00711e2d`

Azure-C-Build-Tools-Gate build 186969307 passed on the PR merge commit 4bdd3282, whose parents are master and 00711e2d.
EBS propagation run 186970231 verified that its c-build-tools checkout was exactly 00711e2d, then ran -Pipeline under Windows PowerShell 5.1 on a real agent:
- The pipeline's own GIT_CONFIG_GLOBAL (bot identity and Azure DevOps extraheader) was kept, and gh auth setup-git completed.
- The SYSTEM_ACCESSTOKEN Azure DevOps preflight passed for both roots.
- Every Azure DevOps git clone succeeded alongside that extraheader, and live discovery built the full 18-repository graph.
- The run then stopped only at the intentional GitHub write-access preflight, because the test token reports permissions.push=false for the Azure upstream repositories. No implementation failure was observed. A full write propagation needs an SSO-authorized GitHub token with push access to those repositories.
update_deps/tests/test_propagate_updates.ps1 passes under Windows PowerShell 5.1 and PowerShell 7.6. Services are mocked, and git runs only against local repositories. The suite covers:
- a local end-to-end update-local-repo run in both modes;
- a loopback HTTP check that git sends exactly one Authorization header.
The suite is registered with CTest as update_deps_propagation_test and update_deps_propagation_pwsh_test when run_unittests=ON. Both pass.
Reverting each fix from the final review makes its test fail.
A read-only smoke test against live services passed in both PowerShell versions. It covered Azure DevOps REST through WAM (PR, policy evaluations, build timeline, and the access check) and GitHub (PR status and checks).
All scripts parse in both versions, and git diff --check is clean. The repository validator reports only its existing findings, none in files touched here.
Not exercised live: interactive write operations (creating real PRs with WAM or a PAT, -Resume, -AutoFix, Ctrl+C). The mocked tests cover their logic.

Earlier validation

The previous head 07a3146d (before deduplication) passed gate build 186829434 and stopped at the same GitHub write-access preflight in EBS run 186828447.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2cc24a4e-c845-4bfe-99ea-c31ce12fb4bd

azure-pipelines · 2026-10-21T00:43:01Z

Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run.

Parth Aggarwal (parth21999) · 2026-10-21T00:43:16Z

/AzurePipelines run

azure-pipelines · 2026-10-21T00:43:23Z

You have several pipelines (over 10) configured to build pull requests in this repository. Specify which pipelines you would like to run by using /azp run [pipelines] command. You can specify multiple pipelines using a comma separated list.

Parth Aggarwal (parth21999) · 2026-10-21T01:13:09Z

/azp run Azure-C-Build-Tools-Gate

azure-pipelines · 2026-10-21T01:13:23Z

Azure Pipelines: Successfully started running 1 pipeline(s).

The previous commit added -Pipeline by copying the Azure-MessagingStore helpers into update_deps/pipeline_helper_scripts, which left two copies of the propagation code. Merge them into helper_scripts, loaded once by load_helpers.ps1, and branch on the mode only where a workstation and a build agent need different behavior. - propagate_updates.ps1 is the only entry point. -Pipeline selects the noninteractive mode and rejects the interactive-only arguments. - Both modes call Azure DevOps through invoke-ado-rest, using a WAM token from the Azure CLI or a PAT interactively and SYSTEM_ACCESSTOKEN in pipelines. Reads retry transient failures, writes retry only 429/503, and a rejected WAM token is refreshed once. The azure-devops CLI extension is no longer needed. - Both modes use the git mirror cache, bounded and retried network git commands, staging limited to submodules and build/, and the most detailed PR description that fits the service limit. - Interactive mode keeps WAM/PAT sign-in, known_graph.json, -Resume, -AutoFix, existing-PR recovery and the live watchers. Pipeline mode keeps the preflight checks, redaction, greedy scheduling and live graph discovery. - Pipeline git credentials go through GIT_CONFIG_PARAMETERS and replace an Azure DevOps extraheader the caller already configured, so git sends a single Authorization header. - Helpers no longer declare script parameters, which overwrote the entry script's parameters when dot-sourced. - Cancelling an Azure PR watch with Ctrl+C now stops watching, and a failure while closing a failed PR no longer recurses. - Add update_deps/tests/test_propagate_updates.ps1, run by CTest under Windows PowerShell and PowerShell 7 when run_unittests is ON. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2cc24a4e-c845-4bfe-99ea-c31ce12fb4bd

Parth Aggarwal (parth21999) · 2026-10-21T19:15:37Z

/azp run Azure-C-Build-Tools-Gate

azure-pipelines · 2026-10-21T19:15:54Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Parth Aggarwal (parth21999) · 2026-10-21T20:50:41Z

+if(WIN32 AND ${run_unittests} AND ("${CMAKE_SOURCE_DIR}" STREQUAL "${CMAKE_CURRENT_SOURCE_DIR}"))
+    add_subdirectory(update_deps/tests)
+endif()
+


top-level projects never add c-build-tools tests anyway...

Parth Aggarwal (parth21999) · 2026-10-21T20:51:48Z

+- **Workflow**: `propagate_updates.ps1` builds the dependency graph (`helper_scripts/build_graph.ps1`) and opens PRs bottom-up
+- **Modes**: Interactive by default (WAM or PAT, `gh auth login`, `-Resume`, `-AutoFix`); `-Pipeline` runs unattended on Azure DevOps agents with `SYSTEM_ACCESSTOKEN` and `GH_TOKEN`
+- **Shared helpers**: Both modes load one helper set via `helper_scripts/load_helpers.ps1`; branch on `test-pipeline-mode` only where behavior must differ, and call Azure DevOps through `invoke-ado-rest`
+- **Tests**: `update_deps/tests/test_propagate_updates.ps1` (CTest `update_deps_propagation_test` when `run_unittests=ON`)


This is describing the implementation, not how to use it. keep this doc minimal with instructional changes only

Parth Aggarwal (parth21999) · 2026-10-21T20:55:18Z

+        }
+    }
+
+    run-test 'command references resolve' {


run-test 'command

can we use the arrange-act-assert pattern and the Steps: pattern to make the tests more readable?

Add pipeline-safe dependency propagation mode

07a3146

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2cc24a4e-c845-4bfe-99ea-c31ce12fb4bd

microsoft-github-policy-service Bot requested a review from anporumb October 21, 2026 00:43

Parth Aggarwal (parth21999) commented Oct 21, 2026

View reviewed changes

Parth Aggarwal (parth21999) mentioned this pull request Oct 22, 2026

[MrBot] Replace update_deps with the Azure-MessagingStore pipeline version #474

Open

Navigation Menu

Repository navigation

[MrBot] Add pipeline-safe dependency propagation mode - #473

[MrBot] Add pipeline-safe dependency propagation mode#473
Parth Aggarwal (parth21999) wants to merge 2 commits into
masterfrom
paaggarwal-microsoft-cbt-propagate-merge

Parth Aggarwal (parth21999) commented Oct 21, 2026 •

edited

Loading

Uh oh!

azure-pipelines Bot commented Oct 21, 2026

Uh oh!

Parth Aggarwal (parth21999) commented Oct 21, 2026

Uh oh!

azure-pipelines Bot commented Oct 21, 2026

Uh oh!

Parth Aggarwal (parth21999) commented Oct 21, 2026

Uh oh!

azure-pipelines Bot commented Oct 21, 2026

Uh oh!

Parth Aggarwal (parth21999) commented Oct 21, 2026

Uh oh!

azure-pipelines Bot commented Oct 21, 2026

Uh oh!

Parth Aggarwal (parth21999) Oct 21, 2026

Uh oh!

Parth Aggarwal (parth21999) Oct 21, 2026

Uh oh!

Parth Aggarwal (parth21999) Oct 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

Parth Aggarwal (parth21999) commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Pipeline migration

Validation on 00711e2d

Earlier validation

Uh oh!

azure-pipelines Bot commented Oct 21, 2026

Uh oh!

Parth Aggarwal (parth21999) commented Oct 21, 2026

Uh oh!

azure-pipelines Bot commented Oct 21, 2026

Uh oh!

Parth Aggarwal (parth21999) commented Oct 21, 2026

Uh oh!

azure-pipelines Bot commented Oct 21, 2026

Uh oh!

Parth Aggarwal (parth21999) commented Oct 21, 2026

Uh oh!

azure-pipelines Bot commented Oct 21, 2026

Uh oh!

Parth Aggarwal (parth21999) Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

Parth Aggarwal (parth21999) Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

Parth Aggarwal (parth21999) Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Parth Aggarwal (parth21999) commented Oct 21, 2026 •

edited

Loading

Validation on `00711e2d`