Skip to content

feat: [1/5] add cloud platform and trust-store capabilities - #9910

Open
Abigail Liang (abigailliang-aks-sig-node) wants to merge 3 commits into
mainfrom
abigailliang/anc-cloud-store
Open

Abigail Liang (abigailliang-aks-sig-node) wants to merge 3 commits into
mainfrom
abigailliang/anc-cloud-store

Conversation

@abigailliang-aks-sig-node

@abigailliang-aks-sig-node Abigail Liang (abigailliang-aks-sig-node) commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

Part [1/5] of the native ANC cloud-initialization series. This replaces the
foundation portion of #9902 with a smaller, independently reviewable layer.
It adds internal capabilities only: no certificate retrieval, refresh CLI,
refresh scheduling, or provisioning-path activation.

  • Add the shared cloud runtime, platform detection, testable file/command
    execution, bounded clients/commands, cancellation, and guest-agent events.
  • Extract the unchanged certificate staging-cache and trust-store helpers.
    Keep cleanup nonrecursive and separate from installed trust-store certificates.
  • Support Ubuntu, Mariner, Azure Linux, Flatcar, and Azure Container Linux.
    Retain raw certificate installation and diagnostic-only public-certificate
    parsing; do not introduce new certificate validation or filtering.
  • Keep runtime/store coverage with the implementation, including direct platform,
    raw-payload, cache-cleanup, logging, and event tests.

The incremental scope is 9 files, +602/-1, down from #9902's 10 files,
+1,285/-1. The existing Go implementation's behavior is unchanged by extraction.
Provisioning, NBC precedence, the native shell template, generated CSE fixtures,
CustomData delivery, and VHD mappings are unchanged.

Migration and PR split (series overview)

The diagrams describe the complete five-part series, not activation in this PR.
Parts 1 through 4 leave provisioning on the existing shell path. Only part 5
switches the native provisioning path to Go.

Migration: initialization moves to Go; the existing provisioning scripts remain.

flowchart TB
    subgraph Before["Before migration - also after PRs 1 through 4"]
        B1["ANC: App.Provision"] --> B2["cse_cmd.sh"]
        B2 --> B3["init-aks-cloud.sh, if present<br/>Cloud initialization responsibilities"]
        B3 --> B4["provision_start.sh<br/>Existing provisioning scripts"]
    end

    subgraph After["After PR 5 - native provisioning path"]
        A1["ANC: App.Provision"] --> A2["initializeAKSCloud (Go)<br/>Initialize certificates"]
        A2 --> A3["Go: configure refresh scheduling, if eligible"]
        A3 --> A4["Go: initialize RepoDepot"]
        A4 --> A5["cse_cmd.sh<br/>Shell initialization call removed"]
        A5 --> A6["provision_start.sh<br/>Existing provisioning scripts"]
    end

    B3 -.->|"Responsibilities reimplemented in Go"| A2

    classDef migrated fill:#CFE4FA,stroke:#0078D4,color:#000000
    classDef unchanged fill:#DFF6DD,stroke:#107C10,color:#000000
    class B3,A2,A3,A4 migrated
    class B4,A6 unchanged
Loading

After part 5, only --provision-config without --nbc-cmd uses the new path.
Legacy CSE/NBC remain unchanged; NBC takes precedence when both flags are supplied.
The original shell artifact remains available for those compatibility paths.

PR split: dependency and merge order, not runtime execution order.

flowchart LR
    MAIN["main"]
    P1["[1/5] #9910<br/>Shared runtime and platform support<br/>Certificate staging and trust stores"]
    P2["[2/5] #9911<br/>WireServer request and retry protocols<br/>Legacy and RCV1P certificate retrieval"]
    P3["[3/5] #9909<br/>Certificate refresh orchestration<br/>Standalone refresh command"]
    P4["[4/5] #9903<br/>RepoDepot configuration<br/>Package-cache refresh and retries"]
    P5["[5/5] #9904<br/>Native Go activation and refresh scheduling<br/>Paired shell-call removal"]

    MAIN -->|"base for"| P1
    P1 -->|"base for"| P2
    P2 -->|"base for"| P3
    P3 -->|"base for"| P4
    P4 -->|"base for"| P5

    classDef capability fill:#CFE4FA,stroke:#0078D4,color:#000000
    classDef activation fill:#E8DAEF,stroke:#5C2D91,color:#000000
    class P1,P2,P3,P4 capability
    class P5 activation
Loading

Part 5 keeps activation atomic: adding the Go call without removing the
shell call would initialize twice; removing the shell call without adding the
Go call would skip initialization.

Series and merge order

  1. [1/5] feat: [1/5] add cloud platform and trust-store capabilities #9910: Platform and trust-store capabilities -- this PR, based on main.
  2. [2/5] feat: [2/5] add cloud certificate retrieval protocols #9911: Certificate retrieval protocols -- based on abigailliang/anc-cloud-store.
  3. [3/5] feat: [3/5] add cloud certificate refresh orchestration and CLI #9909: Certificate refresh and CLI -- based on abigailliang/anc-cloud-protocols.
  4. [4/5] feat: [4/5] add native RepoDepot initialization #9903: RepoDepot capabilities -- based on abigailliang/anc-cloud-refresh.
  5. [5/5] feat: [5/5] enable native cloud initialization before provisioning #9904: Native activation -- based on abigailliang/anc-cloud-repodepot.

Review and merge in order. Part 1 is ready for review; parts 2 through 5 are
dependency drafts. For each dependent PR, merge the updated main into its
branch before retargeting it to main and running required CI, especially if
the prerequisite was squash-merged. Do not merge a child PR into its
prerequisite's feature branch.

The aggregate #9891 and former certificate PR #9902 retain their branches,
commits, and discussions for reference. This split uses additive commits/merges;
no published branch history is rewritten.

Validation

Passed the applicable ANC/parser race suites, CI-pinned golangci-lint v2.14.0
against the layer's base, and Linux AMD64/ARM64 builds. Direct tests cover the
newly isolated runtime/store boundaries. Azure/VHD end-to-end provisioning has
not been run.

Which issue(s) this PR fixes:

  • Work item: AB#40380207 -- [1/5] Add cloud runtime, platform detection and trust-store capabilities
  • Parent user story: AB#40379991 -- Migrate init-aks-cloud initialization to ANC Go and remove the native/scriptless shell dependency

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Windows Unit Test Results

  3 files   18 suites   1m 17s ⏱️
662 tests 662 ✅ 0 💤 0 ❌
685 runs  685 ✅ 0 💤 0 ❌

Results for commit a924c32.

♻️ This comment has been updated with latest results.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Descendant-held output pipes can keep command execution blocked beyond its configured deadline.

1 open issue
What changed in this PR

Adds the internal runtime and trust-store foundation for native ANC cloud initialization. This first layer leaves provisioning on the existing shell path.

Changes:

  • Adds platform detection, HTTP clients, command execution, file writes, and guest-agent events.
  • Adds certificate staging cleanup and platform-specific trust-store installation without filtering raw certificate payloads.
  • Adds runtime/store tests and documents the capability-only scope.
File Description
aks-node-controller/​README.md Documents capabilities and unchanged provisioning behavior.
aks-node-controller/​embeddednodecustomdata.go Shares the Mariner OS identifier constant.
aks-node-controller/​cloud_test_helpers_test.go Adds shared runtime and certificate test helpers.
aks-node-controller/​cloud_runtime.go Adds platform, HTTP, command, file, and event helpers.
aks-node-controller/​cloud_runtime_test.go Tests file writes, platform detection, and response limits.
aks-node-controller/​cloud_runtime_events_test.go Tests operation event emission.
aks-node-controller/​cloud_certificate_store.go Adds staging cleanup, trust-store installation, and public-certificate diagnostics.
aks-node-controller/​cloud_certificate_store_test.go Tests preservation of Ubuntu's bundle symlink.
aks-node-controller/​cloud_certificate_store_behavior_test.go Tests platform installation, cleanup, raw payloads, and logging.

Review effort: Balanced

Comment thread aks-node-controller/cloud_runtime.go

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Subprocesses inheriting stdout or stderr can keep cmd.Run() blocked after cancellation, defeating caller deadlines and the two-minute command limit because process-group cancellation and a finite WaitDelay are missing.

1 open issue

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings October 29, 2026 00:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Background child processes can survive caller cancellation when the parent exits before output draining finishes, because the cancellation callback is no longer invoked and WaitDelay does not terminate the remaining process group.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings October 29, 2026 00:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified for the runtime and certificate trust-store changes.

No open issues 🎉

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified for the foundational runtime and certificate trust-store helpers.

No open issues 🎉

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings October 29, 2026 18:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the runtime and certificate trust-store changes.

No open issues 🎉

Review effort: Balanced

Separate the unchanged cloud runtime and trust-store installation helpers from certificate retrieval and orchestration. Add direct capability coverage without activating provisioning.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: be499bac-1283-498e-b5b1-59d8c91047a4
Copilot AI balanced review requested due to automatic review settings October 29, 2026 21:14
stacked auto-merge was automatically disabled October 29, 2026 21:14

Replaced by a newer auto-merge request

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified in the foundational runtime and certificate trust-store helper changes.

No open issues 🎉

Review effort: Balanced

@github-actions

Copy link
Copy Markdown
Contributor

Code Coverage

Package Line Rate Health
test 0% ❌
windows 27% ❌
staging_cse_windows Package 1 68% ➖
debug 0% ❌
provisioningscripts 2% ❌
parts_windows Package 1 73% ➖
Summary 39% (1819 / 7001) ❌

@github-actions

Copy link
Copy Markdown
Contributor

Code Coverage

Package Line Rate Health
shellspec spec 29% ❌
Summary 29% (4735 / 16396) ❌

@github-actions

Copy link
Copy Markdown
Contributor

Code Coverage

Package Line Rate Health
github.com/Azure/agentbaker/aks-node-controller 73% ➖
github.com/Azure/agentbaker/aks-node-controller/common 100% ✔
github.com/Azure/agentbaker/aks-node-controller/helpers 68% ➖
github.com/Azure/agentbaker/aks-node-controller/parser 88% ✔
github.com/Azure/agentbaker/aks-node-controller/pkg/gpu 100% ✔
github.com/Azure/agentbaker/aks-node-controller/pkg/nodeconfigutils 89% ✔
github.com/Azure/agentbaker/aks-node-controller/utils 0% ❌
github.com/Azure/agentbaker/apiserver 24% ❌
github.com/Azure/agentbaker/cmd 0% ❌
github.com/Azure/agentbaker/cmd/starter 0% ❌
github.com/Azure/agentbaker/fuzz/api 0% ❌
github.com/Azure/agentbaker/hotfix/render-nodecustomdata 35% ❌
github.com/Azure/agentbaker/pkg/agent 78% ➖
github.com/Azure/agentbaker/pkg/agent/datamodel 77% ➖
github.com/Azure/agentbaker/pkg/agent/toggles 0% ❌
github.com/Azure/agentbaker/pkg/vhdbuilder/datamodel 87% ✔
Summary 75% (6209 / 8220) ➖

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants