Skip to content

feat: [1/3] add native cloud certificate capabilities - #9902

Closed
Abigail Liang (abigailliang-aks-sig-node) wants to merge 1 commit into
mainfrom
abigailliang/anc-cloud-certificates
Closed

Abigail Liang (abigailliang-aks-sig-node) wants to merge 1 commit into
mainfrom
abigailliang/anc-cloud-certificates

Conversation

@abigailliang-aks-sig-node

@abigailliang-aks-sig-node Abigail Liang (abigailliang-aks-sig-node) commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Superseded by the five-part series

The certificate work in this PR has been split into three smaller layers.
Review and merge the complete replacement series in this order:

  1. [1/5] feat: [1/5] add cloud platform and trust-store capabilities #9910 -- shared runtime, platform support, and certificate trust stores.
  2. [2/5] feat: [2/5] add cloud certificate retrieval protocols #9911 -- WireServer, legacy, and RCV1P retrieval protocols.
  3. [3/5] feat: [3/5] add cloud certificate refresh orchestration and CLI #9909 -- certificate-refresh orchestration and standalone CLI.
  4. [4/5] feat: [4/5] add native RepoDepot initialization #9903 -- RepoDepot capabilities; original PR ID/history retained.
  5. [5/5] feat: [5/5] enable native cloud initialization before provisioning #9904 -- native Go activation, refresh scheduling, and paired
    shell-call removal; original PR ID/history retained.

Only [5/5] switches the native provisioning path. Parts 1 through 4 retain
the existing shell path. The current English migration and five-part
dependency diagrams are in #9910.

This PR's branch, commits, and discussion are retained for historical
comparison. Its original three-part description and diagrams are preserved
below as historical context, not the current review/merge plan.


What this PR does / why we need it:

Part [1/3] of the native ANC cloud-initialization series split from #9891.
This PR adds certificate capabilities only; it does not switch provisioning away
from init-aks-cloud.sh.

  • Extract the shared cloud runtime and the existing Go certificate implementation
    into focused files without changing the original implementation's behavior.
  • Support legacy and RCV1P certificate retrieval, trust-store installation, cache
    refresh, bounded HTTP/command execution, cancellation, and guest-agent events.
  • Expose the standalone refresh-cloud-certificates --location <region> command.
    It does not configure RepoDepot or install cron/systemd schedules.
  • Keep shell-compatible opt-in handling, raw downloaded certificate installation,
    per-certificate download skipping, and best-effort legacy trust-store updates.
  • Include certificate/runtime/CLI coverage and direct refresh cases across Ubuntu,
    Mariner, Azure Linux, Flatcar, and Azure Container Linux.

Existing provisioning, NBC precedence, the native shell template, generated CSE
fixtures, CustomData delivery, and VHD mappings are unchanged.

Migration and PR split (series overview)

The diagrams below describe the complete series, not activation in this PR.
PRs 1 and 2 add capabilities; the native provisioning-path switch happens only
in PR 3.

Migration: initialization moves to Go; the existing provisioning scripts remain.

flowchart TB
    subgraph Before["Before migration - also after PRs 1 and 2"]
        B1["ANC: App.Provision"] --> B2["cse_cmd.sh"]
        B2 --> B3["init-aks-cloud.sh, if present<br/>Certificates / RepoDepot / refresh scheduling"]
        B3 --> B4["provision_start.sh<br/>Existing provisioning scripts"]
    end

    subgraph After["After PR 3 - native provisioning path"]
        A1["ANC: App.Provision"] --> A2["Go: initializeAKSCloud<br/>1. Initialize certificates<br/>2. Configure refresh scheduling, if eligible<br/>3. Initialize RepoDepot"]
        A2 --> A3["cse_cmd.sh<br/>Shell initialization call removed"]
        A3 --> A4["provision_start.sh<br/>Existing provisioning scripts"]
    end

    B3 -.->|"Initialization responsibilities reimplemented in Go"| A2

    classDef migrated fill:#CFE4FA,stroke:#0078D4,color:#000000
    classDef unchanged fill:#DFF6DD,stroke:#107C10,color:#000000
    class B3,A2 migrated
    class B4,A4 unchanged
Loading

After PR 3, only --provision-config without --nbc-cmd uses the new path.
Legacy CSE/NBC remain unchanged; NBC takes precedence when both flags are supplied.
The original shell artifact remains available for those compatibility paths.

PR split: dependency and merge order, not runtime execution order.

flowchart LR
    MAIN["main"]

    P1["[1/3] #9902<br/>Shared runtime and certificate capabilities<br/>Standalone certificate-refresh command<br/>No provisioning-path switch"]

    P2["[2/3] #9903<br/>RepoDepot configuration<br/>Package-cache refresh and retries<br/>Still no provisioning-path switch"]

    P3["[3/3] #9904<br/>Wire Go initialization into App.Provision<br/>Add cron/systemd refresh scheduling<br/>Remove the native shell-init call<br/>Switch the native provisioning path"]

    MAIN -->|"base for"| P1
    P1 -->|"base for"| P2
    P2 -->|"base for"| P3

    classDef capability fill:#CFE4FA,stroke:#0078D4,color:#000000
    classDef activation fill:#E8DAEF,stroke:#5C2D91,color:#000000
    class P1,P2 capability
    class P3 activation
Loading

PR 3 keeps activation atomic: adding the Go call without removing the shell
call would initialize twice; removing the shell call without adding the Go call
would skip initialization.

Series and merge order

  1. [1/3] feat: [1/3] add native cloud certificate capabilities #9902: Certificate capabilities -- this PR, based on main.
  2. [2/3] feat: [4/5] add native RepoDepot initialization #9903: RepoDepot capabilities -- depends on this PR.
  3. [3/3] feat: [5/5] enable native cloud initialization before provisioning #9904: Native entry-point activation -- depends on [2/3] and switches the Go
    call and shell-template removal together.

The aggregate #9891 remains a reference; review and merge the split series in order.

Validation

From aks-node-controller, passed the ANC/parser race suites, CI-pinned
golangci-lint v2.14.0 against the layer's base, and Linux AMD64/ARM64 builds.
Confirmed no changes to the provisioning template or native status-reporting
implementation. Azure/VHD end-to-end provisioning has not been run.

Which issue(s) this PR fixes:

No linked issue.

Split certificate retrieval, trust-store installation, shared runtime, and the standalone refresh command from #9891. Preserve certificate failure semantics while keeping existing provisioning on the shell path. Include corresponding unit coverage and direct refresh tests for all supported OS families.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: be499bac-1283-498e-b5b1-59d8c91047a4
@github-actions

Copy link
Copy Markdown
Contributor

Windows Unit Test Results

  3 files   18 suites   55s ⏱️
563 tests 563 ✅ 0 💤 0 ❌
585 runs  585 ✅ 0 💤 0 ❌

Results for commit c502637.

@github-actions

Copy link
Copy Markdown
Contributor

Code Coverage

Package Line Rate Health
test 0% ❌
windows 27% ❌
staging_cse_windows Package 1 65% ➖
debug 0% ❌
provisioningscripts 2% ❌
parts_windows Package 1 73% ➖
Summary 38% (1652 / 6834) ❌

@github-actions

Copy link
Copy Markdown
Contributor

Code Coverage

Package Line Rate Health
github.com/Azure/agentbaker/aks-node-controller 74% ➖
github.com/Azure/agentbaker/aks-node-controller/common 100% ✔
github.com/Azure/agentbaker/aks-node-controller/helpers 68% ➖
github.com/Azure/agentbaker/aks-node-controller/parser 88% ✔
github.com/Azure/agentbaker/aks-node-controller/pkg/gpu 100% ✔
github.com/Azure/agentbaker/aks-node-controller/pkg/nodeconfigutils 89% ✔
github.com/Azure/agentbaker/aks-node-controller/utils 0% ❌
github.com/Azure/agentbaker/apiserver 24% ❌
github.com/Azure/agentbaker/cmd 0% ❌
github.com/Azure/agentbaker/cmd/starter 0% ❌
github.com/Azure/agentbaker/fuzz/api 0% ❌
github.com/Azure/agentbaker/hotfix/render-nodecustomdata 35% ❌
github.com/Azure/agentbaker/pkg/agent 78% ➖
github.com/Azure/agentbaker/pkg/agent/datamodel 77% ➖
github.com/Azure/agentbaker/pkg/agent/toggles 0% ❌
github.com/Azure/agentbaker/pkg/vhdbuilder/datamodel 87% ✔
Summary 75% (6305 / 8362) ➖

@github-actions

Copy link
Copy Markdown
Contributor

Code Coverage

Package Line Rate Health
shellspec spec 28% ❌
Summary 28% (4591 / 16082) ❌

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A child process retaining stdout/stderr can keep the refresh blocked beyond both timeout budgets; process-group cancellation, a finite WaitDelay, and a subprocess regression test are needed.

1 open issue
What changed in this PR

Adds native cloud certificate capabilities to ANC as the first part of the cloud-initialization series. Provisioning continues to use init-aks-cloud.sh; repository configuration and refresh scheduling remain out of scope.

Changes:

  • Adds legacy and RCV1P certificate retrieval, caching, and platform-specific trust-store installation.
  • Exposes refresh-cloud-certificates --location <region> with timeouts and guest-agent events.
  • Adds runtime, certificate, and CLI coverage across supported Linux platforms.
File Description
aks-node-controller/​README.md Documents refresh usage, behavior, and scope.
aks-node-controller/​embeddednodecustomdata.go Shares the Mariner OS identifier constant.
aks-node-controller/​cloud_test_helpers_test.go Adds certificate fixtures and mock endpoints.
aks-node-controller/​cloud_runtime.go Adds shared platform, HTTP, command, file, and event helpers.
aks-node-controller/​cloud_runtime_test.go Tests file writes and platform detection.
aks-node-controller/​cloud_refresh.go Implements standalone refresh and location normalization.
aks-node-controller/​cloud_refresh_test.go Tests refresh arguments and CLI error handling.
aks-node-controller/​cloud_certificates.go Implements certificate retrieval, caching, and trust-store updates.
aks-node-controller/​cloud_certificates_test.go Tests platform behavior, retries, cancellation, and certificate handling.
aks-node-controller/​app.go Registers the refresh command and test override.

Review effort: Balanced

Comment on lines +145 to +149
cmd := exec.CommandContext(ctx, name, args...)
cmd.Env = append(os.Environ(), "LC_ALL=C")
cmd.Stdin = bytes.NewReader(input)
var stdout, stderr bytes.Buffer
cmd.Stdout, cmd.Stderr = &stdout, &stderr
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants