Skip to content

[EngSys] Add opt-in Rust native wheel build support - #49397

Merged
jenny (JennyPng) merged 9 commits into
Azure:mainfrom
JennyPng:engsys/rust-native-build-support
Oct 12, 2026
Merged

jenny (JennyPng) merged 9 commits into
Azure:mainfrom
JennyPng:engsys/rust-native-build-support

Conversation

@JennyPng

@JennyPng jenny (JennyPng) commented Oct 7, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Port the reusable native wheel-build support from python-rust-integration #46851 and the prototype in Rust Driver EngSys Prototype #48867 onto current main

  • Route packages with a [tool.cibuildwheel] table through cibuildwheel, including non-setuptools backends such as Maturin.

  • Add opt-in Microsoft Rust toolchain setup, feed authentication, and parameter forwarding through the shared build templates.

  • Forward Rust/Cargo settings into Linux wheel-build containers, expose Windows ARM64 Python import libraries, and allow longer timeout on native builds.

  • Add generic build-routing tests, PowerShell-based pipeline tests, and native-build documentation.

Minor differences made by this PR

  • minor change to build-package-artifacts.yml <- No default Rust Toolchain feed URL is set, Cosmos should explicitly pass it in in their pipeline (reason: unsure if the current 'test-org' feed is intended to be long term, can add default back later)
  • install-msrust-toolchain.yml hard-codes the Crates Io Registry name rather than allowing it to be passed in, since there doesn't appear to currently be a reason for this to change

Scope and follow-ups

This PR does not change any package under sdk/ or activate Cosmos.

Remaining TODO:

  • Source-installing test/documentation jobs
  • nightly version synchronization (version updaters only update Python's version file, not Rust)

TODO FOR COSMOS after this merges:

  • Cosmos should keep their cosmos-sdk-client.yml change on their prototype branch to opt into installing MsRust and signing binaries
    • Keep the Cosmos native package changes, including its build backend,  rust-toolchain.toml , bash scripts for installing msrustup and configuring cargo feed in the linux container, and the cibuildwheel configuration/hooks.

  • forward the parameters to archetype-sdk-client:

InstallMsRustToolchain
MsRustWorkingDirectory
MsRustToolchainFeed
MsRustAdditionalTargets
  • set signBinaries to true

e.g.

extends:
  template: /eng/pipelines/templates/stages/cosmos-sdk-client.yml
  parameters:
    ServiceDirectory: cosmos

    InstallMsRustToolchain: true
    MsRustWorkingDirectory: $(Build.SourcesDirectory)/sdk/cosmos/azure-cosmos
    MsRustToolchainFeed: <APPROVED_ORGANIZATION_LOCAL_TOOLCHAIN_FEED_URL>
    MsRustAdditionalTargets: aarch64-pc-windows-msvc

    Artifacts:
    - name: azure-cosmos
      safeName: azurecosmos
      signBinaries: true

Validation

PYTHONPATH=eng/tools/azure-sdk-tools python -m pytest \
  eng/tools/azure-sdk-tools/tests/test_build_interactions.py \
  eng/tools/azure-sdk-tools/tests/test_parse_functionality.py \
  eng/tools/azure-sdk-tools/tests/test_native_build_pipeline.py -q

Result: 47 passed, 1 skipped. The skip is an existing namespace-discovery test whose expected package path is absent.

Black checks passed for all five changed/new Python files, and git diff --check passed.

No Azure DevOps pipeline or actual native wheel build was run.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 7ac28761-49fd-4c09-9885-101a72746ec0
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
10 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The cross-platform toolchain, credential, container, and ARM64 integration paths have not yet been exercised in Azure DevOps.

0 open findings

What changed in this PR

Adds opt-in Rust native-wheel support to shared build tooling without enabling any SDK package.

Changes:

  • Routes [tool.cibuildwheel] projects through cibuildwheel.
  • Adds Microsoft Rust toolchain setup, authentication, environment forwarding, and extended timeouts.
  • Adds pipeline/build tests and native-build documentation.
File Description
eng/​tools/​azure-sdk-tools/​tests/​test_parse_functionality.py Tests cibuildwheel configuration detection.
eng/​tools/​azure-sdk-tools/​tests/​test_native_build_pipeline.py Tests Rust pipeline configuration and scripts.
eng/​tools/​azure-sdk-tools/​tests/​test_build_interactions.py Tests native build routing and failures.
eng/​tools/​azure-sdk-tools/​ci_tools/​parsing/​parse_functions.py Detects cibuildwheel opt-in.
eng/​tools/​azure-sdk-tools/​ci_tools/​build.py Routes opted-in packages through cibuildwheel.
eng/​pipelines/​templates/​steps/​resolve-build-platforms.yml Selects build logging level.
eng/​pipelines/​templates/​steps/​install-msrust-toolchain.yml Installs and authenticates Rust tooling.
eng/​pipelines/​templates/​steps/​build-package-artifacts.yml Integrates Rust setup into wheel builds.
eng/​pipelines/​templates/​stages/​archetype-sdk-client.yml Exposes Rust build parameters.
eng/​pipelines/​templates/​jobs/​ci.yml Forwards parameters and extends timeouts.
doc/​dev/​packaging.md Documents native-wheel configuration.

🧠 Review effort: Balanced

jenny (JennyPng) and others added 4 commits October 7, 2026 10:22
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 7ac28761-49fd-4c09-9885-101a72746ec0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 7ac28761-49fd-4c09-9885-101a72746ec0
@JennyPng
jenny (JennyPng) marked this pull request as ready for review October 9, 2026 06:02
Copilot AI balanced review requested due to automatic review settings October 9, 2026 06:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Linux native builds from fork PRs receive an empty msrustup credential because they forward unavailable System.AccessToken.

1 open finding

🧠 Review effort: Balanced

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we test these changes on a service to ensure it won't run into pipeline issues - maybe a template-three if needed

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The Linux bootstrap currently selects the oldest available msrustup package rather than the latest compatible version.

1 open finding
1 resolved since last review

🧠 Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings October 12, 2026 18:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The bootstrap selects the oldest msrustup release, and sdist-only Maturin builds incorrectly disable build isolation.

2 open findings

🧠 Review effort: Balanced

Revert aaf2e33 and its formatting follow-up e6fb68a. Remove the Rust fixture, generic PR exclusions, and fixture-driven version synchronization while preserving the original opt-in Rust build support.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 8323733d-4446-4291-afcc-33c49f5afa86
Copilot AI balanced review requested due to automatic review settings October 12, 2026 21:37
@JennyPng

jenny (JennyPng) commented Oct 12, 2026 •

Copy link
Copy Markdown
Member Author

deciding not to add an azure-template-three for now because it requires some shared infra changes that i don't think we should have in main yet. such as

  • maturin reads the version in pyproject.toml and not _version.py, and our dev version updating logic only updates _version.py. we'd need to update this version updating logic which I feel is beyond the needed scope for this PR, which aims just to unblock Cosmos. there is no other SDK that will opt into this in main in the near future
  • Cosmos is just using a test Rust feed URL right now that needs to be explicitly passed in by CI templates when opting in, don't think we want that in main right now

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Rust installation and credential exposure are not restricted to the targeted native artifact.

3 open findings
1 resolved since last review

🧠 Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings October 12, 2026 21:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Credential-bearing cross-platform pipeline changes were not validated with an actual native wheel build or Azure DevOps run.

0 open findings

3 resolved since last review

🧠 Review effort: Balanced

@JennyPng

Copy link
Copy Markdown
Member Author

verifying that normal build path is unaffected by these changes: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=6947514&view=results

@JennyPng
jenny (JennyPng) merged commit 3ba1c04 into Azure:main Oct 12, 2026
54 of 57 checks passed
@JennyPng
jenny (JennyPng) deleted the engsys/rust-native-build-support branch October 12, 2026 23:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants