Add semver check test command for checking API compatibility of stdlib by Kobzol · Pull Request #159671 · rust-lang/rust

Kobzol · 2026-07-21T16:34:15Z

This PR adds a new test to bootstrap (not executed by default at the moment) that checks the API compatibility of the standard library using the https://github.com/obi1kenobi/cargo-semver-checks tool.

The command can be executed using x test std-semver-check.

(Note: I realized that x dist rust-docs-json executed twice in a row invalidates Cargo cache for some reason, that is a separate issue though).

The test is not yet executed on CI, I would do that in a separate PR (maybe pending a t-libs FCP or something).

The test checks that c-s-c is installed, and then uses git to lookup a parent baseline commit (in a future extension, we should make the baseline commit be configurable through config.toml). Then it downloads the corresponding rust-docs-json component of that commit from CI and extracts it. It generates the same JSON docs component from local sources, and then runs c-s-c on those two JSON files to compare their API.

Output when API stability is broken:

Checking semver compatibility of core
    Checking <unknown> v1.99.0-nightly  (d527bc9bf      2026-07-20) -> v1.99.0-dev (assume minor change)
     Checked [   1.282s] 196 checks: 196 pass, 57 skip
     Summary no semver update required
    Finished [   2.368s] <unknown>
Checking semver compatibility of alloc
    Checking <unknown> v1.99.0-nightly  (d527bc9bf      2026-07-20) -> v1.99.0-dev (assume minor change)
     Checked [   0.030s] 196 checks: 196 pass, 57 skip
     Summary no semver update required
    Finished [   0.127s] <unknown>
Checking semver compatibility of std
    Checking <unknown> v1.99.0-nightly  (d527bc9bf      2026-07-20) -> v1.99.0-dev (assume minor change)
     Checked [   0.055s] 196 checks: 195 pass, 1 fail, 0 warn, 57 skip

--- failure function_missing: pub fn removed or renamed ---

Description:
A publicly-visible function cannot be imported by its prior path. A `pub use` may have been removed, or the function itself may have been renamed or removed entirely.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.49.0/src/lints/function_missing.ron

Failed in:
  function std::process::id, previously in file library/std/src/process.rs:2656

     Summary semver requires new major version: 1 major and 0 minor checks failed
    Finished [   0.243s] <unknown>

CC @obi1kenobi @Amanieu

r? @jieyouxu

jieyouxu · 2026-07-22T11:24:29Z

(I'll look at this during the weekend.)

…, r=Mark-Simulacrum Avoid spurious rebuilds of JSON docs in bootstrap Found this while working on rust-lang#159671. Before, repeated runs of e.g. `x dist rust-docs-json` always rebuilt the docs. It was caused by Cargo not knowing that the output should be JSON, so it was trying to find the HTML files. Instead of passing the output format through `RUSTDOCFLAGS`, which Cargo doesn't inspect, we now pass it to Cargo directly, which fixes the issue.

Rollup merge of #159765 - Kobzol:bootstrap-docs-json-rebuild, r=Mark-Simulacrum Avoid spurious rebuilds of JSON docs in bootstrap Found this while working on #159671. Before, repeated runs of e.g. `x dist rust-docs-json` always rebuilt the docs. It was caused by Cargo not knowing that the output should be JSON, so it was trying to find the HTML files. Instead of passing the output format through `RUSTDOCFLAGS`, which Cargo doesn't inspect, we now pass it to Cargo directly, which fixes the issue.

…Simulacrum Avoid spurious rebuilds of JSON docs in bootstrap Found this while working on rust-lang/rust#159671. Before, repeated runs of e.g. `x dist rust-docs-json` always rebuilt the docs. It was caused by Cargo not knowing that the output should be JSON, so it was trying to find the HTML files. Instead of passing the output format through `RUSTDOCFLAGS`, which Cargo doesn't inspect, we now pass it to Cargo directly, which fixes the issue.

jieyouxu

Thanks, this seems reasonable, one test nit
@rustbot author

View changes since this review

theemathas · 2026-07-26T14:12:00Z

We should probably also run cargo semver checks in reverse each release too, and compare the output with the release notes, to make sure we don't accidentally stabilize something. That's probably for a future PR though.

rustbot · 2026-07-27T20:52:35Z

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

jieyouxu

Thanks
@bors r+

View changes since this review

rust-bors · 2026-07-29T07:23:11Z

📌 Commit 73ba963 has been approved by jieyouxu

It is now in the queue for this repository.

@obi1kenobi

Add semver check test command for checking API compatibility of stdlib This PR adds a new test to bootstrap (not executed by default at the moment) that checks the API compatibility of the standard library using the https://github.com/obi1kenobi/cargo-semver-checks tool. The command can be executed using `x test std-semver-check`. (Note: I realized that `x dist rust-docs-json` executed twice in a row invalidates Cargo cache for some reason, that is a separate issue though). The test is not yet executed on CI, I would do that in a separate PR (maybe pending a t-libs FCP or something). The test checks that c-s-c is installed, and then uses git to lookup a parent baseline commit (in a future extension, we should make the baseline commit be configurable through `config.toml`). Then it downloads the corresponding `rust-docs-json` component of that commit from CI and extracts it. It generates the same JSON docs component from local sources, and then runs c-s-c on those two JSON files to compare their API. Output when API stability is broken: ``` Checking semver compatibility of core Checking <unknown> v1.99.0-nightly (d527bc9 2026-07-20) -> v1.99.0-dev (assume minor change) Checked [ 1.282s] 196 checks: 196 pass, 57 skip Summary no semver update required Finished [ 2.368s] <unknown> Checking semver compatibility of alloc Checking <unknown> v1.99.0-nightly (d527bc9 2026-07-20) -> v1.99.0-dev (assume minor change) Checked [ 0.030s] 196 checks: 196 pass, 57 skip Summary no semver update required Finished [ 0.127s] <unknown> Checking semver compatibility of std Checking <unknown> v1.99.0-nightly (d527bc9 2026-07-20) -> v1.99.0-dev (assume minor change) Checked [ 0.055s] 196 checks: 195 pass, 1 fail, 0 warn, 57 skip --- failure function_missing: pub fn removed or renamed --- Description: A publicly-visible function cannot be imported by its prior path. A `pub use` may have been removed, or the function itself may have been renamed or removed entirely. ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.49.0/src/lints/function_missing.ron Failed in: function std::process::id, previously in file library/std/src/process.rs:2656 Summary semver requires new major version: 1 major and 0 minor checks failed Finished [ 0.243s] <unknown> ``` CC @obi1kenobi @Amanieu r? @jieyouxu

@obi1kenobi

Add semver check test command for checking API compatibility of stdlib This PR adds a new test to bootstrap (not executed by default at the moment) that checks the API compatibility of the standard library using the https://github.com/obi1kenobi/cargo-semver-checks tool. The command can be executed using `x test std-semver-check`. (Note: I realized that `x dist rust-docs-json` executed twice in a row invalidates Cargo cache for some reason, that is a separate issue though). The test is not yet executed on CI, I would do that in a separate PR (maybe pending a t-libs FCP or something). The test checks that c-s-c is installed, and then uses git to lookup a parent baseline commit (in a future extension, we should make the baseline commit be configurable through `config.toml`). Then it downloads the corresponding `rust-docs-json` component of that commit from CI and extracts it. It generates the same JSON docs component from local sources, and then runs c-s-c on those two JSON files to compare their API. Output when API stability is broken: ``` Checking semver compatibility of core Checking <unknown> v1.99.0-nightly (d527bc9 2026-07-20) -> v1.99.0-dev (assume minor change) Checked [ 1.282s] 196 checks: 196 pass, 57 skip Summary no semver update required Finished [ 2.368s] <unknown> Checking semver compatibility of alloc Checking <unknown> v1.99.0-nightly (d527bc9 2026-07-20) -> v1.99.0-dev (assume minor change) Checked [ 0.030s] 196 checks: 196 pass, 57 skip Summary no semver update required Finished [ 0.127s] <unknown> Checking semver compatibility of std Checking <unknown> v1.99.0-nightly (d527bc9 2026-07-20) -> v1.99.0-dev (assume minor change) Checked [ 0.055s] 196 checks: 195 pass, 1 fail, 0 warn, 57 skip --- failure function_missing: pub fn removed or renamed --- Description: A publicly-visible function cannot be imported by its prior path. A `pub use` may have been removed, or the function itself may have been renamed or removed entirely. ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.49.0/src/lints/function_missing.ron Failed in: function std::process::id, previously in file library/std/src/process.rs:2656 Summary semver requires new major version: 1 major and 0 minor checks failed Finished [ 0.243s] <unknown> ``` CC @obi1kenobi @Amanieu r? @jieyouxu

Rollup of 6 pull requests Successful merges: - #158460 (Remove llvm_enzyme feature outside of bootstrap) - #159671 (Add semver check test command for checking API compatibility of stdlib) - #159717 (Add `-Zimplicit-sysroot-deps`) - #159994 (Show jobs where a given test was executed in `test-dashboard`) - #160085 (Remove various superfluous lint attributes) - #160123 (add additional license option for third-party dependencies)

@obi1kenobi

Add semver check test command for checking API compatibility of stdlib This PR adds a new test to bootstrap (not executed by default at the moment) that checks the API compatibility of the standard library using the https://github.com/obi1kenobi/cargo-semver-checks tool. The command can be executed using `x test std-semver-check`. (Note: I realized that `x dist rust-docs-json` executed twice in a row invalidates Cargo cache for some reason, that is a separate issue though). The test is not yet executed on CI, I would do that in a separate PR (maybe pending a t-libs FCP or something). The test checks that c-s-c is installed, and then uses git to lookup a parent baseline commit (in a future extension, we should make the baseline commit be configurable through `config.toml`). Then it downloads the corresponding `rust-docs-json` component of that commit from CI and extracts it. It generates the same JSON docs component from local sources, and then runs c-s-c on those two JSON files to compare their API. Output when API stability is broken: ``` Checking semver compatibility of core Checking <unknown> v1.99.0-nightly (d527bc9 2026-07-20) -> v1.99.0-dev (assume minor change) Checked [ 1.282s] 196 checks: 196 pass, 57 skip Summary no semver update required Finished [ 2.368s] <unknown> Checking semver compatibility of alloc Checking <unknown> v1.99.0-nightly (d527bc9 2026-07-20) -> v1.99.0-dev (assume minor change) Checked [ 0.030s] 196 checks: 196 pass, 57 skip Summary no semver update required Finished [ 0.127s] <unknown> Checking semver compatibility of std Checking <unknown> v1.99.0-nightly (d527bc9 2026-07-20) -> v1.99.0-dev (assume minor change) Checked [ 0.055s] 196 checks: 195 pass, 1 fail, 0 warn, 57 skip --- failure function_missing: pub fn removed or renamed --- Description: A publicly-visible function cannot be imported by its prior path. A `pub use` may have been removed, or the function itself may have been renamed or removed entirely. ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.49.0/src/lints/function_missing.ron Failed in: function std::process::id, previously in file library/std/src/process.rs:2656 Summary semver requires new major version: 1 major and 0 minor checks failed Finished [ 0.243s] <unknown> ``` CC @obi1kenobi @Amanieu r? @jieyouxu

…uwer Rollup of 11 pull requests Successful merges: - #158460 (Remove llvm_enzyme feature outside of bootstrap) - #159509 (Generate `valid_range`s for enums sign-agnostically) - #159632 (CFI: Add support for the adt_const_params feature) - #159671 (Add semver check test command for checking API compatibility of stdlib) - #157058 (Rustdoc label badge for notable traits) - #159717 (Add `-Zimplicit-sysroot-deps`) - #159850 (Add regression test for closure in array-length const generic) - #159994 (Show jobs where a given test was executed in `test-dashboard`) - #160110 (convert rustc_hir::Target inherent methods to From impls) - #160123 (add additional license option for third-party dependencies) - #160131 (bootstrap: remove temporary bors email lookup)

@obi1kenobi

Rollup merge of #159671 - Kobzol:semver-checks, r=jieyouxu Add semver check test command for checking API compatibility of stdlib This PR adds a new test to bootstrap (not executed by default at the moment) that checks the API compatibility of the standard library using the https://github.com/obi1kenobi/cargo-semver-checks tool. The command can be executed using `x test std-semver-check`. (Note: I realized that `x dist rust-docs-json` executed twice in a row invalidates Cargo cache for some reason, that is a separate issue though). The test is not yet executed on CI, I would do that in a separate PR (maybe pending a t-libs FCP or something). The test checks that c-s-c is installed, and then uses git to lookup a parent baseline commit (in a future extension, we should make the baseline commit be configurable through `config.toml`). Then it downloads the corresponding `rust-docs-json` component of that commit from CI and extracts it. It generates the same JSON docs component from local sources, and then runs c-s-c on those two JSON files to compare their API. Output when API stability is broken: ``` Checking semver compatibility of core Checking <unknown> v1.99.0-nightly (d527bc9 2026-07-20) -> v1.99.0-dev (assume minor change) Checked [ 1.282s] 196 checks: 196 pass, 57 skip Summary no semver update required Finished [ 2.368s] <unknown> Checking semver compatibility of alloc Checking <unknown> v1.99.0-nightly (d527bc9 2026-07-20) -> v1.99.0-dev (assume minor change) Checked [ 0.030s] 196 checks: 196 pass, 57 skip Summary no semver update required Finished [ 0.127s] <unknown> Checking semver compatibility of std Checking <unknown> v1.99.0-nightly (d527bc9 2026-07-20) -> v1.99.0-dev (assume minor change) Checked [ 0.055s] 196 checks: 195 pass, 1 fail, 0 warn, 57 skip --- failure function_missing: pub fn removed or renamed --- Description: A publicly-visible function cannot be imported by its prior path. A `pub use` may have been removed, or the function itself may have been renamed or removed entirely. ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.49.0/src/lints/function_missing.ron Failed in: function std::process::id, previously in file library/std/src/process.rs:2656 Summary semver requires new major version: 1 major and 0 minor checks failed Finished [ 0.243s] <unknown> ``` CC @obi1kenobi @Amanieu r? @jieyouxu

RalfJung · 2026-08-06T09:21:38Z

We should probably also run cargo semver checks in reverse each release too, and compare the output with the release notes, to make sure we don't accidentally stabilize something. That's probably for a future PR though.

So does c-s-c take into account our "unstable" attribute? Otherwise it'd just report lots of unstable stuff being added which isn't very relevant.

Kobzol · 2026-08-06T10:35:52Z

It does when breakage occurs, when I renamed an unstable function, it didn't report an error.

rustbot assigned jieyouxu Jul 21, 2026

rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Jul 21, 2026

Kobzol mentioned this pull request Jul 23, 2026

Avoid spurious rebuilds of JSON docs in bootstrap #159765

Merged

Kobzol force-pushed the semver-checks branch from a6b308d to 6375718 Compare July 23, 2026 16:42

This comment has been minimized.

Sign in to view

jieyouxu reviewed Jul 26, 2026

View reviewed changes

Comment thread src/bootstrap/src/core/build_steps/test.rs

Comment thread src/bootstrap/src/core/build_steps/test.rs

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 26, 2026

nia-e mentioned this pull request Jul 26, 2026

meta: Running cargo-semver-checks in reverse on std stabilisations rust-lang/libs-team#839

Open

Kobzol added 2 commits July 27, 2026 22:48

Add ./x test std-semver-check test command

1544f42

Add CLI snapshot test for semver check step

73ba963

Kobzol force-pushed the semver-checks branch from 6375718 to 73ba963 Compare July 27, 2026 20:52

jieyouxu approved these changes Jul 29, 2026

View reviewed changes

rust-bors Bot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jul 29, 2026

jhpratt mentioned this pull request Jul 29, 2026

Rollup of 5 pull requests #160128

Closed

jhpratt mentioned this pull request Jul 29, 2026

Rollup of 6 pull requests #160135

Closed

This was referenced Jul 29, 2026

Rollup of 6 pull requests #160138

Closed

Rollup of 11 pull requests #160146

Merged

rust-bors Bot merged commit 14c6694 into rust-lang:main Jul 29, 2026
13 checks passed

rustbot added this to the 1.99.0 milestone Jul 29, 2026

Kobzol deleted the semver-checks branch July 29, 2026 20:46

Kobzol mentioned this pull request Jul 31, 2026

Add CI job for checking stdlib semver compatibility #160253

Open

obi1kenobi mentioned this pull request Aug 1, 2026

Continue resolving cargo-semver-checks blockers for merging into cargo rust-lang/rust-project-goals#104

Open

10 tasks

Navigation Menu

Uh oh!

Uh oh!

Add semver check test command for checking API compatibility of stdlib - #159671

Add semver check test command for checking API compatibility of stdlib#159671
rust-bors[bot] merged 2 commits into
rust-lang:mainfrom
Kobzol:semver-checks

Kobzol commented Jul 21, 2026

Uh oh!

jieyouxu commented Jul 22, 2026

Uh oh!

This comment has been minimized.

jieyouxu left a comment •

edited by rustbot

Loading

Uh oh!

Uh oh!

Uh oh!

theemathas commented Jul 26, 2026 •

edited

Loading

Uh oh!

rustbot commented Jul 27, 2026

Uh oh!

jieyouxu left a comment •

edited by rustbot

Loading

Uh oh!

rust-bors Bot commented Jul 29, 2026

Uh oh!

Uh oh!

RalfJung commented Aug 6, 2026

Uh oh!

Kobzol commented Aug 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Uh oh!

Conversation

Kobzol commented Jul 21, 2026

Uh oh!

jieyouxu commented Jul 22, 2026

Uh oh!

This comment has been minimized.

jieyouxu left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

theemathas commented Jul 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Jul 27, 2026

Uh oh!

jieyouxu left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rust-bors Bot commented Jul 29, 2026

Uh oh!

Uh oh!

RalfJung commented Aug 6, 2026

Uh oh!

Kobzol commented Aug 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

jieyouxu left a comment •

edited by rustbot

Loading

theemathas commented Jul 26, 2026 •

edited

Loading

jieyouxu left a comment •

edited by rustbot

Loading