Proposal: special toolchain names referring to particular override levels

[rami3l]

Part of #4059.

Since #4932 it has been decided that only characters satisfying GeneralSecurityProfile::identifier_allowed() should be allowed in rustup toolchain names.

This has opened up new possibilities for us to assemble our special/reserved toolchain names as needed with special characters such as @ or =.

Following up the above efforts, noting the five levels of rustup toolchain overrides:

1. The +<toolchain> override.
2. The RUSTUP_TOOLCHAIN environment variable.
3. The rustup override set override.
4. The rust-toolchain.toml override.
5. The default toolchain.

... aiming to address #4391, I propose the following changes:

• Make env and default new reserved words in addition to the existing none.
• Introduce the following toolchain syntax:
  • env: Refers to the toolchain that should be activated by the env override.
    • This will not be useful per se but sometime people want to be explicit, e.g. rustup run env cargo instead of cargo.
  • dir=<dir>: Refers to the toolchain that should be activated by the dir override on <dir> or its fallback dirs.
    • e.g. cargo +dir=./crates/download
  • file=<file>: Refers to the toolchain that should be activated by the TOML override in <file> which follows the rust-toolchain.toml schema.
    • e.g. cargo +file=windows.toml, for a Windows-specific rust-toolchain.toml setup.
  • default: Refers to the default toolchain. Bails out if none is set.
    • e.g. cargo +default

Concerns

• How should we handle the error for dir= and file= when the dir/TOML override doesn't exist? Should we bail out or warn and fall back to the next level?
  • My guess is that we should warn and fall back.
• Consider the relation between this proposal and Should default_toolchain in settings.toml allow unqualified names? #4945. Should we resolve the toolchain name when the user issues rustup default file=windows.toml? If so, should we fully resolve or partially resolve it?
  • My guess is that we should partially resolve to an official or a custom toolchain name (that isn't a reserved word and doesn't have special characters like =).
• Can we use this in RUSTUP_TOOLCHAIN (is RUSTUP_TOOLCHAIN not guaranteed to be a parsable toolchain name by downstreams other than rustup)? If we can, we will also resolve Environment variable to disable rust-toolchain.toml #2793 at the same time by setting RUST_TOOLCHAIN=default.
  • I think we probably can.

Note

This also simplifies rust-lang/cargo#10098 because this involves changing the override lookup logic to work with arbitrary base paths. In that case, we'd just need to alter the path for our dir/TOML override source by prefixing it with the path passed via cargo -C.

[ChrisDenton]

I'll think about this some more later but my initial impressions is good. I do like the general idea.

is RUST_TOOLCHAIN not guaranteed to be a parsable toolchain name by downstreams other than rustup

Hm, I do somewhat suspect others do parse this (and the toolchain file). On the other hand we already support things like stable (iirc) so I don't think adding default to the mix really makes things any less rustup specific. However, the more complex syntax (e.g. dir=) might possibly be more of a problem. Then again it's unlikely to match any real-world toolchain if it's not parsed.

[rami3l]

@ChrisDenton In that case we can probably partially resolve and replace it in place until no reserved words or special characters remain, upon the first encounter.

IIRC we already need to convert any toolchain override to a RUSTUP_TOOLCHAIN value anyway.