diff options
| author | bors <bors@rust-lang.org> | 2025-06-26 09:20:07 +0000 |
|---|---|---|
| committer | bors <bors@rust-lang.org> | 2025-06-26 09:20:07 +0000 |
| commit | 8f21a5c92ea55c348c275a1bc4fedbdf181e0d64 (patch) | |
| tree | b46f211a19e18a0de7f5f957c16adc3196858390 | |
| parent | 1e838527f18cd24c81547ce6fbef6815032a80a7 (diff) | |
| parent | e69c0231f0752e180927070495e4b9b3bc9dd578 (diff) | |
| download | rust-8f21a5c92ea55c348c275a1bc4fedbdf181e0d64.tar.gz rust-8f21a5c92ea55c348c275a1bc4fedbdf181e0d64.zip | |
Auto merge of #142581 - Kobzol:bootstrap-std-method, r=jieyouxu
Enforce in bootstrap that build must have stage at least 1 This PR is a step towards https://rust-lang.zulipchat.com/#narrow/channel/326414-t-infra.2Fbootstrap/topic/Proposal.20to.20cleanup.20stages.20and.20steps.20after.20the.20redesign/with/523586917. It's very hard or me to make self-contained changes to bootstrap at this moment, so this PR kind of does several things: 1) (first two commits) Try to reduce the usage of `Std::new` in bootstrap, and replace it with a `Builder::std` method (similar to `Builder::compiler`). This is mostly to remove executions of the `Std` step for stage 0, which doesn't make a lot of sense; I'd like to ideally have the invariant that when a step is invoked, it actually builds or does something. Eventually, I'd like for everything to go through `Builder::std`. (Note: I'm not totally married to this idea, if you don't like it, we can remove it from this PR. I mostly did it right now to remove stage 0 std steps from snapshot tests, which shouldn't be there, but we can also filter them out in a different way) 2) Make sure that when you pass `x build compiler`, only the `Assemble` root level step will be invoked, and not the `Rustc` step. Before, both were invoked, which actually ran `Rustc` twice, once with all `crates` filled, and once with no crates (but both actually represent the same situation). Since the `Rustc::make_run` step actually requests a compile that is one stage below it, this actually made `build compiler --stage 0` work, which we don't want to have anymore. 3) Enforce a bootstrap-global invariant that all `build` commands are always on stage `>=1`. If you try to `build` anything on stage 0, it will print a warning and exit bootstrap. This follows the intuition from the new staging rules after the stage redesign; artifacts that are "stage 0" come outside of bootstrap, and we can't really build something for which we don't have source (although we can still test it, but that's for another PR). Now the logic for build should be quite simple. For pretty much everything except for `Std`, you first use the stage0 compiler to build stage 1. Then you can build a stage 2 <something> using the previously built stage 1 (and then you can continue to stage 3 etc.). And that's it. The default build stage for everything is 1 (modulo download-ci-rustc, but that's a separate can of worms). The snapshot test infra isn't super nice at the moment, as one of next steps I want to create some simple Builder pattern that will allow us to configure the bootstrap invocations in a more "forward-compatible" way (e.g. now it's not possible to modify the config passed to `configure_with_args`). There are some things not yet fully resolved for build stage 0: 1) Cargo is still a `ModeRustc` tool, even though it doesn't really have to be, it is buildable with the stage0 compiler 2) bootstrap tools (`opt-dist`, `build-manifest` etc.) are still called stage0 tools, and in the bootstrap output it says something like "stage 0 rustc builds stage 0 opt-dist". Which is a bit weird, but functionally there's no difference, it's just a slightly inconsistent output. We still haven't decided if we should make these tools ignore staging altogether (which is IMO the right choice) or if we want to allow building stage 1/2/3/... bootstrap tools. r? `@jieyouxu` try-job: x86_64-rust-for-linux
| -rw-r--r-- | src/bootstrap/defaults/bootstrap.library.toml | 1 | ||||
| -rw-r--r-- | src/bootstrap/src/core/build_steps/check.rs | 7 | ||||
| -rw-r--r-- | src/bootstrap/src/core/build_steps/clippy.rs | 6 | ||||
| -rw-r--r-- | src/bootstrap/src/core/build_steps/compile.rs | 22 | ||||
| -rw-r--r-- | src/bootstrap/src/core/build_steps/dist.rs | 24 | ||||
| -rw-r--r-- | src/bootstrap/src/core/build_steps/doc.rs | 15 | ||||
| -rw-r--r-- | src/bootstrap/src/core/build_steps/perf.rs | 4 | ||||
| -rw-r--r-- | src/bootstrap/src/core/build_steps/test.rs | 47 | ||||
| -rw-r--r-- | src/bootstrap/src/core/build_steps/tool.rs | 33 | ||||
| -rw-r--r-- | src/bootstrap/src/core/builder/cargo.rs | 4 | ||||
| -rw-r--r-- | src/bootstrap/src/core/builder/mod.rs | 64 | ||||
| -rw-r--r-- | src/bootstrap/src/core/builder/tests.rs | 1353 | ||||
| -rw-r--r-- | src/bootstrap/src/core/config/config.rs | 8 | ||||
| -rw-r--r-- | src/bootstrap/src/utils/change_tracker.rs | 5 | ||||
| -rw-r--r-- | src/bootstrap/src/utils/tests/mod.rs | 26 | ||||
| -rw-r--r-- | src/ci/docker/host-x86_64/mingw-check-1/Dockerfile | 2 | ||||
| -rwxr-xr-x | src/ci/docker/scripts/rfl-build.sh | 2 |
17 files changed, 943 insertions, 680 deletions
diff --git a/src/bootstrap/defaults/bootstrap.library.toml b/src/bootstrap/defaults/bootstrap.library.toml index 895e50b9a20..6a867093b78 100644 --- a/src/bootstrap/defaults/bootstrap.library.toml +++ b/src/bootstrap/defaults/bootstrap.library.toml @@ -1,7 +1,6 @@ # These defaults are meant for contributors to the standard library and documentation. [build] bench-stage = 1 -build-stage = 1 check-stage = 1 test-stage = 1 diff --git a/src/bootstrap/src/core/build_steps/check.rs b/src/bootstrap/src/core/build_steps/check.rs index 65d25d3fb7e..567416d079b 100644 --- a/src/bootstrap/src/core/build_steps/check.rs +++ b/src/bootstrap/src/core/build_steps/check.rs @@ -1,6 +1,5 @@ //! Implementation of compiling the compiler and standard library, in "check"-based modes. -use crate::core::build_steps::compile; use crate::core::build_steps::compile::{ add_to_sysroot, run_cargo, rustc_cargo, rustc_cargo_env, std_cargo, std_crates_for_run_make, }; @@ -87,7 +86,7 @@ impl Step for Std { } // Reuse the stage0 libstd - builder.ensure(compile::Std::new(compiler, target)); + builder.std(compiler, target); return; } @@ -221,8 +220,8 @@ impl Step for Rustc { // the sysroot for the compiler to find. Otherwise, we're going to // fail when building crates that need to generate code (e.g., build // scripts and their dependencies). - builder.ensure(crate::core::build_steps::compile::Std::new(compiler, compiler.host)); - builder.ensure(crate::core::build_steps::compile::Std::new(compiler, target)); + builder.std(compiler, compiler.host); + builder.std(compiler, target); } else { builder.ensure(Std::new(target)); } diff --git a/src/bootstrap/src/core/build_steps/clippy.rs b/src/bootstrap/src/core/build_steps/clippy.rs index 29a5e1f480b..1e44b5b67a4 100644 --- a/src/bootstrap/src/core/build_steps/clippy.rs +++ b/src/bootstrap/src/core/build_steps/clippy.rs @@ -1,8 +1,8 @@ //! Implementation of running clippy on the compiler, standard library and various tools. +use super::check; use super::compile::{run_cargo, rustc_cargo, std_cargo}; use super::tool::{SourceType, prepare_tool_cargo}; -use super::{check, compile}; use crate::builder::{Builder, ShouldRun}; use crate::core::build_steps::compile::std_crates_for_run_make; use crate::core::builder; @@ -212,8 +212,8 @@ impl Step for Rustc { // the sysroot for the compiler to find. Otherwise, we're going to // fail when building crates that need to generate code (e.g., build // scripts and their dependencies). - builder.ensure(compile::Std::new(compiler, compiler.host)); - builder.ensure(compile::Std::new(compiler, target)); + builder.std(compiler, compiler.host); + builder.std(compiler, target); } else { builder.ensure(check::Std::new(target)); } diff --git a/src/bootstrap/src/core/build_steps/compile.rs b/src/bootstrap/src/core/build_steps/compile.rs index d25cba4f935..8200e154169 100644 --- a/src/bootstrap/src/core/build_steps/compile.rs +++ b/src/bootstrap/src/core/build_steps/compile.rs @@ -211,7 +211,7 @@ impl Step for Std { { trace!(?compiler_to_use, ?compiler, "compiler != compiler_to_use, uplifting library"); - builder.ensure(Std::new(compiler_to_use, target)); + builder.std(compiler_to_use, target); let msg = if compiler_to_use.host == target { format!( "Uplifting library (stage{} -> stage{})", @@ -688,7 +688,7 @@ pub fn std_cargo(builder: &Builder<'_>, target: TargetSelection, stage: u32, car } #[derive(Debug, Clone, PartialEq, Eq, Hash)] -struct StdLink { +pub struct StdLink { pub compiler: Compiler, pub target_compiler: Compiler, pub target: TargetSelection, @@ -699,7 +699,7 @@ struct StdLink { } impl StdLink { - fn from_std(std: Std, host_compiler: Compiler) -> Self { + pub fn from_std(std: Std, host_compiler: Compiler) -> Self { Self { compiler: host_compiler, target_compiler: std.compiler, @@ -1020,6 +1020,12 @@ impl Step for Rustc { } fn make_run(run: RunConfig<'_>) { + // If only `compiler` was passed, do not run this step. + // Instead the `Assemble` step will take care of compiling Rustc. + if run.builder.paths == vec![PathBuf::from("compiler")] { + return; + } + let crates = run.cargo_crates_in_set(); run.builder.ensure(Rustc { build_compiler: run @@ -1065,7 +1071,7 @@ impl Step for Rustc { // Build a standard library for `target` using the `build_compiler`. // This will be the standard library that the rustc which we build *links to*. - builder.ensure(Std::new(build_compiler, target)); + builder.std(build_compiler, target); if builder.config.keep_stage.contains(&build_compiler.stage) { trace!(stage = build_compiler.stage, "`keep-stage` requested"); @@ -1106,10 +1112,10 @@ impl Step for Rustc { // build scripts and proc macros. // If we are not cross-compiling, the Std build above will be the same one as the one we // prepare here. - builder.ensure(Std::new( + builder.std( builder.compiler(self.build_compiler.stage, builder.config.host_target), builder.config.host_target, - )); + ); let mut cargo = builder::Cargo::new( builder, @@ -2077,7 +2083,7 @@ impl Step for Assemble { if builder.download_rustc() { trace!("`download-rustc` requested, reusing CI compiler for stage > 0"); - builder.ensure(Std::new(target_compiler, target_compiler.host)); + builder.std(target_compiler, target_compiler.host); let sysroot = builder.ensure(Sysroot { compiler: target_compiler, force_recompile: false }); // Ensure that `libLLVM.so` ends up in the newly created target directory, @@ -2085,7 +2091,7 @@ impl Step for Assemble { dist::maybe_install_llvm_target(builder, target_compiler.host, &sysroot); // Lower stages use `ci-rustc-sysroot`, not stageN if target_compiler.stage == builder.top_stage { - builder.info(&format!("Creating a sysroot for stage{stage} compiler (use `rustup toolchain link 'name' build/host/stage{stage}`)", stage=target_compiler.stage)); + builder.info(&format!("Creating a sysroot for stage{stage} compiler (use `rustup toolchain link 'name' build/host/stage{stage}`)", stage = target_compiler.stage)); } let mut precompiled_compiler = target_compiler; diff --git a/src/bootstrap/src/core/build_steps/dist.rs b/src/bootstrap/src/core/build_steps/dist.rs index e0f632eda0e..95fc2f1aef9 100644 --- a/src/bootstrap/src/core/build_steps/dist.rs +++ b/src/bootstrap/src/core/build_steps/dist.rs @@ -23,7 +23,7 @@ use crate::core::build_steps::doc::DocumentationFormat; use crate::core::build_steps::tool::{self, Tool}; use crate::core::build_steps::vendor::{VENDOR_DIR, Vendor}; use crate::core::build_steps::{compile, llvm}; -use crate::core::builder::{Builder, Kind, RunConfig, ShouldRun, Step}; +use crate::core::builder::{Builder, Kind, RunConfig, ShouldRun, Step, StepMetadata}; use crate::core::config::TargetSelection; use crate::utils::build_stamp::{self, BuildStamp}; use crate::utils::channel::{self, Info}; @@ -84,6 +84,10 @@ impl Step for Docs { tarball.add_file(builder.src.join("src/doc/robots.txt"), dest, FileType::Regular); Some(tarball.generate()) } + + fn metadata(&self) -> Option<StepMetadata> { + Some(StepMetadata::dist("docs", self.host)) + } } #[derive(Debug, PartialOrd, Ord, Clone, Hash, PartialEq, Eq)] @@ -354,6 +358,10 @@ impl Step for Mingw { Some(tarball.generate()) } + + fn metadata(&self) -> Option<StepMetadata> { + Some(StepMetadata::dist("mingw", self.host)) + } } #[derive(Debug, PartialOrd, Ord, Clone, Hash, PartialEq, Eq)] @@ -540,6 +548,10 @@ impl Step for Rustc { } } } + + fn metadata(&self) -> Option<StepMetadata> { + Some(StepMetadata::dist("rustc", self.compiler.host)) + } } #[derive(Debug, Clone, Hash, PartialEq, Eq)] @@ -711,7 +723,7 @@ impl Step for Std { return None; } - builder.ensure(compile::Std::new(compiler, target)); + builder.std(compiler, target); let mut tarball = Tarball::new(builder, "rust-std", &target.triple); tarball.include_target_in_component_name(true); @@ -723,6 +735,10 @@ impl Step for Std { Some(tarball.generate()) } + + fn metadata(&self) -> Option<StepMetadata> { + Some(StepMetadata::dist("std", self.target).built_by(self.compiler)) + } } /// Tarball containing the compiler that gets downloaded and used by @@ -1002,6 +1018,10 @@ impl Step for Src { tarball.generate() } + + fn metadata(&self) -> Option<StepMetadata> { + Some(StepMetadata::dist("src", TargetSelection::default())) + } } #[derive(Debug, PartialOrd, Ord, Clone, Hash, PartialEq, Eq)] diff --git a/src/bootstrap/src/core/build_steps/doc.rs b/src/bootstrap/src/core/build_steps/doc.rs index 215c155651a..f7c4c5ad0bb 100644 --- a/src/bootstrap/src/core/build_steps/doc.rs +++ b/src/bootstrap/src/core/build_steps/doc.rs @@ -14,7 +14,8 @@ use std::{env, fs, mem}; use crate::core::build_steps::compile; use crate::core::build_steps::tool::{self, SourceType, Tool, prepare_tool_cargo}; use crate::core::builder::{ - self, Alias, Builder, Compiler, Kind, RunConfig, ShouldRun, Step, crate_description, + self, Alias, Builder, Compiler, Kind, RunConfig, ShouldRun, Step, StepMetadata, + crate_description, }; use crate::core::config::{Config, TargetSelection}; use crate::helpers::{submodule_path_of, symlink_dir, t, up_to_date}; @@ -662,6 +663,10 @@ impl Step for Std { } } } + + fn metadata(&self) -> Option<StepMetadata> { + Some(StepMetadata::doc("std", self.target).stage(self.stage)) + } } /// Name of the crates that are visible to consumers of the standard library. @@ -804,7 +809,7 @@ impl Step for Rustc { // Build the standard library, so that proc-macros can use it. // (Normally, only the metadata would be necessary, but proc-macros are special since they run at compile-time.) let compiler = builder.compiler(stage, builder.config.host_target); - builder.ensure(compile::Std::new(compiler, builder.config.host_target)); + builder.std(compiler, builder.config.host_target); let _guard = builder.msg_sysroot_tool( Kind::Doc, @@ -947,7 +952,7 @@ macro_rules! tool_doc { t!(fs::create_dir_all(&out)); let compiler = builder.compiler(stage, builder.config.host_target); - builder.ensure(compile::Std::new(compiler, target)); + builder.std(compiler, target); if true $(&& $rustc_tool)? { // Build rustc docs so that we generate relative links. @@ -1195,7 +1200,7 @@ impl Step for RustcBook { let rustc = builder.rustc(self.compiler); // The tool runs `rustc` for extracting output examples, so it needs a // functional sysroot. - builder.ensure(compile::Std::new(self.compiler, self.target)); + builder.std(self.compiler, self.target); let mut cmd = builder.tool_cmd(Tool::LintDocs); cmd.arg("--src"); cmd.arg(builder.src.join("compiler")); @@ -1272,7 +1277,7 @@ impl Step for Reference { // This is needed for generating links to the standard library using // the mdbook-spec plugin. - builder.ensure(compile::Std::new(self.compiler, builder.config.host_target)); + builder.std(self.compiler, builder.config.host_target); // Run rustbook/mdbook to generate the HTML pages. builder.ensure(RustbookSrc { diff --git a/src/bootstrap/src/core/build_steps/perf.rs b/src/bootstrap/src/core/build_steps/perf.rs index c43043b48f4..4d61b38c876 100644 --- a/src/bootstrap/src/core/build_steps/perf.rs +++ b/src/bootstrap/src/core/build_steps/perf.rs @@ -1,7 +1,7 @@ use std::env::consts::EXE_EXTENSION; use std::fmt::{Display, Formatter}; -use crate::core::build_steps::compile::{Std, Sysroot}; +use crate::core::build_steps::compile::Sysroot; use crate::core::build_steps::tool::{RustcPerf, Rustdoc}; use crate::core::builder::Builder; use crate::core::config::DebuginfoLevel; @@ -152,7 +152,7 @@ Consider setting `rust.debuginfo-level = 1` in `bootstrap.toml`."#); } let compiler = builder.compiler(builder.top_stage, builder.config.host_target); - builder.ensure(Std::new(compiler, builder.config.host_target)); + builder.std(compiler, builder.config.host_target); if let Some(opts) = args.cmd.shared_opts() && opts.profiles.contains(&Profile::Doc) diff --git a/src/bootstrap/src/core/build_steps/test.rs b/src/bootstrap/src/core/build_steps/test.rs index 419839067f9..01b181f55de 100644 --- a/src/bootstrap/src/core/build_steps/test.rs +++ b/src/bootstrap/src/core/build_steps/test.rs @@ -10,7 +10,7 @@ use std::{env, fs, iter}; use clap_complete::shells; -use crate::core::build_steps::compile::run_cargo; +use crate::core::build_steps::compile::{Std, run_cargo}; use crate::core::build_steps::doc::DocumentationFormat; use crate::core::build_steps::gcc::{Gcc, add_cg_gcc_cargo_flags}; use crate::core::build_steps::llvm::get_llvm_version; @@ -19,7 +19,8 @@ use crate::core::build_steps::tool::{self, COMPILETEST_ALLOW_FEATURES, SourceTyp use crate::core::build_steps::toolstate::ToolState; use crate::core::build_steps::{compile, dist, llvm}; use crate::core::builder::{ - self, Alias, Builder, Compiler, Kind, RunConfig, ShouldRun, Step, crate_description, + self, Alias, Builder, Compiler, Kind, RunConfig, ShouldRun, Step, StepMetadata, + crate_description, }; use crate::core::config::TargetSelection; use crate::core::config::flags::{Subcommand, get_completion}; @@ -544,7 +545,7 @@ impl Step for Miri { // We also need sysroots, for Miri and for the host (the latter for build scripts). // This is for the tests so everything is done with the target compiler. let miri_sysroot = Miri::build_miri_sysroot(builder, target_compiler, target); - builder.ensure(compile::Std::new(target_compiler, host)); + builder.std(target_compiler, host); let host_sysroot = builder.sysroot(target_compiler); // Miri has its own "target dir" for ui test dependencies. Make sure it gets cleared when @@ -709,7 +710,7 @@ impl Step for CompiletestTest { // We need `ToolStd` for the locally-built sysroot because // compiletest uses unstable features of the `test` crate. - builder.ensure(compile::Std::new(compiler, host)); + builder.std(compiler, host); let mut cargo = tool::prepare_tool_cargo( builder, compiler, @@ -1009,7 +1010,7 @@ impl Step for RustdocGUI { } fn run(self, builder: &Builder<'_>) { - builder.ensure(compile::Std::new(self.compiler, self.target)); + builder.std(self.compiler, self.target); let mut cmd = builder.tool_cmd(Tool::RustdocGUITest); @@ -1174,6 +1175,10 @@ HELP: to skip test's attempt to check tidiness, pass `--skip src/tools/tidy` to fn make_run(run: RunConfig<'_>) { run.builder.ensure(Tidy); } + + fn metadata(&self) -> Option<StepMetadata> { + Some(StepMetadata::test("tidy", TargetSelection::default())) + } } fn testdir(builder: &Builder<'_>, host: TargetSelection) -> PathBuf { @@ -1236,6 +1241,12 @@ macro_rules! test { }), }) } + + fn metadata(&self) -> Option<StepMetadata> { + Some( + StepMetadata::test(stringify!($name), self.target) + ) + } } }; } @@ -1634,7 +1645,7 @@ NOTE: if you're sure you want to do this, please open an issue as to why. In the if suite == "mir-opt" { builder.ensure(compile::Std::new(compiler, compiler.host).is_for_mir_opt_tests(true)); } else { - builder.ensure(compile::Std::new(compiler, compiler.host)); + builder.std(compiler, compiler.host); } let mut cmd = builder.tool_cmd(Tool::Compiletest); @@ -1642,7 +1653,7 @@ NOTE: if you're sure you want to do this, please open an issue as to why. In the if suite == "mir-opt" { builder.ensure(compile::Std::new(compiler, target).is_for_mir_opt_tests(true)); } else { - builder.ensure(compile::Std::new(compiler, target)); + builder.std(compiler, target); } builder.ensure(RemoteCopyLibs { compiler, target }); @@ -2177,7 +2188,7 @@ impl BookTest { fn run_ext_doc(self, builder: &Builder<'_>) { let compiler = self.compiler; - builder.ensure(compile::Std::new(compiler, compiler.host)); + builder.std(compiler, compiler.host); // mdbook just executes a binary named "rustdoc", so we need to update // PATH so that it points to our rustdoc. @@ -2263,7 +2274,7 @@ impl BookTest { let compiler = self.compiler; let host = self.compiler.host; - builder.ensure(compile::Std::new(compiler, host)); + builder.std(compiler, host); let _guard = builder.msg(Kind::Test, compiler.stage, format!("book {}", self.name), host, host); @@ -2410,7 +2421,7 @@ impl Step for ErrorIndex { drop(guard); // The tests themselves need to link to std, so make sure it is // available. - builder.ensure(compile::Std::new(compiler, compiler.host)); + builder.std(compiler, compiler.host); markdown_test(builder, compiler, &output); } } @@ -2473,7 +2484,7 @@ impl Step for CrateLibrustc { } fn run(self, builder: &Builder<'_>) { - builder.ensure(compile::Std::new(self.compiler, self.target)); + builder.std(self.compiler, self.target); // To actually run the tests, delegate to a copy of the `Crate` step. builder.ensure(Crate { @@ -2483,6 +2494,10 @@ impl Step for CrateLibrustc { crates: self.crates, }); } + + fn metadata(&self) -> Option<StepMetadata> { + Some(StepMetadata::test("CrateLibrustc", self.target)) + } } /// Given a `cargo test` subcommand, add the appropriate flags and run it. @@ -2641,7 +2656,7 @@ impl Step for Crate { // Prepare sysroot // See [field@compile::Std::force_recompile]. - builder.ensure(compile::Std::new(compiler, compiler.host).force_recompile(true)); + builder.ensure(Std::new(compiler, compiler.host).force_recompile(true)); // If we're not doing a full bootstrap but we're testing a stage2 // version of libstd, then what we're actually testing is the libstd @@ -2767,7 +2782,7 @@ impl Step for CrateRustdoc { // using `download-rustc`, the rustc_private artifacts may be in a *different sysroot* from // the target rustdoc (`ci-rustc-sysroot` vs `stage2`). In that case, we need to ensure this // explicitly to make sure it ends up in the stage2 sysroot. - builder.ensure(compile::Std::new(compiler, target)); + builder.std(compiler, target); builder.ensure(compile::Rustc::new(compiler, target)); let mut cargo = tool::prepare_tool_cargo( @@ -2911,7 +2926,7 @@ impl Step for RemoteCopyLibs { return; } - builder.ensure(compile::Std::new(compiler, target)); + builder.std(compiler, target); builder.info(&format!("REMOTE copy libs to emulator ({target})")); @@ -3101,7 +3116,7 @@ impl Step for TierCheck { /// Tests the Platform Support page in the rustc book. fn run(self, builder: &Builder<'_>) { - builder.ensure(compile::Std::new(self.compiler, self.compiler.host)); + builder.std(self.compiler, self.compiler.host); let mut cargo = tool::prepare_tool_cargo( builder, self.compiler, @@ -3334,7 +3349,7 @@ impl Step for CodegenCranelift { let compiler = self.compiler; let target = self.target; - builder.ensure(compile::Std::new(compiler, target)); + builder.std(compiler, target); // If we're not doing a full bootstrap but we're testing a stage2 // version of libstd, then what we're actually testing is the libstd diff --git a/src/bootstrap/src/core/build_steps/tool.rs b/src/bootstrap/src/core/build_steps/tool.rs index 248ee4196b8..83c0525d7c4 100644 --- a/src/bootstrap/src/core/build_steps/tool.rs +++ b/src/bootstrap/src/core/build_steps/tool.rs @@ -20,7 +20,7 @@ use crate::core::build_steps::toolstate::ToolState; use crate::core::build_steps::{compile, llvm}; use crate::core::builder; use crate::core::builder::{ - Builder, Cargo as CargoCommand, RunConfig, ShouldRun, Step, cargo_profile_var, + Builder, Cargo as CargoCommand, RunConfig, ShouldRun, Step, StepMetadata, cargo_profile_var, }; use crate::core::config::{DebuginfoLevel, RustcLto, TargetSelection}; use crate::utils::exec::{BootstrapCommand, command}; @@ -122,14 +122,14 @@ impl Step for ToolBuild { Mode::ToolRustc => { // If compiler was forced, its artifacts should be prepared earlier. if !self.compiler.is_forced_compiler() { - builder.ensure(compile::Std::new(self.compiler, self.compiler.host)); + builder.std(self.compiler, self.compiler.host); builder.ensure(compile::Rustc::new(self.compiler, target)); } } Mode::ToolStd => { // If compiler was forced, its artifacts should be prepared earlier. if !self.compiler.is_forced_compiler() { - builder.ensure(compile::Std::new(self.compiler, target)) + builder.std(self.compiler, target) } } Mode::ToolBootstrap => {} // uses downloaded stage0 compiler libs @@ -479,6 +479,13 @@ macro_rules! bootstrap_tool { } }) } + + fn metadata(&self) -> Option<StepMetadata> { + Some( + StepMetadata::build(stringify!($name), self.target) + .built_by(self.compiler) + ) + } } )+ } @@ -779,6 +786,16 @@ impl Step for Rustdoc { ToolBuildResult { tool_path, build_compiler, target_compiler } } } + + fn metadata(&self) -> Option<StepMetadata> { + Some( + StepMetadata::build("rustdoc", self.compiler.host) + // rustdoc is ToolRustc, so stage N rustdoc is built by stage N-1 rustc + // FIXME: make this stage deduction automatic somehow + // FIXME: log the compiler that actually built ToolRustc steps + .stage(self.compiler.stage.saturating_sub(1)), + ) + } } #[derive(Debug, Clone, Hash, PartialEq, Eq)] @@ -1171,6 +1188,16 @@ macro_rules! tool_extended { None $( .or(Some($add_features)) )?, ) } + + fn metadata(&self) -> Option<StepMetadata> { + // FIXME: refactor extended tool steps to make the build_compiler explicit, + // it is offset by one now for rustc tools + Some( + StepMetadata::build($tool_name, self.target) + .built_by(self.compiler.with_stage(self.compiler.stage.saturating_sub(1))) + .stage(self.compiler.stage) + ) + } } } } diff --git a/src/bootstrap/src/core/builder/cargo.rs b/src/bootstrap/src/core/builder/cargo.rs index 99044e2a253..a878291a33c 100644 --- a/src/bootstrap/src/core/builder/cargo.rs +++ b/src/bootstrap/src/core/builder/cargo.rs @@ -3,8 +3,8 @@ use std::ffi::{OsStr, OsString}; use std::path::{Path, PathBuf}; use super::{Builder, Kind}; +use crate::core::build_steps::test; use crate::core::build_steps::tool::SourceType; -use crate::core::build_steps::{compile, test}; use crate::core::config::SplitDebuginfo; use crate::core::config::flags::Color; use crate::utils::build_stamp; @@ -843,7 +843,7 @@ impl Builder<'_> { // If this is for `miri-test`, prepare the sysroots. if cmd_kind == Kind::MiriTest { - self.ensure(compile::Std::new(compiler, compiler.host)); + self.std(compiler, compiler.host); let host_sysroot = self.sysroot(compiler); let miri_sysroot = test::Miri::build_miri_sysroot(self, compiler, target); cargo.env("MIRI_SYSROOT", &miri_sysroot); diff --git a/src/bootstrap/src/core/builder/mod.rs b/src/bootstrap/src/core/builder/mod.rs index b26f47a3171..7cb7866953a 100644 --- a/src/bootstrap/src/core/builder/mod.rs +++ b/src/bootstrap/src/core/builder/mod.rs @@ -15,6 +15,7 @@ use tracing::instrument; pub use self::cargo::{Cargo, cargo_profile_var}; pub use crate::Compiler; +use crate::core::build_steps::compile::{Std, StdLink}; use crate::core::build_steps::{ check, clean, clippy, compile, dist, doc, gcc, install, llvm, run, setup, test, tool, vendor, }; @@ -139,7 +140,7 @@ pub trait Step: 'static + Clone + Debug + PartialEq + Eq + Hash { /// Metadata that describes an executed step, mostly for testing and tracing. #[allow(unused)] -#[derive(Debug)] +#[derive(Debug, PartialEq, Eq)] pub struct StepMetadata { name: &'static str, kind: Kind, @@ -150,7 +151,23 @@ pub struct StepMetadata { impl StepMetadata { pub fn build(name: &'static str, target: TargetSelection) -> Self { - Self { name, kind: Kind::Build, target, built_by: None, stage: None } + Self::new(name, target, Kind::Build) + } + + pub fn doc(name: &'static str, target: TargetSelection) -> Self { + Self::new(name, target, Kind::Doc) + } + + pub fn dist(name: &'static str, target: TargetSelection) -> Self { + Self::new(name, target, Kind::Dist) + } + + pub fn test(name: &'static str, target: TargetSelection) -> Self { + Self::new(name, target, Kind::Test) + } + + fn new(name: &'static str, target: TargetSelection, kind: Kind) -> Self { + Self { name, kind, target, built_by: None, stage: None } } pub fn built_by(mut self, compiler: Compiler) -> Self { @@ -1350,6 +1367,49 @@ impl<'a> Builder<'a> { resolved_compiler } + /// Obtain a standard library for the given target that will be built by the passed compiler. + /// The standard library will be linked to the sysroot of the passed compiler. + /// + /// Prefer using this method rather than manually invoking `Std::new`. + #[cfg_attr( + feature = "tracing", + instrument( + level = "trace", + name = "Builder::std", + target = "STD", + skip_all, + fields( + compiler = ?compiler, + target = ?target, + ), + ), + )] + pub fn std(&self, compiler: Compiler, target: TargetSelection) { + // FIXME: make the `Std` step return some type-level "proof" that std was indeed built, + // and then require passing that to all Cargo invocations that we do. + + // The "stage 0" std is always precompiled and comes with the stage0 compiler, so we have + // special logic for it, to avoid creating needless and confusing Std steps that don't + // actually build anything. + if compiler.stage == 0 { + if target != compiler.host { + panic!( + r"It is not possible to build the standard library for `{target}` using the stage0 compiler. +You have to build a stage1 compiler for `{}` first, and then use it to build a standard library for `{target}`. +", + compiler.host + ) + } + + // We still need to link the prebuilt standard library into the ephemeral stage0 sysroot + self.ensure(StdLink::from_std(Std::new(compiler, target), compiler)); + } else { + // This step both compiles the std and links it into the compiler's sysroot. + // Yes, it's quite magical and side-effecty.. would be nice to refactor later. + self.ensure(Std::new(compiler, target)); + } + } + pub fn sysroot(&self, compiler: Compiler) -> PathBuf { self.ensure(compile::Sysroot::new(compiler)) } diff --git a/src/bootstrap/src/core/builder/tests.rs b/src/bootstrap/src/core/builder/tests.rs index f1af2b285a2..8adf93ea528 100644 --- a/src/bootstrap/src/core/builder/tests.rs +++ b/src/bootstrap/src/core/builder/tests.rs @@ -10,6 +10,7 @@ use crate::core::build_steps::doc::DocumentationFormat; use crate::core::config::Config; use crate::utils::cache::ExecutedStep; use crate::utils::helpers::get_host_target; +use crate::utils::tests::ConfigBuilder; use crate::utils::tests::git::{GitCtx, git_test}; static TEST_TRIPLE_1: &str = "i686-unknown-haiku"; @@ -193,58 +194,6 @@ fn check_missing_paths_for_x_test_tests() { } #[test] -fn test_exclude() { - let mut config = configure("test", &[TEST_TRIPLE_1], &[TEST_TRIPLE_1]); - config.skip = vec!["src/tools/tidy".into()]; - let cache = run_build(&[], config); - - // Ensure we have really excluded tidy - assert!(!cache.contains::<test::Tidy>()); - - // Ensure other tests are not affected. - assert!(cache.contains::<test::RustdocUi>()); -} - -#[test] -fn test_exclude_kind() { - let path = PathBuf::from("compiler/rustc_data_structures"); - - let mut config = configure("test", &[TEST_TRIPLE_1], &[TEST_TRIPLE_1]); - // Ensure our test is valid, and `test::Rustc` would be run without the exclude. - assert!(run_build(&[], config.clone()).contains::<test::CrateLibrustc>()); - // Ensure tests for rustc are not skipped. - config.skip = vec![path.clone()]; - assert!(run_build(&[], config.clone()).contains::<test::CrateLibrustc>()); - // Ensure builds for rustc are not skipped. - assert!(run_build(&[], config).contains::<compile::Rustc>()); -} - -/// Ensure that if someone passes both a single crate and `library`, all library crates get built. -#[test] -fn alias_and_path_for_library() { - let mut cache = run_build( - &["library".into(), "core".into()], - configure("build", &[TEST_TRIPLE_1], &[TEST_TRIPLE_1]), - ); - assert_eq!( - first(cache.all::<compile::Std>()), - &[ - std!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 0), - std!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 1) - ] - ); - - let mut cache = run_build( - &["library".into(), "core".into()], - configure("doc", &[TEST_TRIPLE_1], &[TEST_TRIPLE_1]), - ); - assert_eq!( - first(cache.all::<doc::Std>()), - &[doc_std!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 1)] - ); -} - -#[test] fn ci_rustc_if_unchanged_invalidate_on_compiler_changes() { git_test(|ctx| { prepare_rustc_checkout(ctx); @@ -316,101 +265,6 @@ mod defaults { use crate::core::builder::*; #[test] - fn build_default() { - let mut cache = run_build(&[], configure("build", &[TEST_TRIPLE_1], &[TEST_TRIPLE_1])); - - let a = TargetSelection::from_user(TEST_TRIPLE_1); - assert_eq!( - first(cache.all::<compile::Std>()), - &[ - std!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 0), - std!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 1), - ] - ); - assert!(!cache.all::<compile::Assemble>().is_empty()); - // Make sure rustdoc is only built once. - assert_eq!( - first(cache.all::<tool::Rustdoc>()), - // Recall that rustdoc stages are off-by-one - // - this is the compiler it's _linked_ to, not built with. - &[tool::Rustdoc { compiler: Compiler::new(1, a) }], - ); - assert_eq!( - first(cache.all::<compile::Rustc>()), - &[rustc!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 0)], - ); - } - - #[test] - fn build_stage_0() { - let config = Config { stage: 0, ..configure("build", &[TEST_TRIPLE_1], &[TEST_TRIPLE_1]) }; - let mut cache = run_build(&[], config); - - let a = TargetSelection::from_user(TEST_TRIPLE_1); - assert_eq!( - first(cache.all::<compile::Std>()), - &[std!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 0)] - ); - assert!(!cache.all::<compile::Assemble>().is_empty()); - assert_eq!( - first(cache.all::<tool::Rustdoc>()), - // This is the beta rustdoc. - // Add an assert here to make sure this is the only rustdoc built. - &[tool::Rustdoc { compiler: Compiler::new(0, a) }], - ); - assert!(cache.all::<compile::Rustc>().is_empty()); - } - - #[test] - fn build_cross_compile() { - let config = Config { - stage: 1, - ..configure("build", &[TEST_TRIPLE_1, TEST_TRIPLE_2], &[TEST_TRIPLE_1, TEST_TRIPLE_2]) - }; - let mut cache = run_build(&[], config); - - let a = TargetSelection::from_user(TEST_TRIPLE_1); - let b = TargetSelection::from_user(TEST_TRIPLE_2); - - // Ideally, this build wouldn't actually have `target: a` - // rustdoc/rustcc/std here (the user only requested a host=B build, so - // there's not really a need for us to build for target A in this case - // (since we're producing stage 1 libraries/binaries). But currently - // bootstrap is just a bit buggy here; this should be fixed though. - assert_eq!( - first(cache.all::<compile::Std>()), - &[ - std!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 0), - std!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 1), - std!(TEST_TRIPLE_1 => TEST_TRIPLE_2, stage = 0), - std!(TEST_TRIPLE_1 => TEST_TRIPLE_2, stage = 1), - ] - ); - assert_eq!( - first(cache.all::<compile::Assemble>()), - &[ - compile::Assemble { target_compiler: Compiler::new(0, a) }, - compile::Assemble { target_compiler: Compiler::new(1, a) }, - compile::Assemble { target_compiler: Compiler::new(1, b) }, - ] - ); - assert_eq!( - first(cache.all::<tool::Rustdoc>()), - &[ - tool::Rustdoc { compiler: Compiler::new(1, a) }, - tool::Rustdoc { compiler: Compiler::new(1, b) }, - ], - ); - assert_eq!( - first(cache.all::<compile::Rustc>()), - &[ - rustc!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 0), - rustc!(TEST_TRIPLE_1 => TEST_TRIPLE_2, stage = 0), - ] - ); - } - - #[test] fn doc_default() { let mut config = configure("doc", &[TEST_TRIPLE_1], &[TEST_TRIPLE_1]); config.compiler_docs = true; @@ -447,326 +301,6 @@ mod dist { } #[test] - fn dist_baseline() { - let mut cache = run_build(&[], configure(&[TEST_TRIPLE_1], &[TEST_TRIPLE_1])); - - let a = TargetSelection::from_user(TEST_TRIPLE_1); - - assert_eq!(first(cache.all::<dist::Docs>()), &[dist::Docs { host: a },]); - assert_eq!(first(cache.all::<dist::Mingw>()), &[dist::Mingw { host: a },]); - assert_eq!( - first(cache.all::<dist::Rustc>()), - &[dist::Rustc { compiler: Compiler::new(2, a) },] - ); - assert_eq!( - first(cache.all::<dist::Std>()), - &[dist::Std { compiler: Compiler::new(1, a), target: a },] - ); - assert_eq!(first(cache.all::<dist::Src>()), &[dist::Src]); - // Make sure rustdoc is only built once. - assert_eq!( - first(cache.all::<tool::Rustdoc>()), - &[tool::Rustdoc { compiler: Compiler::new(2, a) },] - ); - } - - #[test] - fn dist_with_targets() { - let mut cache = - run_build(&[], configure(&[TEST_TRIPLE_1], &[TEST_TRIPLE_1, TEST_TRIPLE_2])); - - let a = TargetSelection::from_user(TEST_TRIPLE_1); - let b = TargetSelection::from_user(TEST_TRIPLE_2); - - assert_eq!( - first(cache.all::<dist::Docs>()), - &[dist::Docs { host: a }, dist::Docs { host: b },] - ); - assert_eq!( - first(cache.all::<dist::Mingw>()), - &[dist::Mingw { host: a }, dist::Mingw { host: b },] - ); - assert_eq!( - first(cache.all::<dist::Rustc>()), - &[dist::Rustc { compiler: Compiler::new(2, a) },] - ); - assert_eq!( - first(cache.all::<dist::Std>()), - &[ - dist::Std { compiler: Compiler::new(1, a), target: a }, - dist::Std { compiler: Compiler::new(2, a), target: b }, - ] - ); - assert_eq!(first(cache.all::<dist::Src>()), &[dist::Src]); - } - - #[test] - fn dist_with_hosts() { - let mut cache = run_build( - &[], - configure(&[TEST_TRIPLE_1, TEST_TRIPLE_2], &[TEST_TRIPLE_1, TEST_TRIPLE_2]), - ); - - let a = TargetSelection::from_user(TEST_TRIPLE_1); - let b = TargetSelection::from_user(TEST_TRIPLE_2); - - assert_eq!( - first(cache.all::<dist::Docs>()), - &[dist::Docs { host: a }, dist::Docs { host: b },] - ); - assert_eq!( - first(cache.all::<dist::Mingw>()), - &[dist::Mingw { host: a }, dist::Mingw { host: b },] - ); - assert_eq!( - first(cache.all::<dist::Rustc>()), - &[ - dist::Rustc { compiler: Compiler::new(2, a) }, - dist::Rustc { compiler: Compiler::new(2, b) }, - ] - ); - assert_eq!( - first(cache.all::<dist::Std>()), - &[ - dist::Std { compiler: Compiler::new(1, a), target: a }, - dist::Std { compiler: Compiler::new(1, a), target: b }, - ] - ); - assert_eq!( - first(cache.all::<compile::Std>()), - &[ - std!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 0), - std!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 1), - std!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 2), - std!(TEST_TRIPLE_1 => TEST_TRIPLE_2, stage = 1), - std!(TEST_TRIPLE_1 => TEST_TRIPLE_2, stage = 2), - ], - ); - assert_eq!(first(cache.all::<dist::Src>()), &[dist::Src]); - } - - #[test] - fn dist_only_cross_host() { - let b = TargetSelection::from_user(TEST_TRIPLE_2); - let mut config = - configure(&[TEST_TRIPLE_1, TEST_TRIPLE_2], &[TEST_TRIPLE_1, TEST_TRIPLE_2]); - config.docs = false; - config.extended = true; - config.hosts = vec![b]; - let mut cache = run_build(&[], config); - - assert_eq!( - first(cache.all::<dist::Rustc>()), - &[dist::Rustc { compiler: Compiler::new(2, b) },] - ); - assert_eq!( - first(cache.all::<compile::Rustc>()), - &[ - rustc!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 0), - rustc!(TEST_TRIPLE_1 => TEST_TRIPLE_2, stage = 1), - ] - ); - } - - #[test] - fn dist_with_targets_and_hosts() { - let mut cache = run_build( - &[], - configure( - &[TEST_TRIPLE_1, TEST_TRIPLE_2], - &[TEST_TRIPLE_1, TEST_TRIPLE_2, TEST_TRIPLE_3], - ), - ); - - let a = TargetSelection::from_user(TEST_TRIPLE_1); - let b = TargetSelection::from_user(TEST_TRIPLE_2); - let c = TargetSelection::from_user(TEST_TRIPLE_3); - - assert_eq!( - first(cache.all::<dist::Docs>()), - &[dist::Docs { host: a }, dist::Docs { host: b }, dist::Docs { host: c },] - ); - assert_eq!( - first(cache.all::<dist::Mingw>()), - &[dist::Mingw { host: a }, dist::Mingw { host: b }, dist::Mingw { host: c },] - ); - assert_eq!( - first(cache.all::<dist::Rustc>()), - &[ - dist::Rustc { compiler: Compiler::new(2, a) }, - dist::Rustc { compiler: Compiler::new(2, b) }, - ] - ); - assert_eq!( - first(cache.all::<dist::Std>()), - &[ - dist::Std { compiler: Compiler::new(1, a), target: a }, - dist::Std { compiler: Compiler::new(1, a), target: b }, - dist::Std { compiler: Compiler::new(2, a), target: c }, - ] - ); - assert_eq!(first(cache.all::<dist::Src>()), &[dist::Src]); - } - - #[test] - fn dist_with_empty_host() { - let config = configure(&[], &[TEST_TRIPLE_3]); - let mut cache = run_build(&[], config); - - let a = TargetSelection::from_user(TEST_TRIPLE_1); - let c = TargetSelection::from_user(TEST_TRIPLE_3); - - assert_eq!(first(cache.all::<dist::Docs>()), &[dist::Docs { host: c },]); - assert_eq!(first(cache.all::<dist::Mingw>()), &[dist::Mingw { host: c },]); - assert_eq!( - first(cache.all::<dist::Std>()), - &[dist::Std { compiler: Compiler::new(2, a), target: c },] - ); - } - - #[test] - fn dist_with_same_targets_and_hosts() { - let mut cache = run_build( - &[], - configure(&[TEST_TRIPLE_1, TEST_TRIPLE_2], &[TEST_TRIPLE_1, TEST_TRIPLE_2]), - ); - - let a = TargetSelection::from_user(TEST_TRIPLE_1); - let b = TargetSelection::from_user(TEST_TRIPLE_2); - - assert_eq!( - first(cache.all::<dist::Docs>()), - &[dist::Docs { host: a }, dist::Docs { host: b },] - ); - assert_eq!( - first(cache.all::<dist::Mingw>()), - &[dist::Mingw { host: a }, dist::Mingw { host: b },] - ); - assert_eq!( - first(cache.all::<dist::Rustc>()), - &[ - dist::Rustc { compiler: Compiler::new(2, a) }, - dist::Rustc { compiler: Compiler::new(2, b) }, - ] - ); - assert_eq!( - first(cache.all::<dist::Std>()), - &[ - dist::Std { compiler: Compiler::new(1, a), target: a }, - dist::Std { compiler: Compiler::new(1, a), target: b }, - ] - ); - assert_eq!(first(cache.all::<dist::Src>()), &[dist::Src]); - assert_eq!( - first(cache.all::<compile::Std>()), - &[ - std!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 0), - std!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 1), - std!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 2), - std!(TEST_TRIPLE_1 => TEST_TRIPLE_2, stage = 1), - std!(TEST_TRIPLE_1 => TEST_TRIPLE_2, stage = 2), - ] - ); - assert_eq!( - first(cache.all::<compile::Assemble>()), - &[ - compile::Assemble { target_compiler: Compiler::new(0, a) }, - compile::Assemble { target_compiler: Compiler::new(1, a) }, - compile::Assemble { target_compiler: Compiler::new(2, a) }, - compile::Assemble { target_compiler: Compiler::new(2, b) }, - ] - ); - } - - /// This also serves as an important regression test for <https://github.com/rust-lang/rust/issues/138123> - /// and <https://github.com/rust-lang/rust/issues/138004>. - #[test] - fn dist_all_cross() { - let cmd_args = - &["dist", "--stage", "2", "--dry-run", "--config=/does/not/exist"].map(str::to_owned); - let config_str = r#" - [rust] - channel = "nightly" - - [build] - extended = true - - build = "i686-unknown-haiku" - host = ["i686-unknown-netbsd"] - target = ["i686-unknown-netbsd"] - "#; - let config = Config::parse_inner(Flags::parse(cmd_args), |&_| toml::from_str(config_str)); - let mut cache = run_build(&[], config); - - // Stage 2 `compile::Rustc` should **NEVER** be cached here. - assert_eq!( - first(cache.all::<compile::Rustc>()), - &[ - rustc!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 0), - rustc!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 1), - rustc!(TEST_TRIPLE_1 => TEST_TRIPLE_3, stage = 1), - ] - ); - } - - #[test] - fn build_all() { - let build = Build::new(configure( - &[TEST_TRIPLE_1, TEST_TRIPLE_2], - &[TEST_TRIPLE_1, TEST_TRIPLE_2, TEST_TRIPLE_3], - )); - let mut builder = Builder::new(&build); - builder.run_step_descriptions( - &Builder::get_step_descriptions(Kind::Build), - &["compiler/rustc".into(), "library".into()], - ); - - assert_eq!(builder.config.stage, 2); - - // `compile::Rustc` includes one-stage-off compiler information as the target compiler - // artifacts get copied from there to the target stage sysroot. - // For example, `stage2/bin/rustc` gets copied from the `stage1-rustc` build directory. - assert_eq!( - first(builder.cache.all::<compile::Rustc>()), - &[ - rustc!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 0), - rustc!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 1), - rustc!(TEST_TRIPLE_1 => TEST_TRIPLE_2, stage = 1), - ] - ); - - assert_eq!( - first(builder.cache.all::<compile::Std>()), - &[ - std!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 0), - std!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 1), - std!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 2), - std!(TEST_TRIPLE_1 => TEST_TRIPLE_2, stage = 1), - std!(TEST_TRIPLE_1 => TEST_TRIPLE_2, stage = 2), - std!(TEST_TRIPLE_1 => TEST_TRIPLE_3, stage = 2), - ] - ); - - assert_eq!( - first(builder.cache.all::<compile::Assemble>()), - &[ - compile::Assemble { - target_compiler: Compiler::new(0, TargetSelection::from_user(TEST_TRIPLE_1),) - }, - compile::Assemble { - target_compiler: Compiler::new(1, TargetSelection::from_user(TEST_TRIPLE_1),) - }, - compile::Assemble { - target_compiler: Compiler::new(2, TargetSelection::from_user(TEST_TRIPLE_1),) - }, - compile::Assemble { - target_compiler: Compiler::new(2, TargetSelection::from_user(TEST_TRIPLE_2),) - }, - ] - ); - } - - #[test] fn llvm_out_behaviour() { let mut config = configure(&[TEST_TRIPLE_1], &[TEST_TRIPLE_2]); config.llvm_from_ci = true; @@ -784,85 +318,6 @@ mod dist { } #[test] - fn build_with_empty_host() { - let config = configure(&[], &[TEST_TRIPLE_3]); - let build = Build::new(config); - let mut builder = Builder::new(&build); - builder.run_step_descriptions(&Builder::get_step_descriptions(Kind::Build), &[]); - - let a = TargetSelection::from_user(TEST_TRIPLE_1); - - assert_eq!( - first(builder.cache.all::<compile::Std>()), - &[ - std!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 0), - std!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 1), - std!(TEST_TRIPLE_1 => TEST_TRIPLE_3, stage = 2), - ] - ); - assert_eq!( - first(builder.cache.all::<compile::Assemble>()), - &[ - compile::Assemble { target_compiler: Compiler::new(0, a) }, - compile::Assemble { target_compiler: Compiler::new(1, a) }, - compile::Assemble { target_compiler: Compiler::new(2, a) }, - ] - ); - assert_eq!( - first(builder.cache.all::<compile::Rustc>()), - &[ - rustc!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 0), - rustc!(TEST_TRIPLE_1 => TEST_TRIPLE_1, stage = 1), - ] - ); - } - - #[test] - fn test_with_no_doc_stage0() { - let mut config = configure(&[TEST_TRIPLE_1], &[TEST_TRIPLE_1]); - config.stage = 0; - config.paths = vec!["library/std".into()]; - config.cmd = Subcommand::Test { - test_args: vec![], - compiletest_rustc_args: vec![], - no_fail_fast: false, - no_doc: true, - doc: false, - bless: false, - force_rerun: false, - compare_mode: None, - rustfix_coverage: false, - pass: None, - run: None, - only_modified: false, - extra_checks: None, - no_capture: false, - }; - - let build = Build::new(config); - let mut builder = Builder::new(&build); - - let host = TargetSelection::from_user(TEST_TRIPLE_1); - - builder.run_step_descriptions( - &[StepDescription::from::<test::Crate>(Kind::Test)], - &["library/std".into()], - ); - - // Ensure we don't build any compiler artifacts. - assert!(!builder.cache.contains::<compile::Rustc>()); - assert_eq!( - first(builder.cache.all::<test::Crate>()), - &[test::Crate { - compiler: Compiler::new(0, host), - target: host, - mode: crate::Mode::Std, - crates: vec!["std".to_owned()], - },] - ); - } - - #[test] fn doc_ci() { let mut config = configure(&[TEST_TRIPLE_1], &[TEST_TRIPLE_1]); config.compiler_docs = true; @@ -889,65 +344,6 @@ mod dist { &[tool::Rustdoc { compiler: Compiler::new(2, a) },] ); } - - #[test] - fn test_docs() { - // Behavior of `x.py test` doing various documentation tests. - let mut config = configure(&[TEST_TRIPLE_1], &[TEST_TRIPLE_1]); - config.cmd = Subcommand::Test { - test_args: vec![], - compiletest_rustc_args: vec![], - no_fail_fast: false, - doc: true, - no_doc: false, - bless: false, - force_rerun: false, - compare_mode: None, - rustfix_coverage: false, - pass: None, - run: None, - only_modified: false, - extra_checks: None, - no_capture: false, - }; - // Make sure rustfmt binary not being found isn't an error. - config.channel = "beta".to_string(); - let build = Build::new(config); - let mut builder = Builder::new(&build); - - builder.run_step_descriptions(&Builder::get_step_descriptions(Kind::Test), &[]); - let a = TargetSelection::from_user(TEST_TRIPLE_1); - - // error_index_generator uses stage 1 to share rustdoc artifacts with the - // rustdoc tool. - assert_eq!( - first(builder.cache.all::<doc::ErrorIndex>()), - &[doc::ErrorIndex { target: a },] - ); - assert_eq!( - first(builder.cache.all::<tool::ErrorIndex>()), - &[tool::ErrorIndex { compiler: Compiler::new(1, a) }] - ); - // Unfortunately rustdoc is built twice. Once from stage1 for compiletest - // (and other things), and once from stage0 for std crates. Ideally it - // would only be built once. If someone wants to fix this, it might be - // worth investigating if it would be possible to test std from stage1. - // Note that the stages here are +1 than what they actually are because - // Rustdoc::run swaps out the compiler with stage minus 1 if --stage is - // not 0. - // - // The stage 0 copy is the one downloaded for bootstrapping. It is - // (currently) needed to run "cargo test" on the linkchecker, and - // should be relatively "free". - assert_eq!( - first(builder.cache.all::<tool::Rustdoc>()), - &[ - tool::Rustdoc { compiler: Compiler::new(0, a) }, - tool::Rustdoc { compiler: Compiler::new(1, a) }, - tool::Rustdoc { compiler: Compiler::new(2, a) }, - ] - ); - } } mod sysroot_target_dirs { @@ -1234,15 +630,104 @@ fn any_debug() { assert_eq!(x.downcast_ref::<MyStruct>(), Some(&MyStruct { x: 7 })); } -/// The staging tests use insta for snapshot testing. +/// These tests use insta for snapshot testing. /// See bootstrap's README on how to bless the snapshots. -mod staging { - use crate::Build; - use crate::core::builder::Builder; +mod snapshot { + use std::path::PathBuf; + + use crate::core::build_steps::{compile, dist, doc, test, tool}; use crate::core::builder::tests::{ - TEST_TRIPLE_1, configure, configure_with_args, render_steps, run_build, + TEST_TRIPLE_1, TEST_TRIPLE_2, TEST_TRIPLE_3, configure, configure_with_args, first, + host_target, render_steps, run_build, }; + use crate::core::builder::{Builder, Kind, StepDescription, StepMetadata}; + use crate::core::config::TargetSelection; + use crate::utils::cache::Cache; + use crate::utils::helpers::get_host_target; use crate::utils::tests::{ConfigBuilder, TestCtx}; + use crate::{Build, Compiler, Config, Flags, Subcommand}; + + #[test] + fn build_default() { + let ctx = TestCtx::new(); + insta::assert_snapshot!( + ctx.config("build") + .render_steps(), @r" + [build] llvm <host> + [build] rustc 0 <host> -> rustc 1 <host> + [build] rustc 1 <host> -> std 1 <host> + [build] rustdoc 0 <host> + "); + } + + #[test] + fn build_cross_compile() { + let ctx = TestCtx::new(); + + insta::assert_snapshot!( + ctx.config("build") + // Cross-compilation fails on stage 1, as we don't have a stage0 std available + // for non-host targets. + .stage(2) + .hosts(&[&host_target(), TEST_TRIPLE_1]) + .targets(&[&host_target(), TEST_TRIPLE_1]) + .render_steps(), @r" + [build] llvm <host> + [build] rustc 0 <host> -> rustc 1 <host> + [build] rustc 1 <host> -> std 1 <host> + [build] rustc 1 <host> -> rustc 2 <host> + [build] rustc 2 <host> -> std 2 <host> + [build] rustc 1 <host> -> std 1 <target1> + [build] rustc 2 <host> -> std 2 <target1> + [build] rustdoc 1 <host> + [build] llvm <target1> + [build] rustc 1 <host> -> rustc 2 <target1> + [build] rustdoc 1 <target1> + "); + } + + #[test] + fn build_with_empty_host() { + let ctx = TestCtx::new(); + insta::assert_snapshot!( + ctx + .config("build") + .hosts(&[]) + .targets(&[TEST_TRIPLE_1]) + .render_steps(), @r" + [build] llvm <host> + [build] rustc 0 <host> -> rustc 1 <host> + [build] rustc 1 <host> -> std 1 <target1> + " + ); + } + + #[test] + fn build_compiler_no_explicit_stage() { + let ctx = TestCtx::new(); + insta::assert_snapshot!( + ctx.config("build") + .path("compiler") + .render_steps(), @r" + [build] llvm <host> + [build] rustc 0 <host> -> rustc 1 <host> + "); + + insta::assert_snapshot!( + ctx.config("build") + .path("rustc") + .render_steps(), @r" + [build] llvm <host> + [build] rustc 0 <host> -> rustc 1 <host> + "); + } + + #[test] + #[should_panic] + fn build_compiler_stage_0() { + let ctx = TestCtx::new(); + ctx.config("build").path("compiler").stage(0).run(); + } #[test] fn build_compiler_stage_1() { @@ -1251,25 +736,624 @@ mod staging { ctx.config("build") .path("compiler") .stage(1) - .get_steps(), @r" - [build] rustc 0 <host> -> std 0 <host> + .render_steps(), @r" + [build] llvm <host> + [build] rustc 0 <host> -> rustc 1 <host> + "); + } + + #[test] + fn build_compiler_stage_2() { + let ctx = TestCtx::new(); + insta::assert_snapshot!( + ctx.config("build") + .path("compiler") + .stage(2) + .render_steps(), @r" + [build] llvm <host> + [build] rustc 0 <host> -> rustc 1 <host> + [build] rustc 1 <host> -> std 1 <host> + [build] rustc 1 <host> -> rustc 2 <host> + "); + } + + #[test] + fn build_library_no_explicit_stage() { + let ctx = TestCtx::new(); + insta::assert_snapshot!( + ctx.config("build") + .path("library") + .render_steps(), @r" + [build] llvm <host> + [build] rustc 0 <host> -> rustc 1 <host> + [build] rustc 1 <host> -> std 1 <host> + "); + } + + #[test] + #[should_panic] + fn build_library_stage_0() { + let ctx = TestCtx::new(); + ctx.config("build").path("library").stage(0).run(); + } + + #[test] + fn build_library_stage_1() { + let ctx = TestCtx::new(); + insta::assert_snapshot!( + ctx.config("build") + .path("library") + .stage(1) + .render_steps(), @r" + [build] llvm <host> + [build] rustc 0 <host> -> rustc 1 <host> + [build] rustc 1 <host> -> std 1 <host> + "); + } + + #[test] + fn build_library_stage_2() { + let ctx = TestCtx::new(); + insta::assert_snapshot!( + ctx.config("build") + .path("library") + .stage(2) + .render_steps(), @r" + [build] llvm <host> + [build] rustc 0 <host> -> rustc 1 <host> + [build] rustc 1 <host> -> std 1 <host> + [build] rustc 1 <host> -> rustc 2 <host> + [build] rustc 2 <host> -> std 2 <host> + "); + } + + #[test] + fn build_miri_no_explicit_stage() { + let ctx = TestCtx::new(); + insta::assert_snapshot!( + ctx.config("build") + .path("miri") + .render_steps(), @r" + [build] llvm <host> + [build] rustc 0 <host> -> rustc 1 <host> + [build] rustc 0 <host> -> miri 1 <host> + "); + } + + #[test] + #[should_panic] + fn build_miri_stage_0() { + let ctx = TestCtx::new(); + ctx.config("build").path("miri").stage(0).run(); + } + + #[test] + fn build_miri_stage_1() { + let ctx = TestCtx::new(); + insta::assert_snapshot!( + ctx.config("build") + .path("miri") + .stage(1) + .render_steps(), @r" + [build] llvm <host> + [build] rustc 0 <host> -> rustc 1 <host> + [build] rustc 0 <host> -> miri 1 <host> + "); + } + + #[test] + fn build_miri_stage_2() { + let ctx = TestCtx::new(); + insta::assert_snapshot!( + ctx.config("build") + .path("miri") + .stage(2) + .render_steps(), @r" + [build] llvm <host> + [build] rustc 0 <host> -> rustc 1 <host> + [build] rustc 1 <host> -> std 1 <host> + [build] rustc 1 <host> -> rustc 2 <host> + [build] rustc 1 <host> -> miri 2 <host> + "); + } + + #[test] + fn build_bootstrap_tool_no_explicit_stage() { + let ctx = TestCtx::new(); + insta::assert_snapshot!( + ctx.config("build") + .path("opt-dist") + .render_steps(), @"[build] rustc 0 <host> -> OptimizedDist <host>"); + } + + #[test] + #[should_panic] + fn build_bootstrap_tool_stage_0() { + let ctx = TestCtx::new(); + ctx.config("build").path("opt-dist").stage(0).run(); + } + + #[test] + fn build_bootstrap_tool_stage_1() { + let ctx = TestCtx::new(); + insta::assert_snapshot!( + ctx.config("build") + .path("opt-dist") + .stage(1) + .render_steps(), @"[build] rustc 0 <host> -> OptimizedDist <host>"); + } + + #[test] + fn build_bootstrap_tool_stage_2() { + let ctx = TestCtx::new(); + insta::assert_snapshot!( + ctx.config("build") + .path("opt-dist") + .stage(2) + .render_steps(), @"[build] rustc 0 <host> -> OptimizedDist <host>"); + } + + #[test] + fn build_default_stage() { + let ctx = TestCtx::new(); + assert_eq!(ctx.config("build").path("compiler").create_config().stage, 1); + } + + /// Ensure that if someone passes both a single crate and `library`, all + /// library crates get built. + #[test] + fn alias_and_path_for_library() { + let ctx = TestCtx::new(); + insta::assert_snapshot!(ctx.config("build") + .paths(&["library", "core"]) + .render_steps(), @r" + [build] llvm <host> + [build] rustc 0 <host> -> rustc 1 <host> + [build] rustc 1 <host> -> std 1 <host> + "); + + insta::assert_snapshot!(ctx.config("build") + .paths(&["std"]) + .render_steps(), @r" + [build] llvm <host> + [build] rustc 0 <host> -> rustc 1 <host> + [build] rustc 1 <host> -> std 1 <host> + "); + + insta::assert_snapshot!(ctx.config("build") + .paths(&["core"]) + .render_steps(), @r" + [build] llvm <host> + [build] rustc 0 <host> -> rustc 1 <host> + [build] rustc 1 <host> -> std 1 <host> + "); + + insta::assert_snapshot!(ctx.config("build") + .paths(&["alloc"]) + .render_steps(), @r" + [build] llvm <host> + [build] rustc 0 <host> -> rustc 1 <host> + [build] rustc 1 <host> -> std 1 <host> + "); + + insta::assert_snapshot!(ctx.config("doc") + .paths(&["library", "core"]) + .render_steps(), @r" + [build] llvm <host> + [build] rustc 0 <host> -> rustc 1 <host> + [build] rustdoc 0 <host> + [doc] std 1 <host> + "); + } + + #[test] + fn build_all() { + let ctx = TestCtx::new(); + insta::assert_snapshot!( + ctx.config("build") + .stage(2) + .paths(&["compiler/rustc", "library"]) + .hosts(&[&host_target(), TEST_TRIPLE_1]) + .targets(&[&host_target(), TEST_TRIPLE_1, TEST_TRIPLE_2]) + .render_steps(), @r" + [build] llvm <host> + [build] rustc 0 <host> -> rustc 1 <host> + [build] rustc 1 <host> -> std 1 <host> + [build] rustc 1 <host> -> rustc 2 <host> + [build] llvm <target1> + [build] rustc 1 <host> -> std 1 <target1> + [build] rustc 1 <host> -> rustc 2 <target1> + [build] rustc 2 <host> -> std 2 <host> + [build] rustc 2 <host> -> std 2 <target1> + [build] rustc 2 <host> -> std 2 <target2> + "); + } + + #[test] + fn dist_default_stage() { + let ctx = TestCtx::new(); + assert_eq!(ctx.config("dist").path("compiler").create_config().stage, 2); + } + + #[test] + fn dist_baseline() { + let ctx = TestCtx::new(); + // Note that stdlib is uplifted, that is why `[dist] rustc 1 <host> -> std <host>` is in + // the output. + insta::assert_snapshot!( + ctx + .config("dist") + .render_steps(), @r" + [build] rustc 0 <host> -> UnstableBookGen <host> + [build] rustc 0 <host> -> Rustbook <host> [build] llvm <host> [build] rustc 0 <host> -> rustc 1 <host> + [build] rustc 1 <host> -> std 1 <host> + [build] rustc 1 <host> -> rustc 2 <host> + [build] rustdoc 1 <host> + [doc] std 2 <host> + [build] rustc 2 <host> -> std 2 <host> + [build] rustc 0 <host> -> LintDocs <host> + [build] rustc 0 <host> -> RustInstaller <host> + [dist] docs <host> + [doc] std 2 <host> + [dist] mingw <host> + [build] rustc 0 <host> -> GenerateCopyright <host> + [dist] rustc <host> + [dist] rustc 1 <host> -> std <host> + [dist] src <> + " + ); + } + + #[test] + fn dist_extended() { + let ctx = TestCtx::new(); + insta::assert_snapshot!( + ctx + .config("dist") + .args(&["--set", "build.extended=true"]) + .render_steps(), @r" + [build] rustc 0 <host> -> UnstableBookGen <host> + [build] rustc 0 <host> -> Rustbook <host> + [build] llvm <host> [build] rustc 0 <host> -> rustc 1 <host> + [build] rustc 0 <host> -> WasmComponentLd <host> + [build] rustc 1 <host> -> std 1 <host> + [build] rustc 1 <host> -> rustc 2 <host> + [build] rustc 1 <host> -> WasmComponentLd <host> + [build] rustdoc 1 <host> + [doc] std 2 <host> + [build] rustc 2 <host> -> std 2 <host> + [build] rustc 0 <host> -> LintDocs <host> + [build] rustc 0 <host> -> RustInstaller <host> + [dist] docs <host> + [doc] std 2 <host> + [dist] mingw <host> + [build] rustc 0 <host> -> GenerateCopyright <host> + [dist] rustc <host> + [dist] rustc 1 <host> -> std <host> + [dist] src <> + [build] rustc 0 <host> -> rustfmt 1 <host> + [build] rustc 0 <host> -> cargo-fmt 1 <host> + [build] rustc 0 <host> -> clippy-driver 1 <host> + [build] rustc 0 <host> -> cargo-clippy 1 <host> + [build] rustc 0 <host> -> miri 1 <host> + [build] rustc 0 <host> -> cargo-miri 1 <host> "); } - impl ConfigBuilder { - fn get_steps(self) -> String { - let config = self.create_config(); + #[test] + fn dist_with_targets() { + let ctx = TestCtx::new(); + insta::assert_snapshot!( + ctx + .config("dist") + .hosts(&[&host_target()]) + .targets(&[&host_target(), TEST_TRIPLE_1]) + .render_steps(), @r" + [build] rustc 0 <host> -> UnstableBookGen <host> + [build] rustc 0 <host> -> Rustbook <host> + [build] llvm <host> + [build] rustc 0 <host> -> rustc 1 <host> + [build] rustc 1 <host> -> std 1 <host> + [build] rustc 1 <host> -> rustc 2 <host> + [build] rustdoc 1 <host> + [doc] std 2 <host> + [doc] std 2 <target1> + [build] rustc 2 <host> -> std 2 <host> + [build] rustc 0 <host> -> LintDocs <host> + [build] rustc 0 <host> -> RustInstaller <host> + [dist] docs <host> + [dist] docs <target1> + [doc] std 2 <host> + [doc] std 2 <target1> + [dist] mingw <host> + [dist] mingw <target1> + [build] rustc 0 <host> -> GenerateCopyright <host> + [dist] rustc <host> + [dist] rustc 1 <host> -> std <host> + [build] rustc 2 <host> -> std 2 <target1> + [dist] rustc 2 <host> -> std <target1> + [dist] src <> + " + ); + } + + #[test] + fn dist_with_hosts() { + let ctx = TestCtx::new(); + insta::assert_snapshot!( + ctx + .config("dist") + .hosts(&[&host_target(), TEST_TRIPLE_1]) + .targets(&[&host_target()]) + .render_steps(), @r" + [build] rustc 0 <host> -> UnstableBookGen <host> + [build] rustc 0 <host> -> Rustbook <host> + [build] llvm <host> + [build] rustc 0 <host> -> rustc 1 <host> + [build] rustc 1 <host> -> std 1 <host> + [build] rustc 1 <host> -> rustc 2 <host> + [build] rustdoc 1 <host> + [doc] std 2 <host> + [build] rustc 2 <host> -> std 2 <host> + [build] rustc 0 <host> -> LintDocs <host> + [build] rustc 1 <host> -> std 1 <target1> + [build] rustc 2 <host> -> std 2 <target1> + [build] rustc 0 <host> -> RustInstaller <host> + [dist] docs <host> + [doc] std 2 <host> + [dist] mingw <host> + [build] rustc 0 <host> -> GenerateCopyright <host> + [dist] rustc <host> + [build] llvm <target1> + [build] rustc 1 <host> -> rustc 2 <target1> + [build] rustdoc 1 <target1> + [dist] rustc <target1> + [dist] rustc 1 <host> -> std <host> + [dist] src <> + " + ); + } + + #[test] + fn dist_with_targets_and_hosts() { + let ctx = TestCtx::new(); + insta::assert_snapshot!( + ctx + .config("dist") + .hosts(&[&host_target(), TEST_TRIPLE_1]) + .targets(&[&host_target(), TEST_TRIPLE_1]) + .render_steps(), @r" + [build] rustc 0 <host> -> UnstableBookGen <host> + [build] rustc 0 <host> -> Rustbook <host> + [build] llvm <host> + [build] rustc 0 <host> -> rustc 1 <host> + [build] rustc 1 <host> -> std 1 <host> + [build] rustc 1 <host> -> rustc 2 <host> + [build] rustdoc 1 <host> + [doc] std 2 <host> + [doc] std 2 <target1> + [build] rustc 2 <host> -> std 2 <host> + [build] rustc 0 <host> -> LintDocs <host> + [build] rustc 1 <host> -> std 1 <target1> + [build] rustc 2 <host> -> std 2 <target1> + [build] rustc 0 <host> -> RustInstaller <host> + [dist] docs <host> + [dist] docs <target1> + [doc] std 2 <host> + [doc] std 2 <target1> + [dist] mingw <host> + [dist] mingw <target1> + [build] rustc 0 <host> -> GenerateCopyright <host> + [dist] rustc <host> + [build] llvm <target1> + [build] rustc 1 <host> -> rustc 2 <target1> + [build] rustdoc 1 <target1> + [dist] rustc <target1> + [dist] rustc 1 <host> -> std <host> + [dist] rustc 1 <host> -> std <target1> + [dist] src <> + " + ); + } - let kind = config.cmd.kind(); - let build = Build::new(config); - let builder = Builder::new(&build); - builder.run_step_descriptions(&Builder::get_step_descriptions(kind), &builder.paths); - render_steps(&builder.cache.into_executed_steps()) + #[test] + fn dist_with_empty_host() { + let ctx = TestCtx::new(); + insta::assert_snapshot!( + ctx + .config("dist") + .hosts(&[]) + .targets(&[TEST_TRIPLE_1]) + .render_steps(), @r" + [build] rustc 0 <host> -> UnstableBookGen <host> + [build] rustc 0 <host> -> Rustbook <host> + [build] llvm <host> + [build] rustc 0 <host> -> rustc 1 <host> + [build] rustc 1 <host> -> std 1 <host> + [build] rustc 1 <host> -> rustc 2 <host> + [build] rustdoc 1 <host> + [doc] std 2 <target1> + [build] rustc 2 <host> -> std 2 <host> + [build] rustc 0 <host> -> RustInstaller <host> + [dist] docs <target1> + [doc] std 2 <target1> + [dist] mingw <target1> + [build] rustc 2 <host> -> std 2 <target1> + [dist] rustc 2 <host> -> std <target1> + "); + } + + /// This also serves as an important regression test for <https://github.com/rust-lang/rust/issues/138123> + /// and <https://github.com/rust-lang/rust/issues/138004>. + #[test] + fn dist_all_cross() { + let ctx = TestCtx::new(); + insta::assert_snapshot!( + ctx + .config("dist") + .hosts(&[TEST_TRIPLE_1]) + .targets(&[TEST_TRIPLE_1]) + .args(&["--set", "rust.channel=nightly", "--set", "build.extended=true"]) + .render_steps(), @r" + [build] rustc 0 <host> -> UnstableBookGen <host> + [build] rustc 0 <host> -> Rustbook <host> + [build] llvm <host> + [build] rustc 0 <host> -> rustc 1 <host> + [build] rustc 0 <host> -> WasmComponentLd <host> + [build] rustc 1 <host> -> std 1 <host> + [build] rustc 1 <host> -> rustc 2 <host> + [build] rustc 1 <host> -> WasmComponentLd <host> + [build] rustdoc 1 <host> + [doc] std 2 <target1> + [build] rustc 2 <host> -> std 2 <host> + [build] rustc 1 <host> -> std 1 <target1> + [build] rustc 2 <host> -> std 2 <target1> + [build] rustc 0 <host> -> LintDocs <host> + [build] rustc 0 <host> -> RustInstaller <host> + [dist] docs <target1> + [doc] std 2 <target1> + [dist] mingw <target1> + [build] llvm <target1> + [build] rustc 1 <host> -> rustc 2 <target1> + [build] rustc 1 <host> -> WasmComponentLd <target1> + [build] rustdoc 1 <target1> + [build] rustc 0 <host> -> GenerateCopyright <host> + [dist] rustc <target1> + [dist] rustc 1 <host> -> std <target1> + [dist] src <> + [build] rustc 0 <host> -> rustfmt 1 <target1> + [build] rustc 0 <host> -> cargo-fmt 1 <target1> + [build] rustc 0 <host> -> clippy-driver 1 <target1> + [build] rustc 0 <host> -> cargo-clippy 1 <target1> + [build] rustc 0 <host> -> miri 1 <target1> + [build] rustc 0 <host> -> cargo-miri 1 <target1> + "); + } + + #[test] + fn test_exclude() { + let ctx = TestCtx::new(); + let steps = ctx.config("test").args(&["--skip", "src/tools/tidy"]).get_steps(); + + let host = TargetSelection::from_user(&host_target()); + steps.assert_contains(StepMetadata::test("RustdocUi", host)); + steps.assert_not_contains(test::Tidy); + } + + #[test] + fn test_exclude_kind() { + let ctx = TestCtx::new(); + let host = TargetSelection::from_user(&host_target()); + + let get_steps = |args: &[&str]| ctx.config("test").args(args).get_steps(); + + // Ensure our test is valid, and `test::Rustc` would be run without the exclude. + get_steps(&[]).assert_contains(StepMetadata::test("CrateLibrustc", host)); + + let steps = get_steps(&["--skip", "compiler/rustc_data_structures"]); + + // Ensure tests for rustc are not skipped. + steps.assert_contains(StepMetadata::test("CrateLibrustc", host)); + steps.assert_contains_fuzzy(StepMetadata::build("rustc", host)); + } +} + +struct ExecutedSteps { + steps: Vec<ExecutedStep>, +} + +impl ExecutedSteps { + fn render(&self) -> String { + render_steps(&self.steps) + } + + #[track_caller] + fn assert_contains<M: Into<StepMetadata>>(&self, metadata: M) { + let metadata = metadata.into(); + if !self.contains(&metadata) { + panic!( + "Metadata `{}` ({metadata:?}) not found in executed steps:\n{}", + render_metadata(&metadata), + self.render() + ); + } + } + + /// Try to match metadata by similarity, it does not need to match exactly. + /// Stages (and built_by compiler) do not need to match, but name, target and + /// kind has to match. + #[track_caller] + fn assert_contains_fuzzy<M: Into<StepMetadata>>(&self, metadata: M) { + let metadata = metadata.into(); + if !self.contains_fuzzy(&metadata) { + panic!( + "Metadata `{}` ({metadata:?}) not found in executed steps:\n{}", + render_metadata(&metadata), + self.render() + ); + } + } + + #[track_caller] + fn assert_not_contains<M: Into<StepMetadata>>(&self, metadata: M) { + let metadata = metadata.into(); + if self.contains(&metadata) { + panic!( + "Metadata `{}` ({metadata:?}) found in executed steps (it should not be there):\n{}", + render_metadata(&metadata), + self.render() + ); } } + + fn contains(&self, metadata: &StepMetadata) -> bool { + self.steps + .iter() + .filter_map(|s| s.metadata.as_ref()) + .any(|executed_metadata| executed_metadata == metadata) + } + + fn contains_fuzzy(&self, metadata: &StepMetadata) -> bool { + self.steps + .iter() + .filter_map(|s| s.metadata.as_ref()) + .any(|executed_metadata| fuzzy_metadata_eq(executed_metadata, metadata)) + } +} + +fn fuzzy_metadata_eq(executed: &StepMetadata, to_match: &StepMetadata) -> bool { + let StepMetadata { name, kind, target, built_by: _, stage: _ } = executed; + *name == to_match.name && *kind == to_match.kind && *target == to_match.target +} + +impl<S: Step> From<S> for StepMetadata { + fn from(step: S) -> Self { + step.metadata().expect("step has no metadata") + } +} + +impl ConfigBuilder { + fn run(self) -> Cache { + let config = self.create_config(); + + let kind = config.cmd.kind(); + let build = Build::new(config); + let builder = Builder::new(&build); + builder.run_step_descriptions(&Builder::get_step_descriptions(kind), &builder.paths); + builder.cache + } + + fn get_steps(self) -> ExecutedSteps { + let cache = self.run(); + ExecutedSteps { steps: cache.into_executed_steps() } + } + + fn render_steps(self) -> String { + self.get_steps().render() + } } /// Renders the executed bootstrap steps for usage in snapshot tests with insta. @@ -1289,23 +1373,34 @@ fn render_steps(steps: &[ExecutedStep]) -> String { return None; }; - let mut record = format!("[{}] ", metadata.kind.as_str()); - if let Some(compiler) = metadata.built_by { - write!(record, "{} -> ", render_compiler(compiler)); - } - let stage = - if let Some(stage) = metadata.stage { format!("{stage} ") } else { "".to_string() }; - write!(record, "{} {stage}<{}>", metadata.name, normalize_target(metadata.target)); - Some(record) + Some(render_metadata(&metadata)) }) .collect::<Vec<_>>() .join("\n") } +fn render_metadata(metadata: &StepMetadata) -> String { + let mut record = format!("[{}] ", metadata.kind.as_str()); + if let Some(compiler) = metadata.built_by { + write!(record, "{} -> ", render_compiler(compiler)); + } + let stage = if let Some(stage) = metadata.stage { format!("{stage} ") } else { "".to_string() }; + write!(record, "{} {stage}<{}>", metadata.name, normalize_target(metadata.target)); + record +} + fn normalize_target(target: TargetSelection) -> String { - target.to_string().replace(&get_host_target().to_string(), "host") + target + .to_string() + .replace(&host_target(), "host") + .replace(TEST_TRIPLE_1, "target1") + .replace(TEST_TRIPLE_2, "target2") } fn render_compiler(compiler: Compiler) -> String { format!("rustc {} <{}>", compiler.stage, normalize_target(compiler.host)) } + +fn host_target() -> String { + get_host_target().to_string() +} diff --git a/src/bootstrap/src/core/config/config.rs b/src/bootstrap/src/core/config/config.rs index d3393afcae0..d7decaa8a98 100644 --- a/src/bootstrap/src/core/config/config.rs +++ b/src/bootstrap/src/core/config/config.rs @@ -1023,7 +1023,7 @@ impl Config { || install_stage.is_some() || check_stage.is_some() || bench_stage.is_some(); - // See https://github.com/rust-lang/compiler-team/issues/326 + config.stage = match config.cmd { Subcommand::Check { .. } => flags_stage.or(check_stage).unwrap_or(0), Subcommand::Clippy { .. } | Subcommand::Fix => flags_stage.or(check_stage).unwrap_or(1), @@ -1051,6 +1051,12 @@ impl Config { | Subcommand::Vendor { .. } => flags_stage.unwrap_or(0), }; + // Now check that the selected stage makes sense, and if not, print a warning and end + if let (0, Subcommand::Build) = (config.stage, &config.cmd) { + eprintln!("WARNING: cannot build anything on stage 0. Use at least stage 1."); + exit!(1); + } + // CI should always run stage 2 builds, unless it specifically states otherwise #[cfg(not(test))] if flags_stage.is_none() && config.is_running_on_ci { diff --git a/src/bootstrap/src/utils/change_tracker.rs b/src/bootstrap/src/utils/change_tracker.rs index 93e01a58077..7c588cfea8c 100644 --- a/src/bootstrap/src/utils/change_tracker.rs +++ b/src/bootstrap/src/utils/change_tracker.rs @@ -426,4 +426,9 @@ pub const CONFIG_CHANGE_HISTORY: &[ChangeInfo] = &[ severity: ChangeSeverity::Info, summary: "Added new option `tool.TOOL_NAME.features` to specify the features to compile a tool with", }, + ChangeInfo { + change_id: 142581, + severity: ChangeSeverity::Warning, + summary: "It is no longer possible to `x build` with stage 0. All build commands have to be on stage 1+.", + }, ]; diff --git a/src/bootstrap/src/utils/tests/mod.rs b/src/bootstrap/src/utils/tests/mod.rs index 91877fd0da4..b8984d1f3aa 100644 --- a/src/bootstrap/src/utils/tests/mod.rs +++ b/src/bootstrap/src/utils/tests/mod.rs @@ -51,12 +51,38 @@ impl ConfigBuilder { self } + pub fn paths(mut self, paths: &[&str]) -> Self { + for path in paths { + self = self.path(path); + } + self + } + + pub fn hosts(mut self, targets: &[&str]) -> Self { + self.args.push("--host".to_string()); + self.args.push(targets.join(",")); + self + } + + pub fn targets(mut self, targets: &[&str]) -> Self { + self.args.push("--target".to_string()); + self.args.push(targets.join(",")); + self + } + pub fn stage(mut self, stage: u32) -> Self { self.args.push("--stage".to_string()); self.args.push(stage.to_string()); self } + pub fn args(mut self, args: &[&str]) -> Self { + for arg in args { + self.args.push(arg.to_string()); + } + self + } + pub fn create_config(mut self) -> Config { // Run in dry-check, otherwise the test would be too slow self.args.push("--dry-run".to_string()); diff --git a/src/ci/docker/host-x86_64/mingw-check-1/Dockerfile b/src/ci/docker/host-x86_64/mingw-check-1/Dockerfile index 9bdcf00dccc..c46a2471e75 100644 --- a/src/ci/docker/host-x86_64/mingw-check-1/Dockerfile +++ b/src/ci/docker/host-x86_64/mingw-check-1/Dockerfile @@ -45,7 +45,7 @@ COPY host-x86_64/mingw-check-1/validate-toolstate.sh /scripts/ # We also skip the x86_64-unknown-linux-gnu target as it is well-tested by other jobs. ENV SCRIPT \ /scripts/check-default-config-profiles.sh && \ - python3 ../x.py build --stage 0 src/tools/build-manifest && \ + python3 ../x.py build --stage 1 src/tools/build-manifest && \ python3 ../x.py test --stage 0 src/tools/compiletest && \ python3 ../x.py check compiletest --set build.compiletest-use-stage0-libtest=true && \ python3 ../x.py check --stage 1 --target=i686-pc-windows-gnu --host=i686-pc-windows-gnu && \ diff --git a/src/ci/docker/scripts/rfl-build.sh b/src/ci/docker/scripts/rfl-build.sh index c5992891398..8acc5040a2f 100755 --- a/src/ci/docker/scripts/rfl-build.sh +++ b/src/ci/docker/scripts/rfl-build.sh @@ -6,7 +6,7 @@ LINUX_VERSION=v6.16-rc1 # Build rustc, rustdoc, cargo, clippy-driver and rustfmt ../x.py build --stage 2 library rustdoc clippy rustfmt -../x.py build --stage 0 cargo +../x.py build --stage 1 cargo BUILD_DIR=$(realpath ./build/x86_64-unknown-linux-gnu) |
