Auto merge of #144787 - Kobzol:codegen-backend-restructure, r=jieyouxu

Refactor codegen backends in bootstrap

This PR refactors the codegen backend steps, in preparation to make more progress on the integration of the GCC codegen backend in bootstrap. It does several things:

1) Splits the `CodegenBackend` step into two, one for clif and another one for gcc. Even though their code is mostly similar, that's IMO mostly fake similarity, and they do (or will) ultimately require different handling. This was already visible in the requirement of building GCC for cg_gcc, of course.
2) It is now possible to build both backends (and dist cranelift) even if they are not specified in `rust.codegen-backends`. It was quite weird that it wasn't possible to even invoke the corresponding codegen backend if the backend wasn't specified in that array, as that array should ideally only change defaults (see later below).
3) Changes the path specification of these steps to an alias. In other words, instead of `compiler/rustc_codegen_cranelift`, the step is now built only using `rustc_codegen_cranelift` or `cg_clif`. This is done to avoid an annoying clash with `x build compiler`, which would otherwise build both codegen backends after the 2) change.
4) Made the copying of codegen backend artifacts more explicit, in particular in the `Assemble` step.
5) Codifies the semantics of `rust.codegen-backends`, which now only affects the defaults of whether a codegen backend will be included in rustc's sysroot and whether it will be disted in `x dist` by default. We can change the behavior later, e.g. to dist cranelift by default in `x dist` once it becomes stabilized. Currently I left the existing behavior that we use on CI, I just tried to document it better.

I don't think that this requires a change tracker entry, because the defaults should work the same as before. It is just now possible to do `x build/dist rustc_codegen_cranelift` even if CLIF is not in the `codegen-backends` array. It is no longer possible to do `./x build compiler/rustc_codegen_cranelift` though, not sure if that requires a change tracker entry.

There is one thing that I didn't touch yet, and that is the fact that `rust.codegen-backends` not only affects the default behavior of `x dist` w.r.t. Cranelift, but also of `x test`. In other words, `x test rustc_codegen_cranelift` still does not hing if cranelift isn't in `rust.codegen-backends`. I plan to take a look at this once I get to refactoring the test steps.

r? `@jieyouxu`

1 files changed, 13 insertions, 5 deletions

diff --git a/bootstrap.example.toml b/bootstrap.example.toml
index 31966af3301..89da6eeb531 100644
--- a/bootstrap.example.toml
+++ b/bootstrap.example.toml
@@ -740,11 +740,19 @@
 # result (broken, compiling, testing) into this JSON file.
 #rust.save-toolstates = <none> (path)
 
-# This is an array of the codegen backends that will be compiled for the rustc
-# that's being compiled. The default is to only build the LLVM codegen backend,
-# and currently the only standard options supported are `"llvm"`, `"cranelift"`
-# and `"gcc"`. The first backend in this list will be used as default by rustc
-# when no explicit backend is specified.
+# This array serves three distinct purposes:
+# - Backends in this list will be automatically compiled and included in the sysroot of each
+# rustc compiled by bootstrap.
+# - The first backend in this list will be configured as the **default codegen backend** by each
+# rustc compiled by bootstrap. In other words, if the first backend is e.g. cranelift, then when
+# we build a stage 1 rustc, it will by default compile Rust programs using the Cranelift backend.
+# This also means that stage 2 rustc would get built by the Cranelift backend.
+# - Running `x dist` (without additional arguments, or with `--include-default-paths`) will produce
+# a dist component/tarball for the Cranelift backend if it is included in this array.
+#
+# Note that the LLVM codegen backend is special and will always be built and distributed.
+#
+# Currently, the only standard options supported here are `"llvm"`, `"cranelift"` and `"gcc"`.
 #rust.codegen-backends = ["llvm"]
 
 # Indicates whether LLD will be compiled and made available in the sysroot for rustc to execute, and