Auto merge of #31123 - alexcrichton:who-doesnt-want-two-build-systems, r=brson

This series of commits adds the initial implementation of a new build system for
the compiler and standard library based on Cargo. The high-level architecture
now looks like:

1. The `./configure` script is run with `--enable-rustbuild` and other standard
   configuration options.
2. A `Makefile` is generate which proxies commands to the new build system.
3. The new build system has a Python script entry point which manages
   downloading both a Rust and Cargo nightly. This initial script also manages
   building the build system itself (which is written in Rust).
4. The build system, written in rust and called `bootstrap`, architects how to
   call `cargo` and manages building all native libraries and such.

One might reasonably ask "why rewrite the build system?", which is a good
question! The Rust project has used Makefiles for as long as I can remember at
least, and while ugly and difficult to use are undeniably robust as they contain
years worth of tweaking and tuning for working on as many platforms in as many
situation as possible. The rationale behind this PR, however is:

* The makefiles are impenetrable to all but a few people on this
  planet. This means that contributions to the build system are almost
  nonexistent, and furthermore if a build system change is needed it's
  incredibly difficult to figure out how to do so. This hindrance prevents us
  from doing some "perhaps fancier" things we may wish to do in make.

* Our build system, while portable, is unfortunately not infinitely portable
  everywhere.  For example the recently-introduced MSVC target is quite unlikely
  to have `make` installed by default (e.g. it requires building inside of an
  MSYS2 shell currently). Conversely, the portability of make comes at a cost of
  crazy and weird hacks to work around all sorts of versions of software
  everywhere, especially when it comes to the configure script and makefiles.
  By rewriting this logic in one of the most robust platforms there is, Rust,
  we get to assuage all of these worries for free!

* There's a standard tool to build Rust crates, Cargo, but the standard library
  and compiler don't use it. This means that they cannot benefit easily from the
  crates.io ecosystem, nor can the ecosystem benefit from a standard way to
  build this repository itself. Moving to Cargo should help assuage both of
  these needs. This has the added benefit of making the compiler more
  approachable for newbies as working on the compiler will just happen to be
  working on a large Cargo project, all the same standard tools and tricks will
  apply.

* There's a huge amount of portability information in the main distribution, for
  example around cross compiling, compiling on new OSes, etc. Pushing this logic
  into standard crates (like `gcc`) enables the community to immediately benefit
  from new build logic.

Despite these benefits, it's going to be a long road to actually replace our
current build system. This PR is just the beginning and doesn't implement the
full suite of functionality as the current one, but there are many more to
follow! The current implementation strategy hopes to look like:

1. Land a second build system in-tree that can be itereated on an and
   contributed to. This will not be used just yet in terms of gating new commits
   to the repo.
2. Over time, bring the second build system to feature parity with the old build
   system, start setting up CI for both build systems.
3. At some point in the future, switch the default to the new build system, but
   keep the old one around.
4. At some further point in the future, delete the entire old build system.

---

Alright, so with all that out of the way, here's some more info on this PR
itself. The inital build system here is contained in the `src/bootstrap`
directory and just adds the necessary minimum bits to bootstrap the compiler
itself. There is currently no support for building documentation, running tests,
or installing, but the implemented support is:

* Compiling LLVM with `cmake` instead of `./configure` + `make`. The LLVM
  project is removing their autotools build system, so we'd have to make this
  transition eventually anyway.

* Compiling compiler-rt with `cmake` as well (for the same rationale as above).

* Adding `Cargo.toml` to map out the dependency graph to all crates, and also
  adding `build.rs` files where appropriate. For example `alloc_jemalloc` has a
  script to build jemalloc, `flate` has a script to build `miniz.c`, `std` will
  build `libbacktrace`, etc.

* Orchestrating all the calls to `cargo` to build the standard distribution,
  following the normal bootstrapping process. This also tracks dependencies
  between steps to ensure cross-compilation targets happen as well.

* Configuration is intended to eventually be done through a `config.toml` file,
  so support is implemented for this. The most likely vector of configuration
  for now, however, is likely through `config.mk` (what `./configure` emits), so
  the build system currently parses this information.

There's still quite a few steps left to do, and I'll open up some follow-up
issues (as well as a tracking issue) for this migration, but hopefully this is a
great start to get going! This PR is currently tested on all the
Windows/Linux/OSX triples for x86\_64 and x86, but more portability is always
welcome!

---

Future functionality left to implement

* [ ] Re-verify that multi-host builds work
* [ ] Verify android build works
* [ ] Verify iOS build work (mostly compiler-rt)
* [ ] Verify sha256 and ideally gpg of downloaded nightly compiler and nightly rustc
* [ ] Implement testing -- this is a huge bullet point with lots of sub-bullets
* [ ] Build and generate documentation (plus the various tools we have in-tree)
* [ ] Move various src/etc scripts into Rust -- not sure how this interacts with `make` build system
* [ ] Implement `make install` - like testing this is also quite massive
* [x] Deduplicate version information with makefiles

0 files changed, 0 insertions, 0 deletions