rust/src/bootstrap/builder, branch 1.66.1

Add a new component, `rust-json-docs`, to distribute the JSON-formatted documentation for std crates in nightly toolchains.

2022-09-17T19:58:34+00:00

We also add a new flag to `x doc`, `--json`, to render the JSON-formatted version alongside the HTML-formatted one.

Move `x test --skip` to be part of `--exclude`

2022-07-31T23:59:30+00:00

`--skip` is inconsistent with the rest of the interface and redundant with `--exclude`. Fix --exclude to work properly for files and directories rather than having a separate flag. If someone needs to use --skip for something other than compiletest, they can use `--test-args --skip` instead.

Allow building single crates for the compiler and standard library

2022-07-03T00:29:39+00:00

- Add `Interned>` and use it for tail args - Refactor `cache.rs` not to need a separate impl for each internable type

Add tests for fixed bugs

2022-06-18T14:58:29+00:00

Pass all paths to `Step::run` at once when using `ShouldRun::krate`

2022-06-18T14:54:35+00:00

This was surprisingly complicated. The main changes are: 1. Invert the order of iteration in `StepDescription::run`. Previously, it did something like: ```python for path in paths: for (step, should_run) in should_runs: if let Some(set) = should_run.pathset_for_path(path): step.run(builder, set) ``` That worked ok for individual paths, but didn't allow passing more than one path at a time to `Step::run` (since `pathset_for_paths` only had one path available to it). Change it to instead look at the intersection of `paths` and `should_run.paths`: ```python for (step, should_run) in should_runs: if let Some(set) = should_run.pathset_for_paths(paths): step.run(builder, set) ``` 2. Change `pathset_for_path` to take multiple pathsets. The goal is to avoid `x test library/alloc` testing *all* library crates, instead of just alloc. The changes here are similarly subtle, to use the intersection between the paths rather than all paths in `should_run.paths`. I added a test for the behavior to try and make it more clear. Note that we use pathsets instead of just paths to allow for sets with multiple aliases (*cough* `all_krates` *cough*). See the documentation added in the next commit for more detail. 3. Change `StepDescription::run` to explicitly handle 0 paths. Before this was implicitly handled by the `for` loop, which just didn't excute when there were no paths. Now it needs a check, to avoid trying to run all steps (this is a problem for steps that use `default_condition`). 4. Change `RunDescription` to have a list of pathsets, rather than a single path. 5. Remove paths as they're matched This allows checking at the end that no invalid paths are left over. Note that if two steps matched the same path, this will no longer run both; but that's a bug anyway. 6. Handle suite paths separately from regular sets. Running multiple suite paths at once instead of in separate `make_run` invocations is both tricky and not particularly useful. The respective test Steps already handle this by introspecting the original paths. Avoid having to deal with it by moving suite handling into a seperate loop than `PathSet::Set` checks.

Auto merge of #96493 - chbaker0:issue-96342-fix, r=Mark-Simulacrum

2022-05-13T01:25:28+00:00

Add compiletest and bootstrap "--skip" option forwarded to libtest With this PR, "x.py test --skip SKIP ..." will run the specified test suite, but forward "--skip SKIP" to the test tool. libtest already supports this option. The PR also adds it to compiletest which itself just forwards it to libtest. Adds the functionality requested in https://github.com/rust-lang/rust/issues/96342. This is useful to work around tests broken upstream. https://github.com/rust-lang/rust/issues/96362#issuecomment-1108609893 is the specific test issue my project is trying to work around.

Fix running bootstrap tests on a fresh clone

2022-05-10T23:13:54+00:00

In #96303, I changed the tests not to manage submodules, with the main goal of avoiding a clone for llvm-project. Unfortunately, there are some tests which depend on submodules - I didn't notice locally because they were already checked out for me, and CI doesn't use submodule handling at all. Fresh clones, however, were impacted: ``` failures: ---- builder::tests::defaults::doc_default stdout ---- thread 'main' panicked at 'fs::read_dir(builder.src.join(&relative_path).join("redirects")) failed with No such file or directory (os error 2)', src/bootstrap/doc.rs:232:21 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace ---- builder::tests::dist::dist_only_cross_host stdout ---- thread 'main' panicked at 'fs::read_to_string(&toml_file_name) failed with No such file or directory (os error 2)', src/bootstrap/lib.rs:1314:20 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace ``` Try and get the best of both worlds by only checking out the submodules actually used in tests.

Add test skip support

2022-05-10T14:29:48+00:00

libtest already supports a "--skip SUBSTRING" arg which excludes any test names matching SUBSTRING. This adds a "--skip" argument to compiletest and bootstrap which is forwarded to libtest.

Use `run_build` helper consistently across most bootstrap tests

2022-04-22T03:18:05+00:00

This is not super important to do, but the consistency is nice. I didn't change any tests that call `configure("dist")` and then override the subcommand - doing that at all is pretty sketchy, but I don't want to mess with it while already doing a refactor.

Add a test for `--exclude test::XXX`

2022-04-22T02:30:54+00:00

I didn't know that the `test::` syntax was valid before, and it doesn't seem to be documented anywhere. Add a test so it doesn't regress accidentally, and as executable documentation.