| Age | Commit message (Collapse) | Author | Lines |
|---|
|
add two new build flags to build clang and enable llvm plugins
Based on the discussion here: https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/Add.20configure.20flag.20to.20build.20clang/near/246439138
It allows building clang (which already is part of the llvm-project) based on the same llvm version which we use to build rustc.
It also allows enabling llvm's plugin interface, which is required for https://enzyme.mit.edu/.
There is no further integration beside of this basic build support.
|
|
rfc3052 followup: Remove authors field from Cargo manifests
Since RFC 3052 soft deprecated the authors field, hiding it from
crates.io, docs.rs, and making Cargo not add it by default, and it is
not generally up to date/useful information for contributors, we may as well
remove it from crates in this repo.
|
|
Ensure `./x.py dist` adheres to `build.tools`
According to `config.toml.example`, the way to produce dist artifacts for both the compiler and a *subset* of tools would be to enable the extended build and manually specify the list of tools to build:
```toml
[build]
extended = true
tools = ["cargo", "rustfmt"]
```
This works as expected for `./x.py build` and `./x.py install`, but *not* for `./x.py dist`. Before this PR `./x.py dist` simply ignored the contents of `build.tools`, building just rustc/rustdoc if `build.extended = false` and all of the tools otherwise. This PR does two things:
* Changes `./x.py dist extended` to only build the tools defined in `build.tools`, if `build.tools` is not empty. The rest of the extended step was refactored to simplify the code.
* Changes how dist jobs for tools are gated: instead of `assert!(builder.config.extended)` to prevent tools from being built with `build.extended = false`, tools are simply built by default depending on `build.extended` and `build.tools`. This also enables to **explicitly** dist tools even with `build.extended = false`.
This PR is best reviewed commit-by-commit.
Fixes #86436
|
|
|
|
|
|
Since RFC 3052 soft deprecated the authors field anyway, hiding it from
crates.io, docs.rs, and making Cargo not add it by default, and it is
not generally up to date/useful information, we should remove it from
crates in this repo.
|
|
The default is then overridden by `should_run`.
|
|
bootstrap.py: change `git log` option to indicate desired behavior
When determining which LLVM artifacts to download, bootstrap.py calls: `git log --author=bors --format=%H -n1 -m --first-parent --
src/llvm-project src/bootstrap/download-ci-llvm-stamp src/version`. However, the `-m` option has no effect, per the `git log` help:
> -m
> This option makes diff output for merge commits to be shown in the
> default format. -m will produce the output only if -p is given as
> well. The default format could be changed using log.diffMerges
> configuration parameter, which default value is separate.
Accordingly, this commit removes use of the -m option in favor of ~~`--diff-merges=off`~~ `--no-patch`, since no diff information is needed, and in fact the presence of a diff breaks the command. Tested using git 2.32, this does not change the output of the command.
The motivation for this change is that some patched versions of git change the behavior of the `-m` flag to imply `-p`, rather than to do nothing unless `-p` is passed. These patched versions of git lead to this script not working. Google's corp-provided git is one such example.
|
|
Don't treat git repos as non-existent when `ignore_git` is set
The new submodule handling depends on `is_git()` to be accurate to
decide whether it should handle submodules at all or not. Unfortunately,
`is_git()` treated "this directory does not have a git repository" and
"this repository should not be used for SHA/version/commit date info"
the same. This changes it to distinguish the two.
To clarify: ignore_get is set by default whenever channel == "dev", which it is by default whenever you're compiling locally. So basically everyone would hit this, not just people who had explicitly configured ignore_git.
Here's an example of an error this fixes:
```
$ x build
Updating only changed submodules
Submodules updated in 0.01 seconds
Finished dev [unoptimized + debuginfo] target(s) in 0.17s
warning: x.py has made several changes recently you may want to look at
help: consider looking at the changes in `src/bootstrap/CHANGELOG.md`
note: to silence this warning, add `changelog-seen = 2` at the top of `config.toml`
Building stage0 std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 0.16s
Copying stage0 std from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Building LLVM for x86_64-unknown-linux-gnu
detected home dir change, cleaning out entire build directory
running: "cmake" "/home/joshua/rustc3/src/llvm-project/llvm" "-G" "Ninja" "-DLLVM_ENABLE_ASSERTIONS=OFF" "-DLLVM_TARGETS_TO_BUILD=AArch64;ARM;BPF;Hexagon;MSP430;Mips;NVPTX;PowerPC;RISCV;Sparc;SystemZ;WebAssembly;X86" "-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=AVR" "-DLLVM_INCLUDE_EXAMPLES=OFF" "-DLLVM_INCLUDE_DOCS=OFF" "-DLLVM_INCLUDE_BENCHMARKS=OFF" "-DLLVM_ENABLE_TERMINFO=OFF" "-DLLVM_ENABLE_LIBEDIT=OFF" "-DLLVM_ENABLE_BINDINGS=OFF" "-DLLVM_ENABLE_Z3_SOLVER=OFF" "-DLLVM_PARALLEL_COMPILE_JOBS=48" "-DLLVM_TARGET_ARCH=x86_64" "-DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-linux-gnu" "-DLLVM_ENABLE_ZLIB=ON" "-DLLVM_ENABLE_LIBXML2=OFF" "-DLLVM_VERSION_SUFFIX=-rust-dev" "-DCMAKE_INSTALL_MESSAGE=LAZY" "-DCMAKE_C_COMPILER=gcc" "-DCMAKE_CXX_COMPILER=g++" "-DCMAKE_ASM_COMPILER=gcc" "-DCMAKE_C_FLAGS=-ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_CXX_FLAGS=-ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_INSTALL_PREFIX=/home/joshua/rustc3/build/x86_64-unknown-linux-gnu/llvm" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_BUILD_TYPE=Release"
CMake Error: The source directory "/home/joshua/rustc3/src/llvm-project/llvm" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
thread 'main' panicked at '
command did not execute successfully, got: exit status: 1
build script failed, must exit now', /home/joshua/.local/lib/cargo/registry/src/github.com-1ecc6299db9ec823/cmake-0.1.44/src/lib.rs:885:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
finished in 0.783 seconds
Build completed unsuccessfully in 0:00:01
```
I *believe* this regression was only introduced in https://github.com/rust-lang/rust/pull/87380, not https://github.com/rust-lang/rust/pull/82653. ``@petrochenkov`` can you check that this fixes the issue you encountered in https://github.com/rust-lang/rust/pull/82653#issuecomment-886113679 ?
r? ``@Mark-Simulacrum``
|
|
Only look for commits by bors that are merge commits, because
those are the only ones with CI artifacts. Also, use
`--first-parent` to avoid traversing stuff like rollup branches.
|
|
Use `git rev-list` instead of `git log` to be more robust against
UI changes in git. Also, use the full email address for bors,
because `--author` uses a substring match.
|
|
When determining which LLVM artifacts to download, bootstrap.py
calls: `git log --author=bors --format=%H -n1 -m --first-parent --
src/llvm-project src/bootstrap/download-ci-llvm-stamp src/version`.
However, the `-m` option has no effect, per the `git log` help:
> -m
> This option makes diff output for merge commits to be shown in the
> default format. -m will produce the output only if -p is given as
> well. The default format could be changed using log.diffMerges
> configuration parameter, which default value is separate.
Accordingly, this commit removes use of the -m option in favor of
`--no-patch`, to make clear that this command should never output
diff information, as the SHA-1 hash is the only desired output.
Tested using git 2.32, this does not change the
output of the command.
The motivation for this change is that some patched versions of git
change the behavior of the `-m` flag to imply `-p`, rather than to do
nothing unless `-p` is passed. These patched versions of git lead to
this script not working. Google's corp-provided git is one such example.
|
|
The new submodule handling depends on `is_git()` to be accurate to
decide whether it should handle submodules at all or not. Unfortunately,
`is_git()` treated "this directory does not have a git repository" and
"this repository should not be used for SHA/version/commit date info"
the same. This changes it to distinguish the two.
To clarify: ignore_get is set by default whenever channel == "dev", which it is by default whenever you're compiling locally. So basically everyone would hit this, not just people who had explicitly configured ignore_git.
Here's an example of an error this fixes:
```
$ x build
Updating only changed submodules
Submodules updated in 0.01 seconds
Finished dev [unoptimized + debuginfo] target(s) in 0.17s
warning: x.py has made several changes recently you may want to look at
help: consider looking at the changes in `src/bootstrap/CHANGELOG.md`
note: to silence this warning, add `changelog-seen = 2` at the top of `config.toml`
Building stage0 std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 0.16s
Copying stage0 std from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Building LLVM for x86_64-unknown-linux-gnu
detected home dir change, cleaning out entire build directory
running: "cmake" "/home/joshua/rustc3/src/llvm-project/llvm" "-G" "Ninja" "-DLLVM_ENABLE_ASSERTIONS=OFF" "-DLLVM_TARGETS_TO_BUILD=AArch64;ARM;BPF;Hexagon;MSP430;Mips;NVPTX;PowerPC;RISCV;Sparc;SystemZ;WebAssembly;X86" "-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=AVR" "-DLLVM_INCLUDE_EXAMPLES=OFF" "-DLLVM_INCLUDE_DOCS=OFF" "-DLLVM_INCLUDE_BENCHMARKS=OFF" "-DLLVM_ENABLE_TERMINFO=OFF" "-DLLVM_ENABLE_LIBEDIT=OFF" "-DLLVM_ENABLE_BINDINGS=OFF" "-DLLVM_ENABLE_Z3_SOLVER=OFF" "-DLLVM_PARALLEL_COMPILE_JOBS=48" "-DLLVM_TARGET_ARCH=x86_64" "-DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-linux-gnu" "-DLLVM_ENABLE_ZLIB=ON" "-DLLVM_ENABLE_LIBXML2=OFF" "-DLLVM_VERSION_SUFFIX=-rust-dev" "-DCMAKE_INSTALL_MESSAGE=LAZY" "-DCMAKE_C_COMPILER=gcc" "-DCMAKE_CXX_COMPILER=g++" "-DCMAKE_ASM_COMPILER=gcc" "-DCMAKE_C_FLAGS=-ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_CXX_FLAGS=-ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_INSTALL_PREFIX=/home/joshua/rustc3/build/x86_64-unknown-linux-gnu/llvm" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_BUILD_TYPE=Release"
CMake Error: The source directory "/home/joshua/rustc3/src/llvm-project/llvm" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
thread 'main' panicked at '
command did not execute successfully, got: exit status: 1
build script failed, must exit now', /home/joshua/.local/lib/cargo/registry/src/github.com-1ecc6299db9ec823/cmake-0.1.44/src/lib.rs:885:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
finished in 0.783 seconds
Build completed unsuccessfully in 0:00:01
```
|
|
Add support for powerpc-unknown-freebsd
- A tier 3 target must have a designated developer or developers (the "target maintainers") on record to be CCed when issues arise regarding the target. (The mechanism to track and CC such developers may evolve over time.)
For all Rust targets on FreeBSD, it's rust@FreeBSD.org.
- Targets must use naming consistent with any existing targets; for instance, a target for the same CPU or OS as an existing Rust target should use the same name for that CPU or OS. Targets should normally use the same names and naming conventions as used elsewhere in the broader ecosystem beyond Rust (such as in other toolchains), unless they have a very good reason to diverge. Changing the name of a target can be highly disruptive, especially once the target reaches a higher tier, so getting the name right is important even for a tier 3 target.
Done.
- Target names should not introduce undue confusion or ambiguity unless absolutely necessary to maintain ecosystem compatibility. For example, if the name of the target makes people extremely likely to form incorrect beliefs about what it targets, the name should be changed or augmented to disambiguate it.
Done
- Tier 3 targets may have unusual requirements to build or use, but must not create legal issues or impose onerous legal terms for the Rust project or for Rust developers or users.
Done.
- The target must not introduce license incompatibilities.
Done.
- Anything added to the Rust repository must be under the standard Rust license (MIT OR Apache-2.0).
Fine with me.
- The target must not cause the Rust tools or libraries built for any other host (even when supporting cross-compilation to the target) to depend on any new dependency less permissive than the Rust licensing policy. This applies whether the dependency is a Rust crate that would require adding new license exceptions (as specified by the tidy tool in the rust-lang/rust repository), or whether the dependency is a native library or binary. In other words, the introduction of the target must not cause a user installing or running a version of Rust or the Rust tools to be subject to any new license requirements.
Done.
- If the target supports building host tools (such as rustc or cargo), those host tools must not depend on proprietary (non-FOSS) libraries, other than ordinary runtime libraries supplied by the platform and commonly used by other binaries built for the target. For instance, rustc built for the target may depend on a common proprietary C runtime library or console output library, but must not depend on a proprietary code generation library or code optimization library. Rust's license permits such combinations, but the Rust project has no interest in maintaining such combinations within the scope of Rust itself, even at tier 3.
Done.
- Targets should not require proprietary (non-FOSS) components to link a functional binary or library.
Done.
- "onerous" here is an intentionally subjective term. At a minimum, "onerous" legal/licensing terms include but are not limited to: non-disclosure requirements, non-compete requirements, contributor license agreements (CLAs) or equivalent, "non-commercial"/"research-only"/etc terms, requirements conditional on the employer or employment of any particular Rust developers, revocable terms, any requirements that create liability for the Rust project or its developers or users, or any requirements that adversely affect the livelihood or prospects of the Rust project or its developers or users.
Fine with me.
- Neither this policy nor any decisions made regarding targets shall create any binding agreement or estoppel by any party. If any member of an approving Rust team serves as one of the maintainers of a target, or has any legal or employment requirement (explicit or implicit) that might affect their decisions regarding a target, they must recuse themselves from any approval decisions regarding the target's tier status, though they may otherwise participate in discussions.
Ok.
- This requirement does not prevent part or all of this policy from being cited in an explicit contract or work agreement (e.g. to implement or maintain support for a target). This requirement exists to ensure that a developer or team responsible for reviewing and approving a target does not face any legal threats or obligations that would prevent them from freely exercising their judgment in such approval, even if such judgment involves subjective matters or goes beyond the letter of these requirements.
Ok.
- Tier 3 targets should attempt to implement as much of the standard libraries as possible and appropriate (core for most targets, alloc for targets that can support dynamic memory allocation, std for targets with an operating system or equivalent layer of system-provided functionality), but may leave some code unimplemented (either unavailable or stubbed out as appropriate), whether because the target makes it impossible to implement or challenging to implement. The authors of pull requests are not obligated to avoid calling any portions of the standard library on the basis of a tier 3 target not implementing those portions.
std is implemented.
- The target must provide documentation for the Rust community explaining how to build for the target, using cross-compilation if possible. If the target supports running tests (even if they do not pass), the documentation must explain how to run tests for the target, using emulation if possible or dedicated hardware if necessary.
Hm, building is possible the same way as other Rust on FreeBSD targets.
- Tier 3 targets must not impose burden on the authors of pull requests, or other developers in the community, to maintain the target. In particular, do not post comments (automated or manual) on a PR that derail or suggest a block on the PR based on a tier 3 target. Do not send automated messages or notifications (via any medium, including via `@)` to a PR author or others involved with a PR regarding a tier 3 target, unless they have opted into such messages.
Ok.
- Backlinks such as those generated by the issue/PR tracker when linking to an issue or PR are not considered a violation of this policy, within reason. However, such messages (even on a separate repository) must not generate notifications to anyone involved with a PR who has not requested such notifications.
Ok.
- Patches adding or updating tier 3 targets must not break any existing tier 2 or tier 1 target, and must not knowingly break another tier 3 target without approval of either the compiler team or the maintainers of the other tier 3 target.
Ok.
- In particular, this may come up when working on closely related targets, such as variations of the same architecture with different features. Avoid introducing unconditional uses of features that another variation of the target may not have; use conditional compilation or runtime detection, as appropriate, to let each target run code supported by that target.
Ok.
|
|
Add missing article
Redo of #87305. I messed up in that PR and wasn't sure how to fix it.
|
|
Don't default to `submodules = true` unless the rust repo has a .git directory
Should hopefully fix https://github.com/rust-lang/rust/pull/82653#issuecomment-885093033 - `@semarie` can you confirm?
r? `@Mark-Simulacrum`
|
|
Fix `--dry-run` when download-ci-llvm is set
Previously it would error out:
```
$ x check --dry-run
thread 'main' panicked at 'std::fs::read_to_string(ci_llvm.join("link-type.txt")) failed with No such file or directory (os error 2) ("CI llvm missing: /home/joshua/rustc3/build/tmp-dry-run/x86_64-unknown-linux-gnu/ci-llvm")', src/bootstrap/config.rs:795:33
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Build completed unsuccessfully in 0:00:10
```
|
|
r=Mark-Simulacrum
Add `--codegen-backends=foo,bar` configure flag
Unfortunately this requires a proper `./configure` flag, as the codegen backends config entry is a list, not a string (breaking `--set`).
|
|
Package LLVM libs for the target rather than the build host
Fixes https://github.com/rust-lang/rust/issues/85250
`dist.rs` uses, in the `rust-dev` stage, `llvm-config --libfiles` to get a list of the LLVM library files built but of course only for the build host. If the target differs we want to package lib files from the target's build tree instead. This is done by splitting/rejoining the paths on their build directories.
At the moment `tree` on the LLVM build directories seems to give almost identical output, but of course this might not be the case in the future. If a file is missing in the target's build tree then this stage will error in the `builder.install()` call. If the target build tree has an extra file then it silently won't be copied and we'll get a linker error when building using this artifact (via `download-ci-llvm = "if-available"`), though we would have received a linker error anyway without this change.
There was also a typo in the example config around this option.
|
|
Fix panics on Windows when the build was cancelled
Fixes #86499
cc `@jyn514`
|
|
|
|
|
|
|
|
Make `x.py d` an alias for `x.py doc`
In rust-lang/cargo#9680, `d` was added to Cargo as an alias for `doc`. This PR adds the same alias to `x.py`. The same considerations of convenience that applied to Cargo also apply to `x.py`, and in any case, the two should be kept symmetrical.
|
|
|
|
|
|
|
|
Previously it would error out:
```
$ x check --dry-run
thread 'main' panicked at 'std::fs::read_to_string(ci_llvm.join("link-type.txt")) failed with No such file or directory (os error 2) ("CI llvm missing: /home/joshua/rustc3/build/tmp-dry-run/x86_64-unknown-linux-gnu/ci-llvm")', src/bootstrap/config.rs:795:33
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Build completed unsuccessfully in 0:00:10
```
|
|
config not being applied
|
|
Fix typo in compile.rs
|
|
Fix NixOS detection
Use `/etc/os-release` instead of `/etc/NIXOS` for detection.
The latter one does not exist on NixOS when using tmpfs as root.
|
|
Update all submodules that rustbuild doesn't depend on lazily
This only updates the submodules the first time they're needed, instead
of unconditionally the first time you run x.py.
Ideally, this would move *all* submodules to rustbuild and not exclude some tools and
backtrace. Unfortunately, cargo requires all `Cargo.toml` files in the
whole workspace to be present to build any crate.
On my machine, this takes the time for an initial submodule clone (for
`x.py --help`) from 55.70 to 15.87 seconds.
Helps with https://github.com/rust-lang/rust/issues/76653. Builds on https://github.com/rust-lang/rust/pull/86015 and should not be merged before (only the last commit is relevant).
|
|
This only updates the submodules the first time they're needed, instead
of unconditionally the first time you run x.py.
Ideally, this would move *all* submodules and not exclude some tools and
backtrace. Unfortunately, cargo requires all `Cargo.toml` files in the
whole workspace to be present to build any crate.
On my machine, this takes the time for an initial submodule clone (for
`x.py --help`) from 55.70 to 15.87 seconds.
This uses exactly the same logic as the LLVM update used, modulo some
minor cleanups:
- Use a local variable for `src.join(relative_path)`
- Remove unnecessary arrays for `book!` macro and make the macro simpler to use
- Add more comments
|
|
Use `/etc/os-release` instead of `/etc/NIXOS`.
The latter one does not exist on NixOS when using tmpfs as root.
|
|
|
|
Unfortunately this requires a proper ./configure flag, as the codegen
backends config entry is a list, not a string (breaking --set).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Fix implementors display
Part of https://github.com/rust-lang/rust/issues/86632.
This PR does a few things:
* It fixes of the JS rendered implementors.
* It generates anchors for JS rendered implementors to make it coherent with the others.
* It adds a test to ensure that we won't have the same issue again.
* It changes the way we render the rustdoc-gui crates to simplify it a bit and also to allow to have dependencies without going through compiletest.
Before:
After:
I plan to add the `[src]` links in another PR because this one is already big enough.
cc `@Mark-Simulacrum` (for the bootstrap changes)
r? `@Nemo157`
|
|
Fix rust-analyzer install when not available.
This changes it so that `x.py install` won't fail if rust-analyzer isn't available. This was changed in #86568 to handle the case where installing on stable/beta, and `extended=true`, to skip rust-analyzer. But I neglected to update the install part to also ignore it.
Fixes #86999
|
|
|
|
|
|
|
