| Age | Commit message (Collapse) | Author | Lines |
|
Do array-slice equality via array equality, rather than always via slices
~~Draft because it needs a rebase after #91766 eventually gets through bors.~~
This enables the optimizations from #85828 to be used for array-to-slice comparisons too, not just array-to-array.
For example, <https://play.rust-lang.org/?version=nightly&mode=release&edition=2021&gist=5f9ba69b3d5825a782f897c830d3a6aa>
```rust
pub fn demo(x: &[u8], y: [u8; 4]) -> bool {
*x == y
}
```
Currently writes the array to stack for no reason:
```nasm
sub rsp, 4
mov dword ptr [rsp], edx
cmp rsi, 4
jne .LBB0_1
mov eax, dword ptr [rdi]
cmp eax, dword ptr [rsp]
sete al
add rsp, 4
ret
.LBB0_1:
xor eax, eax
add rsp, 4
ret
```
Whereas with the change in this PR it just compares it directly:
```nasm
cmp rsi, 4
jne .LBB1_1
cmp dword ptr [rdi], edx
sete al
ret
.LBB1_1:
xor eax, eax
ret
```
|
|
Stabilize asm! and global_asm!
Tracking issue: #72016
It's been almost 2 years since the original [RFC](https://github.com/rust-lang/rfcs/pull/2850) was posted and we're finally ready to stabilize this feature!
The main changes in this PR are:
- Removing `asm!` and `global_asm!` from the prelude as per the decision in #87228.
- Stabilizing the `asm` and `global_asm` features.
- Removing the unstable book pages for `asm` and `global_asm`. The contents are moved to the [reference](https://github.com/rust-lang/reference/pull/1105) and [rust by example](https://github.com/rust-lang/rust-by-example/pull/1483).
- All links to these pages have been removed to satisfy the link checker. In a later PR these will be replaced with links to the reference or rust by example.
- Removing the automatic suggestion for using `llvm_asm!` instead of `asm!` if you're still using the old syntax, since it doesn't work anymore with `asm!` no longer being in the prelude. This only affects code that predates the old LLVM-style `asm!` being renamed to `llvm_asm!`.
- Updating `stdarch` and `compiler-builtins`.
- Updating all the tests.
r? `@joshtriplett`
|
|
This'll still go via slices eventually for large arrays, but this way slice comparisons to short arrays can use the same memcmp-avoidance tricks.
Added some tests for all the combinations to make sure I didn't accidentally infinitely-recurse something.
|
|
Update LLVM submodule
Update LLVM submodule with recent cherry-picks. In particular:
* https://github.com/rust-lang/llvm-project/pull/123
* https://github.com/rust-lang/llvm-project/pull/124
|
|
They are also removed from the prelude as per the decision in
https://github.com/rust-lang/rust/issues/87228.
stdarch and compiler-builtins are updated to work with the new, stable
asm! and global_asm! macros.
|
|
|
|
|
|
...because alignment is always nonzero.
This helps eliminate redundant runtime alignment checks, when a DST
is a field of a struct whose remaining fields have alignment 1.
|
|
std: Stabilize the `thread_local_const_init` feature
This commit is intended to follow the stabilization disposition of the
FCP that has now finished in #84223. This stabilizes the ability to flag
thread local initializers as `const` expressions which enables the macro
to generate more efficient code for accessing it, notably removing
runtime checks for initialization.
More information can also be found in #84223 as well as the tests where
the feature usage was removed in this PR.
Closes #84223
|
|
|
|
|
|
|
|
fix sparc64 ABI for aggregates with floating point members
Fixes #86163
|
|
|
|
This commit is intended to follow the stabilization disposition of the
FCP that has now finished in #84223. This stabilizes the ability to flag
thread local initializers as `const` expressions which enables the macro
to generate more efficient code for accessing it, notably removing
runtime checks for initialization.
More information can also be found in #84223 as well as the tests where
the feature usage was removed in this PR.
Closes #84223
|
|
LLVM has built-in heuristics for adding stack canaries to functions. These
heuristics can be selected with LLVM function attributes. This patch adds a
rustc option `-Z stack-protector={none,basic,strong,all}` which controls the use
of these attributes. This gives rustc the same stack smash protection support as
clang offers through options `-fno-stack-protector`, `-fstack-protector`,
`-fstack-protector-strong`, and `-fstack-protector-all`. The protection this can
offer is demonstrated in test/ui/abi/stack-protector.rs. This fills a gap in the
current list of rustc exploit
mitigations (https://doc.rust-lang.org/rustc/exploit-mitigations.html),
originally discussed in #15179.
Stack smash protection adds runtime overhead and is therefore still off by
default, but now users have the option to trade performance for security as they
see fit. An example use case is adding Rust code in an existing C/C++ code base
compiled with stack smash protection. Without the ability to add stack smash
protection to the Rust code, the code base artifacts could be exploitable in
ways not possible if the code base remained pure C/C++.
Stack smash protection support is present in LLVM for almost all the current
tier 1/tier 2 targets: see
test/assembly/stack-protector/stack-protector-target-support.rs. The one
exception is nvptx64-nvidia-cuda. This patch follows clang's example, and adds a
warning message printed if stack smash protection is used with this target (see
test/ui/stack-protector/warn-stack-protector-unsupported.rs). Support for tier 3
targets has not been checked.
Since the heuristics are applied at the LLVM level, the heuristics are expected
to add stack smash protection to a fraction of functions comparable to C/C++.
Some experiments demonstrating how Rust code is affected by the different
heuristics can be found in
test/assembly/stack-protector/stack-protector-heuristics-effect.rs. There is
potential for better heuristics using Rust-specific safety information. For
example it might be reasonable to skip stack smash protection in functions which
transitively only use safe Rust code, or which uses only a subset of functions
the user declares safe (such as anything under `std.*`). Such alternative
heuristics could be added at a later point.
LLVM also offers a "safestack" sanitizer as an alternative way to guard against
stack smashing (see #26612). This could possibly also be included as a
stack-protection heuristic. An alternative is to add it as a sanitizer (#39699).
This is what clang does: safestack is exposed with option
`-fsanitize=safe-stack`.
The options are only supported by the LLVM backend, but as with other codegen
options it is visible in the main codegen option help menu. The heuristic names
"basic", "strong", and "all" are hopefully sufficiently generic to be usable in
other backends as well.
Reviewed-by: Nikita Popov <nikic@php.net>
Extra commits during review:
- [address-review] make the stack-protector option unstable
- [address-review] reduce detail level of stack-protector option help text
- [address-review] correct grammar in comment
- [address-review] use compiler flag to avoid merging functions in test
- [address-review] specify min LLVM version in fortanix stack-protector test
Only for Fortanix test, since this target specifically requests the
`--x86-experimental-lvi-inline-asm-hardening` flag.
- [address-review] specify required LLVM components in stack-protector tests
- move stack protector option enum closer to other similar option enums
- rustc_interface/tests: sort debug option list in tracking hash test
- add an explicit `none` stack-protector option
Revert "set LLVM requirements for all stack protector support test revisions"
This reverts commit a49b74f92a4e7d701d6f6cf63d207a8aff2e0f68.
|
|
|
|
can vectorize
For small types with padding, the current implementation is UB because it does integer operations on uninit values. But LLVM has gotten smarter since I wrote the previous implementation in 2017, so remove all the manual magic and just write it in such a way that LLVM will vectorize. This code is much simpler (albeit nuanced) and has very little `unsafe`, and is actually faster to boot!
|
|
Because after PR 86041, the optimizer no longer load-merges at the LLVM IR level, which might be part of the perf loss. (I'll run perf and see if this makes a difference.)
Also I added a codegen test so this hopefully won't regress in future -- it passes on stable and with my change here, but not on the 2021-11-09 nightly.
|
|
|
|
Add LLVM CFI support to the Rust compiler
This PR adds LLVM Control Flow Integrity (CFI) support to the Rust compiler. It initially provides forward-edge control flow protection for Rust-compiled code only by aggregating function pointers in groups identified by their number of arguments.
Forward-edge control flow protection for C or C++ and Rust -compiled code "mixed binaries" (i.e., for when C or C++ and Rust -compiled code share the same virtual address space) will be provided in later work as part of this project by defining and using compatible type identifiers (see Type metadata in the design document in the tracking issue #89653).
LLVM CFI can be enabled with -Zsanitizer=cfi and requires LTO (i.e., -Clto).
Thank you, `@eddyb` and `@pcc,` for all the help!
|
|
This commit adds LLVM Control Flow Integrity (CFI) support to the Rust
compiler. It initially provides forward-edge control flow protection for
Rust-compiled code only by aggregating function pointers in groups
identified by their number of arguments.
Forward-edge control flow protection for C or C++ and Rust -compiled
code "mixed binaries" (i.e., for when C or C++ and Rust -compiled code
share the same virtual address space) will be provided in later work as
part of this project by defining and using compatible type identifiers
(see Type metadata in the design document in the tracking issue #89653).
LLVM CFI can be enabled with -Zsanitizer=cfi and requires LTO (i.e.,
-Clto).
|
|
|
|
|
|
Co-authored-by: r00ster <r00ster91@protonmail.com>
|
|
|
|
Add `const_eval_select` intrinsic
Adds an intrinsic that calls a given function when evaluated at compiler time, but generates a call to another function when called at runtime.
See https://github.com/rust-lang/const-eval/issues/7 for previous discussion.
r? `@oli-obk.`
|
|
|
|
|
|
Create more accurate debuginfo for vtables.
Before this PR all vtables would have the same name (`"vtable"`) in debuginfo. Now they get an unambiguous name that identifies the implementing type and the trait that is being implemented.
This is only one of several possible improvements:
- This PR describes vtables as arrays of `*const u8` pointers. It would nice to describe them as structs where function pointer is represented by a field with a name indicative of the method it maps to. However, this requires coming up with a naming scheme that avoids clashes between methods with the same name (which is possible if the vtable contains multiple traits).
- The PR does not update the debuginfo we generate for the vtable-pointer field in a fat `dyn` pointer. Right now there does not seem to be an easy way of getting ahold of a vtable-layout without also knowing the concrete self-type of a trait object.
r? `@wesleywiser`
|
|
Before this commit all vtables would have the same name "vtable" in
debuginfo. Now they get a name that identifies the implementing type
and the trait that is being implemented.
|
|
|
|
Add `pie` as another `relocation-model` value
MCP: https://github.com/rust-lang/compiler-team/issues/461
|
|
|
|
|
|
Check whether a call/invoke of the function exists, but don't
match a leftover function declaration.
Also remove the CHECK-LABELs: In panic-in-drop=unwind mode the
call will not actually be in either of those functions, so
remove the restriction and look for any calls.
|
|
This fixes compiling things like the `snap` crate after
https://reviews.llvm.org/D105462. I added a test that verifies the
additional attribute gets specified, and confirmed that I can build
cargo with both LLVM 13 and 14 with this change applied.
|
|
|
|
add support for clobbering xer, cr, and cr[0-7] for asm! on OpenPower/PowerPC
Fixes #88315
|
|
Fixes #88315
|
|
|
|
|
|
Add needs-asm-support to more tests
These were found as test failures on s390x for RHEL and Fedora.
|
|
These were found as test failures on s390x for RHEL and Fedora.
|
|
This has regressed due to https://bugs.llvm.org/show_bug.cgi?id=51211.
It's pretty likely that we'll have to eat this regression for this
release.
|
|
|
|
Fix `reachable_set` for non-function items in non-library crates
I unintentionally changed `reachable_set` to ignore non-function items when `!self.any_library` in https://github.com/rust-lang/rust/pull/86492, which can lead to "undefined reference" errors in non-library (`cdylib`/`staticlib`/`bin`) crates, for example: https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=6bb2c5065a9be7e40943d0541e161b5a
This PR restores the behavior of `reachable_set` for non-function items.
Fixes https://github.com/rust-lang/rust/issues/88016.
<details>
<summary>The modified test will fail with this output without the `reachable_set` change</summary>
```
---- [codegen] codegen/external-no-mangle-statics.rs#staticlib stdout ----
error in revision `staticlib`: verification with 'FileCheck' failed
status: exit status: 1
command: "/checkout/build/x86_64-unknown-linux-gnu/ci-llvm/bin/FileCheck" "--input-file" "/checkout/build/x86_64-unknown-linux-gnu/test/codegen/external-no-mangle-statics.staticlib/external-no-mangle-statics.ll" "/checkout/src/test/codegen/external-no-mangle-statics.rs" "--check-prefixes" "CHECK,NONMSVC,staticlib"
stdout:
------------------------------------------
------------------------------------------
stderr:
------------------------------------------
/checkout/src/test/codegen/external-no-mangle-statics.rs:10:11: error: CHECK: expected string not found in input
// CHECK: `@A` = local_unnamed_addr constant
^
/checkout/build/x86_64-unknown-linux-gnu/test/codegen/external-no-mangle-statics.staticlib/external-no-mangle-statics.ll:1:1: note: scanning from here
; ModuleID = 'external_no_mangle_statics.b50529d3-cgu.0'
^
/checkout/build/x86_64-unknown-linux-gnu/test/codegen/external-no-mangle-statics.staticlib/external-no-mangle-statics.ll:1:6: note: possible intended match here
; ModuleID = 'external_no_mangle_statics.b50529d3-cgu.0'
^
Input file: /checkout/build/x86_64-unknown-linux-gnu/test/codegen/external-no-mangle-statics.staticlib/external-no-mangle-statics.ll
Check file: /checkout/src/test/codegen/external-no-mangle-statics.rs
-dump-input=help explains the following input dump.
Input was:
<<<<<<
1: ; ModuleID = 'external_no_mangle_statics.b50529d3-cgu.0'
check:10'0 X~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found
check:10'1 ? possible intended match
2: source_filename = "external_no_mangle_statics.b50529d3-cgu.0"
check:10'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3: target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
check:10'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
4: target triple = "x86_64-unknown-linux-gnu"
check:10'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
5:
check:10'0 ~
6: !llvm.module.flags = !{!0, !1}
check:10'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.
.
.
>>>>>>
------------------------------------------
failures:
[codegen] codegen/external-no-mangle-statics.rs#staticlib
```
</details>
|
|
Add support for clobber_abi to asm!
This PR adds the `clobber_abi` feature that was proposed in #81092.
Fixes #81092
cc `@rust-lang/wg-inline-asm`
r? `@nagisa`
|
|
|
|
Lint against named asm labels
This adds a deny-by-default lint to prevent the use of named labels in inline `asm!`. Without a solution to #81088 about whether the compiler should rewrite named labels or a special syntax for labels, a lint against them should prevent users from writing assembly that could break for internal compiler reasons, such as inlining or anything else that could change the number of actual inline assembly blocks emitted.
This does **not** resolve the issue with rewriting labels, that still needs a decision if the compiler should do any more work to try to make them work.
|