| Age | Commit message (Collapse) | Author | Lines |
|---|
|
Fix `array::IntoIter::fold` to use the optimized `Range::fold`
It was using `Iterator::by_ref` in the implementation, which ended up pessimizing it enough that, for example, it didn't vectorize when we tried it in the <https://rust-lang.zulipchat.com/#narrow/stream/257879-project-portable-simd/topic/Reducing.20sum.20into.20wider.20types> conversation.
Demonstration that the codegen test doesn't pass on the current nightly: <https://rust.godbolt.org/z/Taxev5eMn>
|
|
Like we have `add`/`sub` which are the `usize` version of `offset`, this adds the `usize` equivalent of `offset_from`. Like how `.add(d)` replaced a whole bunch of `.offset(d as isize)`, you can see from the changes here that it's fairly common that code actually knows the order between the pointers and *wants* a `usize`, not an `isize`.
As a bonus, this can do `sub nuw`+`udiv exact`, rather than `sub`+`sdiv exact`, which can be optimized slightly better because it doesn't have to worry about negatives. That's why the slice iterators weren't using `offset_from`, though I haven't updated that code in this PR because slices are so perf-critical that I'll do it as its own change.
This is an intrinsic, like `offset_from`, so that it can eventually be allowed in CTFE. It also allows checking the extra safety condition -- see the test confirming that CTFE catches it if you pass the pointers in the wrong order.
|
|
|
|
|
|
|
|
Drop support for legacy PM with LLVM 15
LLVM 15 already removes some of the legacy PM APIs we're using. This patch forces use of NewPM with LLVM 15 (with `-Z new-llvm-pass-manager=no` throwing a warning) and stubs out various FFI methods with a report_fatal_error on LLVM 15.
For LLVMPassManagerBuilderPopulateLTOPassManager() I went with adding our own wrapper, as the alternative would be to muck about with weak symbols, which seems to be non-trivial as far as cross-platform support is concerned (std has `weak!` for this purpose, but only as an internal utility.)
Fixes #96072.
Fixes #96362.
|
|
Check if call return type is visibly uninhabited when building MIR
The main motivation behind the change is to expose information about diverging
calls to the generator transform and match the precision of drop range tracking
which already understands that call expressions with visibly uninhabited types
diverges.
This change should also accept strictly more programs than before. That is
programs that were previously rejected due to errors raised by control-flow
sensitive checks in a code that is no longer considered reachable.
Fixes #93161.
|
|
The linked issue has an accurate description of the situation,
drop the inaccurate comment.
|
|
asm: Add a kreg0 register class on x86 which includes k0
Previously we only exposed a kreg register class which excludes the k0
register since it can't be used in many instructions. However k0 is a
valid register and we need to have a way of marking it as clobbered for
clobber_abi.
Fixes #94977
|
|
Previously we only exposed a kreg register class which excludes the k0
register since it can't be used in many instructions. However k0 is a
valid register and we need to have a way of marking it as clobbered for
clobber_abi.
Fixes #94977
|
|
|
|
Add codegen tests for additional cases where noop iterators get optimized away
Optimizations have improved over time and now LLVM manages to optimize more in-place-collect noop-iterators to O(1) functions. This updates the codegen test to match.
Many but not all cases reported in #79308 work now.
|
|
Speed up Vec::clear().
Currently it just calls `truncate(0)`. `truncate()` is (a) not marked as
`#[inline]`, and (b) more general than needed for `clear()`.
This commit changes `clear()` to do the work itself. This modest change
was first proposed in rust-lang#74172, where the reviewer rejected it because
there was insufficient evidence that `Vec::clear()`'s performance
mattered enough to justify the change. Recent changes within rustc have
made `Vec::clear()` hot within `macro_parser.rs`, so the change is now
clearly worthwhile.
Although it doesn't show wins on CI perf runs, this seems to be because they
use PGO. But not all platforms currently use PGO. Also, local builds don't use
PGO, and `truncate` sometimes shows up in an over-represented fashion in local
profiles. So local profiling will be made easier by this change.
Note that this will also benefit `String::clear()`, because it just
calls `Vec::clear()`.
Finally, the commit removes the `vec-clear.rs` codegen test. It was
added in #52908. From before then until now, `Vec::clear()` just called
`Vec::truncate()` with a zero length. The body of Vec::truncate() has
changed a lot since then. Now that `Vec::clear()` is doing actual work
itself, and not just calling `Vec::truncate()`, it's not surprising that
its generated code includes a load and an icmp. I think it's reasonable
to remove this test.
r? `@m-ou-se`
|
|
|
|
Optimization have improved over time and now LLVM manages to optimize more
in-place-collect noop-iterators to O(1) functions. This updates the codegen test to match.
|
|
Currently it just calls `truncate(0)`. `truncate()` is (a) not marked as
`#[inline]`, and (b) more general than needed for `clear()`.
This commit changes `clear()` to do the work itself. This modest change
was first proposed in rust-lang#74172, where the reviewer rejected it because
there was insufficient evidence that `Vec::clear()`'s performance
mattered enough to justify the change. Recent changes within rustc have
made `Vec::clear()` hot within `macro_parser.rs`, so the change is now
clearly worthwhile.
Although it doesn't show wins on CI perf runs, this seems to be because they
use PGO. But not all platforms currently use PGO. Also, local builds don't use
PGO, and `truncate` sometimes shows up in an over-represented fashion in local
profiles. So local profiling will be made easier by this change.
Note that this will also benefit `String::clear()`, because it just
calls `Vec::clear()`.
Finally, the commit removes the `vec-clear.rs` codegen test. It was
added in #52908. From before then until now, `Vec::clear()` just called
`Vec::truncate()` with a zero length. The body of Vec::truncate() has
changed a lot since then. Now that `Vec::clear()` is doing actual work
itself, and not just calling `Vec::truncate()`, it's not surprising that
its generated code includes a load and an icmp. I think it's reasonable
to remove this test.
|
|
Fix miscompilation of inline assembly with outputs in cases where we emit an invoke instead of call instruction.
We ran into this bug where rustc would segfault while trying to compile certain uses of inline assembly.
Here is a simple repro that demonstrates the issue:
```rust
#![feature(asm_unwind)]
fn main() {
let _x = String::from("string here just cause we need something with a non-trivial drop");
let foo: u64;
unsafe {
std::arch::asm!(
"mov {}, 1",
out(reg) foo,
options(may_unwind)
);
}
println!("{}", foo);
}
```
([playground link](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=7d6641e83370d2536a07234aca2498ff))
But crucially `feature(asm_unwind)` is not actually needed and this can be triggered on stable as a result of the way async functions/generators are handled in the compiler. e.g.:
```rust
extern crate futures; // 0.3.21
async fn bar() {
let foo: u64;
unsafe {
std::arch::asm!(
"mov {}, 1",
out(reg) foo,
);
}
println!("{}", foo);
}
fn main() {
futures::executor::block_on(bar());
}
```
([playground link](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=1c7781c34dd4a3e80ae4bd936a0c82fc))
An example of the incorrect LLVM generated:
```llvm
bb1: ; preds = %start
%1 = invoke i64 asm sideeffect alignstack inteldialect unwind "mov ${0:q}, 1", "=&r,~{dirflag},~{fpsr},~{flags},~{memory}"()
to label %bb2 unwind label %cleanup, !srcloc !9
store i64 %1, i64* %foo, align 8
bb2:
[...snip...]
```
The store should not be placed after the asm invoke but rather should be in the normal control flow basic block (`bb2` in this case).
[Here](https://gist.github.com/luqmana/be1af5b64d2cda5a533e3e23a7830b44) is a writeup of the investigation that lead to finding this.
|
|
|
|
|
|
It was using `Iterator::by_ref` in the implementation, which ended up pessimizing it enough that, for example, it didn't vectorize when we tried it in the <https://rust-lang.zulipchat.com/#narrow/stream/257879-project-portable-simd/topic/Reducing.20sum.20into.20wider.20types> conversation.
Demonstration that the codegen test doesn't pass on the current nightly: <https://rust.godbolt.org/z/Taxev5eMn>
|
|
async: Give predictable name to binding generated from .await expressions.
This name makes it to debuginfo and allows debuggers to identify such bindings and their captured versions in suspended async fns.
This will be useful for async stack traces, as discussed in https://internals.rust-lang.org/t/async-debugging-logical-stack-traces-setting-goals-collecting-examples/15547.
I don't know if this needs some discussion by ````@rust-lang/compiler,```` e.g. about the name of the binding (`__awaitee`) or about the fact that this PR introduces a (soft) guarantee about a compiler generated name. Although, regarding the later, I think the same reasoning applies here as it does for debuginfo in general.
r? ````@tmandry````
|
|
expressions.
This name makes it to debuginfo and allows debuggers to identify such bindings and
their captured versions in suspended async fns.
|
|
|
|
|
|
Signed-off-by: codehorseman <cricis@yeah.net>
|
|
works for them
|
|
This commit
- changes names to use di_node instead of metadata
- uniformly names all functions that build new debuginfo nodes build_xyz_di_node
- renames CrateDebugContext to CodegenUnitDebugContext (which is more accurate)
- moves TypeMap and functions that work directly work with it to a new type_map module
- moves and reimplements enum related builder functions to a new enums module
- splits enum debuginfo building for the native and cpp-like cases, since they are mostly separate
- uses SmallVec instead of Vec in many places
- removes the old infrastructure for dealing with recursion cycles (create_and_register_recursive_type_forward_declaration(), RecursiveTypeDescription, set_members_of_composite_type(), MemberDescription, MemberDescriptionFactory, prepare_xyz_metadata(), etc)
- adds type_map::build_type_with_children() as a replacement for dealing with recursion cycles
- adds many (doc-)comments explaining what's going on
- changes cpp-like naming for C-Style enums so they don't get a enum$<...> name (because the NatVis visualizer does not apply to them)
- fixes detection of what is a C-style enum because some enums where classified as C-style even though they have fields
- changes the position of discriminant debuginfo node so it is consistently nested inside the top-level union instead of, sometimes, next to it
|
|
This change is somewhat extensive, since it affects MIR -- since this is called to determine Copy vs Move -- so any test that's `no_core` needs to actually have the normal `impl`s it uses.
|
|
We can't do it for everything, but it would be nice to at least stop making calls to clone methods in debug from things like derived-clones.
|
|
This ensures that information about target features configured with
`-C target-feature=...` or detected with `-C target-cpu=native` is
retained for subsequent consumers of LLVM bitcode.
This is crucial for linker plugin LTO, since this information is not
conveyed to the plugin otherwise.
|
|
Reopen 91719
Reopened #91719, which was closed inadvertently due to technical difficulties.
|
|
Add !align metadata on loads of &/&mut/Box
Note that this refers to the alignment of what the loaded value points
to, _not_ the alignment of the loaded value itself.
r? `@ghost` (blocked on #94158)
|
|
|
|
Co-authored-by: Scott McMurray <scottmcm@users.noreply.github.com>
|
|
|
|
|
|
|
|
r=michaelwoerister,davidtwco
Restore the local filter on mono item sorting
In `CodegenUnit::items_in_deterministic_order`, there's a comment that
only local HirIds should be taken into account, but #90408 removed the
`as_local` call that sets others to None. Restoring that check fixes the
s390x hangs seen in [RHBZ 2058803].
[RHBZ 2058803]: https://bugzilla.redhat.com/show_bug.cgi?id=2058803
|
|
|
|
|
|
|
|
This changed in upstream change https://reviews.llvm.org/D98152 (aka
https://github.com/llvm/llvm-project/commit/a266af721153fab6452094207b09ed265ab0be7b)
wherein LLVM got smarter about using intrinsics. As best I can tell the
change I've made here preserves the intent of the test on LLVM 14 and
before while also passing on LLVM 15 and later.
|
|
Revert "Auto merge of #92419 - erikdesjardins:coldland, r=nagisa"
Should fix (untested) #94390
Reopens #46515, #87055
r? `@ehuss`
|
|
Note that this refers to the alignment of what the loaded value points
to, _not_ the alignment of the loaded value itself.
|
|
more complete sparc64 ABI fix for aggregates with floating point members
Previous fix didn't handle nested structures at all.
|
|
Apply noundef metadata to loads of types that do not permit raw init
This matches the noundef attributes we apply on arguments/return types.
Fixes (partially) #74378.
|
|
|
|
This reverts commit 4f49627c6fe2a32d1fed6310466bb0e1c535c0c0, reversing
changes made to 028c6f1454787c068ff5117e9000a1de4fd98374.
|
|
Apply noundef attribute to all scalar types which do not permit raw init
Beyond `&`/`&mut`/`Box`, this covers `char`, enum discriminants, `NonZero*`, etc.
All such types currently cause a Miri error if left uninitialized,
and an `invalid_value` lint in cases like `mem::uninitialized::<char>()`.
Note that this _does not_ change whether or not it is UB for `u64` (or
other integer types with no invalid values) to be undef.
Fixes (partially) #74378.
r? `@ghost` (blocked on #94127)
`@rustbot` label S-blocked
|
|
For MIRI, cfg out the swap vectorization logic from 94212
Because of #69488 the swap logic from #94212 doesn't currently work in MIRI.
Copying in smaller pieces is probably much worse for its performance anyway, so it'd probably rather just use the simple path regardless.
Part of #94371, though another PR will be needed for the CTFE aspect.
r? `@oli-obk`
cc `@RalfJung`
|
