| Age | Commit message (Collapse) | Author | Lines |
|---|
|
|
|
This commit adjusts the naming of various lang items so that they are
consistent and don't include prefixes containing the target or
"LangItem". In addition, lang item variants are no longer exported from
the `lang_items` module.
Signed-off-by: David Wood <david@davidtw.co>
|
|
|
|
|
|
Currently, the code spills operands onto the stack for the purpose of
debuginfo. However, naked functions can only contain an asm block. Therefore,
attempting to spill the operands on the stack is undefined behavior.
Fixes https://github.com/rust-lang/rust/issues/42779
cc https://github.com/rust-lang/rust/issues/32408
|
|
Emit == null instead of <= null for niche check
When the niche maximum is zero, emit a "== zero" check instead of a "<= zero" check. In particular, this avoids the awkward case of "<= null". While LLVM does canonicalize this to "== null", this apparently doesn't happen for constant expressions, leading to the issue in #74425. While that can be addressed on the LLVM side, it still seems prudent to emit sensible IR here, because this will allow null checks to be optimized earlier in the pipeline.
Fixes #74425.
|
|
When the niche maximum is zero, emit a "== zero" check instead of
a "<= zero" check. In particular, this avoid the awkward case of
"<= null". While LLVM does canonicalize this to "!= null", this
appently doesn't happen for constant expressions, leading to the
issue in #74425. While that can be addressed on the LLVM side, it
still seems prudent to emit sensible IR here, because this will
allow null checks to be optimized earlier in the pipeline.
Fixes #74425.
|
|
This commit improves code generation for WebAssembly targets when
translating floating to integer casts. This improvement is only relevant
when the `nontrapping-fptoint` feature is not enabled, but the feature
is not enabled by default right now. Additionally this improvement only
affects safe casts since unchecked casts were improved in #74659.
Some more background for this issue is present on #73591, but the
general gist of the issue is that in LLVM the `fptosi` and `fptoui`
instructions are defined to return an `undef` value if they execute on
out-of-bounds values; they notably do not trap. To implement these
instructions for WebAssembly the LLVM backend must therefore generate
quite a few instructions before executing `i32.trunc_f32_s` (for
example) because this WebAssembly instruction traps on out-of-bounds
values. This codegen into wasm instructions happens very late in the
code generator, so what ends up happening is that rustc inserts its own
codegen to implement Rust's saturating semantics, and then LLVM also
inserts its own codegen to make sure that the `fptosi` instruction
doesn't trap. Overall this means that a function like this:
#[no_mangle]
pub unsafe extern "C" fn cast(x: f64) -> u32 {
x as u32
}
will generate this WebAssembly today:
(func $cast (type 0) (param f64) (result i32)
(local i32 i32)
local.get 0
f64.const 0x1.fffffffep+31 (;=4.29497e+09;)
f64.gt
local.set 1
block ;; label = @1
block ;; label = @2
local.get 0
f64.const 0x0p+0 (;=0;)
local.get 0
f64.const 0x0p+0 (;=0;)
f64.gt
select
local.tee 0
f64.const 0x1p+32 (;=4.29497e+09;)
f64.lt
local.get 0
f64.const 0x0p+0 (;=0;)
f64.ge
i32.and
i32.eqz
br_if 0 (;@2;)
local.get 0
i32.trunc_f64_u
local.set 2
br 1 (;@1;)
end
i32.const 0
local.set 2
end
i32.const -1
local.get 2
local.get 1
select)
This PR improves the situation by updating the code generation for
float-to-int conversions in rustc, specifically only for WebAssembly
targets and only for some situations (float-to-u8 still has not great
codegen). The fix here is to use basic blocks and control flow to avoid
speculatively executing `fptosi`, and instead LLVM's raw intrinsic for
the WebAssembly instruction is used instead. This effectively extends
the support added in #74659 to checked casts. After this commit the
codegen for the above Rust function looks like:
(func $cast (type 0) (param f64) (result i32)
(local i32)
block ;; label = @1
local.get 0
f64.const 0x0p+0 (;=0;)
f64.ge
local.tee 1
i32.const 1
i32.xor
br_if 0 (;@1;)
local.get 0
f64.const 0x1.fffffffep+31 (;=4.29497e+09;)
f64.le
i32.eqz
br_if 0 (;@1;)
local.get 0
i32.trunc_f64_u
return
end
i32.const -1
i32.const 0
local.get 1
select)
For reference, in Rust 1.44, which did not have saturating
float-to-integer casts, the codegen LLVM would emit is:
(func $cast (type 0) (param f64) (result i32)
block ;; label = @1
local.get 0
f64.const 0x1p+32 (;=4.29497e+09;)
f64.lt
local.get 0
f64.const 0x0p+0 (;=0;)
f64.ge
i32.and
i32.eqz
br_if 0 (;@1;)
local.get 0
i32.trunc_f64_u
return
end
i32.const 0)
So we're relatively close to the original codegen, although it's
slightly different because the semantics of the function changed where
we're emulating the `i32.trunc_sat_f32_s` instruction rather than always
replacing out-of-bounds values with zero.
There is still work that could be done to improve casts such as `f32` to
`u8`. That form of cast still uses the `fptosi` instruction which
generates lots of branch-y code. This seems less important to tackle now
though. In the meantime this should take care of most use cases of
floating-point conversion and as a result I'm going to speculate that
this...
Closes #73591
|
|
Miri rename undef uninit
Renamed parts of code within the `librustc_middle/mir/interpret/` directory.
Related issue [#71193](https://github.com/rust-lang/rust/issues/71193)
|
|
|
|
Renamed the function ScalarMaybeUninit::not_undef to ScalarMaybeUninit::check_init in the file src/librustc_middle/mir/interpret/value.rs, to reflect changes in terminology used.
Related issue rust-lang#71193
|
|
functions
This patch extends the existing `type_i8p` method so that it requires an
explicit address space to be specified. Before this patch, the
`type_i8p` method implcitily assumed the default address space, which is
not a safe transformation on all targets, namely AVR.
The Rust compiler already has support for tracking the "instruction
address space" on a per-target basis. This patch extends the code
generation routines so that an address space must always be specified.
In my estimation, around 15% of the callers of `type_i8p` produced
invalid code on AVR due to the loss of address space prior to LLVM final
code generation. This would lead to unavoidable assertion errors
relating to invalid bitcasts.
With this patch, the address space is always either 1) explicitly set to
the instruction address space because the logic is dealing with functions
which must be placed there, or 2) explicitly set to the default address
space 0 because the logic can only operate on data space pointers and thus
we keep the existing semantics of assuming the default, "data" address space.
|
|
This commit implements the `unused_generic_params` query, an initial
version of polymorphization which detects when an item does not use
generic parameters and is being needlessly monomorphized as a result.
Signed-off-by: David Wood <david@davidtw.co>
|
|
This commit avoids a natural, free-range double substitution error by
monomorphizing the projection element before getting the type.
Signed-off-by: David Wood <david@davidtw.co>
|
|
|
|
|
|
Note that the output of `unpretty-debug.stdout` has changed. In that
test the hash values are normalized from a symbol numbers to small
numbers like "0#0" and "0#1". The increase in the number of static
symbols must have caused the original numbers to contain more digits,
resulting in different pretty-printing prior to normalization.
|
|
Use WASM's saturating casts if they are available
WebAssembly supports saturating floating point to integer casts behind a target feature. The feature is already available on many browsers. Beginning with 1.45 Rust will start defining the behavior of floating point to integer casts to be saturating as well. For this Rust constructs additional checks on top of the `fptoui` / `fptosi` instructions it emits. Here we introduce the possibility for the codegen backend to construct saturating casts itself and only fall back to constructing the checks ourselves if that is not possible.
Resolves part of #73591
|
|
WebAssembly supports saturating floating point to integer casts behind a
target feature. The feature is already available on many browsers.
Beginning with 1.45 Rust will start defining the behavior of floating
point to integer casts to be saturating as well. For this Rust
constructs additional checks on top of the `fptoui` / `fptosi`
instructions it emits. Here we introduce the possibility for the codegen
backend to construct saturating casts itself and only fall back to
constructing the checks ourselves if that is not possible.
|
|
added regions with counter expressions and counters.
Added codegen_llvm/coverageinfo mod for upcoming coverage map
Move coverage region collection to CodegenCx finalization
Moved from `query coverageinfo` (renamed from `query coverage_data`),
as discussed in the PR at:
https://github.com/rust-lang/rust/pull/73684#issuecomment-649882503
Address merge conflict in MIR instrument_coverage test
The MIR test output format changed for int types.
moved debug messages out of block.rs
This makes the block.rs calls to add coverage mapping data to the
CodegenCx much more concise and readable.
move coverage intrinsic handling into llvm impl
I realized that having half of the coverage intrinsic handling in
`rustc_codegen_ssa` and half in `rustc_codegen_llvm` meant that any
non-llvm backend would be bound to the same decisions about how the
coverage-related MIR terminators should be handled.
To fix this, I moved the non-codegen portion of coverage intrinsic
handling into its own trait, and implemented it in `rustc_codegen_llvm`
alongside `codegen_intrinsic_call`.
I also added the (required?) stubs for the new intrinsics to
`IntrepretCx::emulate_intrinsic()`, to ensure calls to this function do
not fail if called with these new but known intrinsics.
address PR Feedback on 28 June 2020 2:48pm PDT
|
|
propagation
|
|
|
|
first stage of implementing LLVM code coverage
This PR replaces #70680 (WIP toward LLVM Code Coverage for Rust) since I am re-implementing the Rust LLVM code coverage feature in a different part of the compiler (in MIR pass(es) vs AST).
This PR updates rustc with `-Zinstrument-coverage` option that injects the llvm intrinsic `instrprof.increment()` for code generation.
This initial version only injects counters at the top of each function, and does not yet implement the required coverage map.
Upcoming PRs will add the coverage map, and add more counters and/or counter expressions for each conditional code branch.
Rust compiler MCP https://github.com/rust-lang/compiler-team/issues/278
Relevant issue: #34701 - Implement support for LLVMs code coverage instrumentation
***[I put together some development notes here, under a separate branch.](https://github.com/richkadel/rust/blob/cfa0b21d34ee64e4ebee226101bd2ef0c6757865/src/test/codegen/coverage-experiments/README-THIS-IS-TEMPORARY.md)***
|
|
tag/niche terminology cleanup
The term "discriminant" was used in two ways throughout the compiler:
* every enum variant has a corresponding discriminant, that can be given explicitly with `Variant = N`.
* that discriminant is then encoded in memory to store which variant is active -- but this encoded form of the discriminant was also often called "discriminant", even though it is conceptually quite different (e.g., it can be smaller in size, or even use niche-filling).
After discussion with @eddyb, this renames the second term to "tag". The way the tag is encoded can be either `TagEncoding::Direct` (formerly `DiscriminantKind::Tag`) or `TagEncoding::Niche` (formerly `DiscrimianntKind::Niche`).
This finally resolves some long-standing confusion I had about the handling of variant indices and discriminants, which surfaced in https://github.com/rust-lang/rust/pull/72419.
(There is also a `DiscriminantKind` type in libcore, it remains unaffected. I think this corresponds to the discriminant, not the tag, so that seems all right.)
r? @eddyb
|
|
|
|
|
|
As suggested in PR feedback:
https://github.com/rust-lang/rust/pull/73011#discussion_r435728923
This allows count_code_region() to be handled like a normal intrinsic so
the InstanceDef::InjectedCode variant is no longer needed.
|
|
This initial version only injects counters at the top of each function.
Rust Coverage will require injecting additional counters at each
conditional code branch.
|
|
Fix #[thread_local] statics as asm! sym operands
The `asm!` RFC specifies that `#[thread_local]` statics may be used as `sym` operands for inline assembly.
This also fixes a regression in the handling of `#[thread_local]` during monomorphization which caused link-time errors with multiple codegen units, most likely introduced by #71192.
r? @oli-obk
|
|
Track span of function in method calls, and use this in #[track_caller]
Fixes #69977
When we parse a chain of method calls like `foo.a().b().c()`, each
`MethodCallExpr` gets assigned a span that starts at the beginning of
the call chain (`foo`). While this is useful for diagnostics, it means
that `Location::caller` will return the same location for every call
in a call chain.
This PR makes us separately record the span of the function name and
arguments for a method call (e.g. `b()` in `foo.a().b().c()`). This
`Span` is passed through HIR lowering and MIR building to
`TerminatorKind::Call`, where it is used in preference to
`Terminator.source_info.span` when determining `Location::caller`.
This new span is also useful for diagnostics where we want to emphasize
a particular method call - for an example, see
https://github.com/rust-lang/rust/pull/72389#discussion_r436035990
|
|
Fixes #69977
When we parse a chain of method calls like `foo.a().b().c()`, each
`MethodCallExpr` gets assigned a span that starts at the beginning of
the call chain (`foo`). While this is useful for diagnostics, it means
that `Location::caller` will return the same location for every call
in a call chain.
This PR makes us separately record the span of the function name and
arguments for a method call (e.g. `b()` in `foo.a().b().c()`). This
`Span` is passed through HIR lowering and MIR building to
`TerminatorKind::Call`, where it is used in preference to
`Terminator.source_info.span` when determining `Location::caller`.
This new span is also useful for diagnostics where we want to emphasize
a particular method call - for an example, see
https://github.com/rust-lang/rust/pull/72389#discussion_r436035990
|
|
Fixes more of:
clippy::unused_unit
clippy::op_ref
clippy::useless_format
clippy::needless_return
clippy::useless_conversion
clippy::bind_instead_of_map
clippy::into_iter_on_ref
clippy::redundant_clone
clippy::nonminimal_bool
clippy::redundant_closure
clippy::option_as_ref_deref
clippy::len_zero
clippy::iter_cloned_collect
clippy::filter_next
|
|
|
|
|
|
Make TLS accesses explicit in MIR
r? @rust-lang/wg-mir-opt
cc @RalfJung @vakaras for miri thread locals
cc @bjorn3 for cranelift
fixes #70685
|
|
Improve inline asm error diagnostics
Previously we were just using the raw LLVM error output (with line, caret, etc) as the diagnostic message, which ends up looking rather out of place with our existing diagnostics.
The new diagnostics properly format the diagnostics and also take advantage of LLVM's per-line `srcloc` attribute to map an error in inline assembly directly to the relevant line of source code.
Incidentally also fixes #71639 by disabling `srcloc` metadata during LTO builds since we don't know what crate it might have come from. We can only resolve `srcloc`s from the currently crate since it indexes into the source map for the current crate.
Fixes #72664
Fixes #71639
r? @petrochenkov
### Old style
```rust
#![feature(llvm_asm)]
fn main() {
unsafe {
let _x: i32;
llvm_asm!(
"mov $0, $1
invalid_instruction $0, $1
mov $0, $1"
: "=&r" (_x)
: "r" (0)
:: "intel"
);
}
}
```
```
error: <inline asm>:3:14: error: invalid instruction mnemonic 'invalid_instruction'
invalid_instruction ecx, eax
^~~~~~~~~~~~~~~~~~~
--> src/main.rs:6:9
|
6 | / llvm_asm!(
7 | | "mov $0, $1
8 | | invalid_instruction $0, $1
9 | | mov $0, $1"
... |
12 | | :: "intel"
13 | | );
| |__________^
```
### New style
```rust
#![feature(asm)]
fn main() {
unsafe {
asm!(
"mov {0}, {1}
invalid_instruction {0}, {1}
mov {0}, {1}",
out(reg) _,
in(reg) 0i64,
);
}
}
```
```
error: invalid instruction mnemonic 'invalid_instruction'
--> test.rs:7:14
|
7 | invalid_instruction {0}, {1}
| ^
|
note: instantiated into assembly here
--> <inline asm>:3:14
|
3 | invalid_instruction rax, rcx
| ^^^^^^^^^^^^^^^^^^^
```
|
|
|
|
Properly handle InlineAsmOperand::SymFn when collecting monomorphized items
Fixes #72484
|
|
|
|
|
|
Pass more `Copy` types by value.
There are a lot of locations where we pass `&T where T: Copy` by reference,
which should both be slightly less performant and less readable IMO.
This PR currently consists of three fairly self contained commits:
- passes `ty::Predicate` by value and stops depending on `AsRef<ty::Predicate>`.
- changes `<&List<_>>::into_iter` to iterate over the elements by value. This would break `List`s
of non copy types. But as the only list constructor requires `T` to be copy anyways, I think
the improved readability is worth this potential future restriction.
- passes `mir::PlaceElem` by value. Mir currently has quite a few copy types which are passed by reference, e.g. `Local`. As I don't have a lot of experience working with MIR, I mostly did this to get some feedback from people who use MIR more frequently
- tries to reuse `ty::Predicate` in case it did not change in some places, which should hopefully
fix the regression caused by #72055
r? @nikomatsakis for the first commit, which continues the work of #72055 and makes adding `PredicateKind::ForAll` slightly more pleasant. Feel free to reassign though
|
|
|
|
Fixes #72484
|
|
|
|
Dumb NRVO
This is a very simple version of an NRVO pass, which scans backwards from the `return` terminator to see if there is an an assignment like `_0 = _1`. If a basic block with two or more predecessors is encountered during this scan without first seeing an assignment to the return place, we bail out. This avoids running a full "reaching definitions" dataflow analysis.
I wanted to see how much `rustc` would benefit from even a very limited version of this optimization. We should be able to use this as a point of comparison for more advanced versions that are based on live ranges.
r? @ghost
|
|
|
|
|
|
|
|
|
|
|
