about summary refs log tree commit diff
path: root/compiler/rustc_codegen_llvm/src/intrinsic.rs
AgeCommit message (Collapse)AuthorLines
2022-02-24Introduce Bx::switch_to_blockbjorn3-50/+44
2022-02-20Remove build_sibling_blockbjorn3-16/+26
2022-02-19Adopt let else in more placesest31-6/+4
2022-02-15Overhaul `TyS` and `Ty`.Nicholas Nethercote-2/+2
Specifically, change `Ty` from this: ``` pub type Ty<'tcx> = &'tcx TyS<'tcx>; ``` to this ``` pub struct Ty<'tcx>(Interned<'tcx, TyS<'tcx>>); ``` There are two benefits to this. - It's now a first class type, so we can define methods on it. This means we can move a lot of methods away from `TyS`, leaving `TyS` as a barely-used type, which is appropriate given that it's not meant to be used directly. - The uniqueness requirement is now explicit, via the `Interned` type. E.g. the pointer-based `Eq` and `Hash` comes from `Interned`, rather than via `TyS`, which wasn't obvious at all. Much of this commit is boring churn. The interesting changes are in these files: - compiler/rustc_middle/src/arena.rs - compiler/rustc_middle/src/mir/visit.rs - compiler/rustc_middle/src/ty/context.rs - compiler/rustc_middle/src/ty/mod.rs Specifically: - Most mentions of `TyS` are removed. It's very much a dumb struct now; `Ty` has all the smarts. - `TyS` now has `crate` visibility instead of `pub`. - `TyS::make_for_test` is removed in favour of the static `BOOL_TY`, which just works better with the new structure. - The `Eq`/`Ord`/`Hash` impls are removed from `TyS`. `Interned`s impls of `Eq`/`Hash` now suffice. `Ord` is now partly on `Interned` (pointer-based, for the `Equal` case) and partly on `TyS` (contents-based, for the other cases). - There are many tedious sigil adjustments, i.e. adding or removing `*` or `&`. They seem to be unavoidable.
2022-01-24Merge add_handler into catch_switchbjorn3-3/+2
Some codegen backends may require all handlers to be immediately known
2022-01-18Rollup merge of #92425 - calebzulawski:simd-cast, r=workingjubileeMatthias Krüger-9/+21
Improve SIMD casts * Allows `simd_cast` intrinsic to take `usize` and `isize` * Adds `simd_as` intrinsic, which is the same as `simd_cast` except for saturating float-to-int conversions (matching the behavior of `as`). cc `@workingjubilee`
2022-01-12Remove deprecated LLVM-style inline assemblyTomasz Miąsko-2/+1
2022-01-09eplace usages of vec![].into_iter with [].into_iterLucas Kent-1/+1
2022-01-04Add simd_as intrinsicCaleb Zulawski-5/+5
2022-01-04Allow isize/usize in simd_castCaleb Zulawski-4/+16
2021-12-16Remove `in_band_lifetimes` from `rustc_codegen_llvm`LegionMammal978-16/+19
See #91867 for more information.
2021-12-03LLVM codgen support for unwinding inline assemblycynecx-0/+2
2021-11-14Remove workaround for the forward progress handling in LLVMAndreas Jonson-9/+0
2021-11-10Add comment regarding bit orderCaleb Zulawski-0/+3
2021-11-06Fix off-by-one error uncovered by std::simd testsCaleb Zulawski-2/+2
2021-11-06Allow simd_select_bitmask to take byte arraysCaleb Zulawski-20/+31
2021-11-06Allow simd_bitmask to return byte arraysCaleb Zulawski-12/+35
2021-10-25Add LLVM CFI support to the Rust compilerRamon de C Valle-0/+8
This commit adds LLVM Control Flow Integrity (CFI) support to the Rust compiler. It initially provides forward-edge control flow protection for Rust-compiled code only by aggregating function pointers in groups identified by their number of arguments. Forward-edge control flow protection for C or C++ and Rust -compiled code "mixed binaries" (i.e., for when C or C++ and Rust -compiled code share the same virtual address space) will be provided in later work as part of this project by defining and using compatible type identifiers (see Type metadata in the design document in the tracking issue #89653). LLVM CFI can be enabled with -Zsanitizer=cfi and requires LTO (i.e., -Clto).
2021-10-07Rollup merge of #89298 - gcohara:issue89193, r=workingjubileeGuillaume Gomez-8/+25
Issue 89193 - Fix ICE when using `usize` and `isize` with SIMD gathers closes #89193 r? `@workingjubilee`
2021-10-01Fix clippy lintsGuillaume Gomez-4/+4
2021-09-27Issue 89193George O'Hara-8/+25
2021-09-22Avoid a couple of Symbol::as_str calls in cg_llvmbjorn3-6/+9
2021-09-19Auto merge of #88575 - eddyb:fn-abi-queries, r=nagisabors-2/+2
Querify `FnAbi::of_{fn_ptr,instance}` as `fn_abi_of_{fn_ptr,instance}`. *Note: opening this PR as draft because it's based on #88499* This more or less replicates the `LayoutOf::layout_of` setup from #88499, to replace `FnAbi::of_{fn_ptr,instance}` with `FnAbiOf::fn_abi_of_{fn_ptr,instance}`, and also route them through queries (which `layout_of` has used for a while). The two changes at the use sites (other than the names) are: * return type is now wrapped in `&'tcx` * the value *is* interned, which may affect performance * the `extra_args` list is now an interned `&'tcx ty::List<Ty<'tcx>>` * should be cheap (it's empty for anything other than C variadics) Theoretically, a `FnAbiOfHelpers` implementer could choose to keep the `Result<...>` instead of eagerly erroring, but the only existing users of these APIs are codegen backends, so they don't (want to) take advantage of this. At least miri could make use of this, since it prefers propagating errors (it "just" doesn't use `FnAbi` yet - cc `@RalfJung).` The way this is done is probably less efficient than what is possible, because the queries handle the correctness-oriented API (i.e. the split into `fn` pointers vs instances), whereas a lower-level query could end up with more reuse between different instances with identical signatures. r? `@nagisa` cc `@oli-obk` `@bjorn3`
2021-09-19Rollup merge of #88855 - calebzulawski:feature/simd_shuffle, r=nagisaYuki Okushi-4/+21
Allow simd_shuffle to accept vectors of any length cc ``@rust-lang/project-portable-simd`` ``@workingjubilee``
2021-09-18Querify `fn_abi_of_{fn_ptr,instance}`.Eduard-Mihai Burtescu-1/+1
2021-09-18ty::layout: replicate `layout_of` setup for `fn_abi_of_{fn_ptr,instance}`.Eduard-Mihai Burtescu-2/+2
2021-09-11Allow simd_shuffle to accept vectors of any lengthCaleb Zulawski-4/+21
2021-09-09Make `abi::Abi` `Copy` and remove a *lot* of refsAndreas Liljeqvist-1/+1
fix fix Remove more refs and clones fix more fix
2021-09-02rustc_target: move `LayoutOf` to `ty::layout`.Eduard-Mihai Burtescu-2/+2
2021-08-22Fix typos “an”→“a” and a few different ones that appeared in the ↵Frank Steffahn-1/+1
same search
2021-08-22Fix more “a”/“an” typosFrank Steffahn-1/+1
2021-08-22Fix typos “a”→“an”Frank Steffahn-1/+1
2021-08-12Implement `black_box` using intrinsicGary Guo-0/+26
The new implementation allows some `memcpy`s to be optimized away, so the uninit value in ui/sanitize/memory.rs is constructed directly onto the return place. Therefore the sanitizer now says that the value is allocated by `main` rather than `random`.
2021-08-05Remove the `decl` arg from `FnAbi::llvm_type`Josh Stone-1/+1
We can apply the `c_variadic` fix all the time, rather than trying to distinguish between declarations and any other use.
2021-08-05Prepare call/invoke for opaque pointersJosh Stone-94/+80
Rather than relying on `getPointerElementType()` from LLVM function pointers, we now pass the function type explicitly when building `call` or `invoke` instructions.
2021-08-04Prepare inbounds_gep for opaque pointersTomasz Miąsko-4/+12
Implement inbounds_gep using LLVMBuildInBoundsGEP2 which takes an explicit type argument instead of deriving it from a pointer type.
2021-07-09Pass type when creating loadNikita Popov-8/+11
This makes load generation compatible with opaque pointers. The generation of nontemporal copies still accesses the pointer element type, as fixing this requires more movement.
2021-07-08Adjust the threshold to look at the ABI, not just the sizeScott McMurray-12/+24
2021-07-08Stop generating `alloca`s+`memcmp` for simple array equalityScott McMurray-0/+26
2021-06-30Add support for leaf fn frame pointer eliminationSimonas Kazlauskas-1/+1
This PR adds ability for the target specifications to specify frame pointer emission type that's not just “always” or “whatever cg decides”. In particular there's a new mode that allows omission of the frame pointer for leaf functions (those that don't call any other functions). We then set this new mode for Aarch64-based Apple targets. Fixes #86196
2021-05-17rustc_codegen_ssa: append blocks to functions w/o creating a builder.Eduard-Mihai Burtescu-1/+2
2021-04-17Don't set `fast`(-math) for certain simd opsSimonas Kazlauskas-21/+19
`fast-math` implies things like functions not being able to accept as an argument or return as a result, say, `inf` which made these functions confusingly named or behaving incorrectly, depending on how you interpret it. Since the time when these intrinsics have been implemented the intrinsics user's (stdsimd) approach has changed significantly and so now it is required that these intrinsics operate normally rather than in "whatever" way. Fixes #84268
2021-04-14Add more SIMD math.h intrinsicsJubilee Young-2/+6
LLVM supports many functions from math.h in its IR. Many of these have single-instruction variants on various platforms. So, let's add them so std::arch can use them. Yes, exact comparison is intentional: rounding must always return a valid integer-equal value, except for inf/NAN.
2021-03-19correct macro namesSparrowLii-4/+4
2021-03-19Add simd_neg platform intrinsicSparrowLii-2/+21
2021-03-10Remove the -Zinsert-sideeffectSimonas Kazlauskas-9/+5
This removes all of the code we had in place to work-around LLVM's handling of forward progress. From this removal excluded is a workaround where we'd insert a `sideeffect` into clearly infinite loops such as `loop {}`. This code remains conditionally effective when the LLVM version is earlier than 12.0, which fixed the forward progress related miscompilations at their root.
2021-01-23Make declare_cfn more flexibleCaleb Zulawski-6/+8
2021-01-03Implement floating point SIMD intrinsics over all vector widths, and limit ↵Caleb Zulawski-78/+55
SIMD vector lengths.
2020-12-11use strip_prefix over slicing (clippy::manual_strip)Matthias Krüger-2/+2
2020-11-29Auto merge of #78863 - KodrAus:feat/simd-array, r=oli-obkbors-50/+55
Support repr(simd) on ADTs containing a single array field This is a squash and rebase of `@gnzlbg's` #63531 I've never actually written code in the compiler before so just fumbled my way around until it would build 😅 I imagine there'll be some work we need to do in `rustc_codegen_cranelift` too for this now, but might need some input from `@bjorn3` to know what that is. cc `@rust-lang/project-portable-simd` ----- This PR allows using `#[repr(simd)]` on ADTs containing a single array field: ```rust #[repr(simd)] struct S0([f32; 4]); #[repr(simd)] struct S1<const N: usize>([f32; N]); #[repr(simd)] struct S2<T, const N: usize>([T; N]); ``` This should allow experimenting with portable packed SIMD abstractions on nightly that make use of const generics.