about summary refs log tree commit diff
path: root/compiler/rustc_codegen_llvm/src/intrinsic.rs
AgeCommit message (Collapse)AuthorLines
2021-10-07Rollup merge of #89298 - gcohara:issue89193, r=workingjubileeGuillaume Gomez-8/+25
Issue 89193 - Fix ICE when using `usize` and `isize` with SIMD gathers closes #89193 r? `@workingjubilee`
2021-10-01Fix clippy lintsGuillaume Gomez-4/+4
2021-09-27Issue 89193George O'Hara-8/+25
2021-09-22Avoid a couple of Symbol::as_str calls in cg_llvmbjorn3-6/+9
2021-09-19Auto merge of #88575 - eddyb:fn-abi-queries, r=nagisabors-2/+2
Querify `FnAbi::of_{fn_ptr,instance}` as `fn_abi_of_{fn_ptr,instance}`. *Note: opening this PR as draft because it's based on #88499* This more or less replicates the `LayoutOf::layout_of` setup from #88499, to replace `FnAbi::of_{fn_ptr,instance}` with `FnAbiOf::fn_abi_of_{fn_ptr,instance}`, and also route them through queries (which `layout_of` has used for a while). The two changes at the use sites (other than the names) are: * return type is now wrapped in `&'tcx` * the value *is* interned, which may affect performance * the `extra_args` list is now an interned `&'tcx ty::List<Ty<'tcx>>` * should be cheap (it's empty for anything other than C variadics) Theoretically, a `FnAbiOfHelpers` implementer could choose to keep the `Result<...>` instead of eagerly erroring, but the only existing users of these APIs are codegen backends, so they don't (want to) take advantage of this. At least miri could make use of this, since it prefers propagating errors (it "just" doesn't use `FnAbi` yet - cc `@RalfJung).` The way this is done is probably less efficient than what is possible, because the queries handle the correctness-oriented API (i.e. the split into `fn` pointers vs instances), whereas a lower-level query could end up with more reuse between different instances with identical signatures. r? `@nagisa` cc `@oli-obk` `@bjorn3`
2021-09-19Rollup merge of #88855 - calebzulawski:feature/simd_shuffle, r=nagisaYuki Okushi-4/+21
Allow simd_shuffle to accept vectors of any length cc ``@rust-lang/project-portable-simd`` ``@workingjubilee``
2021-09-18Querify `fn_abi_of_{fn_ptr,instance}`.Eduard-Mihai Burtescu-1/+1
2021-09-18ty::layout: replicate `layout_of` setup for `fn_abi_of_{fn_ptr,instance}`.Eduard-Mihai Burtescu-2/+2
2021-09-11Allow simd_shuffle to accept vectors of any lengthCaleb Zulawski-4/+21
2021-09-09Make `abi::Abi` `Copy` and remove a *lot* of refsAndreas Liljeqvist-1/+1
fix fix Remove more refs and clones fix more fix
2021-09-02rustc_target: move `LayoutOf` to `ty::layout`.Eduard-Mihai Burtescu-2/+2
2021-08-22Fix typos “an”→“a” and a few different ones that appeared in the ↵Frank Steffahn-1/+1
same search
2021-08-22Fix more “a”/“an” typosFrank Steffahn-1/+1
2021-08-22Fix typos “a”→“an”Frank Steffahn-1/+1
2021-08-12Implement `black_box` using intrinsicGary Guo-0/+26
The new implementation allows some `memcpy`s to be optimized away, so the uninit value in ui/sanitize/memory.rs is constructed directly onto the return place. Therefore the sanitizer now says that the value is allocated by `main` rather than `random`.
2021-08-05Remove the `decl` arg from `FnAbi::llvm_type`Josh Stone-1/+1
We can apply the `c_variadic` fix all the time, rather than trying to distinguish between declarations and any other use.
2021-08-05Prepare call/invoke for opaque pointersJosh Stone-94/+80
Rather than relying on `getPointerElementType()` from LLVM function pointers, we now pass the function type explicitly when building `call` or `invoke` instructions.
2021-08-04Prepare inbounds_gep for opaque pointersTomasz Miąsko-4/+12
Implement inbounds_gep using LLVMBuildInBoundsGEP2 which takes an explicit type argument instead of deriving it from a pointer type.
2021-07-09Pass type when creating loadNikita Popov-8/+11
This makes load generation compatible with opaque pointers. The generation of nontemporal copies still accesses the pointer element type, as fixing this requires more movement.
2021-07-08Adjust the threshold to look at the ABI, not just the sizeScott McMurray-12/+24
2021-07-08Stop generating `alloca`s+`memcmp` for simple array equalityScott McMurray-0/+26
2021-06-30Add support for leaf fn frame pointer eliminationSimonas Kazlauskas-1/+1
This PR adds ability for the target specifications to specify frame pointer emission type that's not just “always” or “whatever cg decides”. In particular there's a new mode that allows omission of the frame pointer for leaf functions (those that don't call any other functions). We then set this new mode for Aarch64-based Apple targets. Fixes #86196
2021-05-17rustc_codegen_ssa: append blocks to functions w/o creating a builder.Eduard-Mihai Burtescu-1/+2
2021-04-17Don't set `fast`(-math) for certain simd opsSimonas Kazlauskas-21/+19
`fast-math` implies things like functions not being able to accept as an argument or return as a result, say, `inf` which made these functions confusingly named or behaving incorrectly, depending on how you interpret it. Since the time when these intrinsics have been implemented the intrinsics user's (stdsimd) approach has changed significantly and so now it is required that these intrinsics operate normally rather than in "whatever" way. Fixes #84268
2021-04-14Add more SIMD math.h intrinsicsJubilee Young-2/+6
LLVM supports many functions from math.h in its IR. Many of these have single-instruction variants on various platforms. So, let's add them so std::arch can use them. Yes, exact comparison is intentional: rounding must always return a valid integer-equal value, except for inf/NAN.
2021-03-19correct macro namesSparrowLii-4/+4
2021-03-19Add simd_neg platform intrinsicSparrowLii-2/+21
2021-03-10Remove the -Zinsert-sideeffectSimonas Kazlauskas-9/+5
This removes all of the code we had in place to work-around LLVM's handling of forward progress. From this removal excluded is a workaround where we'd insert a `sideeffect` into clearly infinite loops such as `loop {}`. This code remains conditionally effective when the LLVM version is earlier than 12.0, which fixed the forward progress related miscompilations at their root.
2021-01-23Make declare_cfn more flexibleCaleb Zulawski-6/+8
2021-01-03Implement floating point SIMD intrinsics over all vector widths, and limit ↵Caleb Zulawski-78/+55
SIMD vector lengths.
2020-12-11use strip_prefix over slicing (clippy::manual_strip)Matthias Krüger-2/+2
2020-11-29Auto merge of #78863 - KodrAus:feat/simd-array, r=oli-obkbors-50/+55
Support repr(simd) on ADTs containing a single array field This is a squash and rebase of `@gnzlbg's` #63531 I've never actually written code in the compiler before so just fumbled my way around until it would build 😅 I imagine there'll be some work we need to do in `rustc_codegen_cranelift` too for this now, but might need some input from `@bjorn3` to know what that is. cc `@rust-lang/project-portable-simd` ----- This PR allows using `#[repr(simd)]` on ADTs containing a single array field: ```rust #[repr(simd)] struct S0([f32; 4]); #[repr(simd)] struct S1<const N: usize>([f32; N]); #[repr(simd)] struct S2<T, const N: usize>([T; N]); ``` This should allow experimenting with portable packed SIMD abstractions on nightly that make use of const generics.
2020-11-16compiler: fold by valueBastian Kauschke-3/+3
2020-11-13remove internal simd_size_and_ty from llvm backendAshley Mannix-34/+17
2020-11-10Changed unwrap_or to unwrap_or_else in some places.Nicholas-Baron-6/+8
The discussion seems to have resolved that this lint is a bit "noisy" in that applying it in all places would result in a reduction in readability. A few of the trivial functions (like `Path::new`) are fine to leave outside of closures. The general rule seems to be that anything that is obviously an allocation (`Box`, `Vec`, `vec![]`) should be in a closure, even if it is a 0-sized allocation.
2020-11-08Collapse all uses of `target.options.foo` into `target.foo`Vadim Petrochenkov-1/+1
with an eye on merging `TargetOptions` into `Target`. `TargetOptions` as a separate structure is mostly an implementation detail of `Target` construction, all its fields logically belong to `Target` and available from `Target` through `Deref` impls.
2020-11-08Support repr(simd) on ADTs containing a single array fieldgnzlbg-50/+72
This PR allows using `#[repr(simd)]` on ADTs containing a single array field: ```rust #[repr(simd)] struct S0([f32; 4]); #[repr(simd)] struct S1<const N: usize>([f32; N]); #[repr(simd)] struct S2<T, const N: usize>([T; N]); ``` This should allow experimenting with portable packed SIMD abstractions on nightly that make use of const generics.
2020-10-16Review commentsJack Huey-3/+2
2020-10-16Use map_bound(_ref) instead of Binder::bind when possibleJack Huey-12/+14
2020-10-16Auto merge of #77972 - Mark-Simulacrum:side-effect-loop, r=nagisabors-5/+5
Prevent miscompilation in trivial loop {} Ideally, we would want to handle a broader set of cases to fully fix the underlying bug here. That is currently relatively expensive at compile and runtime, so we don't do that for now. Performance results indicate this is not a major regression, if at all, so it should be safe to land. cc #28728
2020-10-15Prevent miscompilation in trivial loop {}Mark Rousskov-5/+5
Ideally, we would want to handle a broader set of cases to fully fix the underlying bug here. That is currently relatively expensive at compile and runtime, so we don't do that for now.
2020-10-15Replace target.target with target and target.ptr_width with target.pointer_widthest31-3/+3
Preparation for a subsequent change that replaces rustc_target::config::Config with its wrapped Target. On its own, this commit breaks the build. I don't like making build-breaking commits, but in this instance I believe that it makes review easier, as the "real" changes of this PR can be seen much more easily. Result of running: find compiler/ -type f -exec sed -i -e 's/target\.target\([)\.,; ]\)/target\1/g' {} \; find compiler/ -type f -exec sed -i -e 's/target\.target$/target/g' {} \; find compiler/ -type f -exec sed -i -e 's/target.ptr_width/target.pointer_width/g' {} \; ./x.py fmt
2020-10-03Support vectors with fewer than 8 elements for simd_select_bitmaskAmanieu d'Antras-3/+7
2020-09-15Extract some intrinsics out of rustc_codegen_llvmkhyperia-520/+7
A significant amount of intrinsics do not actually need backend-specific behaviors to be implemented, instead relying on methods already in rustc_codegen_ssa. So, extract those methods out to rustc_codegen_ssa, so that each backend doesn't need to reimplement the same code.
2020-09-04Change ty.kind to a methodLeSeulArtichaut-26/+26
2020-09-01Auto merge of #76071 - khyperia:configurable_to_immediate, r=eddybbors-2/+2
Make to_immediate/from_immediate configurable by backends `librustc_codegen_ssa` has the concept of an immediate vs. memory type, and `librustc_codegen_llvm` uses this distinction to implement `bool`s being `i8` in memory, and `i1` in immediate contexts. However, some of that implementation leaked into `codegen_ssa` when converting to/from immediate values. So, move those methods into builder traits, so that behavior can be configured by backends. This is useful if a backend is able to keep bools as bools, or, needs to do more trickery than just bools to bytes. (Note that there's already a large amount of things abstracted with "immediate types" - this is just bringing this particular thing in line to be abstracted as well) --- Pinging @eddyb since that's who I was talking about this change with when they suggested I submit a PR.
2020-08-30mv compiler to compiler/mark-0/+2241