summary refs log tree commit diff
path: root/compiler/rustc_codegen_cranelift/src/intrinsics
AgeCommit message (Collapse)AuthorLines
2023-02-27Unify all validity check intrinsicsNilstrieb-49/+31
Also merges the inhabitedness check into the query to further unify the code paths.
2023-02-27Rollup merge of #108364 - Nilstrieb:validity-checks-refactor, r=compiler-errorsMatthias Krüger-3/+7
Unify validity checks into a single query Previously, there were two queries to check whether a type allows the 0x01 or zeroed bitpattern. I am planning on adding a further initness to check in #100423, truly uninit for MaybeUninit, which would make this three queries. This seems overkill for such a small feature, so this PR unifies them into one. I am not entirely happy with the naming and key type and open for improvements. r? oli-obk
2023-02-24Rename many interner functions.Nicholas Nethercote-1/+1
(This is a large commit. The changes to `compiler/rustc_middle/src/ty/context.rs` are the most important ones.) The current naming scheme is a mess, with a mix of `_intern_`, `intern_` and `mk_` prefixes, with little consistency. In particular, in many cases it's easy to use an iterator interner when a (preferable) slice interner is available. The guiding principles of the new naming system: - No `_intern_` prefixes. - The `intern_` prefix is for internal operations. - The `mk_` prefix is for external operations. - For cases where there is a slice interner and an iterator interner, the former is `mk_foo` and the latter is `mk_foo_from_iter`. Also, `slice_interners!` and `direct_interners!` can now be `pub` or non-`pub`, which helps enforce the internal/external operations division. It's not perfect, but I think it's a clear improvement. The following lists show everything that was renamed. slice_interners - const_list - mk_const_list -> mk_const_list_from_iter - intern_const_list -> mk_const_list - substs - mk_substs -> mk_substs_from_iter - intern_substs -> mk_substs - check_substs -> check_and_mk_substs (this is a weird one) - canonical_var_infos - intern_canonical_var_infos -> mk_canonical_var_infos - poly_existential_predicates - mk_poly_existential_predicates -> mk_poly_existential_predicates_from_iter - intern_poly_existential_predicates -> mk_poly_existential_predicates - _intern_poly_existential_predicates -> intern_poly_existential_predicates - predicates - mk_predicates -> mk_predicates_from_iter - intern_predicates -> mk_predicates - _intern_predicates -> intern_predicates - projs - intern_projs -> mk_projs - place_elems - mk_place_elems -> mk_place_elems_from_iter - intern_place_elems -> mk_place_elems - bound_variable_kinds - mk_bound_variable_kinds -> mk_bound_variable_kinds_from_iter - intern_bound_variable_kinds -> mk_bound_variable_kinds direct_interners - region - intern_region (unchanged) - const - mk_const_internal -> intern_const - const_allocation - intern_const_alloc -> mk_const_alloc - layout - intern_layout -> mk_layout - adt_def - intern_adt_def -> mk_adt_def_from_data (unusual case, hard to avoid) - alloc_adt_def(!) -> mk_adt_def - external_constraints - intern_external_constraints -> mk_external_constraints Other - type_list - mk_type_list -> mk_type_list_from_iter - intern_type_list -> mk_type_list - tup - mk_tup -> mk_tup_from_iter - intern_tup -> mk_tup
2023-02-23Unify validity checks into a single queryNilstrieb-3/+7
Previously, there were two queries to check whether a type allows the 0x01 or zeroed bitpattern. I am planning on adding a further initness to check, truly uninit for MaybeUninit, which would make this three queries. This seems overkill for such a small feature, so this PR unifies them into one.
2023-02-18Stop implementing _with_overflow intrinsics in codegen backends.Camille GILLOT-14/+0
2023-02-17Replace `mk_foo` calls with `infer_foo` where possible.Nicholas Nethercote-1/+1
There are several `mk_foo`/`intern_foo` pairs, where the former takes an iterator and the latter takes a slice. (This naming convention is bad, but that's a fix for another PR.) This commit changes several `mk_foo` occurrences into `intern_foo`, avoiding the need for some `.iter()`/`.into_iter()` calls. Affected cases: - mk_type_list - mk_tup - mk_substs - mk_const_list
2023-02-15Auto merge of #108012 - compiler-errors:issue-107999, r=oli-obkbors-3/+10
Don't ICE in `might_permit_raw_init` if reference is polymorphic Emitting optimized MIR for a polymorphic function may require computing layout of a type that isn't (yet) known. This happens in the instcombine pass, for example. Let's fail gracefully in that condition. cc `@saethlin` fixes #107999
2023-02-14Make permit_uninit/zero_init fallibleMichael Goulet-3/+10
2023-02-14s/eval_usize/eval_target_usize/ for clarityOli Scherer-2/+2
2023-02-09Merge commit '7d53619064ab7045c383644cb445052d2a3d46db' into ↵bjorn3-19/+12
sync_cg_clif-2023-02-09
2023-01-27Auto merge of #107269 - bjorn3:sync_cg_clif-2023-01-24, r=bjorn3bors-15/+122
Sync rustc_codegen_cranelift For cg_clif itself there have been a couple of bug fixes since the last sync, a Cranelift update and implemented all remaining simd platform intrinsics used by `std::simd`. (`std::arch` still misses a lot though) Most of the diff is from reworking of the cg_clif build system though. r? `@ghost` `@rustbot` label +A-codegen +A-cranelift +T-compiler
2023-01-24Merge commit '598f0909568a51de8a2d1148f55a644fd8dffad0' into ↵bjorn3-15/+122
sync_cg_clif-2023-01-24
2023-01-23Thread a ParamEnv down to might_permit_raw_initBen Kimock-2/+5
2022-12-21Auto merge of #105613 - Nilstrieb:rename-assert_uninit_valid, r=RalfJungbors-2/+4
Rename `assert_uninit_valid` intrinsic It's not about "uninit" anymore but about "filling with 0x01 bytes" so the name should at least try to reflect that. This is actually not fully correct though, as it does still panic for all uninit with `-Zstrict-init-checks`. I'm not sure what the best way is to deal with that not causing confusion. I guess we could just remove the flag? I don't think having it makes a lot of sense anymore with the direction that we have chose to go. It could be relevant again if #100423 lands so removing it may be a bit over eager. r? `@RalfJung`
2022-12-14Merge commit '2bb3996244cf1b89878da9e39841e9f6bf061602' into ↵bjorn3-248/+450
sync_cg_clif-2022-12-14
2022-12-13Rename `assert_uninit_valid` intrinsicNilstrieb-2/+4
It's not about "uninit" anymore but about "filling with 0x01 bytes" so the name should at least try to reflect that.
2022-10-23Merge commit '266e96785ab71834b917bf474f130a6d8fdecd4b' into ↵bjorn3-39/+158
sync_cg_clif-2022-10-23
2022-09-10Auto merge of #101483 - oli-obk:guaranteed_opt, r=fee1-deadbors-8/+1
The `<*const T>::guaranteed_*` methods now return an option for the unknown case cc https://github.com/rust-lang/rust/issues/53020#issuecomment-1236932443 I chose `0` for "not equal" and `1` for "equal" and left `2` for the unknown case so backends can just forward to raw pointer equality and it works ✨ r? `@fee1-dead` or `@lcnr` cc `@rust-lang/wg-const-eval`
2022-09-09The `<*const T>::guaranteed_*` methods now return an option for the unknown caseOli Scherer-8/+1
2022-09-06Lower the assume intrinsic to a MIR statementOli Scherer-3/+0
2022-08-31Correct typoDezhi Wu-1/+1
2022-08-31Fix a bunch of typoDezhi Wu-1/+1
This PR will fix some typos detected by [typos]. I only picked the ones I was sure were spelling errors to fix, mostly in the comments. [typos]: https://github.com/crate-ci/typos
2022-08-30Rollup merge of #101101 - RalfJung:read-pointer-as-bytes, r=oli-obkDylan DPC-1/+4
interpret: make read-pointer-as-bytes a CTFE-only error with extra information Next step in the reaction to https://github.com/rust-lang/rust/issues/99923. Also teaches Miri to implicitly strip provenance in more situations when transmuting pointers to integers, which fixes https://github.com/rust-lang/miri/issues/2456. Pointer-to-int transmutation during CTFE now produces a message like this: ``` = help: this code performed an operation that depends on the underlying bytes representing a pointer = help: the absolute address of a pointer is not known at compile-time, so such operations are not supported ``` r? ``@oli-obk``
2022-08-28Auto merge of #96946 - WaffleLapkin:ptr_mask, r=scottmcmbors-0/+7
Add pointer masking convenience functions This PR adds the following public API: ```rust impl<T: ?Sized> *const T { fn mask(self, mask: usize) -> *const T; } impl<T: ?Sized> *mut T { fn mask(self, mask: usize) -> *const T; } // mod intrinsics fn mask<T>(ptr: *const T, mask: usize) -> *const T ``` This is equivalent to `ptr.map_addr(|a| a & mask)` but also uses a cool llvm intrinsic. Proposed in https://github.com/rust-lang/rust/pull/95643#issuecomment-1121562352 cc `@Gankra` `@scottmcm` `@RalfJung` r? rust-lang/libs-api
2022-08-27interpret: make read-pointer-as-bytes *always* work in MiriRalf Jung-1/+4
and show some extra information when it happens in CTFE
2022-08-24Merge commit 'e9d1a0a7b0b28dd422f1a790ccde532acafbf193' into ↵bjorn3-25/+58
sync_cg_clif-2022-08-24
2022-08-21fix cg craneliftWaffle Maybe-0/+2
Co-authored-by: bjorn3 <17426603+bjorn3@users.noreply.github.com>
2022-08-21use shorter `ptr_mask` impl in cg craneliftWaffle Maybe-3/+1
Co-authored-by: bjorn3 <bjorn3@users.noreply.github.com>
2022-08-21Add pointer masking convenience functionsMaybe Waffle-0/+7
This commit adds the following functions all of which have a signature `pointer, usize -> pointer`: - `<*mut T>::mask` - `<*const T>::mask` - `intrinsics::ptr_mask` These functions are equivalent to `.map_addr(|a| a & mask)` but they utilize `llvm.ptrmask` llvm intrinsic. *masks your pointers*
2022-07-26Merge commit 'd3a2366ee877075c59b38bd8ced55f224fc7ef51' into ↵bjorn3-461/+705
sync_cg_clif-2022-07-26
2022-07-25Merge commit 'c19edfd71a1d0ddef86c2c67fdb40718d40a72b4' into ↵bjorn3-2/+20
sync_cg_clif-2022-07-25
2022-07-20Implement vtable_size and vtable_align intrinsics for cg_clifbjorn3-0/+10
2022-07-14Use constant eval to do strict validity checks5225225-13/+2
2022-07-13Rename `debugging_opts` to `unstable_opts`Joshua Nelson-2/+2
This is no longer used only for debugging options (e.g. `-Zoutput-width`, `-Zallow-features`). Rename it to be more clear.
2022-05-24Add flag for stricter checks on uninit/zeroed5225225-2/+13
2022-05-23Refactor call terminator to always hold a destination placeJakob Degen-28/+34
2022-05-15Merge commit '63734fcdd718cca089f84c42f3a42c0096cfd431' into ↵bjorn3-31/+31
sync_cg_clif-2022-05-15
2022-05-11Rename `unsigned_offset_from` to `sub_ptr`Scott McMurray-1/+1
2022-05-11Add `unsigned_offset_from` on pointersScott McMurray-3/+10
Like we have `add`/`sub` which are the `usize` version of `offset`, this adds the `usize` equivalent of `offset_from`. Like how `.add(d)` replaced a whole bunch of `.offset(d as isize)`, you can see from the changes here that it's fairly common that code actually knows the order between the pointers and *wants* a `usize`, not an `isize`. As a bonus, this can do `sub nuw`+`udiv exact`, rather than `sub`+`sdiv exact`, which can be optimized slightly better because it doesn't have to worry about negatives. That's why the slice iterators weren't using `offset_from`, though I haven't updated that code in this PR because slices are so perf-critical that I'll do it as its own change. This is an intrinsic, like `offset_from`, so that it can eventually be allowed in CTFE. It also allows checking the extra safety condition -- see the test confirming that CTFE catches it if you pass the pointers in the wrong order.
2022-04-22Merge commit 'f2cdd4a78d89c009342197cf5844a21f8aa813df' into ↵bjorn3-34/+19
sync_cg_clif-2022-04-22
2022-03-20Merge commit '370c397ec9169809e5ad270079712e0043514240' into ↵bjorn3-37/+56
sync_cg_clif-2022-03-20
2022-03-07Clarify `Layout` interning.Nicholas Nethercote-1/+1
`Layout` is another type that is sometimes interned, sometimes not, and we always use references to refer to it so we can't take any advantage of the uniqueness properties for hashing or equality checks. This commit renames `Layout` as `LayoutS`, and then introduces a new `Layout` that is a newtype around an `Interned<LayoutS>`. It also interns more layouts than before. Previously layouts within layouts (via the `variants` field) were never interned, but now they are. Hence the lifetime on the new `Layout` type. Unlike other interned types, these ones are in `rustc_target` instead of `rustc_middle`. This reflects the existing structure of the code, which does layout-specific stuff in `rustc_target` while `TyAndLayout` is generic over the `Ty`, allowing the type-specific stuff to occur in `rustc_middle`. The commit also adds a `HashStable` impl for `Interned`, which was needed. It hashes the contents, unlike the `Hash` impl which hashes the pointer.
2022-03-07Introduce `ConstAllocation`.Nicholas Nethercote-1/+1
Currently some `Allocation`s are interned, some are not, and it's very hard to tell at a use point which is which. This commit introduces `ConstAllocation` for the known-interned ones, which makes the division much clearer. `ConstAllocation::inner()` is used to get the underlying `Allocation`. In some places it's natural to use an `Allocation`, in some it's natural to use a `ConstAllocation`, and in some places there's no clear choice. I've tried to make things look as nice as possible, while generally favouring `ConstAllocation`, which is the type that embodies more information. This does require quite a few calls to `inner()`. The commit also tweaks how `PartialOrd` works for `Interned`. The previous code was too clever by half, building on `T: Ord` to make the code shorter. That caused problems with deriving `PartialOrd` and `Ord` for `ConstAllocation`, so I changed it to build on `T: PartialOrd`, which is slightly more verbose but much more standard and avoided the problems.
2022-02-23Merge commit '35d9c6bf256968e1b40e0d554607928bdf9cebea' into ↵bjorn3-525/+584
sync_cg_clif-2022-02-23
2022-02-16Move ty::print methods to Drop-based scope guardsMark Rousskov-15/+21
2022-01-19remove `is_noop`lcnr-1/+1
2021-12-20Merge commit '97e504549371d7640cf011d266e3c17394fdddac' into ↵bjorn3-19/+46
sync_cg_clif-2021-12-20
2021-10-12Add const_eval_select intrinsicDeadbeef-3/+1
2021-09-12Auto merge of #88839 - nbdd0121:alignof, r=nagisabors-1/+1
Introduce NullOp::AlignOf This PR introduces `Rvalue::NullaryOp(NullOp::AlignOf, ty)`, which will be lowered from `align_of`, similar to `size_of` lowering to `Rvalue::NullaryOp(NullOp::SizeOf, ty)`. The changes are originally part of #88700 but since it's not dependent on other changes and could have performance impact on its own, it's separated into its own PR.
2021-09-13Introduce NullOp::AlignOfGary Guo-1/+1