| Age | Commit message (Collapse) | Author | Lines |
|---|
|
Don't store locals that have been moved from in generators
This avoids reserving storage in generators for locals that are moved
out of (and not re-initialized) prior to yield points. Fixes #59123.
This adds a new dataflow analysis, `RequiresStorage`, to determine whether the storage of a local can be destroyed without being observed by the program. The rules are:
1. StorageLive(x) => mark x live
2. StorageDead(x) => mark x dead
3. If a local is moved from, _and has never had its address taken_, mark it dead
4. If (any part of) a local is initialized, mark it live'
This is used to determine whether to save a local in the generator object at all, as well as which locals can be overlapped in the generator layout.
Here's the size in bytes of all testcases included in the change, before and after the change:
async fn test |Size before |Size after
-----------------|------------|----------
single | 1028 | 1028
single_with_noop | 2056 | 1032
joined | 5132 | 3084
joined_with_noop | 8208 | 3084
generator test |Size before |Size after
----------------------------|------------|----------
move_before_yield | 1028 | 1028
move_before_yield_with_noop | 2056 | 1032
overlap_move_points | 3080 | 2056
## Future work
Note that there is a possible extension to this optimization, which modifies rule 3 to read: "If a local is moved from, _**and either has never had its address taken, or is Freeze and has never been mutably borrowed**_, mark it dead." This was discussed at length in #59123 and then #61849. Because this would cause some behavior to be UB which was not UB before, it's a step that needs to be taken carefully.
A more immediate priority for me is inlining `std::mem::size_of_val(&x)` so it becomes apparent that the address of `x` is not taken. This way, using `size_of_val` to look at the size of your inner futures does not affect the size of your outer future.
cc @cramertj @eddyb @Matthias247 @nikomatsakis @RalfJung @Zoxc
|
|
|
|
Use a more efficient iteration order for forward dataflow
Currently, dataflow begins by visiting each block in order of ID (`BasicBlock(0)`, `BasicBlock(1)`, etc.). This PR changes that initial iteration to reverse post-order (see [this blog post](https://eli.thegreenplace.net/2015/directed-graph-traversal-orderings-and-applications-to-data-flow-analysis/#data-flow-analysis) for more info). This ensures that the effects of all predecessors will be applied before a basic block is visited if the CFG has no back-edges, and should result in less total iterations even when back-edges exist. This should not change the results of dataflow analysis.
The current ordering for basic blocks may be pretty close to RPO already--`BasicBlock(0)` is already the start block--in which case the cost of doing the traversal up front will outweigh the efficiency gains.
A perf run is needed to check this.
r? @pnkfelix (I think).
|
|
|
|
|
|
Currently, dataflow begins by visiting each block in order of ID
(`BasicBlock(0)`, `BasicBlock(1)`, etc.). This PR changes that initial
iteration to reverse post-order. This ensures that the effects of all
predecessors will be applied before a basic block is visited if the CFG
has no back-edges, and should result in less total iterations even when
back-edges exist. This should not change the results of dataflow
analysis.
The current ordering for basic blocks is pretty close to RPO
already--`BasicBlock(0)` is already the start block, so the gains from
this are pretty small, especially since we need to do an extra traversal
up front.
Note that some basic blocks are unreachable from the `START_BLOCK`
during dataflow. We add these blocks to the work queue as well to
preserve the original behavior.
|
|
This avoids reserving storage in generators for locals that are moved
out of (and not re-initialized) prior to yield points.
|
|
|
|
|
|
|
|
rustc_mir: Hide initial block state when defining transfer functions
This PR addresses [this FIXME](https://github.com/rust-lang/rust/blob/2887008e0ce0824be4e0e9562c22ea397b165c97/src/librustc_mir/dataflow/mod.rs#L594-L596).
This makes `sets.on_entry` inaccessible in `{before_,}{statement,terminator}_effect`. This field was meant to allow implementors of `BitDenotation` to access the initial state for each block (optionally with the effect of all previous statements applied via `accumulates_intrablock_state`) while defining transfer functions. However, the ability to set the initial value for the entry set of each basic block (except for START_BLOCK) no longer exists. As a result, this functionality is mostly useless, and when it *was* used it was used erroneously (see #62007).
Since `on_entry` is now useless, we can also remove `BlockSets`, which held the `gen`, `kill`, and `on_entry` bitvectors and replace it with a `GenKill` struct. Variables of this type are called `trans` since they represent a transfer function. `GenKill`s are stored contiguously in `AllSets`, which reduces the number of bounds checks and may improve cache performance: one is almost never accessed without the other.
Replacing `BlockSets` with `GenKill` allows us to define some new helper functions which streamline dataflow iteration and the dataflow-at-location APIs. Notably, `state_for_location` used a subtle side-effect of the `kill`/`kill_all` setters to apply the transfer function, and could be incorrect if a transfer function depended on effects of previous statements in the block on `gen_set`.
Additionally, this PR merges `BitSetOperator` and `InitialFlow` into one trait. Since the value of `InitialFlow` defines the semantics of the `join` operation, there's no reason to have seperate traits for each. We can add a default impl of `join` which branches based on `BOTTOM_VALUE`. This should get optimized away.
|
|
Since the value of `InitialFlow` defines the semantics of the `join`
operation, there's no reason to have seperate traits for each. We can
add a default impl of `join` which branches based on `BOTTOM_VALUE`.
This should get optimized away.
|
|
This commit makes `sets.on_entry` inaccessible in
`{before_,}{statement,terminator}_effect`. This field was meant to allow
implementors of `BitDenotation` to access the initial state for each
block (optionally with the effect of all previous statements applied via
`accumulates_intrablock_state`) while defining transfer functions.
However, the ability to set the initial value for the entry set of each
basic block (except for START_BLOCK) no longer exists. As a result, this
functionality is mostly useless, and when it *was* used it was used
erroneously (see #62007).
Since `on_entry` is now useless, we can also remove `BlockSets`, which
held the `gen`, `kill`, and `on_entry` bitvectors and replace it with a
`GenKill` struct. Variables of this type are called `trans` since they
represent a transfer function. `GenKill`s are stored contiguously in
`AllSets`, which reduces the number of bounds checks and may improve
cache performance: one is almost never accessed without the other.
Replacing `BlockSets` with `GenKill` allows us to define some new helper
functions which streamline dataflow iteration and the
dataflow-at-location APIs. Notably, `state_for_location` used a subtle
side-effect of the `kill`/`kill_all` setters to apply the transfer
function, and could be incorrect if a transfer function depended on
effects of previous statements in the block on `gen_set`.
|
|
|
|
Resolves #62007.
Due to a bug, the previous version of this check did not actually kill
any conflicting borrows unless the borrowed place had no projections.
Specifically, `entry_set` will always be empty when `statement_effect`
is called. It does not contain the set of borrows which are live at this
point in the program.
|
|
Remove unnecessary lift calls
Note that some of these might be useful for sanity checking that there's no infer types or regions.
r? @eddyb
|
|
|
|
|
|
|
|
Optimize matches
Attempt to fix or improve #60571
This is breaking some diagnostics because the MIR for match arms isn't in source order any more.
cc @centril
|
|
|
|
|
|
|
|
We never have more than one imaginary target, so we have no reason for
a `Vec`
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Emit StorageDead along unwind paths for generators
Completion of the work done in #60840. That PR made a change to implicitly consider a local `StorageDead` after Drop, but that was incorrect for DropAndReplace (see also #61060 which tried to fix this in a different way).
This finally enables the optimization implemented in #60187.
r? @eddyb
cc @Zoxc @cramertj @RalfJung
|
|
This reverts commit dd2eabc49d415dd30cea0953df5d7659d4d9440f.
|
|
Make MIR drop terminators borrow the dropped location
r? @eddyb
cc @tmandry
|
|
Rename Place::local to Place::local_or_deref_local
r? @oli-obk
|
|
The commit should have changed comments as well.
At the time of writting, it passes the tidy and check tool.
Revisions asked by eddyb :
- Renamed of all the occurences of {visit/super}_mir
- Renamed test structures `CachedMir` to `Cached`
Fixing the missing import on `AggregateKind`
|
|
|
|
|
|
|
|
Rollup of 6 pull requests
Successful merges:
- #61092 (Make sanitize_place iterate instead of recurse)
- #61093 (Make borrow_of_local_data iterate instead of recurse)
- #61094 (Make find_local iterate instead of recurse)
- #61099 (Make ignore_borrow iterate instead of recurse)
- #61103 (Make find iterate instead of recurse)
- #61104 (Make eval_place_to_op iterate instead of recurse)
Failed merges:
r? @ghost
|
|
Make find iterate instead of recurse
r? @oli-obk
|
|
|
|
Once upon a time (commit 9bd35c07c26) there were two kinds of
projection: one for places, and one for constants. It therefore made
sense to share the `Projection` struct for both. Although the different
use-cases used different concrete types, sharing was made possible by
type-parameterisation of `Projection`.
Since then, however, the usage of projections in constants has
disappeared, meaning that (forgetting lifetimes for a moment) the
parameterised type is only every instantiated under one guise. So it may
as well be a concrete type.
|
|
|
|
Add match arm scopes and other scope fixes
* Add drop and lint scopes for match arms.
* Lint attributes are now respected on match arms.
* Make sure we emit a StorageDead if we diverge when initializing a temporary.
* Adjust MIR pretty printing of scopes for locals.
* Don't generate duplicate lint scopes for `let statements`.
* Add some previously missing fake borrows for matches.
closes #46525
cc @rust-lang/compiler
|
|
|
|
|
|
|
|
|
