Rollup merge of #135481 - Zalathar:node-flow, r=oli-obk

coverage: Completely overhaul counter assignment, using node-flow graphs

The existing code for choosing where to put physical counter-increments gets the job done, but is very ad-hoc and hard to modify without introducing tricky regressions.

This PR replaces all of that with a more principled approach, based on the algorithm described in "Optimal measurement points for program frequency counts" (Knuth & Stevenson, 1973).

---

We start by ensuring that our graph has “balanced flow”, i.e. each node's flow (execution count) is equal to the sum of all its in-edge flows, and equal to the sum of all its out-edge flows. That isn't naturally true of control-flow graphs, so we introduce a wrapper type `BalancedFlowGraph` to fix that by introducing synthetic nodes and edges as needed.

Once our graph has balanced flow, the next step is to create another view of that graph in which each node's successors have all been merged into one “supernode”. Consequently, each node's out-edges can be coalesced into a single out-edge to one of those supernodes. Because of the balanced-flow property, the flow of that coalesced edge is equal to the flow of the original node.

Having expressed all of our node flows as edge flows, we can then analyze node flows using techniques for analyzing edge flows. We incrementally build a spanning tree over the merged supernodes, such that each new edge in the spanning tree represents a node whose flow can be computed from that of other nodes.

When this is done, we end up with a list of “counter terms” for each node, describing which nodes need physical counters, and how the remaining nodes can have their flow calculated by adding and subtracting those physical counters.

---

The re-blessed coverage tests show that this results in modest or major improvements for our test programs. Some tests need fewer physical counters, some tests need fewer expression nodes for the same number of physical counters, and some tests show striking reductions in both.

3 files changed, 53 insertions, 0 deletions

diff --git a/compiler/rustc_data_structures/src/graph/iterate/mod.rs b/compiler/rustc_data_structures/src/graph/iterate/mod.rs
index ecda7d3fba8..7b4573d7a84 100644
--- a/compiler/rustc_data_structures/src/graph/iterate/mod.rs
+++ b/compiler/rustc_data_structures/src/graph/iterate/mod.rs
@@ -125,6 +125,16 @@ where
     pub fn visited(&self, node: G::Node) -> bool {
         self.visited.contains(node)
     }
+
+    /// Returns a reference to the set of nodes that have been visited, with
+    /// the same caveats as [`Self::visited`].
+    ///
+    /// When incorporating the visited nodes into another bitset, using bulk
+    /// operations like `union` or `intersect` can be more efficient than
+    /// processing each node individually.
+    pub fn visited_set(&self) -> &DenseBitSet<G::Node> {
+        &self.visited
+    }
 }
 
 impl<G> std::fmt::Debug for DepthFirstSearch<G>
diff --git a/compiler/rustc_data_structures/src/graph/mod.rs b/compiler/rustc_data_structures/src/graph/mod.rs
index 103ddd917bf..92035e8bc48 100644
--- a/compiler/rustc_data_structures/src/graph/mod.rs
+++ b/compiler/rustc_data_structures/src/graph/mod.rs
@@ -4,6 +4,7 @@ pub mod dominators;
 pub mod implementation;
 pub mod iterate;
 mod reference;
+pub mod reversed;
 pub mod scc;
 pub mod vec_graph;
 
diff --git a/compiler/rustc_data_structures/src/graph/reversed.rs b/compiler/rustc_data_structures/src/graph/reversed.rs
new file mode 100644
index 00000000000..9b726deaa15
--- /dev/null
+++ b/compiler/rustc_data_structures/src/graph/reversed.rs
@@ -0,0 +1,42 @@
+use crate::graph::{DirectedGraph, Predecessors, Successors};
+
+/// View that reverses the direction of edges in its underlying graph, so that
+/// successors become predecessors and vice-versa.
+///
+/// Because of `impl<G: Graph> Graph for &G`, the underlying graph can be
+/// wrapped by-reference instead of by-value if desired.
+#[derive(Clone, Copy, Debug)]
+pub struct ReversedGraph<G> {
+    pub inner: G,
+}
+
+impl<G> ReversedGraph<G> {
+    pub fn new(inner: G) -> Self {
+        Self { inner }
+    }
+}
+
+impl<G: DirectedGraph> DirectedGraph for ReversedGraph<G> {
+    type Node = G::Node;
+
+    fn num_nodes(&self) -> usize {
+        self.inner.num_nodes()
+    }
+}
+
+// Implementing `StartNode` is not possible in general, because the start node
+// of an underlying graph is instead an _end_ node in the reversed graph.
+// But would be possible to define another wrapper type that adds an explicit
+// start node to its underlying graph, if desired.
+
+impl<G: Predecessors> Successors for ReversedGraph<G> {
+    fn successors(&self, node: Self::Node) -> impl Iterator<Item = Self::Node> {
+        self.inner.predecessors(node)
+    }
+}
+
+impl<G: Successors> Predecessors for ReversedGraph<G> {
+    fn predecessors(&self, node: Self::Node) -> impl Iterator<Item = Self::Node> {
+        self.inner.successors(node)
+    }
+}