Auto merge of #89499 - Mark-Simulacrum:with-llvm-13, r=nikic

Split out LLVM PGO step and use clang 13 to compile LLVM

We're seeing a PGO version mismatch error in CI logs:

    LLVM Profile Error: Runtime and instrumentation version mismatch : expected 5, but get 7

which is likely due to the version bumped here differing from that used by
rustc.

This PR fixes this by splitting out the PGO step for LLVM into a separate phase of the pgo.sh script, which nets no change to performance (see [these results](https://perf.rust-lang.org/compare.html?start=c34ac8747ca96d09cb08b8f5adddead826e77c06&end=e272c2af45f40c74dab83948235903ffbe3ad57f)). Then, it follows that up with an upgrade to LLVM/clang version 13 as our bootstrap compiler, which yields the performance improvements for this PR -- around 5%. This depends on the first step here, because otherwise we end up somehow clobbering or otherwise hurting our ability to effectively collect performance data, yielding reductions in performance for a subset of benchmarks -- it is not clear what the cause here was precisely, but the split only costs ~10 minutes and seems worthwhile.

2 files changed, 31 insertions, 9 deletions

diff --git a/src/ci/docker/host-x86_64/dist-x86_64-linux/build-clang.sh b/src/ci/docker/host-x86_64/dist-x86_64-linux/build-clang.sh
index ed5edfec4e1..562be752f84 100755
--- a/src/ci/docker/host-x86_64/dist-x86_64-linux/build-clang.sh
+++ b/src/ci/docker/host-x86_64/dist-x86_64-linux/build-clang.sh
@@ -4,7 +4,7 @@ set -ex
 
 source shared.sh
 
-LLVM=llvmorg-12.0.1
+LLVM=llvmorg-13.0.0
 
 mkdir llvm-project
 cd llvm-project
diff --git a/src/ci/pgo.sh b/src/ci/pgo.sh
index e35e3e670cc..29ef13a60fb 100755
--- a/src/ci/pgo.sh
+++ b/src/ci/pgo.sh
@@ -4,9 +4,13 @@ set -euxo pipefail
 
 rm -rf /tmp/rustc-pgo
 
+# We collect LLVM profiling information and rustc profiling information in
+# separate phases. This increases build time -- though not by a huge amount --
+# but prevents any problems from arising due to different profiling runtimes
+# being simultaneously linked in.
+
 python3 ../x.py build --target=$PGO_HOST --host=$PGO_HOST \
     --stage 2 library/std \
-    --rust-profile-generate=/tmp/rustc-pgo \
     --llvm-profile-generate
 
 # Profile libcore compilation in opt-level=0 and opt-level=3
@@ -15,6 +19,29 @@ RUSTC_BOOTSTRAP=1 ./build/$PGO_HOST/stage2/bin/rustc --edition=2018 \
 RUSTC_BOOTSTRAP=1 ./build/$PGO_HOST/stage2/bin/rustc --edition=2018 \
     --crate-type=lib -Copt-level=3 ../library/core/src/lib.rs
 
+# Merge the profile data we gathered for LLVM
+# Note that this uses the profdata from the clang we used to build LLVM,
+# which likely has a different version than our in-tree clang.
+/rustroot/bin/llvm-profdata \
+    merge -o /tmp/llvm-pgo.profdata ./build/$PGO_HOST/llvm/build/profiles
+
+# Rustbuild currently doesn't support rebuilding LLVM when PGO options
+# change (or any other llvm-related options); so just clear out the relevant
+# directories ourselves.
+rm -r ./build/$PGO_HOST/llvm ./build/$PGO_HOST/lld
+
+# Okay, LLVM profiling is done, switch to rustc PGO.
+
+python3 ../x.py build --target=$PGO_HOST --host=$PGO_HOST \
+    --stage 2 library/std \
+    --rust-profile-generate=/tmp/rustc-pgo
+
+# Profile libcore compilation in opt-level=0 and opt-level=3
+RUSTC_BOOTSTRAP=1 ./build/$PGO_HOST/stage2/bin/rustc --edition=2018 \
+    --crate-type=lib ../library/core/src/lib.rs
+RUSTC_BOOTSTRAP=1 ./build/$PGO_HOST/stage2/bin/rustc --edition=2018 \
+    --crate-type=lib -Copt-level=3 ../library/core/src/lib.rs
+
 cp -r /tmp/rustc-perf ./
 chown -R $(whoami): ./rustc-perf
 cd rustc-perf
@@ -46,18 +73,13 @@ cd /checkout/obj
 ./build/$PGO_HOST/llvm/bin/llvm-profdata \
     merge -o /tmp/rustc-pgo.profdata /tmp/rustc-pgo
 
-# Merge the profile data we gathered for LLVM
-# Note that this uses the profdata from the clang we used to build LLVM,
-# which likely has a different version than our in-tree clang.
-/rustroot/bin/llvm-profdata \
-    merge -o /tmp/llvm-pgo.profdata ./build/$PGO_HOST/llvm/build/profiles
-
 # Rustbuild currently doesn't support rebuilding LLVM when PGO options
 # change (or any other llvm-related options); so just clear out the relevant
 # directories ourselves.
 rm -r ./build/$PGO_HOST/llvm ./build/$PGO_HOST/lld
 
-# This produces the actual final set of artifacts.
+# This produces the actual final set of artifacts, using both the LLVM and rustc
+# collected profiling data.
 $@ \
     --rust-profile-use=/tmp/rustc-pgo.profdata \
     --llvm-profile-use=/tmp/llvm-pgo.profdata