Auto merge of #117771 - tmandry:fuchsia-gha, r=Mark-Simulacrum

Build Fuchsia in CI

Fittingly, when I first put this up it was failing due to discovering an ICE in clippy (looks like fixed in https://github.com/rust-lang/rust-clippy/pull/11760), probably more fallout from recent type system changes. Other recent regressions this would have caught include

- #117455 and #117493
- #117602

Originally we discussed basing this on cargotest, but they ended up not sharing anything. Fuchsia has its own tool to manage checkouts and its own build system. What it requires is a fully "install"ed toolchain with a host and fuchsia target. We share logic from the dist-various-2 builder to build the fuchsia target.

Right now this runs clippy and skips linking a bunch of targets, since most issues we catch are in the frontend. In theory we could probably get the build CPU time down quite a bit with this approach, but right now some linked targets are creeping into the dependencies anyway and we don't have a good way of preventing that yet.

The approach is basically to get a checkout at a pinned commit and then run a [script](https://fuchsia-review.git.corp.google.com/c/fuchsia/+/943833/6/scripts/rust/build_fuchsia_from_rust_ci.sh) at a predetermined location. I would like to update that pin every few weeks. Partial checkouts are used to minimize clone time, but we don't filter out prebuilt packages.

r? `@Mark-Simulacrum`

Based on discussion in [this Zulip thread](https://rust-lang.zulipchat.com/#narrow/stream/242791-t-infra/topic/Putting.20Fuchsia.20in.20crater).

2 files changed, 102 insertions, 0 deletions

diff --git a/src/ci/docker/scripts/build-fuchsia-toolchain.sh b/src/ci/docker/scripts/build-fuchsia-toolchain.sh
new file mode 100755
index 00000000000..beea2f522fd
--- /dev/null
+++ b/src/ci/docker/scripts/build-fuchsia-toolchain.sh
@@ -0,0 +1,60 @@
+#!/usr/bin/env bash
+
+set -ex
+source shared.sh
+
+FUCHSIA_SDK_URL=https://chrome-infra-packages.appspot.com/dl/fuchsia/sdk/core/linux-amd64
+FUCHSIA_SDK_ID=MrhQwtmP8CpZre-i_PNOREcThbUcrX3bA-45d6WQr-cC
+FUCHSIA_SDK_SHA256=32b850c2d98ff02a59adefa2fcf34e44471385b51cad7ddb03ee3977a590afe7
+FUCHSIA_SDK_USR_DIR=/usr/local/core-linux-amd64-fuchsia-sdk
+CLANG_DOWNLOAD_URL=\
+https://chrome-infra-packages.appspot.com/dl/fuchsia/third_party/clang/linux-amd64
+CLANG_DOWNLOAD_ID=Tpc85d1ZwSlZ6UKl2d96GRUBGNA5JKholOKe24sRDr0C
+CLANG_DOWNLOAD_SHA256=4e973ce5dd59c12959e942a5d9df7a19150118d03924a86894e29edb8b110ebd
+
+install_clang() {
+  mkdir -p clang_download
+  pushd clang_download > /dev/null
+
+  # Download clang+llvm
+  curl -LO "${CLANG_DOWNLOAD_URL}/+/${CLANG_DOWNLOAD_ID}"
+  echo "$(echo ${CLANG_DOWNLOAD_SHA256}) ${CLANG_DOWNLOAD_ID}" | sha256sum --check --status
+  unzip -qq ${CLANG_DOWNLOAD_ID} -d clang-linux-amd64
+
+  # Other dists currently depend on our Clang... moving into /usr/local for other
+  #  dist usage instead of a Fuchsia /usr/local directory
+  chmod -R 777 clang-linux-amd64/.
+  cp -a clang-linux-amd64/. /usr/local
+
+  # CFLAGS and CXXFLAGS env variables in main Dockerfile handle sysroot linking
+  for arch in x86_64 aarch64; do
+    for tool in clang clang++; do
+      ln -s /usr/local/bin/${tool} /usr/local/bin/${arch}-unknown-fuchsia-${tool}
+    done
+    ln -s /usr/local/bin/llvm-ar /usr/local/bin/${arch}-unknown-fuchsia-ar
+  done
+
+  popd > /dev/null
+  rm -rf clang_download
+}
+
+install_zircon_libs() {
+  mkdir -p zircon
+  pushd zircon > /dev/null
+
+  # Download Fuchsia SDK (with Zircon libs)
+  curl -LO "${FUCHSIA_SDK_URL}/+/${FUCHSIA_SDK_ID}"
+  echo "$(echo ${FUCHSIA_SDK_SHA256}) ${FUCHSIA_SDK_ID}" | sha256sum --check --status
+  unzip -qq ${FUCHSIA_SDK_ID} -d core-linux-amd64
+
+  # Moving SDK into Docker's user-space
+  mkdir -p ${FUCHSIA_SDK_USR_DIR}
+  chmod -R 777 core-linux-amd64/.
+  cp -r core-linux-amd64/* ${FUCHSIA_SDK_USR_DIR}
+
+  popd > /dev/null
+  rm -rf zircon
+}
+
+hide_output install_clang
+hide_output install_zircon_libs
diff --git a/src/ci/docker/scripts/shared.sh b/src/ci/docker/scripts/shared.sh
new file mode 100644
index 00000000000..9969659088d
--- /dev/null
+++ b/src/ci/docker/scripts/shared.sh
@@ -0,0 +1,42 @@
+#!/bin/false
+# shellcheck shell=bash
+
+# This file is intended to be sourced with `. shared.sh` or
+# `source shared.sh`, hence the invalid shebang and not being
+# marked as an executable file in git.
+
+function hide_output {
+  { set +x; } 2>/dev/null
+  on_err="
+echo ERROR: An error was encountered with the build.
+cat /tmp/build.log
+exit 1
+"
+  trap "$on_err" ERR
+  bash -c "while true; do sleep 30; echo \$(date) - building ...; done" &
+  PING_LOOP_PID=$!
+  "$@" &> /tmp/build.log
+  trap - ERR
+  kill $PING_LOOP_PID
+  set -x
+}
+
+# See https://unix.stackexchange.com/questions/82598
+# Duplicated in src/ci/shared.sh
+function retry {
+  echo "Attempting with retry:" "$@"
+  local n=1
+  local max=5
+  while true; do
+    "$@" && break || {
+      if [[ $n -lt $max ]]; then
+        sleep $n  # don't retry immediately
+        ((n++))
+        echo "Command failed. Attempt $n/$max:"
+      else
+        echo "The command has failed after $n attempts."
+        return 1
+      fi
+    }
+  done
+}