diff options
| -rw-r--r-- | src/doc/rustc-dev-guide/src/tests/ci.md | 24 |
1 files changed, 20 insertions, 4 deletions
diff --git a/src/doc/rustc-dev-guide/src/tests/ci.md b/src/doc/rustc-dev-guide/src/tests/ci.md index 9dde407895e..a4b22392f19 100644 --- a/src/doc/rustc-dev-guide/src/tests/ci.md +++ b/src/doc/rustc-dev-guide/src/tests/ci.md @@ -322,7 +322,7 @@ Our CI workflow uses various caching mechanisms, mainly for two things: ### Docker images caching The Docker images we use to run most of the Linux-based builders take a *long* -time to fully build. To speed up the build, we cache it using [Docker registry +time to fully build. To speed up the build, we cache them using [Docker registry caching], with the intermediate artifacts being stored on [ghcr.io]. We also push the built Docker images to ghcr, so that they can be reused by other tools (rustup) or by developers running the Docker build locally (to speed up their @@ -334,6 +334,13 @@ override the cache for the others. Instead, we store the images under different tags, identifying them with a custom hash made from the contents of all the Dockerfiles and related scripts. +The CI calculates a hash key, so that the cache of a Docker image is +invalidated if one of the following changes: + +- Dockerfile +- Files copied into the Docker image in the Dockerfile +- The architecture of the GitHub runner (x86 or ARM) + [ghcr.io]: https://github.com/rust-lang-ci/rust/pkgs/container/rust-ci [Docker registry caching]: https://docs.docker.com/build/cache/backends/registry/ @@ -341,9 +348,18 @@ Dockerfiles and related scripts. We build some C/C++ stuff in various CI jobs, and we rely on [sccache] to cache the intermediate LLVM artifacts. Sccache is a distributed ccache developed by -Mozilla, which can use an object storage bucket as the storage backend. In our -case, the artefacts are uploaded to an S3 bucket that we control -(`rust-lang-ci-sccache2`). +Mozilla, which can use an object storage bucket as the storage backend. + +With sccache there's no need to calculate the hash key ourselves. Sccache +invalidates the cache automatically when it detects changes to relevant inputs, +such as the source code, the version of the compiler, and important environment +variables. +So we just pass the sccache wrapper on top of cargo and sccache does the rest. + +We store the persistent artifacts on the S3 bucket `rust-lang-ci-sccache2`. So +when the CI runs, if sccache sees that LLVM is being compiled with the same C/C++ +compiler and the LLVM source code is the same, sccache retrieves the individual +compiled translation units from S3. [sccache]: https://github.com/mozilla/sccache |