| Age | Commit message (Collapse) | Author | Lines |
|---|
|
|
|
|
|
CI: fix publishing of toolstate history
Hopefully fixes the upload of toolstate history that I broke in https://github.com/rust-lang/rust/pull/125145. The problem is that the toolstate environment variables need to be available not just when updating `latest.json` through the Bash/Python script, but also in the main CI workflow, where `history` is updated in bootstrap (the toolstate logic is distributed in two places :/ ).
The only tricky thing is how to pass the token secret to the CI job, as I need to enable it only for privileged (`auto`/`try`) builds. I assume that the secret is filled only on `rust-lang-ci`, not on `rust-lang`, so I chose the easiest way of just configuring the environment variable globally. We'll see if that works on PR CI.
r? `@ehuss`
|
|
|
|
Remove unused(?) `~/rustsrc` folder from docker script
Every time I run a docker image, the script creates an empty `~/rustsrc` folder. It doesn't seem to be referenced anywhere else, so I'd like to remove it.
Incidentally, this also documents DIST_TRY_BUILD as I fail to find it from a simple search any time I look for it to eg enable tests on try builds.
|
|
|
|
|
|
|
|
Reduce builder size of jobs that take less than an hour
The current longest build time is ~2hr for the `dist-x86_64-linux-alt` build. This is already on a 16-core builder, so we can't make it any faster (by throwing more hardware at it).
Given that overall build times will be at least 2hrs, we can reduce build costs by reducing the builder size for any job that takes less than 1hr since it will still complete before `dist-x86_64-linux-alt` does.
Note that scaling isn't linear, halving the core count increases end-to-end build times by about 25-50%. In [this sample build](https://github.com/rust-lang/rust/actions/runs/9037235792/usage?pr=124985) `arm-android` went from ~52m to 1h 5m and `dist-arm-linux` went from ~55m to 1h 17m (then failed due to missing metrics).
Current job builder sizes and times and proposed new sizes:
| Job | Size | Proposed | Run 1 | Run 2 | Run 3 | Run 4 |
|-----|------|----------|-------|-------|-------|-------|
| aarch64-gnu | - | | 1h 9m 1s | 1h 8m 47s | 1h 8m 45s | 1h 9m 6s |
| arm-android | 8c | 4c | 52m 32s | 52m 38s | 51m 30s | 53m 13s |
| armhf-gnu | 8c | 4c | 37m 30s | 37m 40s | 38m 41s | 37m 56s |
| dist-aarch64-linux | 8c | 4c | 57m 11s | 56m 48s | 55m 53s | 56m 19s |
| dist-android | 8c | 4c | 24m 37s | 25m 13s | 25m 15s | 24m 17s |
| dist-arm-linux | 16c | 8c | 53m 34s | 55m 11s | 56m 1s | 54m 29s |
| dist-armhf-linux | 8c | 4c | 42m 1s | 43m 32s | 43m 27s | 41m 55s |
| dist-armv7-linux | 8c | 4c | 44m 51s | 44m 35s | 43m 34s | 46m 2s |
| dist-i586-gnu-i586-i686-musl | 8c | 4c | 37m 59s | 37m 56s | 38m 4s | 38m 24s |
| dist-i686-linux | 8c | 4c | 52m 20s | 51m 3s | 52m 53s | 50m 38s |
| dist-loongarch64-linux | 8c | 4c | 40m 39s | 40m 20s | 41m 6s | 40m 44s |
| dist-ohos | 8c | 4c | 25m 5s | 24m 34s | 25m 18s | 23m 40s |
| dist-powerpc-linux | 8c | 4c | 42m 31s | 43m 53s | 42m 35s | 41m 56s |
| dist-powerpc64-linux | 8c | 4c | 42m 52s | 44m 36s | 45m 32s | 43m 51s |
| dist-powerpc64le-linux | 8c | 4c | 43m 41s | 44m 11s | 43m 2s | 44m 21s |
| dist-riscv64-linux | 8c | 4c | 41m 25s | 42m 41s | 41m 52s | 43m 47s |
| dist-s390x-linux | 8c | 4c | 46m 48s | 47m 18s | 47m 27s | 46m 49s |
| dist-various-1 | 8c | 4c | 42m 14s | 43m 20s | 43m 20s | 41m 41s |
| dist-various-2 | 8c | 4c | 36m 18s | 38m 15s | 37m 41s | 39m 28s |
| dist-x86_64-freebsd | 8c | 4c | 39m 21s | 39m 40s | 40m 1s | 40m 2s |
| dist-x86_64-illumos | 8c | 4c | 45m 35s | 46m 43s | 46m 2s | 46m 4s |
| dist-x86_64-linux | 16c | | 1h 53m 10s | 1h 51m 15s | 1h 52m 18s | 1h 52m 26s |
| dist-x86_64-linux-alt | 16c | | 2h 3m 33s | 2h 3m 31s | 2h 4m 12s | 2h 2m 21s |
| dist-x86_64-musl | 8c | | 1h 5m 42s | 1h 6m 13s | 1h 7m 49s | 1h 6m 6s |
| dist-x86_64-netbsd | 8c | 4c | 40m 4s | 39m 48s | 40m 16s | 39m 43s |
| i686-gnu | 8c | | 1h 13m 38s | 1h 13m 39s | 1h 13m 48s | 1h 13m 12s |
| i686-gnu-nopt | 8c | | 1h 17m 44s | 1h 18m 14s | 1h 19m 55s | 1h 18m 44s |
| mingw-check | 4c | | 28m 15s | 27m 39s | 28m 36s | 28m 38s |
| test-various | 8c | 4c | 37m 45s | 37m 17s | 38m 26s | 38m 11s |
| x86_64-gnu | 4c | | 1h 34m 1s | 1h 31m 51s | 1h 30m 35s | 1h 32m 53s |
| x86_64-gnu-stable | 4c | | 1h 28m 26s | 1h 28m 11s | 1h 29m 40s | 1h 46m 28s |
| x86_64-gnu-aux | 4c | | 1h 33m 32s | 1h 31m 57s | 1h 34m 8s | 1h 32m 57s |
| x86_64-gnu-integration | 8c | | 1h 22m 2s | 1h 20m 14s | 1h 19m 46s | 1h 21m 24s |
| x86_64-gnu-debug | 8c | 4c | 52m 41s | 53m 40s | 51m 51s | 56m 9s |
| x86_64-gnu-distcheck | 8c | | 1h 9m 14s | 1h 5m 31s | 1h 6m 29s | 1h 5m 50s |
| x86_64-gnu-llvm-18 | 8c | | 1h 39m 47s | 1h 37m 57s | 1h 38m 40s | 1h 37m 38s |
| x86_64-gnu-llvm-17 | 8c | | 1h 41m 50s | 1h 45m 43s | 1h 45m 4s | 1h 43m 4s |
| x86_64-gnu-nopt | 4c | | 1h 20m 42s | 1h 21m 38s | 1h 20m 4s | 1h 22m 11s |
| x86_64-gnu-tools | 8c | | 1h 5m 0s | 1h 5m 30s | 1h 3m 1s | 1h 3m 20s |
| dist-x86_64-apple | xl | | 1h 35m 1s | 1h 39m 57s | 2h 2m 31s | 1h 47m 37s |
| dist-apple-various | xl | | 1h 18m 54s | 1h 22m 31s | 1h 13m 19s | 1h 38m 18s |
| x86_64-apple-1 | xl | | 1h 32m 8s | 1h 40m 12s | 1h 51m 28s | 1h 40m 26s |
| x86_64-apple-2 | xl | | 1h 0m 32s | 1h 4m 5s | 1h 9m 0s | 1h 7m 17s |
| dist-aarch64-apple | m1 | | 1h 3m 9s | 1h 1m 14s | 1h 2m 6s | 1h 2m 24s |
| aarch64-apple | m1 | | 53m 38s | 1h 1m 5s | 1h 3m 15s | 1h 6m 11s |
| x86_64-msvc | 8c | | 1h 27m 48s | 1h 29m 38s | 1h 29m 55s | 1h 28m 4s |
| i686-msvc | 8c | | 1h 38m 28s | 1h 34m 7s | 1h 39m 19s | 1h 39m 28s |
| x86_64-msvc-ext | 8c | | 1h 44m 5s | 1h 38m 40s | 1h 45m 21s | 1h 44m 19s |
| i686-mingw | 8c | | 1h 49m 57s | 1h 45m 1s | 1h 52m 4s | 1h 51m 4s |
| x86_64-mingw | 8c | | 1h 44m 2s | 1h 37m 36s | 1h 49m 58s | 1h 47m 5s |
| dist-x86_64-msvc | 8c | | 1h 57m 14s | 1h 49m 43s | 1h 52m 53s | 1h 52m 35s |
| dist-i686-msvc | 8c | | 1h 8m 5s | 1h 4m 9s | 1h 9m 26s | 1h 12m 0s |
| dist-aarch64-msvc | 8c | | 1h 18m 40s | 1h 14m 4s | 1h 22m 1s | 1h 19m 6s |
| dist-i686-mingw | 8c | | 1h 15m 36s | 1h 14m 36s | 1h 16m 38s | 1h 16m 2s |
| dist-x86_64-mingw | 8c | | 1h 11m 54s | 1h 16m 12s | 1h 16m 54s | 1h 18m 2s |
| dist-x86_64-msvc-alt | 8c | | 1h 11m 17s | 1h 10m 0s | 1h 11m 8s | 1h 13m 14s |
|
|
|
|
|
|
Upgrade pre-built Clang used in MSVC and MacOS builds, move MSVC builds to Server 2022
Fixes #92948
Example working MacOS and Windows builds: <https://github.com/rust-lang/rust/actions/runs/8989360201>
There is a [bug in Clang 18](https://github.com/llvm/llvm-project/pull/81849) that causes issues when building for Arm64 in later parts of the build (specifically `libgit2`). As a workaround, we will still use the pre-built Clang to build LLVM but will use MSVC for the rest of the Arm64 build.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Use Server 2022 in CI for mingw jobs
From my [previous testing](https://github.com/rust-lang/rust/issues/92948#issuecomment-2077771347), there is nothing blocking using Server 2022 for the Windows MinGW jobs.
This change:
* Adds new `os` entries for Server 2022.
* Renamed the Server 2019 `os` entries (as we'll be removing these over time).
* Moves the mingw jobs to Server 2022 and leaves the msvc jobs on Server 2019.
Partially fixes #92948
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Neither `env.DEPLOY` nor `env.DEPLOY_ALT` should be present in this job.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
CI: add a script for dynamically computing CI job matrix
It would be great if was easier to run specific CI workflows locally, and also to allow us to spawn a specific CI workflow by bors, to enable running arbitrary try builds. See discussion [here](https://rust-lang.zulipchat.com/#narrow/stream/242791-t-infra/topic/CI.20workflows.20refactoring).
This PR is a first step in that direction.
- Moves the definition of CI runners and (for now) PR jobs into a separate `jobs.yml` file.
- Adds a simple Python script that reads the file, decides which jobs should be active for the current CI workflow, and prints them as JSON to their output.
- The PR job then reads this output and generates its job matrix based on it.
By moving the job definitions from `ci.yml` into a separate file, we can handle it programmatically, which should make it easier to both do local execution of CI jobs and also to do arbitrary try builds.
|
|
|
|
|
|
|
