summary refs log tree commit diff
path: root/src/ci/citool
AgeCommit message (Collapse)AuthorLines
2025-03-27Add CI metadata to bootstrap metricsJakub Beránek-2/+2
This will allow us to provide links to CI workflows, jobs and summaries in the post-merge analysis report.
2025-03-27Add a note about interpreting job duration changesJakub Beránek-1/+9
2025-03-27Add cache for job metricsJakub Beránek-1/+19
2025-03-27Add job duration changes stats in post-merge analysisJakub Beránek-5/+57
2025-03-26Rollup merge of #138930 - Kobzol:analyze-bootstrap-diffs, r=marcoieniStuart Cook-28/+140
Add bootstrap step diff to CI job analysis This PR adds another analysis to the job analysis report in GitHub summary. It compares (diffs) bootstrap steps executed by the parent run and by the current commit. This will help us figure out if the bootstrap invocation did something different than before, and also how did the duration of individual steps and bootstrap invocations change. Can be tested on the https://github.com/rust-lang/rust/pull/119899 PR like this: ```bash $ curl https://ci-artifacts.rust-lang.org/rustc-builds/3d3394eb64ee2f99ad1a2b849b376220fd38263e/metrics-mingw-check.json > metrics.json $ cargo run --manifest-path src/ci/citool/Cargo.toml postprocess-metrics metrics.json --job-name mingw-check --parent 961351c76c812e3aeb65bfb542742500a6436aed > out.md ``` r? `@marcoie`
2025-03-25Add diff of bootstrap stepsJakub Beránek-28/+140
2025-03-22Group test diffs by stage in post-merge analysisJakub Beránek-18/+25
2025-03-18Remove double nesting in post-merge workflowJakub Beránek-8/+1
2025-03-17Rollup merge of #138533 - Kobzol:try-job-auto-tests, r=marcoieniMatthias Krüger-12/+19
Only use `DIST_TRY_BUILD` for try jobs that were not selected explicitly Some CI jobs (x64 Linux, ARM64 Linux and x64 MSVC) use the `opt-dist` tool to build an optimized toolchain using PGO and BOLT. When performing a default try build for x64 Linux, in most cases we want to run perf. on that artifact. To reduce the latency of this common use-case, `opt-dist` skips building several components not needed for perf., and it also skips running post-optimization tests, when it detects that the job is executed as a try job (not a merge/auto job). This is useful, but it also means that if you *want* to run the tests, you had to go to `jobs.yml` and manually comment this environment variable, create a WIP commit, do a try build, and then remove the WIP commit, which is annoying (in the similar way that modifying what gets run in try builds was annoying before we had the `try-job` annotations). I thought that we could introduce some additional PR description marker like `try-job-run-tests`, but it's hard to discover that such things exist. Instead, I think that there's a much simpler heuristic for determining whether `DIST_TRY_BUILD` should be used (that I implemented in this PR): - If you do just ``@bors` try`, without any custom try jobs selected, `DIST_TRY_BUILD` will be activated, to finish the build as fast as possible. - If you specify any custom try jobs, you are most likely doing experiments and you want to see if tests pass and everything builds as it should. The `DIST_TRY_BUILD` variable will thus *not* be set in this case. In this way, if you want to run dist tests, you can just add the `try-job: dist-x86_64-linux` line to the PR description, and you don't need to create any WIP commits. r? `@marcoieni`
2025-03-17Small review improvementsJakub Beránek-2/+2
2025-03-15Only use `DIST_TRY_BUILD` for try jobs that were not selected explicitlyJakub Beránek-12/+19
2025-03-15Reformat codeJakub Beránek-7/+10
2025-03-15Do not error out on missing parent metricsJakub Beránek-5/+18
2025-03-15Put test differences into a `<details>` section and add better explanation ↵Jakub Beránek-34/+56
of the post merge report
2025-03-15Add a helper function for outputting detailsJakub Beránek-9/+18
2025-03-15Add bootstrap stage to test namesJakub Beránek-4/+11
2025-03-15Print test diffs into GitHub summaryJakub Beránek-6/+35
So that we can also observe them for try builds, before merging a PR.
2025-03-15Use first-level heading for test differences headerJakub Beránek-1/+1
2025-03-15Print number of found test diffsJakub Beránek-0/+1
2025-03-15Refactor metrics and analysis in citool to distinguish them betterJakub Beránek-219/+196
2025-03-15Print metrics postprocessing to stdoutJakub Beránek-28/+12
This allows the code to be simplified a little bit.
2025-03-15Rollup merge of #138454 - Kobzol:post-merge-workflow-fixes, r=jieyouxuLeón Orell Valerian Liehr-77/+138
Improve post-merge workflow Contains various fixes for the post-merge workflow implemented in https://github.com/rust-lang/rust/pull/138013, which were suggested on Zulip. This PR changes the grouping of test diffs and ignores doctests, as they are too noisy. I'll post an example output (before/after this PR) in comments below. r? ```@jieyouxu```
2025-03-14Add clarification about doctestsJakub Beránek-1/+1
2025-03-13Add `doc_url` attribute to CI jobsJakub Beránek-1/+7
2025-03-13Do not print doctest diffs in the reportJakub Beránek-12/+44
When they are moved around in code, their name changes, which produces too noisy diffs.
2025-03-13Group diffs by tests, rather than job groupsJakub Beránek-76/+86
2025-03-13Add cache for downloading job metricsJakub Beránek-0/+17
To make it easier to experiment locally.
2025-03-13Print the compared SHAsJakub Beránek-0/+2
2025-03-13Fix pluralization of testsJakub Beránek-1/+1
2025-03-11Rollup merge of #138307 - Kobzol:citool-alias, r=marcoieniJakub Beránek-232/+343
Allow specifying glob patterns for try jobs This PR modifies the `try-job` lookup logic to allow glob patterns. So you can e.g. request all MSVC-related jobs with `try-job: *msvc*`. Best reviewed commit by commit. r? ``````@marcoieni`````` try-job: `*msvc*`
2025-03-10Ignore job duplicatesJakub Beránek-1/+5
2025-03-10Handle backticks in try job patternsJakub Beránek-2/+9
2025-03-10Allow using glob aliases for custom try jobsJakub Beránek-34/+116
2025-03-10Move job handling to a separate moduleJakub Beránek-230/+242
2025-03-10Make all keys explicit in citoolJakub Beránek-6/+12
Just to avoid surprises, the amount of used keys is not large.
2025-03-10Handle empty test suites in job summary reportJakub Beránek-3/+7
2025-03-07Add post-merge analysis CI workflowJakub Beránek-9/+282
2025-03-06Rollup merge of #138053 - Kobzol:more-try-jobs, r=marcoieniMichael Goulet-2/+6
Increase the max. custom try jobs requested to `20` Requested [here](https://rust-lang.zulipchat.com/#narrow/channel/242791-t-infra/topic/Increase.20try-job.20limit). r? ````@marcoieni````
2025-03-05Increase the max. custom try jobs requested to `20`Jakub Beránek-2/+6
2025-03-05Apply review remarksJakub Beránek-2/+3
2025-03-05Upload Datadog average CPU usage metric in citoolJakub Beránek-15/+785
2025-03-05Make citool compilation fasterJakub Beránek-0/+4
2025-03-04Record bootstrap step durations into GitHub summary in citoolJakub Beránek-2/+28
2025-03-04Postprocess test suite metrics into GitHub summaryJakub Beránek-0/+164
2025-02-17Trim channelJakub Beránek-1/+3
To avoid including a newline at the end, which broke `only_on_channel` comparison.
2025-02-17Trim try-job namesJakub Beránek-1/+1
2025-02-17Avoid double serialization of environment stringsJakub Beránek-21/+16
2025-02-17Add testsJakub Beránek-39/+324
2025-02-17Add comment to [workspace]Jakub Beránek-0/+3
2025-02-17Apply review commentsJakub Beránek-2/+7