| Age | Commit message (Collapse) | Author | Lines |
|---|
|
Signed-off-by: onur-ozkan <work@onurozkan.dev>
|
|
Signed-off-by: onur-ozkan <work@onurozkan.dev>
|
|
|
|
Signed-off-by: onur-ozkan <work@onurozkan.dev>
|
|
In accordance with RFC 3771.
I also added a stub doc page for the target and renamed the
windows-gnullvm page for consistency.
|
|
|
|
It is useless after the removal of the parallel compiler configuration.
|
|
By nobuild, I mean that the type annotations are all in comments,
not in the "native" typescript syntax. This is a bit uglier,
but it lets you rapid-prototype without tsc, works with all
the native browser debugging tools, and keeps Node out of Rust's
bootstrap chain.
This pull request mostly just adds ts-ignore annotations
and type declarations. To actually take good advantage of
typescript, we'll want to "burn down" this pile of unsafe code
until we eventually have a version with almost none of these.
This PR also adds tsc to the mingw-check Dockerfile, so that
it can't fall out of date like the Closure annotations did.
https://rust-lang.zulipchat.com/#narrow/channel/266220-t-rustdoc/topic/typescript
|
|
This will tell you if license-metadata.json is out of date.
|
|
|
|
|
|
Signed-off-by: onur-ozkan <work@onurozkan.dev>
|
|
Signed-off-by: onur-ozkan <work@onurozkan.dev>
|
|
Signed-off-by: onur-ozkan <work@onurozkan.dev>
|
|
Use reuse tool 4.0
This change upgrades us to reuse-tool 4.0.3, which has a new TOML format configuration instead of the old `.reuse/dep5` Debian-style file.
* Updated requirements file to install reuse-4.0.3
* Ran `reuse convert-dep5` to switch to new file format
* Switched over to `override` so the `REUSE.toml` file takes precedence over whatever random Copyright strings `reuse` finds in the source tree.
Should fix https://github.com/rust-lang/rust/issues/127361
|
|
) brew install python@3.10
) python3.10 -m venv /tmp/myenv
) source /tmp/myenv/bin/activate
) pip install pip-tools
) /tmp/myenv/bin/pip-compile --allow-unsafe --generate-hashes reuse-requirements.in
|
|
Signed-off-by: onur-ozkan <work@onurozkan.dev>
|
|
`debug-logging` conflicts with `download-rustc` option, and doesn't really
make sense to enable it for a profile that is used for tool development.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
|
|
Signed-off-by: onur-ozkan <work@onurozkan.dev>
|
|
) Updated requirements file
) Ran `reuse convert-dep5` to switch to new file format
|
|
|
|
|
|
enable clippy for bootstrap on CI PRs (in `mingw-check` image)
Let's keep the bootstrap codebase cleaner.
|
|
|
|
Signed-off-by: onur-ozkan <work@onurozkan.dev>
|
|
Previously this command was linting compiler and library together.
As we no longer run clippy on the entire tree unless it's explicitly
requested, we need to update this command by adding `library` path.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
|
|
rustdoc-search: shard the search result descriptions
## Preview
This makes no visual changes to rustdoc search. It's a pure perf improvement.
<details><summary>old</summary>
Preview: <http://notriddle.com/rustdoc-html-demo-10/doc/std/index.html?search=vec>
WebPageTest Comparison with before branch on a sort of worst case (searching `vec`, winds up downloading most of the shards anyway): <https://www.webpagetest.org/video/compare.php?tests=240317_AiDc61_2EM,240317_AiDcM0_2EN>
Waterfall diagram:
</details>
Preview: <http://notriddle.com/rustdoc-html-demo-10/doc2/std/index.html?search=vec>
WebPageTest Comparison with before branch on a sort of worst case (searching `vec`, winds up downloading most of the shards anyway): <https://www.webpagetest.org/video/compare.php?tests=240322_BiDcCH_13R,240322_AiDcJY_104>
## Description
r? `@GuillaumeGomez`
The descriptions are, on almost all crates[^1], the majority of the size of the search index, even though they aren't really used for searching. This makes it relatively easy to separate them into their own files.
Additionally, this PR pulls out information about whether there's a description into a bitmap. This allows us to sort, truncate, *then* download.
This PR also bumps us to ES8. Out of the browsers we support, all of them support async functions according to caniuse.
https://caniuse.com/async-functions
[^1]:
<https://microsoft.github.io/windows-docs-rs/>, a crate with
44MiB of pure names and no descriptions for them, is an outlier
and should not be counted. But this PR should improve it, by replacing a long line of empty strings with a compressed bitmap with a single Run section. Just not very much.
## Detailed sizes
```console
$ cat test.sh
set -ex
cp ../search-index*.js search-index.js
awk 'FNR==NR {a++;next} FNR<a-3' search-index.js{,} | awk 'NR>1 {gsub(/\],\\$/,""); gsub(/^\["[^"]+",/,""); print} {next}' | sed -E "s:\\\\':':g" > search-index.json
jq -c '.t' search-index.json > t.json
jq -c '.n' search-index.json > n.json
jq -c '.q' search-index.json > q.json
jq -c '.D' search-index.json > D.json
jq -c '.e' search-index.json > e.json
jq -c '.i' search-index.json > i.json
jq -c '.f' search-index.json > f.json
jq -c '.c' search-index.json > c.json
jq -c '.p' search-index.json > p.json
jq -c '.a' search-index.json > a.json
du -hs t.json n.json q.json D.json e.json i.json f.json c.json p.json a.json
$ bash test.sh
+ cp ../search-index1.78.0.js search-index.js
+ awk 'FNR==NR {a++;next} FNR<a-3' search-index.js search-index.js
+ awk 'NR>1 {gsub(/\],\\$/,""); gsub(/^\["[^"]+",/,""); print} {next}'
+ sed -E 's:\\'\'':'\'':g'
+ jq -c .t search-index.json
+ jq -c .n search-index.json
+ jq -c .q search-index.json
+ jq -c .D search-index.json
+ jq -c .e search-index.json
+ jq -c .i search-index.json
+ jq -c .f search-index.json
+ jq -c .c search-index.json
+ jq -c .p search-index.json
+ jq -c .a search-index.json
+ du -hs t.json n.json q.json D.json e.json i.json f.json c.json p.json a.json
64K t.json
800K n.json
8.0K q.json
4.0K D.json
16K e.json
192K i.json
544K f.json
4.0K c.json
36K p.json
20K a.json
```
These are, roughly, the size of each section in the standard library (this tool actually excludes libtest, for parsing-json-with-awk reasons, but libtest is tiny so it's probably not important).
t = item type, like "struct", "free fn", or "type alias". Since one byte is used for every item, this implies that there are approximately 64 thousand items in the standard library.
n = name, and that's now the largest section of the search index with the descriptions removed from it
q = parent *module* path, stored parallel to the items within
D = the size of each description shard, stored as vlq hex numbers
e = empty description bit flags, stored as a roaring bitmap
i = parent *type* index as a link into `p`, stored as decimal json numbers; used only for associated types; might want to switch to vlq hex, since that's shorter, but that would be a separate pr
f = function signature, stored as lists of lists that index into `p`
c = deprecation flag, stored as a roaring bitmap
p = parent *type*, stored separately and linked into from `i` and `f`
a = alias, as [[key, value]] pairs
## Search performance
http://notriddle.com/rustdoc-html-demo-11/perf-shard/index.html
For example, in stm32f4:
<table><thead><tr><th>before<th>after</tr></thead>
<tbody><tr><td>
```
Testing T -> U ... in_args = 0, returned = 0, others = 200
wall time = 617
Testing T, U ... in_args = 0, returned = 0, others = 200
wall time = 198
Testing T -> T ... in_args = 0, returned = 0, others = 200
wall time = 282
Testing crc32 ... in_args = 0, returned = 0, others = 0
wall time = 426
Testing spi::pac ... in_args = 0, returned = 0, others = 0
wall time = 673
```
</td><td>
```
Testing T -> U ... in_args = 0, returned = 0, others = 200
wall time = 716
Testing T, U ... in_args = 0, returned = 0, others = 200
wall time = 207
Testing T -> T ... in_args = 0, returned = 0, others = 200
wall time = 289
Testing crc32 ... in_args = 0, returned = 0, others = 0
wall time = 418
Testing spi::pac ... in_args = 0, returned = 0, others = 0
wall time = 687
```
</td></tr><tr><td>
```
user: 005.345 s
sys: 002.955 s
wall: 006.899 s
child_RSS_high: 583664 KiB
group_mem_high: 557876 KiB
```
</td><td>
```
user: 004.652 s
sys: 000.565 s
wall: 003.865 s
child_RSS_high: 538696 KiB
group_mem_high: 511724 KiB
```
</td></tr>
</table>
This perf tester is janky and unscientific enough that the apparent differences might just be noise. If it's not an order of magnitude, it's probably not real.
## Future possibilities
* Currently, results are not shown until the descriptions are downloaded. Theoretically, the description-less results could be shown. But actually doing that, and making sure it works properly, would require extra work (we have to be careful to avoid layout jumps).
* More than just descriptions can be sharded this way. But we have to be careful to make sure the size wins are worth the round trips. Ideally, data that’s needed only for display should be sharded while data needed for search isn’t.
* [Full text search](https://internals.rust-lang.org/t/full-text-search-for-rustdoc-and-doc-rs/20427) also needs this kind of infrastructure. A good implementation might store a compressed bloom filter in the search index, then download the full keyword in shards. But, we have to be careful not just of the amount readers have to download, but also of the amount that [publishers](https://gist.github.com/notriddle/c289e77f3ed469d1c0238d1d135d49e1) have to store.
|
|
The descriptions are, on almost all crates[^1], the majority
of the size of the search index, even though they aren't really
used for searching. This makes it relatively easy to separate
them into their own files.
This commit also bumps us to ES8. Out of the browsers we support,
all of them support async functions according to caniuse.
https://caniuse.com/async-functions
[^1]:
<https://microsoft.github.io/windows-docs-rs/>, a crate with
44MiB of pure names and no descriptions for them, is an outlier
and should not be counted.
|
|
|
|
|
|
|
|
|
|
|
|
This PR saves library docs as github artifacts so they can be easily
viewed for review.
Discussed in <https://rust-lang.zulipchat.com/#narrow/stream/242791-t-infra/topic/Building.20docs.20for.20PR.20CI>
|
|
* architecture
* configures
* preparation
* toolstate
* unknown
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
|
|
|
|
|
|
|
|
|
|
Fix maintainer validation message
State clearly why maintainer verification is not activating, ruling out the user thinking they misconfigured something.
|
|
fix remove token check for maintainers
fix: rm validate_maintainers function
|
|
|
|
|
|
This duplicates mingw-check into two jobs where one job
runs `tidy` only while the other job does not. The tidy
job will not cancel other jobs on failure.
|
|
Run `x test tidy` sooner in mingw-check
It takes less time to run than the other tests and is more likely to fail. `expand-yaml-anchors` is still run first to make sure the CI files are internally consistent.
Note that changing to `--stage 0` doesn't actually do anything since bootstrap tools are always built with the bootstrap compiler, this just makes it less confusing.
cc https://github.com/rust-lang/rust/pull/105058/commits/83bab41b5b2d4752d187dd91b05c88ac74cf3783
|
|
|
|
It takes less time to run than the other tests and is more likely to fail.
`expand-yaml-anchors` is still run first to make sure the CI files are internally consistent.
|
|
|
|
|
|
|
