| Age | Commit message (Collapse) | Author | Lines |
|---|
|
|
|
`TokenStream` is now almost identical to `ThinTokenStream`. This commit
removes the latter, replacing it with the former.
|
|
Make `TokenStream` less recursive.
`TokenStream` is currently recursive in *two* ways:
- the `TokenTree` variant contains a `ThinTokenStream`, which can
contain a `TokenStream`;
- the `TokenStream` variant contains a `Vec<TokenStream>`.
The latter is not necessary and causes significant complexity. This
commit replaces it with the simpler `Vec<(TokenTree, IsJoint)>`.
This reduces complexity significantly. In particular, `StreamCursor` is
eliminated, and `Cursor` becomes much simpler, consisting now of just a
`TokenStream` and an index.
The commit also removes the `Extend` impl for `TokenStream`, because it
is only used in tests. (The commit also removes those tests.)
Overall, the commit reduces the number of lines of code by almost 200.
|
|
|
|
This avoids 770,000 allocations when compiling the `html5ever`
benchmark, reducing instruction counts by up to 2%.
|
|
`TokenStream` is currently recursive in *two* ways:
- the `TokenTree` variant contains a `ThinTokenStream`, which can
contain a `TokenStream`;
- the `TokenStream` variant contains a `Vec<TokenStream>`.
The latter is not necessary and causes significant complexity. This
commit replaces it with the simpler `Vec<(TokenTree, IsJoint)>`.
This reduces complexity significantly. In particular, `StreamCursor` is
eliminated, and `Cursor` becomes much simpler, consisting now of just a
`TokenStream` and an index.
The commit also removes the `Extend` impl for `TokenStream`, because it
is only used in tests. (The commit also removes those tests.)
Overall, the commit reduces the number of lines of code by almost 200.
|
|
Found with `git grep -P '\b([a-z]+)\s+\1\b'`
|
|
|
|
Resolve `$crate`s for pretty-printing at more appropriate time
Doing it in `BuildReducedGraphVisitor` wasn't a good idea, identifiers wasn't actually visited half of the time.
As a result some `$crate`s weren't resolved and were therefore pretty-printed as `$crate` literally, which turns into two tokens during re-parsing of the pretty-printed text.
Now we are visiting and resolving `$crate` identifiers in an item right before sending that item to a proc macro attribute or derive.
Fixes https://github.com/rust-lang/rust/issues/57089
|
|
|
|
|
|
|
|
|
|
Fix a number of uncovered deficiencies in diagnostics
|
|
|
|
|
|
`TokenStream` improvements
Some `TokenStream` improvements: shrinking `TokenStream` and some other types, and some other code clean-ups.
|
|
overhaul external doc attribute diagnostics
This PR improves the error handling and spans for the external doc attribute. Many cases that silently failed before now emit errors, spans are tightened, and the errors have help and suggestions.
I tried to address all the cases that users ran into in the tracking issue.
cc #44732
r? @QuietMisdreavus
|
|
`TokenStream::new` is a better name for the former, and the latter is
now just equivalent to `TokenStream::Stream`.
|
|
|
|
Also, provide a suggestion for the correct syntax.
|
|
Because it's an extra type layer that doesn't really help; in a couple
of places it actively gets in the way, and overall removing it makes the
code nicer. It does, however, move `tokenstream::TokenTree` further away
from the `TokenTree` in `quote.rs`.
More importantly, this change reduces the size of `TokenStream` from 48
bytes to 40 bytes on x86-64, which is enough to slightly reduce
instruction counts on numerous benchmarks, the best by 1.5%.
Note that `open_tt` and `close_tt` have gone from being methods on
`Delimited` to associated methods of `TokenTree`.
|
|
Various minor/cosmetic improvements to code
r? @Centril 😄
|
|
|
|
This commit replaces many usages of `File::open` and reading or writing
with `fs::read_to_string`, `fs::read` and `fs::write`. This reduces code
complexity, and will improve performance for most reads, since the
functions allocate the buffer to be the size of the file.
I believe that this commit will not impact behavior in any way, so some
matches will check the error kind in case the file was not valid UTF-8.
Some of these cases may not actually care about the error.
|
|
attributes
|
|
libsyntax_pos: A few tweaks
|
|
r=michaelwoerister
53956 panic on include bytes of own file
fix #53956
When using `include_bytes!` on a source file in the project, compiler would panic on subsequent compilations because `expand_include_bytes` would overwrite files in the source_map with no source. This PR changes `expand_include_bytes` to check source_map and use the already existing src, if any.
|
|
fixes remaining test failures
|
|
|
|
includes the source if it can convert bytes to string
|
|
|
|
|
|
`Edition` is not a public API, we want users to break when a new edition is added
|
|
`CrateRoot` -> `PathRoot`, `::` doesn't necessarily mean crate root now
`SelfValue` -> `SelfLower`, `SelfType` -> `SelfUpper`, both `self` and `Self` can be used in type and value namespaces now
|
|
Currently libproc_macro does not use `DotEq` token.
https://github.com/rust-lang/rust/pull/49545 changed libproc_macro
to not generate `DotEq` token.
|
|
Reuse the `P` in `InvocationCollector::fold_{,opt_}expr`.
This requires adding a new method, `P::filter_map`.
This commit reduces instruction counts for various benchmarks by up to
0.7%.
|
|
|
|
|
|
Point at end of macro arm when encountering EOF
Fix #52866.
|
|
|
|
|
|
|
|
|
|
|
|
Fix #52866
|
|
|
|
|
|
This requires adding a new method, `P::filter_map`.
This commit reduces instruction counts for various benchmarks by up to
0.7%.
|
|
macro paths
|
