| Age | Commit message (Collapse) | Author | Lines |
|---|
|
`TokenStream` is currently recursive in *two* ways:
- the `TokenTree` variant contains a `ThinTokenStream`, which can
contain a `TokenStream`;
- the `TokenStream` variant contains a `Vec<TokenStream>`.
The latter is not necessary and causes significant complexity. This
commit replaces it with the simpler `Vec<(TokenTree, IsJoint)>`.
This reduces complexity significantly. In particular, `StreamCursor` is
eliminated, and `Cursor` becomes much simpler, consisting now of just a
`TokenStream` and an index.
The commit also removes the `Extend` impl for `TokenStream`, because it
is only used in tests. (The commit also removes those tests.)
Overall, the commit reduces the number of lines of code by almost 200.
|
|
|
|
|
|
|
|
Remove `TokenStream::JointTree`.
This is done by adding a new `IsJoint` field to `TokenStream::Tree`,
which simplifies a lot of `match` statements. And likewise for
`CursorKind`.
The commit also adds a new method `TokenTree:stream()` which can replace
a choice between `.into()` and `.joint()`.
|
|
This is done by adding a new `IsJoint` field to `TokenStream::Tree`,
which simplifies a lot of `match` statements. And likewise for
`CursorKind`.
The commit also adds a new method `TokenTree:stream()` which can replace
a choice between `.into()` and `.joint()`.
|
|
Do not point at delim spans for complete correct blocks
Fix #56834.
|
|
|
|
`TokenStream::new` is a better name for the former, and the latter is
now just equivalent to `TokenStream::Stream`.
|
|
Because it's an extra type layer that doesn't really help; in a couple
of places it actively gets in the way, and overall removing it makes the
code nicer. It does, however, move `tokenstream::TokenTree` further away
from the `TokenTree` in `quote.rs`.
More importantly, this change reduces the size of `TokenStream` from 48
bytes to 40 bytes on x86-64, which is enough to slightly reduce
instruction counts on numerous benchmarks, the best by 1.5%.
Note that `open_tt` and `close_tt` have gone from being methods on
`Delimited` to associated methods of `TokenTree`.
|
|
|
|
cleanup: remove static lifetimes from consts
A follow-up to https://github.com/rust-lang/rust/pull/56497.
|
|
r=michaelwoerister
53956 panic on include bytes of own file
fix #53956
When using `include_bytes!` on a source file in the project, compiler would panic on subsequent compilations because `expand_include_bytes` would overwrite files in the source_map with no source. This PR changes `expand_include_bytes` to check source_map and use the already existing src, if any.
|
|
|
|
fixes remaining test failures
|
|
|
|
|
|
|
|
|
|
rustdoc: Replaces fn main search and extern crate search with proper parsing during doctests.
Fixes #21299.
Fixes #33731.
Let me know if there's any additional changes you'd like made!
|
|
|
|
|
|
|
|
|
|
|
|
proc_macro::Group::span_open and span_close
Before this addition, every delimited group like `(`...`)` `[`...`]` `{`...`}` has only a single Span that covers the full source location from opening delimiter to closing delimiter. This makes it impossible for a procedural macro to trigger an error pointing to just the opening or closing delimiter. The Rust compiler does not seem to have the same limitation:
```rust
mod m {
type T =
}
```
```console
error: expected type, found `}`
--> src/main.rs:3:1
|
3 | }
| ^
```
On that same input, a procedural macro would be forced to trigger the error on the last token inside the block, on the entire block, or on the next token after the block, none of which is really what you want for an error like above.
This commit adds `group.span_open()` and `group.span_close()` which access the Span associated with just the opening delimiter and just the closing delimiter of the group. Relevant to Syn as we implement real error messages for when parsing fails in a procedural macro: https://github.com/dtolnay/syn/issues/476.
```diff
impl Group {
fn span(&self) -> Span;
+ fn span_open(&self) -> Span;
+ fn span_close(&self) -> Span;
}
```
Fixes #48187
r? @alexcrichton
|
|
|
|
|
|
* When encountering EOF, point at the last opening brace that does not
have the same indentation level as its close delimiter.
* When encountering the wrong type of close delimiter, point at the
likely correct open delimiter to give a better idea of what went
wrong.
|
|
|
|
|
|
or "".into()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Tweak unclosed delimiter parser error
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
