| Age | Commit message (Collapse) | Author | Lines |
|
Makes progress towards #43081
In PR #73084, we started recursively expanded nonterminals during the
pretty-print/reparse check, allowing them to be properly compared
against the reparsed tokenstream.
Unfortunately, the recursive logic in that PR only handles the case
where a nonterminal appears inside a `TokenTree::Delimited`. If a
nonterminal appears directly in the expanded tokens of another
nonterminal, the inner nonterminal will not be expanded.
This PR fixes the recursive expansion of nonterminals, ensuring that
they are expanded wherever they occur.
|
|
Delimiters cannot appear as isolated tokens in a token stream
|
|
|
|
If the right-hand iterator has exactly one more element than the
left-hand iterator, then both iterators will be fully consumed, but
the extra element will never be compared.
|
|
|
|
Ignore `|` and `+` tokens during proc-macro pretty-print check
Fixes #76182
This is an alternative to PR #76188
These tokens are not preserved in the AST in certain cases
(e.g. a leading `|` in a pattern or a trailing `+` in a trait bound).
This PR ignores them entirely during the pretty-print/reparse check
to avoid spuriously using the re-parsed tokenstream.
|
|
We currently only attach tokens when parsing a `:stmt` matcher for a
`macro_rules!` macro. Proc-macro attributes on statements are still
unstable, and need additional work.
|
|
A `Visibility` does not have outer attributes, so we only capture tokens
when parsing a `macro_rules!` matcher
|
|
|
|
An `AttrItem` does not have outer attributes, so we only capture tokens
when parsing a `macro_rules!` matcher
|
|
An `NtLiteral` just wraps an `Expr`, so we don't need to add a new `tokens`
field to an AST struct.
|
|
A `Ty` does not have outer attributes, so we only capture tokens
when parsing a `macro_rules!` matcher
|
|
A `Block` does not have outer attributes, so we only capture tokens when
parsing a `macro_rules!` matcher
|
|
Fixes #76182
This is an alternative to PR #76188
These tokens are not preserved in the AST in certain cases
(e.g. a leading `|` in a pattern or a trailing `+` in a trait bound).
This PR ignores them entirely during the pretty-print/reparse check
to avoid spuriously using the re-parsed tokenstream.
|
|
small typo fix in rustc_parse docs
small typo in rustc_parse::new_parser_from_file's documentation
I'm not sure a PR is the way to do this though.
|
|
|
|
To match better naming from proc-macro
|
|
|
|
|
|
|