summary refs log tree commit diff
path: root/compiler/rustc_parse/src/lib.rs
AgeCommit message (Collapse)AuthorLines
2020-09-28Fix recursive nonterminal expansion during pretty-print/reparse checkAaron Hill-23/+34
Makes progress towards #43081 In PR #73084, we started recursively expanded nonterminals during the pretty-print/reparse check, allowing them to be properly compared against the reparsed tokenstream. Unfortunately, the recursive logic in that PR only handles the case where a nonterminal appears inside a `TokenTree::Delimited`. If a nonterminal appears directly in the expanded tokens of another nonterminal, the inner nonterminal will not be expanded. This PR fixes the recursive expansion of nonterminals, ensuring that they are expanded wherever they occur.
2020-09-26pretty-print-reparse hack: Remove an impossible caseVadim Petrochenkov-4/+1
Delimiters cannot appear as isolated tokens in a token stream
2020-09-26pretty-print-reparse hack: Rename some variables for clarityVadim Petrochenkov-16/+21
2020-09-21Don't use `zip` to compare iterators during pretty-print hackAaron Hill-8/+5
If the right-hand iterator has exactly one more element than the left-hand iterator, then both iterators will be fully consumed, but the extra element will never be compared.
2020-09-17Remove redundant #![feature(...)] 's from compiler/est31-1/+0
2020-09-13Auto merge of #76585 - Aaron1011:ignore-vert-plus, r=petrochenkovbors-0/+6
Ignore `|` and `+` tokens during proc-macro pretty-print check Fixes #76182 This is an alternative to PR #76188 These tokens are not preserved in the AST in certain cases (e.g. a leading `|` in a pattern or a trailing `+` in a trait bound). This PR ignores them entirely during the pretty-print/reparse check to avoid spuriously using the re-parsed tokenstream.
2020-09-10Attach tokens to `ast::Stmt`Aaron Hill-0/+7
We currently only attach tokens when parsing a `:stmt` matcher for a `macro_rules!` macro. Proc-macro attributes on statements are still unstable, and need additional work.
2020-09-10Attach `TokenStream` to `ast::Visibility`Aaron Hill-1/+1
A `Visibility` does not have outer attributes, so we only capture tokens when parsing a `macro_rules!` matcher
2020-09-10Attach `TokenStream` to `ast::Path`Aaron Hill-0/+1
2020-09-10Attach tokens to `NtMeta` (`ast::AttrItem`)Aaron Hill-0/+1
An `AttrItem` does not have outer attributes, so we only capture tokens when parsing a `macro_rules!` matcher
2020-09-10Collect tokens when handling `:literal` matcherAaron Hill-1/+1
An `NtLiteral` just wraps an `Expr`, so we don't need to add a new `tokens` field to an AST struct.
2020-09-10Attach `TokenStream` to `ast::Ty`Aaron Hill-0/+1
A `Ty` does not have outer attributes, so we only capture tokens when parsing a `macro_rules!` matcher
2020-09-10Attach `TokenStream` to `ast::Block`Aaron Hill-0/+1
A `Block` does not have outer attributes, so we only capture tokens when parsing a `macro_rules!` matcher
2020-09-10Ignore `|` and `+` tokens during proc-macro pretty-print checkAaron Hill-0/+6
Fixes #76182 This is an alternative to PR #76188 These tokens are not preserved in the AST in certain cases (e.g. a leading `|` in a pattern or a trailing `+` in a trait bound). This PR ignores them entirely during the pretty-print/reparse check to avoid spuriously using the re-parsed tokenstream.
2020-09-10Rollup merge of #76563 - yokodake:patch-1, r=jonas-schievinkTyler Mandry-1/+1
small typo fix in rustc_parse docs small typo in rustc_parse::new_parser_from_file's documentation I'm not sure a PR is the way to do this though.
2020-09-10small typo fix in rustc_parse docsNanami-1/+1
2020-09-03Rename IsJoint -> SpacingAleksey Kladov-2/+2
To match better naming from proc-macro
2020-09-03Condense StringReader's API to a single functionAleksey Kladov-2/+7
2020-09-01Remove trivia tokensAleksey Kladov-7/+0
2020-08-30mv compiler to compiler/mark-0/+595