summary refs log tree commit diff
path: root/src/librustc_ast/tokenstream.rs
AgeCommit message (Collapse)AuthorLines
2020-05-29Revert "Move functions to librustc_parse"Aaron Hill-0/+121
This reverts commit 7a4c1865fb2f58c57e3f09645515dec8be3022c6.
2020-05-29Revert "Recursively expand nonterminals"Aaron Hill-0/+2
This reverts commit 2af0218bf1ffca0750a352554f20a07b760a30a8.
2020-05-29Revert "Fix rebase fallout"Aaron Hill-0/+2
This reverts commit 5685e4dd90ad2f4978c63b9097f0b568e4ce6e5c.
2020-05-22Fix rebase falloutAaron Hill-2/+0
2020-05-22Recursively expand nonterminalsAaron Hill-2/+0
2020-05-22Move functions to librustc_parseAaron Hill-121/+0
2020-05-19Use a fixed-point iteration when breaking tokensAaron Hill-5/+40
Some tokens need to be broken in a loop until we reach 'unbreakable' tokens.
2020-05-19Break tokens before checking if they are 'probably equal'Aaron Hill-2/+32
Fixes #68489 When checking two `TokenStreams` to see if they are 'probably equal', we ignore the `IsJoint` information associated with each `TokenTree`. However, the `IsJoint` information determines whether adjacent tokens will be 'glued' (if possible) when construction the `TokenStream` - e.g. `[Gt Gt]` can be 'glued' to `BinOp(Shr)`. Since we are ignoring the `IsJoint` information, 'glued' and 'unglued' tokens are equivalent for determining if two `TokenStreams` are 'probably equal'. Therefore, we need to 'unglue' all tokens in the stream to avoid false negatives (which cause us to throw out the cached tokens, losing span information).
2020-03-09Use `Token::uninterpolate` in couple more places matching on `(Nt)Ident`Vadim Petrochenkov-0/+7
2020-02-29Move directory `libsyntax` -> `librustc_ast`Vadim Petrochenkov-0/+486