about summary refs log tree commit diff
path: root/src/libsyntax/parse/token.rs
AgeCommit message (Collapse)AuthorLines
2019-08-27Simplify with Symbol/Token::is_book_lit.Mazdak Farrokhzad-4/+7
2019-08-27token: refactor with is_non_raw_ident_where.Mazdak Farrokhzad-19/+13
2019-08-19glue tokens when building token streamAleksey Kladov-1/+1
2019-08-05add unknown tokenAleksey Kladov-1/+3
2019-07-31Address review comments.Mazdak Farrokhzad-2/+2
2019-07-30Account for maybe_whole_expr in range patterns.Mazdak Farrokhzad-0/+13
2019-07-10Remove duplicate attr_to_stringMark Rousskov-1/+1
attribute_to_string exists.
2019-07-10Move literal_to_string to fmt::DisplayMark Rousskov-0/+28
2019-06-23let_chains: readd kw::let to ident_can_begin_expr.Mazdak Farrokhzad-2/+1
2019-06-23let_chains: Comment out Let in ident_can_begin_expr.Mazdak Farrokhzad-1/+2
2019-06-23let_chains: Add support for parsing let expressions.Mazdak Farrokhzad-0/+1
2019-06-08syntax: Move some `Token` methods aroundVadim Petrochenkov-48/+40
2019-06-08syntax: Remove `Deref` impl from `Token`Vadim Petrochenkov-18/+11
2019-06-08syntax: Move most of the `TokenKind` methods to `Token`Vadim Petrochenkov-75/+41
2019-06-06Some code cleanup and tidy/test fixesVadim Petrochenkov-2/+1
2019-06-06syntax: Switch function parameter order in `TokenTree::token`Vadim Petrochenkov-4/+4
2019-06-06syntax: Remove duplicate span from `token::Ident`Vadim Petrochenkov-35/+59
2019-06-06syntax: Remove duplicate span from `token::Lifetime`Vadim Petrochenkov-17/+26
2019-06-06syntax: Add some helper methods to `Token`Vadim Petrochenkov-1/+17
2019-06-06syntax: Use `Token` in `TokenTree::Token`Vadim Petrochenkov-6/+15
2019-06-06syntax: Rename `TokenAndSpan` into `Token`Vadim Petrochenkov-0/+12
2019-06-06syntax: Rename `Token` into `TokenKind`Vadim Petrochenkov-14/+14
2019-06-06Always use token kinds through `token` module rather than `Token` typeVadim Petrochenkov-2/+2
2019-05-27Use `Symbol` equality in `is_ident_named`.Nicholas Nethercote-2/+2
2019-05-23syntax: Some code cleanupVadim Petrochenkov-7/+9
2019-05-23syntax: Turn `token::Lit` into a structVadim Petrochenkov-44/+51
2019-05-23syntax: Remove an obsolete hack from literal comparisonsVadim Petrochenkov-9/+1
2019-05-23syntax: Return named errors from literal parsing functionsVadim Petrochenkov-2/+23
2019-05-22Simplify use of keyword symbolsVadim Petrochenkov-37/+37
2019-05-19Improve type size assertionsVadim Petrochenkov-3/+3
Now they - Tell what the new size is, when it changes - Do not require passing an identifier
2019-05-12Auto merge of #60767 - Centril:rollup-4cbsb73, r=Centrilbors-10/+4
Rollup of 4 pull requests Successful merges: - #60694 (Fix HIR printing of existential type #60662) - #60750 (syntax: Remove some legacy nonterminal tokens) - #60751 (Assorted cleanup in parser & AST validation) - #60752 (Fix minor typos for ItemLocalId) Failed merges: r? @ghost
2019-05-12syntax: Remove some legacy nonterminal tokensVadim Petrochenkov-10/+4
2019-05-11Simplify conversions between tokens and semantic literalsVadim Petrochenkov-0/+7
2019-05-11Keep the original token in `ast::Lit`Vadim Petrochenkov-0/+5
2019-05-07Implement built-in await syntaxTaylor Cramer-0/+5
Adds support for .await under the existing async_await feature gate. Moves macro-like await! syntax to the await_macro feature gate. Removes support for `await` as a non-keyword under the `async_await` feature.
2019-03-06Simplify codeEsteban Küber-18/+3
2019-03-06Emit missing unclosed delimiter errorsEsteban Küber-6/+6
2019-02-18Make `interpolated_to_tokenstream` a method on `Nonterminal`.Nicholas Nethercote-82/+81
2019-02-18Remove `LazyTokenStream`.Nicholas Nethercote-85/+28
It's present within `Token::Interpolated` as an optimization, so that if a nonterminal is converted to a `TokenStream` multiple times, the first-computed value is saved and reused. But in practice it's not needed. `interpolated_to_tokenstream()` is a cold function: it's only called a few dozen times while compiling rustc itself, and a few hundred times across the entire `rustc-perf` suite. Furthermore, when it is called, it is almost always the first conversion, so no benefit is gained from it. So this commit removes `LazyTokenStream`, along with the now-unnecessary `Token::interpolated()`. As well as a significant simplification, the removal speeds things up slightly, mostly due to not having to `drop` the `LazyTokenStream` instances.
2019-02-18Change `Token::interpolated_to_tokenstream()`.Nicholas Nethercote-9/+3
It is currently a method of `Token`, but it only is valid to call if `self` is a `Token::Interpolated`. This commit eliminates the possibility of misuse by changing it to an associated function that takes a `Nonterminal`, which also simplifies the call sites. This requires splitting out a new function, `nonterminal_to_string`.
2019-02-10rustc: doc commentsAlexander Regueiro-10/+10
2019-02-09Auto merge of #57944 - estebank:unclosed-delim-the-quickening, r=oli-obkbors-5/+11
Deduplicate mismatched delimiter errors Delay unmatched delimiter errors until after the parser has run to deduplicate them when parsing and attempt recovering intelligently. Second attempt at #54029, follow up to #53949. Fix #31528.
2019-02-07Make name resolution handle consts in GenericParamsFromOuterFunction properlyvarkor-1/+2
2019-02-07Adjust parser generic parameter errorsvarkor-1/+0
2019-02-07Parse const genericsvarkor-0/+14
Co-Authored-By: Gabriel Smith <yodaldevoid@users.noreply.github.com>
2019-02-07unify error handling to single methodEsteban Küber-18/+4
2019-02-07Deduplicate mismatched delimiter errorsEsteban Küber-5/+25
Delay unmatched delimiter errors until after the parser has run to deduplicate them when parsing and attempt recovering intelligently.
2019-02-07libsyntax => 2018Taiki Endo-16/+20
2019-02-06Overhaul `syntax::fold::Folder`.Nicholas Nethercote-1/+1
This commit changes `syntax::fold::Folder` from a functional style (where most methods take a `T` and produce a new `T`) to a more imperative style (where most methods take and modify a `&mut T`), and renames it `syntax::mut_visit::MutVisitor`. The first benefit is speed. The functional style does not require any reallocations, due to the use of `P::map` and `MoveMap::move_{,flat_}map`. However, every field in the AST must be overwritten; even those fields that are unchanged are overwritten with the same value. This causes a lot of unnecessary memory writes. The imperative style reduces instruction counts by 1--3% across a wide range of workloads, particularly incremental workloads. The second benefit is conciseness; the imperative style is usually more concise. E.g. compare the old functional style: ``` fn fold_abc(&mut self, abc: ABC) { ABC { a: fold_a(abc.a), b: fold_b(abc.b), c: abc.c, } } ``` with the imperative style: ``` fn visit_abc(&mut self, ABC { a, b, c: _ }: &mut ABC) { visit_a(a); visit_b(b); } ``` (The reductions get larger in more complex examples.) Overall, the patch removes over 200 lines of code -- even though the new code has more comments -- and a lot of the remaining lines have fewer characters. Some notes: - The old style used methods called `fold_*`. The new style mostly uses methods called `visit_*`, but there are a few methods that map a `T` to something other than a `T`, which are called `flat_map_*` (`T` maps to multiple `T`s) or `filter_map_*` (`T` maps to 0 or 1 `T`s). - `move_map.rs`/`MoveMap`/`move_map`/`move_flat_map` are renamed `map_in_place.rs`/`MapInPlace`/`map_in_place`/`flat_map_in_place` to reflect their slightly changed signatures. - Although this commit renames the `fold` module as `mut_visit`, it keeps it in the `fold.rs` file, so as not to confuse git. The next commit will rename the file.
2019-01-20Mark incorrect recovered `char` literals as `TyErr` to avoid type errorsYuki Okushi-2/+1
generated by cgit-pink 1.4.1-3-g733a5 (git 2.37.1) at 2026-02-23 05:59:31 +0000