about summary refs log tree commit diff
path: root/src/librustc_parse/lexer/mod.rs
AgeCommit message (Collapse)AuthorLines
2020-08-30mv compiler to compiler/mark-598/+0
2020-08-30Rollup merge of #76057 - matklad:remove-retokenize, r=petrochenkovDylan DPC-28/+2
Move retokenize hack to save_analysis closes #76046
2020-08-29Move retokenize hack to save_analysisAleksey Kladov-28/+2
2020-08-21Auto merge of #75642 - matklad:lexer-comments, r=petrochenkovbors-41/+55
Move doc comment parsing to rustc_lexer Plain comments are trivia, while doc comments are not, so it feels like this belongs to the rustc_lexer. The specific reason to do this is the desire to use rustc_lexer in rustdoc for syntax highlighting, without duplicating "is this a doc comment?" logic there. r? @ghost
2020-08-20Rename rustc_lexer::TokenKind::Not to BangAleksey Kladov-1/+1
All other tokens are named by the punctuation they use, rather than by semantics operation they stand for. `!` is the only exception to the rule, let's fix it.
2020-08-19Simplify search for bare `\r` in doc commentsAleksey Kladov-14/+11
Outer `if` is the fast path -- it calls into hyperoptimized memchr. The inner loop is just the simplest code possible -- it doesn't generated the tightest code, but that shouldn't matter if we are going to error anyhow.
2020-08-19Move doc comment parsing to rustc_lexerAleksey Kladov-41/+58
Plain comments are trivial, while doc comments are not, so it feels like this belongs to the rustc_lexer. The specific reason to do this is the desire to use rustc_lexer in rustdoc for syntax highlighting, without duplicating "is this a doc comment?" logic there.
2020-08-15replaced log with tracingGurpreet Singh-1/+1
2020-08-06Add some comments for magic numbers + Add testsVadim Petrochenkov-0/+3
2020-08-06rustc_ast: Stop using "string typing" for doc comment tokensVadim Petrochenkov-10/+11
Explicitly store their kind and style retrieved during lexing in the token
2020-07-04Create new E0768 error code for "no valid digits found for number" errorGuillaume Gomez-1/+8
2020-06-25Add E0766 error for unterminated double quote byte stringGuillaume Gomez-6/+9
2020-06-21Create E0765 error for unterminated double quote stringsGuillaume Gomez-2/+9
2020-06-16Create new E0763 error code for unterminated byte constantGuillaume Gomez-2/+9
2020-06-11Rollup merge of #73172 - matthiaskrgr:cl9ppy, r=Dylan-DPCDylan DPC-1/+1
Fix more clippy warnings Fixes more of: clippy::unused_unit clippy::op_ref clippy::useless_format clippy::needless_return clippy::useless_conversion clippy::bind_instead_of_map clippy::into_iter_on_ref clippy::redundant_clone clippy::nonminimal_bool clippy::redundant_closure clippy::option_as_ref_deref clippy::len_zero clippy::iter_cloned_collect clippy::filter_next r? @Dylan-DPC
2020-06-10Create new error code E0762 for unterminated char literalsGuillaume Gomez-1/+9
2020-06-09Fix more clippy warningsMatthias Krüger-1/+1
Fixes more of: clippy::unused_unit clippy::op_ref clippy::useless_format clippy::needless_return clippy::useless_conversion clippy::bind_instead_of_map clippy::into_iter_on_ref clippy::redundant_clone clippy::nonminimal_bool clippy::redundant_closure clippy::option_as_ref_deref clippy::len_zero clippy::iter_cloned_collect clippy::filter_next
2020-06-07Create new error code E0758 for unterminated multi-line commentsGuillaume Gomez-1/+9
2020-06-01Cleanup: Inline `struct_span_fatal()`, which is only called once, and remove ↵Julian Wollersberger-24/+5
an outdated FIXME.
2020-06-01Simplify raw string error reporting.Julian Wollersberger-27/+24
This makes `UnvalidatedRawStr` and `ValidatedRawStr` unnecessary and removes 70 lines.
2020-05-16Rollup merge of #72047 - ↵Ralf Jung-122/+30
Julian-Wollersberger:literal_error_reporting_cleanup, r=petrochenkov Literal error reporting cleanup While doing some performance work, I noticed some code duplication in `librustc_parser/lexer/mod.rs`, so I cleaned it up. This PR is probably best reviewed commit by commit. I'm not sure what the API stability practices for `librustc_lexer` are. Four public methods in `unescape.rs` can be removed, but two are used by clippy, so I left them in for now. I could open a PR for Rust-Analyzer when this one lands. But how do I open a PR for clippy? (Git submodules are frustrating to work with)
2020-05-14cleanup stale FIXME(#64197)sergey-melnychuk-2/+5
2020-05-13Unified `validate_{byte,str,raw_str,raw_byte_str}_escape` methods into one ↵Julian Wollersberger-122/+30
method `validate_literal_escape` with a mode argument. This enables simplifying the `match` in `cook_lexer_literal()` and it eliminates 90 lines of repetition :)
2020-05-03Implement `confusable_idents` lint.Charles Lew-1/+2
2020-04-01Rollup merge of #70522 - rcoh:60762-raw-string-errors, r=petrochenkovMazdak Farrokhzad-34/+59
Improve error messages for raw strings (#60762) This diff improves error messages around raw strings in a few ways: - Catch extra trailing `#` in the parser. This can't be handled in the lexer because we could be in a macro that actually expects another # (see test) - Refactor & unify error handling in the lexer between ByteStrings and RawByteStrings - Detect potentially intended terminators (longest sequence of "#*" is suggested) Fixes #60762 cc @estebank who reviewed the original (abandoned) PR for the same ticket. r? @Centril
2020-03-30remove obsolete commentTshepang Lekhonkhobe-3/+0
Made obsolete by b5e35b128efeed4bfdb4b1ee9d0697389ec9f164
2020-03-29Cleanup error messages, improve docstringsRussell Cohen-5/+4
2020-03-29Improve error messages for raw strings (#60762)Russell Cohen-34/+60
This diff improves error messages around raw strings in a few ways: - Catch extra trailing `#` in the parser. This can't be handled in the lexer because we could be in a macro that actually expects another # (see test) - Refactor & unify error handling in the lexer between ByteStrings and RawByteStrings - Detect potentially intended terminators (longest sequence of "#*" is suggested)
2020-03-22Rollup merge of #70254 - matthiaskrgr:cl4ppy, r=CentrilDylan DPC-8/+4
couple more clippy fixes (let_and_return, if_same_then_else) * summarize if-else-code with identical blocks (clippy::if_same_then_else) * don't create variable bindings just to return the bound value immediately (clippy::let_and_return)
2020-03-22don't create variable bindings just to return the bound value immediately ↵Matthias Krüger-8/+4
(clippy::let_and_return)
2020-03-20parse/lexer: support `StringReader::retokenize` called on external files.Eduard-Mihai Burtescu-4/+12
2020-02-29Rename `syntax` to `rustc_ast` in source codeVadim Petrochenkov-2/+2
2020-02-29Make it build againVadim Petrochenkov-1/+1
2020-02-28Add E0747 error codeGuillaume Gomez-2/+6
2020-02-27don't use .into() to convert types into identical types.Matthias Krüger-10/+5
example: let s: String = format!("hello").into();
2020-02-09Make issue references consistentMatthias Prechtl-2/+3
2020-02-011. move node_id to syntaxMazdak Farrokhzad-1/+1
2. invert rustc_session & syntax deps 3. drop rustc_session dep in rustc_hir
2020-01-01Rename `syntax_pos` to `rustc_span` in source codeVadim Petrochenkov-3/+3
2019-12-31Inline and remove `nfc_symbol_from` method.Charles Lew-8/+1
2019-12-31Add symbol normalization for proc_macro_server.Charles Lew-8/+12
2019-12-26Normalize identifiers in librustc_parse.Charles Lew-2/+15
2019-12-22Format the worldMark Rousskov-101/+99
2019-12-05rustc_parser: cleanup importsMazdak Farrokhzad-6/+5
2019-11-10move syntax::parse -> librustc_parseMazdak Farrokhzad-0/+643
also move MACRO_ARGUMENTS -> librustc_parse
generated by cgit-pink 1.4.1-3-g733a5 (git 2.37.1) at 2026-02-27 06:34:16 +0000