about summary refs log tree commit diff
path: root/src/libsyntax/parse/lexer
AgeCommit message (Collapse)AuthorLines
2018-09-29Improve error display for codeblocks in rustdocGuillaume Gomez-0/+11
2018-09-09Don't compute padding of braces unless they are unmatchedEsteban Küber-26/+23
2018-09-09Auto merge of #53902 - dtolnay:group, r=petrochenkovbors-3/+3
proc_macro::Group::span_open and span_close Before this addition, every delimited group like `(`...`)` `[`...`]` `{`...`}` has only a single Span that covers the full source location from opening delimiter to closing delimiter. This makes it impossible for a procedural macro to trigger an error pointing to just the opening or closing delimiter. The Rust compiler does not seem to have the same limitation: ```rust mod m { type T = } ``` ```console error: expected type, found `}` --> src/main.rs:3:1 | 3 | } | ^ ``` On that same input, a procedural macro would be forced to trigger the error on the last token inside the block, on the entire block, or on the next token after the block, none of which is really what you want for an error like above. This commit adds `group.span_open()` and `group.span_close()` which access the Span associated with just the opening delimiter and just the closing delimiter of the group. Relevant to Syn as we implement real error messages for when parsing fails in a procedural macro: https://github.com/dtolnay/syn/issues/476. ```diff impl Group { fn span(&self) -> Span; + fn span_open(&self) -> Span; + fn span_close(&self) -> Span; } ``` Fixes #48187 r? @alexcrichton
2018-09-08Track distinct spans for open and close delimiterDavid Tolnay-3/+3
2018-09-05Change wording of unclosed delimiter labelEsteban Küber-1/+4
2018-09-05Provide more context for unenclosed delimitersEsteban Küber-2/+43
* When encountering EOF, point at the last opening brace that does not have the same indentation level as its close delimiter. * When encountering the wrong type of close delimiter, point at the likely correct open delimiter to give a better idea of what went wrong.
2018-09-05Reword un-closed delimiter labelEsteban Küber-2/+2
2018-08-28Use FxHash{Map,Set} instead of the default Hash{Map,Set} everywhere in rustc.Eduard-Mihai Burtescu-3/+3
2018-08-23use String::new() instead of String::from(""), "".to_string(), "".to_owned() ↵Matthias Krüger-1/+1
or "".into()
2018-08-20Set applicability for more suggestions.Sébastien Duquette-2/+6
2018-08-19fix tidy errorsDonato Sciarra-6/+6
2018-08-19mv codemap source_mapDonato Sciarra-2/+2
2018-08-19mv codemap() source_map()Donato Sciarra-3/+3
2018-08-19mv (mod) codemap source_mapDonato Sciarra-3/+3
2018-08-19mv filemap source_fileDonato Sciarra-23/+23
2018-08-19mv FileMap SourceFileDonato Sciarra-5/+5
2018-08-19mv CodeMap SourceMapDonato Sciarra-20/+20
2018-08-17Rollup merge of #53373 - estebank:unclosed, r=petrochenkovkennytm-11/+21
Tweak unclosed delimiter parser error
2018-08-15Do not emit "incorrect close delimiter" twice in the same placeEsteban Küber-11/+19
2018-08-15Tweak unclosed delimiter parser errorEsteban Küber-2/+4
2018-08-13A few cleanups and minor improvements for the lexerljedrz-74/+134
2018-07-29Replace push loops with collect() and extend() where possibleljedrz-3/+1
2018-07-23Allow by default, fix testsmark-0/+1
2018-07-20proc_macro: move some implementation details to a rustc module.Eduard-Mihai Burtescu-6/+0
2018-07-14Remove some tests using AST comparisons, fix other testsVadim Petrochenkov-2/+4
2018-07-14Remove most of `PartialEq` impls from AST and HIR structuresVadim Petrochenkov-1/+1
2018-07-10Deny bare trait objects in in src/libsyntaxljedrz-1/+1
2018-06-28Fix FileMap::line_begin_pos().Michael Woerister-2/+4
The method relied on the FileMap still being under construction in order for it to do what the name promises. It's now independent of the current state.
2018-06-27Make FileMap::{lines, multibyte_chars, non_narrow_chars} non-mutable.Michael Woerister-19/+1
2018-06-26migrate codebase to `..=` inclusive range patternsZack M. Davis-2/+2
These were stabilized in March 2018's #47813, and are the Preferred Way to Do It going forward (q.v. #51043).
2018-06-15Auto merge of #50296 - cmdd:master, r=nikomatsakisbors-0/+14
Add error message for using >= 65535 hashes for raw string literal escapes Fixes #50111.
2018-06-09Crate-ify and delete unused code in syntax::parseMark Simulacrum-24/+22
2018-06-08provide error message when using more than 65535 hash symbols for raw stringsDavid Cao-0/+14
2018-05-31Rollup merge of #51240 - nnethercote:parse-2, r=nikomatsakisGuillaume Gomez-5/+3
Two minor parsing tweaks
2018-05-31Avoid an unnecessary `match` when lexing "<-".Nicholas Nethercote-5/+1
2018-05-31Tweak identifer lexing.Nicholas Nethercote-0/+2
By calling `bump()` after getting the first char, to avoid a redundant `ident_continue()` test on it.
2018-05-28Auto merge of #50724 - zackmdavis:applicability_rush, r=Manishearthbors-6/+7
add suggestion applicabilities to librustc and libsyntax A down payment on #50723. Interested in feedback on whether my `MaybeIncorrect` vs. `MachineApplicable` judgement calls are well-calibrated (and that we have a consensus on what this means). r? @Manishearth cc @killercup @estebank
2018-05-26Fix testEsteban Küber-4/+8
2018-05-25in which we check for confusable Unicodepoints in float literal exponentZack M. Davis-5/+12
The `FatalError.raise()` might seem unmotivated (in most places in the compiler, `err.emit()` suffices), but it's actually used to maintain behavior (viz., stop lexing, don't emit potentially spurious errors looking for the next token after the bad Unicodepoint in the exponent): the previous revision's `self.err_span_` ultimately calls `Handler::emit`, which aborts if the `Handler`'s continue_after_error flag is set, which seems to typically be true during lexing (see `phase_1_parse_input` and and how `CompileController::basic` has `continue_parse_after_error: false` in librustc_driver). Also, let's avoid apostrophes in error messages (the present author would argue that users expect a reassuringly detached, formal, above-it-all tone from a Serious tool like a compiler), and use an RLS-friendly structured suggestion. Resolves #49746.
2018-05-22Auto merge of #50838 - alexcrichton:token-impls, r=eddybbors-17/+40
rustc: Fix joint-ness of stringified token-streams This commit fixes `StringReader`'s parsing of tokens which have been stringified through procedural macros. Whether or not a token tree is joint is defined by span information, but when working with procedural macros these spans are often dummy and/or overridden which means that they end up considering all operators joint if they can! The fix here is to track the raw source span as opposed to the overridden span. With this information we can more accurately classify `Punct` structs as either joint or not. Closes #50700
2018-05-20suggestion applicabilities for libsyntax and librustc, run-rustfix testsZack M. Davis-6/+7
Consider this a down payment on #50723. To recap, an `Applicability` enum was recently (#50204) added, to convey to Rustfix and other tools whether we think it's OK for them to blindly apply the suggestion, or whether to prompt a human for guidance (because the suggestion might contain placeholders that we can't infer, or because we think it has a sufficiently high probability of being wrong even though it's— presumably—right often enough to be worth emitting in the first place). When a suggestion is marked as `MaybeIncorrect`, we try to use comments to indicate precisely why (although there are a few places where we just say `// speculative` because the present author's subjective judgement balked at the idea that the suggestion has no false positives). The `run-rustfix` directive is opporunistically set on some relevant UI tests (and a couple tests that were in the `test/ui/suggestions` directory, even if the suggestions didn't originate in librustc or libsyntax). This is less trivial than it sounds, because a surprising number of test files aren't equipped to be tested as fixed even when they contain successfully fixable errors, because, e.g., there are more, not-directly-related errors after fixing. Some test files need an attribute or underscore to avoid unused warnings tripping up the "fixed code is still producing diagnostics" check despite the fixes being correct; this is an interesting contrast-to/inconsistency-with the behavior of UI tests (which secretly pass `-A unused`), a behavior which we probably ought to resolve one way or the other (filed issue #50926). A few suggestion labels are reworded (e.g., to avoid phrasing it as a question, which which is discouraged by the style guidelines listed in `.span_suggestion`'s doc-comment).
2018-05-20lexer: Fix span override for the first token in a stringVadim Petrochenkov-9/+12
2018-05-18rustc: Fix joint-ness of stringified token-streamsAlex Crichton-17/+40
This commit fixes `StringReader`'s parsing of tokens which have been stringified through procedural macros. Whether or not a token tree is joint is defined by span information, but when working with procedural macros these spans are often dummy and/or overridden which means that they end up considering all operators joint if they can! The fix here is to track the raw source span as opposed to the overridden span. With this information we can more accurately classify `Punct` structs as either joint or not. Closes #50700
2018-05-18Auto merge of #50307 - petrochenkov:keyhyg2, r=nikomatsakisbors-1/+1
Implement edition hygiene for keywords Determine "keywordness" of an identifier in its hygienic context. cc https://github.com/rust-lang/rust/pull/49611 I've resurrected `proc` as an Edition-2015-only keyword for testing purposes, but it should probably be buried again. EDIT: `proc` is removed again.
2018-05-18Auto merge of #50566 - nnethercote:bump, r=petrochenkovbors-65/+55
Streamline `StringReader::bump` These patches make `bump` smaller and nicer. They speed up most runs for coercions and tuple-stress by 1--3%.
2018-05-17Turn some functions from `token.rs` into methods on `Ident`Vadim Petrochenkov-1/+1
2018-05-15Represent lifetimes as two joint tokens in proc macrosVadim Petrochenkov-4/+1
2018-05-15proc_macro: Validate inputs to `Punct::new` and `Ident::new`Vadim Petrochenkov-0/+9
2018-05-14Remove `StringReader::col`.Nicholas Nethercote-7/+13
It only has a single use, within code handling indented block comments. We can replace that with the new `FileMap::col_pos()`, which computes the col position (BytePos instead of CharPos) based on the record of the last newline char (which we already record). This is actually an improvement, because `trim_whitespace_prefix_and_push_line()` was using `col`, which is a `CharPos`, as a slice index, which is a byte/char confusion.
2018-05-14Make `nextnextch()` more closely resemble `nextch()`.Nicholas Nethercote-8/+7