about summary refs log tree commit diff
path: root/src/libsyntax/parse/lexer
AgeCommit message (Collapse)AuthorLines
2019-04-04Simplify doc comment lexingAleksey Kladov-17/+5
is_doc_comment function checks the first four chars, but this is redundant, `doc_comment` local var has the same info.
2019-04-04cleanup shebang handling in the lexerAleksey Kladov-11/+5
2019-04-03make StringReader methods privateAleksey Kladov-2/+2
2019-04-03make StringReader fields privateAleksey Kladov-6/+6
2019-04-02Update more unit test to new APIOliver Scherer-0/+1
2019-03-20Tweak incorrect escaped char diagnosticEsteban Küber-19/+19
2019-03-16syntax: Introduce `Ident::can_be_raw`Vadim Petrochenkov-8/+4
2019-03-13Rollup merge of #58876 - estebank:numeric-lifetime, r=petrochenkovMazdak Farrokhzad-3/+15
Parse lifetimes that start with a number and give specific error Fix #58786.
2019-03-09review commentsEsteban Küber-4/+4
2019-03-08Parse lifetimes that start with a number and give specific errorEsteban Küber-6/+18
2019-03-07Fix with_emitter callersEsteban Küber-1/+1
2019-02-16help suggestion when trying to delimit string literals with directed unicode ↵Patrick McCarter-9/+44
quotes #58436
2019-02-13Rollup merge of #58273 - taiki-e:rename-dependency, r=matthewjasperMazdak Farrokhzad-4/+3
Rename rustc_errors dependency in rust 2018 crates I think this is a better solution than `use rustc_errors as errors` in `lib.rs` and `use crate::errors` in modules. Related: rust-lang/cargo#5653 cc #58099 r? @Centril
2019-02-13Cleanup importsTaiki Endo-2/+2
2019-02-13Rename rustc_errors dependency in rust 2018 cratesTaiki Endo-3/+2
2019-02-10rustc: doc commentsAlexander Regueiro-9/+9
2019-02-07Deduplicate mismatched delimiter errorsEsteban Küber-15/+26
Delay unmatched delimiter errors until after the parser has run to deduplicate them when parsing and attempt recovering intelligently.
2019-02-07libsyntax => 2018Taiki Endo-33/+41
2019-01-26remove `_with_applicability` from suggestion fnsAndy Russell-3/+3
2019-01-20Auto merge of #56884 - euclio:codeblock-diagnostics, r=QuietMisdreavusbors-13/+0
rustdoc: overhaul code block lexing errors Fixes #53919. This PR moves the reporting of code block lexing errors from rendering time to an early pass, so we can use the compiler's error reporting mechanisms. This dramatically improves the diagnostics in this situation: we now de-emphasize the lexing errors as a note under a warning that has a span and suggestion instead of just emitting errors at the top level. Additionally, this PR generalizes the markdown -> source span calculation function, which should allow other rustdoc warnings to use better spans in the future. Last, the PR makes sure that the code block is always emitted in the docs, even if it fails to highlight correctly. Of note: - The new pass unfortunately adds another pass over the docs to gather the doc blocks for syntax-checking. I wonder if this could be combined with the pass that looks for testable blocks? I'm not familiar with that code, so I don't know how feasible that is. - `pulldown_cmark` doesn't make it easy to find the spans of the code blocks, so the code that calculates the spans is a little nasty. It works for all the test cases I threw at it, but I wouldn't be surprised if an edge case would break it. Should have a thorough review. - This PR worsens the state of #56885, since those certain fatal lexing errors are now emitted before docs get generated at all.
2019-01-20Revert changeYuki Okushi-3/+2
2019-01-20[WIP] Improve error behaviorYuki Okushi-6/+8
2019-01-18Continue chekingYuki Okushi-1/+1
2019-01-16Cancel processYuki Okushi-1/+1
2019-01-14rustdoc: check code block syntax in early passAndy Russell-13/+0
2019-01-14Rollup merge of #57540 - estebank:eval-more, r=petrochenkovMazdak Farrokhzad-1/+1
Modify some parser diagnostics to continue evaluating beyond the parser Continue evaluating further errors after parser errors on: - trailing type argument attribute - lifetime in incorrect location - incorrect binary literal - missing `for` in `impl Trait for Foo` - type argument in `where` clause - incorrect float literal - incorrect `..` in pattern - associated types - incorrect discriminator value variant error and others. All of these were found by making `continue-parse-after-error` `true` by default to identify errors that would need few changes. There are now only a handful of errors that have any change with `continue-parse-after-error` enabled. These changes make it so `rust` _won't_ stop evaluation after finishing parsing, enabling type checking errors to be displayed on the existing code without having to fix the parse errors. Each commit has an individual diagnostic change with their corresponding tests. CC #48724.
2019-01-11Continue evaluating after parsing incorrect binary literalEsteban Küber-1/+1
2019-01-08Make `TokenStream` less recursive.Nicholas Nethercote-4/+4
`TokenStream` is currently recursive in *two* ways: - the `TokenTree` variant contains a `ThinTokenStream`, which can contain a `TokenStream`; - the `TokenStream` variant contains a `Vec<TokenStream>`. The latter is not necessary and causes significant complexity. This commit replaces it with the simpler `Vec<(TokenTree, IsJoint)>`. This reduces complexity significantly. In particular, `StreamCursor` is eliminated, and `Cursor` becomes much simpler, consisting now of just a `TokenStream` and an index. The commit also removes the `Extend` impl for `TokenStream`, because it is only used in tests. (The commit also removes those tests.) Overall, the commit reduces the number of lines of code by almost 200.
2018-12-31Do not use unicode character in diagnostic helpEsteban Küber-1/+1
2018-12-31Use structured suggestion for braceless unicode escape squenceEsteban Küber-6/+30
2018-12-25Remove licensesMark Rousskov-40/+0
2018-12-23Rollup merge of #56964 - nnethercote:TokenStream-IsJoint, r=petrochenkovMazdak Farrokhzad-3/+2
Remove `TokenStream::JointTree`. This is done by adding a new `IsJoint` field to `TokenStream::Tree`, which simplifies a lot of `match` statements. And likewise for `CursorKind`. The commit also adds a new method `TokenTree:stream()` which can replace a choice between `.into()` and `.joint()`.
2018-12-20Remove `TokenStream::JointTree`.Nicholas Nethercote-3/+2
This is done by adding a new `IsJoint` field to `TokenStream::Tree`, which simplifies a lot of `match` statements. And likewise for `CursorKind`. The commit also adds a new method `TokenTree:stream()` which can replace a choice between `.into()` and `.joint()`.
2018-12-19Rollup merge of #56910 - estebank:unclosed-eof, r=oli-obkPietro Albini-1/+9
Do not point at delim spans for complete correct blocks Fix #56834.
2018-12-17Do not point at delim spans for complete correct blocksEsteban Küber-1/+9
2018-12-12Rename `TokenStream::concat` and remove `TokenStream::concat_rc_vec`.Nicholas Nethercote-3/+3
`TokenStream::new` is a better name for the former, and the latter is now just equivalent to `TokenStream::Stream`.
2018-12-10Remove `tokenstream::Delimited`.Nicholas Nethercote-4/+5
Because it's an extra type layer that doesn't really help; in a couple of places it actively gets in the way, and overall removing it makes the code nicer. It does, however, move `tokenstream::TokenTree` further away from the `TokenTree` in `quote.rs`. More importantly, this change reduces the size of `TokenStream` from 48 bytes to 40 bytes on x86-64, which is enough to slightly reduce instruction counts on numerous benchmarks, the best by 1.5%. Note that `open_tt` and `close_tt` have gone from being methods on `Delimited` to associated methods of `TokenTree`.
2018-12-07Various minor/cosmetic improvements to codeAlexander Regueiro-1/+1
2018-12-06Rollup merge of #56500 - ljedrz:cleanup_rest_of_const_lifetimes, r=zackmdavisPietro Albini-1/+1
cleanup: remove static lifetimes from consts A follow-up to https://github.com/rust-lang/rust/pull/56497.
2018-12-06Auto merge of #54517 - mcr431:53956-panic-on-include_bytes-of-own-file, ↵bors-1/+1
r=michaelwoerister 53956 panic on include bytes of own file fix #53956 When using `include_bytes!` on a source file in the project, compiler would panic on subsequent compilations because `expand_include_bytes` would overwrite files in the source_map with no source. This PR changes `expand_include_bytes` to check source_map and use the already existing src, if any.
2018-12-05Fix pretty testShotaro Yamada-1/+1
2018-12-04adds DocTest filename variant, refactors doctest_offset out of source_map, ↵Matthew Russo-1/+1
fixes remaining test failures
2018-12-04cleanup: remove static lifetimes from constsljedrz-1/+1
2018-12-04Use iterator and pattern APIs instead of `char_at`Shotaro Yamada-38/+26
2018-11-26Update outdated code comments in StringReaderJason Langenauer-3/+2
2018-11-14Clean up some non-mod-rs stuff.Eric Huss-1/+0
2018-11-04Auto merge of #54861 - rep-nop:find_main_in_doctest, r=estebankbors-1/+22
rustdoc: Replaces fn main search and extern crate search with proper parsing during doctests. Fixes #21299. Fixes #33731. Let me know if there's any additional changes you'd like made!
2018-11-01buffer errors from initial tokenization when parsingQuietMisdreavus-1/+22
2018-10-29Rename other occs of (Code/File)Map to Source(Map/File) #51574David Lavati-54/+54
2018-09-29Add a sentence before rustc errorsGuillaume Gomez-1/+3