about summary refs log tree commit diff
path: root/src/libsyntax/parse/mod.rs
AgeCommit message (Collapse)AuthorLines
2019-05-22Rollup merge of #60995 - topecongiro:parser-from-stream-and-base-dir, ↵Mazdak Farrokhzad-0/+17
r=michaelwoerister Add stream_to_parser_with_base_dir This PR adds `stream_to_parser_with_base_dir`, which creates a parser from a token stream and a base directory. Context: I would like to parse `cfg_if!` macro and get a list of modules defined inside it from rustfmt so that rustfmt can format those modules (cc https://github.com/rust-lang/rustfmt/issues/3253). To do so, I need to create a parser from `TokenStream` and set the directory of `Parser` to the same directory as the parent directory of a file which contains `cfg_if!` invocation. AFAIK there is no way to achieve this, and hence this PR. Alternatively, I could change the visibility of `Parser.directory` from `crate` to `pub` so that the value can be modified after initializing a parser. I don't have a preference over either approach (or others, as long as it works).
2019-05-21Move `edition` outside the hygiene lock and avoid accessing itJohn Kåre Alsaker-12/+15
2019-05-21Fix tidy: remove a trailing whitespacetopecongiro-1/+1
2019-05-21Add doc commenttopecongiro-0/+10
2019-05-21Add stream_to_parser_with_base_dirtopecongiro-0/+7
2019-05-16Auto merge of #60763 - matklad:tt-parser, r=petrochenkovbors-6/+6
Move token tree related lexer state to a separate struct Just a types-based refactoring. We only used a bunch of fields when tokenizing into a token tree, so let's move them out of the base lexer
2019-05-13move span and token to tt readerAleksey Kladov-2/+1
2019-05-13Move token tree related lexer state to a separate structAleksey Kladov-4/+5
We only used a bunch of fields when tokenizing into a token tree, so let's move them out of the base lexer
2019-05-13Remove the equality operation between `Symbol` and strings.Nicholas Nethercote-7/+9
And also the equality between `Path` and strings, because `Path` is made up of `Symbol`s.
2019-05-13Pass a `Symbol` to `check_name`, `emit_feature_err`, and related functions.Nicholas Nethercote-2/+4
2019-05-11Move literal parsing code into a separate fileVadim Petrochenkov-345/+7
Remove some dead code
2019-05-11Simplify conversions between tokens and semantic literalsVadim Petrochenkov-79/+133
2019-05-11Keep the original token in `ast::Lit`Vadim Petrochenkov-0/+1
2019-05-09Rollup merge of #60188 - estebank:recover-block, r=varkorMazdak Farrokhzad-2/+25
Identify when a stmt could have been parsed as an expr There are some expressions that can be parsed as a statement without a trailing semicolon depending on the context, which can lead to confusing errors due to the same looking code being accepted in some places and not others. Identify these cases and suggest enclosing in parenthesis making the parse non-ambiguous without changing the accepted grammar. Fix #54186, cc #54482, fix #59975, fix #47287.
2019-05-06review comments: fix typo and add commentsEsteban Küber-2/+2
2019-05-06Auto merge of #60261 - matklad:one-escape, r=petrochenkovbors-222/+47
introduce unescape module A WIP PR to gauge early feedback Currently, we deal with escape sequences twice: once when we [lex](https://github.com/rust-lang/rust/blob/112f7e9ac564e2cfcfc13d599c8376a219fde1bc/src/libsyntax/parse/lexer/mod.rs#L928-L1065) a string, and a second time when we [unescape](https://github.com/rust-lang/rust/blob/112f7e9ac564e2cfcfc13d599c8376a219fde1bc/src/libsyntax/parse/mod.rs#L313-L366) literals. Note that we also produce different sets of diagnostics in these two cases. This PR aims to remove this duplication, by introducing a new `unescape` module as a single source of truth for character escaping rules. I think this would be a useful cleanup by itself, but I also need this for https://github.com/rust-lang/rust/pull/59706. In the current state, the PR has `unescape` module which fully (modulo bugs) deals with string and char literals. I am quite happy about the state of this module What this PR doesn't have yet are: * [x] handling of byte and byte string literals (should be simple to add) * [x] good diagnostics * [x] actual removal of code from lexer (giant `scan_char_or_byte` should go away completely) * [x] performance check * [x] general cleanup of the new code Diagnostics will be the most labor-consuming bit here, but they are mostly a question of just correctly adjusting spans to sub-tokens. The current setup for diagnostics is that `unescape` produces a plain old `enum` with various problems, and they are rendered into `Handler` separately. This bit is not actually required (it is possible to just pass the `Handler` in), but I like the separation between diagnostics and logic this approach imposes, and such separation should again be useful for #59706 cc @eddyb , @petrochenkov
2019-05-02Deduplicate needed parentheses suggestion codeEsteban Küber-1/+22
2019-05-02don't amplify errors in format! with bad literalsAleksey Kladov-7/+14
2019-05-02introduce unescape moduleAleksey Kladov-222/+40
Currently, we deal with escape sequences twice: once when we lex a string, and a second time when we unescape literals. This PR aims to remove this duplication, by introducing a new `unescape` mode as a single source of truth for character escaping rules
2019-05-02Rollup merge of #60348 - agnxy:refactor-parser, r=petrochenkovMazdak Farrokhzad-0/+1
move some functions from parser.rs to diagostics.rs Starting with a few functions mentioned in https://github.com/rust-lang/rust/issues/60015#issuecomment-484259773. We might refactor parser.rs further in subsequent changes. r? @petrochenkov
2019-05-01move some functions from parser.rs to diagostics.rsAndrew Xu-0/+1
parser.rs is too big. Some functions only for error reporting and error recovery are being moved to diagostics.rs.
2019-04-29Identify when a stmt could have been parsed as an exprEsteban Küber-1/+3
There are some expressions that can be parsed as a statement without a trailing semicolon depending on the context, which can lead to confusing errors due to the same looking code being accepted in some places and not others. Identify these cases and suggest enclosing in parenthesis making the parse non-ambiguous without changing the accepted grammar.
2019-04-23reduce visibilityAleksey Kladov-1/+1
2019-03-09Expose new_sub_parser_from_filetopecongiro-1/+1
This function is useful when external tools like rustfmt want to parse internal files without parsing a whole crate.
2019-03-09Auto merge of #59012 - pietroalbini:rollup, r=pietroalbinibors-1/+1
Rollup of 24 pull requests Successful merges: - #58080 (Add FreeBSD armv6 and armv7 targets) - #58204 (On return type `impl Trait` for block with no expr point at last semi) - #58269 (Add librustc and libsyntax to rust-src distribution.) - #58369 (Make the Entry API of HashMap<K, V> Sync and Send) - #58861 (Expand where negative supertrait specific error is shown) - #58877 (Suggest removal of `&` when borrowing macro and appropriate) - #58883 (Suggest appropriate code for unused field when destructuring pattern) - #58891 (Remove stray ` in the docs for the FromIterator implementation for Option) - #58893 (race condition in thread local storage example) - #58906 (Monomorphize generator field types for debuginfo) - #58911 (Regression test for #58435.) - #58912 (Regression test for #58813) - #58916 (Fix release note problems noticed after merging.) - #58918 (Regression test added for an async ICE.) - #58921 (Add an explicit test for issue #50582) - #58926 (Make the lifetime parameters of tcx consistent.) - #58931 (Elide invalid method receiver error when it contains TyErr) - #58940 (Remove JSBackend from config.toml) - #58950 (Add self to mailmap) - #58961 (On incorrect cfg literal/identifier, point at the right span) - #58963 (libstd: implement Error::source for io::Error) - #58970 (delay_span_bug in wfcheck's ty.lift_to_tcx unwrap) - #58984 (Teach `-Z treat-err-as-bug` to take a number of errors to emit) - #59007 (Add a test for invalid const arguments) Failed merges: - #58959 (Add release notes for PR #56243) r? @ghost
2019-03-06Make `-Z treat-err-as-bug` take a number of errors to be emittedEsteban Küber-1/+1
`-Z treat-err-as-bug=0` will cause `rustc` to panic after the first error is reported. `-Z treat-err-as-bug=2` will cause `rustc` to panic after 3 errors have been reported.
2019-03-06Simplify codeEsteban Küber-2/+9
2019-02-13Rollup merge of #58273 - taiki-e:rename-dependency, r=matthewjasperMazdak Farrokhzad-1/+1
Rename rustc_errors dependency in rust 2018 crates I think this is a better solution than `use rustc_errors as errors` in `lib.rs` and `use crate::errors` in modules. Related: rust-lang/cargo#5653 cc #58099 r? @Centril
2019-02-13Cleanup importsTaiki Endo-1/+1
2019-02-13Rename rustc_errors dependency in rust 2018 cratesTaiki Endo-1/+1
2019-02-10rustc: doc commentsAlexander Regueiro-22/+21
2019-02-07unify error handling to single methodEsteban Küber-0/+1
2019-02-07Deduplicate mismatched delimiter errorsEsteban Küber-17/+40
Delay unmatched delimiter errors until after the parser has run to deduplicate them when parsing and attempt recovering intelligently.
2019-02-07libsyntax => 2018Taiki Endo-30/+32
2019-01-24Remove quote_*! macros and associated APIsMark Simulacrum-22/+17
2019-01-20Auto merge of #57651 - JohnTitor:give-char-type, r=estebankbors-0/+1
Implement new literal type `Err` Fixes #57384 I removed `return Ok`, otherwise, two errors occur. Any solutions? r? @estebank
2019-01-20Mark incorrect recovered `char` literals as `TyErr` to avoid type errorsYuki Okushi-1/+1
2019-01-19Rollup merge of #57486 - nnethercote:simplify-TokenStream-more, r=petrochenkovMazdak Farrokhzad-3/+3
Simplify `TokenStream` some more These commits simplify `TokenStream`, remove `ThinTokenStream`, and avoid some clones. The end result is simpler code and a slight perf win on some benchmarks. r? @petrochenkov
2019-01-16Add new literal type ErrYuki Okushi-0/+1
2019-01-14Remove `ThinTokenStream`.Nicholas Nethercote-3/+3
`TokenStream` is now almost identical to `ThinTokenStream`. This commit removes the latter, replacing it with the former.
2019-01-11Add label for invalid literal suffixEsteban Küber-0/+2
2019-01-11Continue evaluating after incorrect float literalEsteban Küber-1/+5
2018-12-25Remove licensesMark Rousskov-10/+0
2018-12-17Tweak query code for performanceJohn Kåre Alsaker-0/+1
2018-12-17Auto merge of #56737 - nnethercote:TokenStream-improvements, r=petrochenkovbors-3/+3
`TokenStream` improvements Some `TokenStream` improvements: shrinking `TokenStream` and some other types, and some other code clean-ups.
2018-12-14Rollup merge of #56658 - Xanewok:non-panicking-file-parser, r=petrochenkovkennytm-8/+29
Add non-panicking `maybe_new_parser_from_file` variant Add (seemingly?) missing `maybe_new_parser_from_file` constructor variant. Disclaimer: I'm not certain this is the correct approach - just found out we don't have this when working on a Rustfmt PR to catch/prevent more Rust parser panics: https://github.com/rust-lang/rustfmt/pull/3240 and tried to make it work somehow.
2018-12-12Rename `TokenStream::concat` and remove `TokenStream::concat_rc_vec`.Nicholas Nethercote-3/+3
`TokenStream::new` is a better name for the former, and the latter is now just equivalent to `TokenStream::Stream`.
2018-12-10Remove `tokenstream::Delimited`.Nicholas Nethercote-32/+29
Because it's an extra type layer that doesn't really help; in a couple of places it actively gets in the way, and overall removing it makes the code nicer. It does, however, move `tokenstream::TokenTree` further away from the `TokenTree` in `quote.rs`. More importantly, this change reduces the size of `TokenStream` from 48 bytes to 40 bytes on x86-64, which is enough to slightly reduce instruction counts on numerous benchmarks, the best by 1.5%. Note that `open_tt` and `close_tt` have gone from being methods on `Delimited` to associated methods of `TokenTree`.
2018-12-09Add missing, non-panicking `maybe_new_parser_from_file` variantIgor Matuszewski-8/+29
2018-12-06Auto merge of #54517 - mcr431:53956-panic-on-include_bytes-of-own-file, ↵bors-4/+6
r=michaelwoerister 53956 panic on include bytes of own file fix #53956 When using `include_bytes!` on a source file in the project, compiler would panic on subsequent compilations because `expand_include_bytes` would overwrite files in the source_map with no source. This PR changes `expand_include_bytes` to check source_map and use the already existing src, if any.
generated by cgit-pink 1.4.1-3-g733a5 (git 2.37.1) at 2026-02-23 12:31:28 +0000