about summary refs log tree commit diff
path: root/src/libsyntax/parse/lexer
AgeCommit message (Collapse)AuthorLines
2015-01-02std: Stabilize the prelude moduleAlex Crichton-1/+2
This commit is an implementation of [RFC 503][rfc] which is a stabilization story for the prelude. Most of the RFC was directly applied, removing reexports. Some reexports are kept around, however: * `range` remains until range syntax has landed to reduce churn. * `Path` and `GenericPath` remain until path reform lands. This is done to prevent many imports of `GenericPath` which will soon be removed. * All `io` traits remain until I/O reform lands so imports can be rewritten all at once to `std::io::prelude::*`. This is a breaking change because many prelude reexports have been removed, and the RFC can be consulted for the exact list of removed reexports, as well as to find the locations of where to import them. [rfc]: https://github.com/rust-lang/rfcs/blob/master/text/0503-prelude-stabilization.md [breaking-change] Closes #20068
2014-12-21Fallout of std::str stabilizationAlex Crichton-17/+17
2014-12-19libsyntax: use `#[deriving(Copy)]`Jorge Aparicio-3/+1
2014-12-15Remove all shadowed lifetimes.Niko Matsakis-2/+2
2014-12-13libsyntax: use unboxed closuresJorge Aparicio-2/+6
2014-12-11Register new snapshotsAlex Crichton-8/+4
2014-12-09auto merge of #19644 : pcwalton/rust/oibit3, r=nikomatsakisbors-0/+2
2014-12-08Revert "Register new snapshots"Alex Crichton-3/+3
This reverts commit 9b443289cf32cbcff16768614340f0c844675340.
2014-12-08librustc: Make `Copy` opt-in.Niko Matsakis-0/+2
This change makes the compiler no longer infer whether types (structures and enumerations) implement the `Copy` trait (and thus are implicitly copyable). Rather, you must implement `Copy` yourself via `impl Copy for MyType {}`. A new warning has been added, `missing_copy_implementations`, to warn you if a non-generic public type has been added that could have implemented `Copy` but didn't. For convenience, you may *temporarily* opt out of this behavior by using `#![feature(opt_out_copy)]`. Note though that this feature gate will never be accepted and will be removed by the time that 1.0 is released, so you should transition your code away from using it. This breaks code like: #[deriving(Show)] struct Point2D { x: int, y: int, } fn main() { let mypoint = Point2D { x: 1, y: 1, }; let otherpoint = mypoint; println!("{}{}", mypoint, otherpoint); } Change this code to: #[deriving(Show)] struct Point2D { x: int, y: int, } impl Copy for Point2D {} fn main() { let mypoint = Point2D { x: 1, y: 1, }; let otherpoint = mypoint; println!("{}{}", mypoint, otherpoint); } This is the backwards-incompatible part of #13231. Part of RFC #3. [breaking-change]
2014-12-08auto merge of #19378 : japaric/rust/no-as-slice, r=alexcrichtonbors-22/+20
Now that we have an overloaded comparison (`==`) operator, and that `Vec`/`String` deref to `[T]`/`str` on method calls, many `as_slice()`/`as_mut_slice()`/`to_string()` calls have become redundant. This patch removes them. These were the most common patterns: - `assert_eq(test_output.as_slice(), "ground truth")` -> `assert_eq(test_output, "ground truth")` - `assert_eq(test_output, "ground truth".to_string())` -> `assert_eq(test_output, "ground truth")` - `vec.as_mut_slice().sort()` -> `vec.sort()` - `vec.as_slice().slice(from, to)` -> `vec.slice(from_to)` --- Note that e.g. `a_string.push_str(b_string.as_slice())` has been left untouched in this PR, since we first need to settle down whether we want to favor the `&*b_string` or the `b_string[]` notation. This is rebased on top of #19167 cc @alexcrichton @aturon
2014-12-06libsyntax: remove unnecessary `to_string()` callsJorge Aparicio-11/+11
2014-12-06libsyntax: remove unnecessary `as_slice()` callsJorge Aparicio-11/+9
2014-12-05Register new snapshotsAlex Crichton-3/+3
2014-12-03syntax: support ES6-style unicode escapesCorey Richardson-3/+78
First half of bootstrapping https://github.com/rust-lang/rfcs/pull/446
2014-11-30Adjust some error messages to start with a lowercase letter and not finish ↵P1start-1/+1
with a full stop
2014-11-25Deprecate MaybeOwned[Vector] in favor of CowJorge Aparicio-4/+4
2014-11-21unicode: Rename is_XID_start to is_xid_start, is_XID_continue to is_xid_continueBrian Anderson-3/+3
2014-11-21unicode: Add stability attributes to u_charBrian Anderson-2/+2
Free functions deprecated. UnicodeChar experimental pending final decisions about prelude.
2014-11-21core: Convert Char::escape_default, escape_unicode to iteratorsBrian Anderson-2/+2
[breaking-change]
2014-11-21Fix various deprecation warnings from char changesBrian Anderson-3/+3
2014-11-20Switch numeric suffix parsing to use the new system.Huon Wilson-70/+4
This moves errors and all handling of numeric suffixes into the parser rather than the lexer.
2014-11-20Parse and store suffixes on literals.Huon Wilson-28/+84
This adds an optional suffix at the end of a literal token: `"foo"bar`. An actual use of a suffix in a expression (or other literal that the compiler reads) is rejected in the parser. This doesn't switch the handling of numbers to this system, and doesn't outlaw illegal suffixes for them yet.
2014-11-19Switch to an independent enum for `Lit*` subtokens.Huon Wilson-20/+21
2014-11-17Switch to purely namespaced enumsSteven Fackler-0/+2
This breaks code that referred to variant names in the same namespace as their enum. Reexport the variants in the old location or alter code to refer to the new locations: ``` pub enum Foo { A, B } fn main() { let a = A; } ``` => ``` pub use self::Foo::{A, B}; pub enum Foo { A, B } fn main() { let a = A; } ``` or ``` pub enum Foo { A, B } fn main() { let a = Foo::A; } ``` [breaking-change]
2014-11-13Add error message specific to \<carriage return>.Huon Wilson-0/+7
This can crop-up with a misconfigured editor or an unexpected interaction between version control and certain operating systems. Closes #11669.
2014-11-05Register snapshots.Eduard Burtescu-6/+0
2014-11-04libsyntax: Forbid escapes in the inclusive range `\x80`-`\xff` inPatrick Walton-4/+19
Unicode characters and strings. Use `\u0080`-`\u00ff` instead. ASCII/byte literals are unaffected. This PR introduces a new function, `escape_default`, into the ASCII module. This was necessary for the pretty printer to continue to function. RFC #326. Closes #18062. [breaking-change]
2014-11-03Ignore whitespace tokens when re-computing spans in save_analysisNick Cameron-0/+13
2014-10-30Use common variants for open and close delimitersBrendan Zabarauskas-6/+6
This common representation for delimeters should make pattern matching easier. Having a separate `token::DelimToken` enum also allows us to enforce the invariant that the opening and closing delimiters must be the same in `ast::TtDelimited`, removing the need to ensure matched delimiters when working with token trees.
2014-10-29Rename fail! to panic!Steve Klabnik-7/+7
https://github.com/rust-lang/rfcs/pull/221 The current terminology of "task failure" often causes problems when writing or speaking about code. You often want to talk about the possibility of an operation that returns a Result "failing", but cannot because of the ambiguity with task failure. Instead, you have to speak of "the failing case" or "when the operation does not succeed" or other circumlocutions. Likewise, we use a "Failure" header in rustdoc to describe when operations may fail the task, but it would often be helpful to separate out a section describing the "Err-producing" case. We have been steadily moving away from task failure and toward Result as an error-handling mechanism, so we should optimize our terminology accordingly: Result-producing functions should be easy to describe. To update your code, rename any call to `fail!` to `panic!` instead. Assuming you have not created your own macro named `panic!`, this will work on UNIX based systems: grep -lZR 'fail!' . | xargs -0 -l sed -i -e 's/fail!/panic!/g' You can of course also do this by hand. [breaking-change]
2014-10-28Move token-to-string functions into print::pprustBrendan Zabarauskas-2/+2
2014-10-28Use an enum rather than a bool in token::IdentBrendan Zabarauskas-25/+36
2014-10-28Convert some token functions into methodsBrendan Zabarauskas-6/+5
2014-10-28Use PascalCase for token variantsBrendan Zabarauskas-94/+94
2014-10-24Add a lint for not using field pattern shorthandsP1start-1/+1
Closes #17792.
2014-10-19Remove a large amount of deprecated functionalityAlex Crichton-11/+11
Spring cleaning is here! In the Fall! This commit removes quite a large amount of deprecated functionality from the standard libraries. I tried to ensure that only old deprecated functionality was removed. This is removing lots and lots of deprecated features, so this is a breaking change. Please consult the deprecation messages of the deleted code to see how to migrate code forward if it still needs migration. [breaking-change]
2014-09-30librustc: Forbid `..` in range patterns.Patrick Walton-1/+1
This breaks code that looks like: match foo { 1..3 => { ... } } Instead, write: match foo { 1...3 => { ... } } Closes #17295. [breaking-change]
2014-09-18Fix fallout in tests from removing the use of Gc in ExpnInfo.Eduard Burtescu-3/+3
2014-09-03Fix spelling errors and capitalization.Joseph Crail-1/+1
2014-08-14librustc: Implement simple `where` clauses.Patrick Walton-12/+12
These `where` clauses are accepted everywhere generics are currently accepted and desugar during type collection to the type parameter bounds we have today. A new keyword, `where`, has been added. Therefore, this is a breaking change. Change uses of `where` to other identifiers. [breaking-change]
2014-08-13quote_expr macro: embed Ident using special encoding that preserves hygiene.Felix S. Klock II-0/+105
This adds support to `quote_expr!` and friends for round-trip hygienic preservation of Ident. Here are the pieces of the puzzle: * adding a method for encoding Ident for re-reading into token tree. * Support for reading such encoded Idents in the lexer. Note that one must peek ahead for MOD_SEP after scan_embedded_hygienic_ident. * To ensure that encoded Idents are only read when we are in the midst of expanding a `quote_expr` or similar, added a `read_embedded_ident` flag on `StringReader`. * pprust support for encoding Ident's as (uint,uint) pairs (for hygiene).
2014-08-06Use byte literals in libsyntaxnham-2/+2
2014-07-15Deprecate `str::from_utf8_owned`Adolfo Ochagavía-1/+1
Use `String::from_utf8` instead [breaking-change]
2014-07-11Add scaffolding for assigning alpha-numeric codes to rustc diagnosticsJakub Wieczorek-1/+1
2014-07-09Fix all the test falloutCorey Richardson-7/+17
2014-07-09ast: make Name its own typeCorey Richardson-25/+25
2014-07-09lexer: lex WS/COMMENT/SHEBANG rather than skippingCorey Richardson-57/+85
Now, the lexer will categorize every byte in its input according to the grammar. The parser skips over these while parsing, thus avoiding their presence in the input to syntax extensions.
2014-07-09syntax: don't parse numeric literals in the lexerCorey Richardson-180/+162
This removes a bunch of token types. Tokens now store the original, unaltered numeric literal (that is still checked for correctness), which is parsed into an actual number later, as needed, when creating the AST. This can change how syntax extensions work, but otherwise poses no visible changes. [breaking-change]
2014-07-09syntax: don't process string/char/byte/binary litsCorey Richardson-47/+53
This shuffles things around a bit so that LIT_CHAR and co store an Ident which is the original, unaltered literal in the source. When creating the AST, unescape and postprocess them. This changes how syntax extensions can work, slightly, but otherwise poses no visible changes. To get a useful value out of one of these tokens, call `parse::{char_lit, byte_lit, bin_lit, str_lit}` [breaking-change]
2014-07-09lexer: add ident_from and ident_from_to methodsCorey Richardson-0/+14