rust/compiler/rustc_lexer, branch perf-tmphttps://github.com/rust-lang/rust
http://git.dreamy.place/mirrors/rust/atom?h=perf-tmp2025-09-22T19:10:41+00:00Fix a crash/mislex when more than one frontmatter closing possibility is considered2025-09-22T19:10:41+00:00Alexhuangalex409@gmail.com2025-09-22T19:10:41+00:00urn:sha1:2d18c886f5754258109fad2e1e0c3bcca0cc9fa5Rollup merge of #146106 - epage:whitespace, r=fee1-dead2025-09-03T13:08:10+00:00Stuart CookZalathar@users.noreply.github.com2025-09-03T13:08:10+00:00urn:sha1:56213a553ea242e61af853258052913d0fd75451
fix(lexer): Only allow horizontal whitespace in frontmatter
In writing up the reference for frontmatter, I realized that we probably
shouldn't be accepting Unicode Line Ending characters between the code
fence and infostring or trailing after the infostring or a code fence.
In digging into the unicode specification we use for Whitespace, it
divides it up into categories, so I'm deferring to what it says for
horizontal whitespace for what should be used within a line.
Note, I am leaving out support for Unicode Default Ignorable characters.
I figure that can be discussed outside of this change within the
reference and tracking issue.
Fixes rust-lang/rust#145971
Frontmatter tracking issue: rust-lang/rust#136889
Revert introduction of `[workspace.dependencies]`.2025-09-02T09:12:54+00:00Nicholas Nethercoten.nethercote@gmail.com2025-09-02T09:09:13+00:00urn:sha1:301655eafe56a4b5064adebd412becf94b59221c
This was done in #145740 and #145947. It is causing problems for people
using r-a on anything that uses the rustc-dev rustup package, e.g. Miri,
clippy.
This repository has lots of submodules and subtrees and various
different projects are carved out of pieces of it. It seems like
`[workspace.dependencies]` will just be more trouble than it's worth.
fix(lexer): Only allow horizontal whitespace in frontmatter2025-09-02T01:51:39+00:00Ed Pageeopage@gmail.com2025-08-28T20:15:41+00:00urn:sha1:6f0da976c5f422ff5b1124d409e7d7e063fc4225
In writing up the reference for frontmatter, I realized that we probably
shouldn't be accepting Unicode Line Ending characters between the code
fence and infostring or trailing after the infostring or a code fence.
In digging into the unicode specification we use for Whitespace, it
divides it up into categories, so I'm deferring to what it says for
horizontal whitespace for what should be used within a line.
Note, I am leaving out support for Unicde Default Ignorable characters.
I figure that can be discussed outside of this change within the
reference and tracking issue.docs(lexer): Organize and document whitespace by Pattern_White_Space2025-09-02T01:51:39+00:00Ed Pageeopage@gmail.com2025-08-28T19:18:42+00:00urn:sha1:428e413414e0719a7758676ac49e07b85eb86decfix(lexer): Don't require frontmatters to be escaped with indented fences2025-08-28T19:08:33+00:00Ed Pageeopage@gmail.com2025-08-22T15:46:47+00:00urn:sha1:142e25e35621b7043cc86bdc3d74f463ca2032d1
The RFC only limits hyphens at the beginning of lines and not if they
are indented or embedded in other content.
Sticking to that approach was confirmed by the T-lang liason at
https://github.com/rust-lang/rust/issues/141367#issuecomment-3202217544
There is a regression in error message quality which I'm leaving for
someone if they feel this needs improving.
Remove unnecessary `[dependencies.unicode-properties]` entries.2025-08-27T22:08:40+00:00Nicholas Nethercoten.nethercote@gmail.com2025-08-27T22:08:40+00:00urn:sha1:b4c8fe2b4bdbf8ff3e0b6a38e6027a431d1aca32
The Cargo style guide says to put dependencies on a single line if they
fit.
Add `memchr` to `[workspace.dependencies]`.2025-08-27T03:59:32+00:00Nicholas Nethercoten.nethercote@gmail.com2025-08-21T21:26:26+00:00urn:sha1:dfa748e910b7b1cda37bfe709cd026da66a63e4cfix(lexer): Allow '-' in the infostring continue set2025-08-22T14:26:19+00:00Ed Pageeopage@gmail.com2025-08-22T14:26:19+00:00urn:sha1:f43f974b9e14af328b7982d6320cc4d7a9104410
This more closely matches the RFC and what our T-lang contact has asked
for, see https://github.com/rust-lang/rust/issues/136889#issuecomment-3212715312test(lexer): Add frontmatter unit test2025-07-10T15:25:29+00:00Ed Pageeopage@gmail.com2025-07-10T15:25:29+00:00urn:sha1:df53b3dc04950d980bd28ec4b04f48cd0cd01237