Rollup merge of #146899 - Teapot4195:issue-146847-fix, r=nnethercote

Fix a crash/mislex when more than one frontmatter closing possibility is considered

When the less fortunate recovery path for frontmatters are taken, if the lexer considers more than one possible frontmatter closing possibility, the current index is entirely mis-tracked and can result in bump_bytes landing in the middle of a multichar unicode character.

This fixes it by tracking the actual base index and updating it as it considers additional closing possibilities.

fixes rust-lang/rust#146847

1 files changed, 3 insertions, 1 deletions

diff --git a/compiler/rustc_lexer/src/lib.rs b/compiler/rustc_lexer/src/lib.rs
index c29ab569f47..f6790f7ed1e 100644
--- a/compiler/rustc_lexer/src/lib.rs
+++ b/compiler/rustc_lexer/src/lib.rs
@@ -599,14 +599,16 @@ impl Cursor<'_> {
             if potential_closing.is_none() {
                 // a less fortunate recovery if all else fails which finds any dashes preceded by whitespace
                 // on a standalone line. Might be wrong.
+                let mut base_index = 0;
                 while let Some(closing) = rest.find("---") {
                     let preceding_chars_start = rest[..closing].rfind("\n").map_or(0, |i| i + 1);
                     if rest[preceding_chars_start..closing].chars().all(is_horizontal_whitespace) {
                         // candidate found
-                        potential_closing = Some(closing);
+                        potential_closing = Some(closing + base_index);
                         break;
                     } else {
                         rest = &rest[closing + 3..];
+                        base_index += closing + 3;
                     }
                 }
             }