Skip to content

Commit

Permalink
fix: don't OOM if file ends in a null byte not preceded by whitespace
Browse files Browse the repository at this point in the history
Initially, I couldn't reproduce this because Neovim inserts a new line
at the end of the file. This prevents the `text` external rule to match
with 0 width, hence preventing the OOM.

Since at the start of `scan_text` we call `lexer->mark_end`, advancing
the state of the lexer inside the while loop doesn't increase the size
of the identified token until the next iteration. Before this commit,
when the last character of a file wasn't whitespace, the `text` rule
matched with 0 width, producing the OOM. Making the loop a `while
(true)` and checking at the end of it for `lexer->eof(lexer)` solves
this problem.
  • Loading branch information
TheLeoP committed Oct 29, 2024
1 parent d9bce8c commit c5336a1
Showing 1 changed file with 6 additions and 5 deletions.
11 changes: 6 additions & 5 deletions src/scanner.c
Original file line number Diff line number Diff line change
Expand Up @@ -377,7 +377,7 @@ static bool scan_text(Scanner *scanner, TSLexer *lexer) {
: scanner->end_delimiter.size;
int start_i = 0;
int end_i = 0;
while (!lexer->eof(lexer)) {
while (true) {
int ith_start = get_delimiter(scanner->start_delimiter, start_i,
DEFAULT_START_DELIMITER);
if (lexer->lookahead == ith_start) {
Expand All @@ -389,9 +389,8 @@ static bool scan_text(Scanner *scanner, TSLexer *lexer) {
}
start_i = 0;
}
if (start_i == start_delimiter_max) {
if (start_i == start_delimiter_max)
break;
}

int ith_end =
get_delimiter(scanner->end_delimiter, end_i, DEFAULT_END_DELIMITER);
Expand All @@ -404,9 +403,11 @@ static bool scan_text(Scanner *scanner, TSLexer *lexer) {
}
end_i = 0;
}
if (end_i == end_delimiter_max) {
if (end_i == end_delimiter_max)
break;

if (lexer->eof(lexer))
break;
}

lexer->advance(lexer, false);
}
Expand Down

0 comments on commit c5336a1

Please sign in to comment.