Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
fix: don't OOM if file ends in a null byte not preceded by whitespace
Initially, I couldn't reproduce this because Neovim inserts a new line at the end of the file. This prevents the `text` external rule to match with 0 width, hence preventing the OOM. Since at the start of `scan_text` we call `lexer->mark_end`, advancing the state of the lexer inside the while loop doesn't increase the size of the identified token until the next iteration. Before this commit, when the last character of a file wasn't whitespace, the `text` rule matched with 0 width, producing the OOM. Making the loop a `while (true)` and checking at the end of it for `lexer->eof(lexer)` solves this problem.
- Loading branch information