fix: align linear token sequence semantics - #113
Conversation
There was a problem hiding this comment.
Pull request overview
This PR tightens and aligns the Linear Token Sequence (LTS) fast-path semantics with the intended Java/JDK regex behavior by (a) matching JDK-style whitespace for \s/\S, (b) making the bracket-tail shortcut enforce the literal-space delimiters it recognizes, and (c) preventing LTS admission of wildcard-dot patterns unless the pattern is globally DOTALL.
Changes:
- Align LTS whitespace/non-whitespace classification with JDK-style whitespace (and add JDK-differential regression coverage).
- Require literal spaces around the bracket-tail shortcut and add adversarial coverage.
- Refuse LTS routing for default-dot wildcard patterns (only allow wildcard LTS when globally DOTALL).
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| reggie-runtime/src/test/java/com/datadoghq/reggie/runtime/LinearTokenSequenceMatcherTest.java | Adds JDK-differential tests for whitespace semantics, bracket-tail delimiter strictness, and DOTALL/default-dot routing. |
| reggie-runtime/src/main/java/com/datadoghq/reggie/runtime/RuntimeCompiler.java | Tightens LTS admission rules around wildcard-dot usage and DOTALL requirements. |
| reggie-runtime/src/main/java/com/datadoghq/reggie/runtime/LinearTokenSequenceMatcher.java | Updates LTS whitespace handling and enforces literal-space delimiters for the bracket-tail shortcut; adds SKIP_ANY_EXCEPT_NEWLINE execution path. |
| reggie-codegen/src/test/java/com/datadoghq/reggie/codegen/analysis/PatternCategorizerTest.java | Updates categorizer fixture to explicitly be DOTALL for bracket-tail shortcut eligibility. |
| reggie-codegen/src/main/java/com/datadoghq/reggie/codegen/analysis/PatternCategorizer.java | Splits dotall vs default-dot wildcard categorization; requires DOTALL .* for bracket-tail shortcut detection. |
| reggie-codegen/src/main/java/com/datadoghq/reggie/codegen/analysis/PatternAtom.java | Introduces ANY_STAR_EXCEPT_NEWLINE atom kind. |
| reggie-codegen/src/main/java/com/datadoghq/reggie/codegen/analysis/LinearTokenSequencePlan.java | Adds SKIP_ANY_EXCEPT_NEWLINE op kind and mapping from the new atom kind. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a87ffa492c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
a87ffa4 to
47e531b
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## agent/logs-reggie-l1b #113 +/- ##
=========================================================
- Coverage 84.4% 84.4% -0.1%
Complexity 1 1
=========================================================
Files 147 147
Lines 42991 42993 +2
Branches 5859 5860 +1
=========================================================
Hits 36296 36296
- Misses 4906 4911 +5
+ Partials 1789 1786 -3
Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
47e531b to
81f6dc5
Compare
Summary
Validation
Stacked on #112 (which is stacked on #111).