Skip to content

fix: align linear token sequence semantics - #113

Open
jbachorik wants to merge 1 commit into
agent/logs-reggie-l1bfrom
agent/logs-reggie-l1c
Open

fix: align linear token sequence semantics#113
jbachorik wants to merge 1 commit into
agent/logs-reggie-l1bfrom
agent/logs-reggie-l1c

Conversation

@jbachorik

Copy link
Copy Markdown
Collaborator

Summary

  • aligns Linear Token Sequence \s and \S handling with default JDK whitespace semantics
  • makes the bracket-tail shortcut require the literal-space delimiters it recognizes
  • restricts wildcard LTS admission to globally DOTALL patterns, preserving default-dot semantics
  • adds JDK differential and adversarial regression coverage

Validation

  • `./gradlew jacocoVerify build`
  • focused LTS and categorizer tests after final cleanup
  • final implementation-plan validation and code review run independently

Stacked on #112 (which is stacked on #111).

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR tightens and aligns the Linear Token Sequence (LTS) fast-path semantics with the intended Java/JDK regex behavior by (a) matching JDK-style whitespace for \s/\S, (b) making the bracket-tail shortcut enforce the literal-space delimiters it recognizes, and (c) preventing LTS admission of wildcard-dot patterns unless the pattern is globally DOTALL.

Changes:

  • Align LTS whitespace/non-whitespace classification with JDK-style whitespace (and add JDK-differential regression coverage).
  • Require literal spaces around the bracket-tail shortcut and add adversarial coverage.
  • Refuse LTS routing for default-dot wildcard patterns (only allow wildcard LTS when globally DOTALL).

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
reggie-runtime/src/test/java/com/datadoghq/reggie/runtime/LinearTokenSequenceMatcherTest.java Adds JDK-differential tests for whitespace semantics, bracket-tail delimiter strictness, and DOTALL/default-dot routing.
reggie-runtime/src/main/java/com/datadoghq/reggie/runtime/RuntimeCompiler.java Tightens LTS admission rules around wildcard-dot usage and DOTALL requirements.
reggie-runtime/src/main/java/com/datadoghq/reggie/runtime/LinearTokenSequenceMatcher.java Updates LTS whitespace handling and enforces literal-space delimiters for the bracket-tail shortcut; adds SKIP_ANY_EXCEPT_NEWLINE execution path.
reggie-codegen/src/test/java/com/datadoghq/reggie/codegen/analysis/PatternCategorizerTest.java Updates categorizer fixture to explicitly be DOTALL for bracket-tail shortcut eligibility.
reggie-codegen/src/main/java/com/datadoghq/reggie/codegen/analysis/PatternCategorizer.java Splits dotall vs default-dot wildcard categorization; requires DOTALL .* for bracket-tail shortcut detection.
reggie-codegen/src/main/java/com/datadoghq/reggie/codegen/analysis/PatternAtom.java Introduces ANY_STAR_EXCEPT_NEWLINE atom kind.
reggie-codegen/src/main/java/com/datadoghq/reggie/codegen/analysis/LinearTokenSequencePlan.java Adds SKIP_ANY_EXCEPT_NEWLINE op kind and mapping from the new atom kind.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread reggie-runtime/src/main/java/com/datadoghq/reggie/runtime/RuntimeCompiler.java Outdated
@jbachorik
jbachorik marked this pull request as ready for review August 14, 2026 12:53

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a87ffa492c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread reggie-codegen/src/main/java/com/datadoghq/reggie/codegen/automaton/CharSet.java Outdated
@jbachorik
jbachorik force-pushed the agent/logs-reggie-l1c branch from a87ffa4 to 47e531b Compare August 14, 2026 13:05
@codecov-commenter

codecov-commenter commented Aug 14, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 68.88889% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.4%. Comparing base (44e3604) to head (81f6dc5).

Files with missing lines Patch % Lines
...hq/reggie/codegen/analysis/PatternCategorizer.java 50.0% 0 Missing and 5 partials ⚠️
.../com/datadoghq/reggie/runtime/RuntimeCompiler.java 50.0% 1 Missing and 4 partials ⚠️
...ghq/reggie/runtime/LinearTokenSequenceMatcher.java 81.8% 0 Missing and 4 partials ⚠️
Additional details and impacted files
@@                    Coverage Diff                    @@
##             agent/logs-reggie-l1b    #113     +/-   ##
=========================================================
- Coverage                     84.4%   84.4%   -0.1%     
  Complexity                       1       1             
=========================================================
  Files                          147     147             
  Lines                        42991   42993      +2     
  Branches                      5859    5860      +1     
=========================================================
  Hits                         36296   36296             
- Misses                        4906    4911      +5     
+ Partials                      1789    1786      -3     
Files with missing lines Coverage Δ
...ggie/codegen/analysis/LinearTokenSequencePlan.java 82.0% <100.0%> (-0.6%) ⬇️
...datadoghq/reggie/codegen/analysis/PatternAtom.java 87.5% <100.0%> (+0.4%) ⬆️
...ghq/reggie/runtime/LinearTokenSequenceMatcher.java 80.6% <81.8%> (+0.6%) ⬆️
...hq/reggie/codegen/analysis/PatternCategorizer.java 59.3% <50.0%> (-0.1%) ⬇️
.../com/datadoghq/reggie/runtime/RuntimeCompiler.java 87.9% <50.0%> (-0.4%) ⬇️

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 44e3604...81f6dc5. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jbachorik
jbachorik force-pushed the agent/logs-reggie-l1c branch from 47e531b to 81f6dc5 Compare August 14, 2026 13:24
@jbachorik
jbachorik requested a review from Zenithar August 14, 2026 14:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants