How to recognize a token without consuming the terminating characters.

[Korporal]

I'm puzzled when dealing with some inline assembler grammar.

This is a snippet:

procedure breakpoint(X) intrinsic(arm) ;

        ; this is a comment
        
        /* embed these hard coded values, they */
        /* implement simple jump table   */

        0x123   ; this is dodgy
        0X123
        0111b
        0123h
        023H
        0o1256
        01123d
        11001101010100b
        10101010101010y
        LDR R1, =#0x20001000    /* Load address 0x20001000 to R1 */
        LDR R2, =#0h20001004    /* Load address 0x20001004 to R2 */
        LDR R3, =#0o20001008    /* Load address 0x20001008 to R3 */
        LDR R0, [R1]            /* Load data at the address pointing by R1, save to R0 */
        LDR R1, [R2]            /* Load data at the address pointing by R2, save to R1 */
        ADD R0, R1              /* Add R0 to R1, save to R0 */
        STR R0, [R3]            ; Store R0 to the address pointing by R3
end;

line comments are either // text or ; text but must be terminated by a newline.

ASSEMBLER_LINE_COMMENT:     ((LCOM | SEMICOLON)(.*?)([\r\n])) -> channel(HIDDEN);

The problem I have is that the parser wants to see 'statements' here as a sequence of any tokens followed by a newline. This works for all tokens except the line comments because the act of recognizing these consumes the newline.

assemblerStmt
  : assemblerToken* ASSEMBLER_NEWLINE  
  ;
  

If we have several lines like this:

procedure breakpoint(X) intrinsic(arm) ;
        STR R0, [R3]            ; Store R0 to the address pointing by R3
        STR R0, [R3]            ; Store R0 to the address pointing by R3        
        STR R0, [R3]            ; Store R0 to the address pointing by R3
end;

the parser never sees the newline for each line so just accumulates the tokens.

How can I solve this? is there way to perform some action for the line comment token that means "push newline" so that the chars are used in the recognition but not consumed, being seen when the next token is processed? The parser isn't going to parse the assembly text as such, it just wants to see a stream of tokens without being concerned what they are or their order or anything.