Backtracking

[kisp]

Hi, I am not sure whether I should expect backtracking to occur.

For instance, in this example

(import lexgen)

(define (err s)
  (print "lexical error on stream: " s)
  (list))

(lex (seq (star (range #\0 #\9)) (char #\2)) err "123")

I get the error

lexical error on stream: ((3 2 1) #!eof)

instead of getting the possible result:

((#\1 #\2) (#\3))

[iraikov]

Hello, thank you for using lexgen. There is no provision for general backtracking, as that usually incurs a significant performance penalty. In principle, the error continuation can be modified to return the unmodified input stream and/or apply alternate lexer functions.