How to match unicode strings?

[agieocean]

I'm trying to match unicode strings with a terminal.

What I've tried is:

put: "put" UNICODE INT INT
UNICODE: /[\u0000-\uffff]+/

put: "put" STRING INT INT
STRING: /[.*]+/

Both of these return:
lark.exceptions.UnexpectedCharacters: No terminal matches '👽' in the current parser context, at line 3 col 5
When parsing:
put 👽 5 5

[erezsh]

Did you add %ignore " " ?

[agieocean]

Just tried that and got the same error unfortunately, this is the full lark file I'm using:

start: instruction+
instruction: line | fill | write | put
line: "line" INT INT INT
fill: "fill" INT INT
write: "write" FILENAME                                               
put: "put" STRING INT INT

FILENAME: /[a-zA-Z0-9\.]+/                                            
UNICODE: /[\u0000-\uffff]+/
STRING: /[.*]+/                                                                                                                             
%import common.WORD   // imports from terminal library                
%import common.INT
%import common.WS                                                     
%ignore WS                                                            
%ignore " "

[erezsh]

In this grammar you're not using UNICODE ...

[agieocean]

I get the same error whether I use UNICODE or not. So if I use this grammar:

start: instruction+                                                   
instruction: line | fill | write | put
line: "line" INT INT INT
fill: "fill" INT INT                                                  
write: "write" FILENAME                                               
put: "put" UNICODE INT INT                                                                                                                  
FILENAME: /[a-zA-Z0-9\.]+/                                            
UNICODE: /[\u0000-\uffff]+/                                           
STRING: /[.*]+/                                                                                                                             
%import common.WORD   // imports from terminal library
%import common.INT                                                    
%import common.WS
%ignore WS                                                            
%ignore " "

I still get:
lark.exceptions.UnexpectedCharacters: No terminal matches '👽' in the current parser context, at line 3 col 5

[agieocean]

start: instruction+
instruction: line | fill | write | put
line: "line" INT INT INT
fill: "fill" INT INT
write: "write" FILENAME                                               
put: "put" UNICODE_STRING INT INT

FILENAME: /[a-zA-Z0-9\.]+/                                            
UNICODE: /[\u00000000-\uffffffff]/                                    
UNICODE_STRING: UNICODE+                                              
STRING: /[.*]+/                                                                                                                             
%import common.WORD   // imports from terminal library
%import common.INT
%import common.WS                                                     
%ignore WS                                                            
%ignore " "

With this grammar it's able to match some emojis, for instance ✅ but others like 🐇 or 👽 still don't match.

[erezsh]

This one works:

from lark import Lark

grammar = r"""
start: EMOJI+

EMOJI: /[\u0000-\U0010FFFF]/+

%import common.WS
%ignore WS
"""

parser = Lark(grammar)

tests = ["✅", "✅ 🐇", "👽", "😀😎🚀", "👨‍👩‍👧‍👦", "👍🏽", "⚽️"]
for s in tests:
    print(s, "=>", parser.parse(s).pretty())