error when another rule alternative works.

[mikecargal]

(Some additional details here ANTLR Java 8, odd behavior parsing with returnStatement as start rule, including screen shots of pare trees)

While answering a StackOverflow ANTLR question, I encountered the following: (Using the Java8Parser grammar from the ANTLR parsers GitHub page)

When trying to parser return a.b.c(); using start rule returnStatement, I got an error on the ( and ) characters, even though this is clearly a syntactically correct return statement.

I backed up and tried { return a.b.c(); } using the block start rules, and the parse was successful.

Bart Kiers noted that by changing the methodInvocation_lfno_primary from:

methodInvocation_lfno_primary
    :   methodName '(' argumentList? ')'
    |   typeName '.' typeArguments? Identifier '(' argumentList? ')'       // ALT 2
    |   expressionName '.' typeArguments? Identifier '(' argumentList? ')' // ALT 3
    |   'super' '.' typeArguments? Identifier '(' argumentList? ')'
    |   typeName '.' 'super' '.' typeArguments? Identifier '(' argumentList? ')'
    ;

to

methodInvocation_lfno_primary
    :   methodName '(' argumentList? ')'
    |   expressionName '.' typeArguments? Identifier '(' argumentList? ')'
    |   typeName '.' typeArguments? Identifier '(' argumentList? ')'
    |   'super' '.' typeArguments? Identifier '(' argumentList? ')'
    |   typeName '.' 'super' '.' typeArguments? Identifier '(' argumentList? ')'
    ;

the issue resolves itself. (Though I don't think either of us are convinced this should be necessary)

This puts the matching alternative before the alternative that gave the error (reference the SO question to see parse trees from IntelliJ plugin).

Based on my understanding, even with the first rule, if a match failed on Alt 2, ANTLR should then try the next alternative. (i.e. you should not have to re-order the alternatives if one fails to match.)

Is there a reasonable explanation for this behavior?

BTW, I noticed a other "oddities with the grammar, such as:

typeName: Identifier | packageOrTypeName '.' Identifier;

packageOrTypeName
    : Identifier
    | packageOrTypeName '.' Identifier
    ;

notice the duplicated packageOrTypeName '.' Identifier alternatives in both rules.

[kaby76]

Sorry, I just found this thread today. I was wondering what happened to the discussion over in StackOverflow.

The problem is that AdaptivePredict() wants to direct typeName/packageOrTypeName to consume too much of the string sequence "a.b.c()". In the "normal" parse of a compilationUnit, packageOrTypeName calls AdaptivePredict() and recognizes "a", so typeName matches "a.b", and methodInvocation_lfno_primary can match up with the second alt "a.b.c()". In the "abnormal" parse from returnStatement, AdaptivePredict() causes packageOrTypeName to match "a.b", typeName to match "a.b.c", which then causes a fault in methodInvocation_lfno_primary. Why AdaptivePredict tells to parse packageOrTypeName as "a.b" rather than "a", I haven't read. But, I suspect there is a bug with the parse engine with epsilon-transitions to the final state of an ATN in the context of other similar contexts. This is hard to get right, which I found while writing a lookahead predictor for "code completion".

There are two ways that fix this problem, both of which worked for me:

1. Change typeName to typeName: Identifier ('.' Identifier)*;. The parse of return a.b.c(); from returnStatement works.
2. Or, add an EOF-augmented rule for returnStatement and use that returnStatementEOF : 'return' expression? ';' EOF. The parse of return a.b.c(); from returnStatementEOF also works.

This tends to be an argument why one should always augment start rules with EOF.