Why do lexical rules for keywords conflict with identifiers even if I specify the special case (keywords) before the more general identifier rule?
Created May 4, 2012
Terence Parr What is wrong with the following? Doesn't the lexer know by the order
in which I specify the two tokens that I want BEGIN to be a special
case of an ID?
class TestLexer extends Lexer; ... BEGIN : "begin" ; ID : ('a'..'z')+ ;
There is no order dependency between the rules in ANTLR. "begin" predicts both BEGIN and ID, unfortuantely.
Use the token {...} section:
class TestLexer extends Lexer;
...
tokens {
BEGIN="begin";
}
ID : ('a'..'z')+
;
The lexer will automatically return BEGIN from the lexer if it sees "begin". The lexer maintains a literals hashtable with all the keywords and other strings and then tests for literals before returning to the parser.