Why doesn't the lexer match any of the literals from my "tokens" section?

Created Dec 15, 2000

Terence Parr

The most common problem is that your lexer does not have a rule that can match a pattern that includes your literals. For example, there is no rule that can match the text for "blort", hence, ANTLR cannot test any rule's matched text against the literals table.

class MyLexer extends Lexer;
options {
        charVocabulary = '3'..'377';
}

tokens {
        "blort" ;
}
BEGIN : '%';
END : '*';
WS      :       (' '|'	'|'
'{newline();})+
                {$setType(Token.SKIP);}
        ;

Here, only WS, '%', and '*' can be matched by the lexer. Add an IDENTIFIER rule and ANTLR can then check the text against the literals table.