Hi! I am pretty new to antlr, so I hope I have not missed other documentation about this phenomenon but I have stumbled upon some strange behavior I want to ask about.

In defining my simple grammar it seems that I need to ensure that the order of tokens in my VocabTokenTypes.java file and ParserTokenTypes.java is the same or my tokens have the wrong type in the parser.

I noticed this by using the ParseView tree where my tokens had the wrong type.

When I put the lexer rules in the same order as the tokens appear first in the parser then the input files are parsed correctly. If I put all the symbols in the end then it does not work.

any comments about this?

Simple grammar below

class MakefileParser extends Parser;

options {

k = 1;

}



file

: (block)* EOF;



block

: include;


include

: INCLUDE (DOLLAR LPAREN IDENTIFIER RPAREN | IDENTIFIER );

class MakefileLexer extends Lexer;


options {

charVocabulary='3'..'377';

k=2;

}


tokens {

INCLUDE="include";

}


DOLLAR: '$';

LPAREN: '(';



IDENTIFIER: ('a'..'z'|'A'..'Z'|'0'..'9'|'.') ('a'..'z'|'A'..'Z'|'0'..'9'|'.'|'_'|'-')*;


RPAREN: ')';


WS : (' '

| ' ')

{ _ttype = Token.SKIP; }

;


NEWLINE: "
" { _ttype = Token.SKIP; newline();

// System.out.println("Found newline");

}

| ( '
' | '
' ) { _ttype = Token.SKIP; newline();

// System.out.println("Found newline");

}

;