You will have to subclass your generated lexer and override nextToken() to provide that behavior. Or you could delegate that responsibility. The main thing is that the interface between parser and lexer is one method called nextToken(), and that's what you have to implement to get the functionality you want.