How do I use a non-ANTLR-generated (C++) lexer into an ANTLR-generated parser?

Ric Klaren

Subclass your scanner from antlr::TokenStream and make it adhere to the interface defined by TokenStream. Basically this comes down to defining a nextToken function in your scanner class.

I use (+/-) the following to link a Cocktail rex scanner to antlr:

#include <antlr/TokenStream.hpp>

class Scanner : public antlr::TokenStream
   // implement a nextToken function..
   antlr::RefToken nextToken( void )
      int tok;
      char yytext[MAX_STRING];
      // do scanner magic...
      ... // set tok and yytext..
      // Return a token
      antlr::RefToken ret(new antlr::CommonToken(tok,yytext ));
      return ret;