Why can't I get my lexer to recognize 8-bit characters or binary bytes even though I set my character vocabulary to '\0'..'\377'?

Terence Parr

It's probably the type of stream you are using. Check to make sure you're creating a DataInputStream not a Reader:

DataLexer lexer = new DataLexer(new DataInputStream(System.in));
DataParser parser = new DataParser(lexer);

Someone reported that you can change the standard mapping for your ascii-based machine:

new InputStreamReader(new FileInputStream("TestFile"), "8859_1"));

It should read ascii + 8-bit binary as well. This lets you use the Java Reader hierarchy.