dcsimg
BUG in Java 1.2 and 1.3 recognisers
1 posts in topic
Flat View  Flat View
TOPIC ACTIONS:
 

Posted By:   Anonymous
Posted On:   Tuesday, May 14, 2002 06:29 PM

In the rule NUM_INT, there is an optional sub-rule to match hex or octal digits following a leading zero. The optional sub-rule is missing a semantic action as follows: {isDecimal=false;} The later part of the rule, following a comment: // only check to see if it's a float if looks like decimal so far is therefore executed even when the number is hex or octal. Consequently, this output: 0x01234.45e-67: 138 null: 1 is given by this program: import java.io.*; import antlr.*; public class TestJava { public static void main( String[] args ) throws Exception { Reader r = new St   More>>

In the rule NUM_INT, there is an optional sub-rule to match hex or octal digits following a leading zero. The optional sub-rule is missing a semantic action as follows:

			
{isDecimal=false;}

The later part of the rule, following a comment:
			
// only check to see if it's a float if looks like decimal so far

is therefore executed even when the number is hex or octal.


Consequently, this output:

			
0x01234.45e-67: 138
null: 1

is given by this program:
			
import java.io.*;
import antlr.*;

public class TestJava
{
public static void main( String[] args ) throws Exception
{
Reader r = new StringReader( "0x01234.45e-67" );
JavaLexer lexer = new JavaLexer( r );

Token tok;
do
{
tok = lexer.nextToken();
System.out.println( tok.getText() + ": " + tok.getType() );
} while( tok.getType() != Token.EOF_TYPE );
}
}


Not a huge problem, since anything incorrectly matched by this rule is an incorrect program anyway, but it's a bug nonetheless. It doesn't conform to the Language Spec.


Cheers,

George

   <<Less

Re: BUG in Java 1.2 and 1.3 recognisers

Posted By:   Anonymous  
Posted On:   Tuesday, May 14, 2002 07:27 PM

The bug still stands, but I made a small error in describing it, and I have found another bug since.


Of course, the rule should check for a float suffix following octal digits, because the integer part of a float can have leading zeroes, looking like octal. Therefore, the semantic action should be associated with matching hex digits only, not with hex and octal digits.


Additional bug: according to the Language Spec, a floating point number can begin with a string of digits, followed by a decimal point. It can include leading zeroes. However, the rule supplied in the Java 1.2 and 1.3 recognisers decides that a number is octal as soon as it encounters a leading zero, without considering the possibility that it is the start of a decimal floating point number. From then on, it only accepts octal digits before the decimal point. This means that 0123.45 and 789.45 are correctly tokenised, but 0789.45 is not.


In my opinion this is a much more severe bug, not least because people like me, who are looking for a quick way to parse the hell that is C-style numeric literals, are likely to copy the entire thing into their own grammars.


Cheers,

George

About | Sitemap | Contact