How does lookahead depth affect generated DFA's (i.e. _tokenSet_##)?
1 posts in topic
Flat View  Flat View
Older Post First |   Reply to Topic
TOPIC ACTIONS:
 

Posted By:   Adam_McClure
Posted On:   Thursday, August 29, 2002 04:01 PM

I seem to have a hard time getting some rules to bottom out. For example, I have a rule defined as an identifier which is checked at the end of an expression. The generated code parses the token as part of the expression, but fails and claims it doesn't recognize the next token. I'm using nested expressions to represent a functional language. expr: comp ('>' comp)* ; comp: unary ('+' unary)* ; unary: ID ; Now feed in the following expression: a + b > c The generated code will recognize a+b but throw an exception when trying to recognize '>'. I've tried playing around with the look   More>>

I seem to have a hard time getting some rules to bottom out. For example, I have a rule defined as an identifier which is checked at the end of an expression. The generated code parses the token as part of the expression, but fails and claims it doesn't recognize the next token. I'm using nested expressions to represent a functional language.



expr: comp ('>' comp)* ;

comp: unary ('+' unary)* ;

unary: ID ;


Now feed in the following expression:


a + b > c


The generated code will recognize a+b but throw an exception when trying to recognize '>'.


I've tried playing around with the lookahead depth thinking that will change things but it only seems to shift the errors around. I've done some debugging and narrowed the problem to the tokenSet.member(LA(#)) tests following successful evaluation of the ID rule.


I think the question then comes back to, how are the DFA's generated and is there a way to clarify my error? What impact does the lookahead depth have on the DFA?

   <<Less
Report | Quote This | Send private message | Reply | Print

Re: How does lookahead depth affect generated DFA's (i.e. _tokenSet_##)?

Posted By:   Monty_Zukowski  
Posted On:   Friday, August 30, 2002 07:33 AM

In your options section add this option:

        codeGenBitsetTestThreshold = 999999;

This will tell antlr not to make the bitsets unless there are more than 999999 possibilities. The only exception to that is negated sets ~(a|b) which always use bitsets.


Without the bitsets you will see statements like


if (LA(1)==A || LA(1)==B)

and this should give you a better idea of what antlr is thinking. Your problem might be related to EOF handling or it might be from the rule calling expr.


The next step, if you can't figure out the problem, is to post a small example grammar and sample input so I can try it on my machine and see exactly what is going on.

Report | Quote This | Send private message | Reply | Print
About | Sitemap | Contact