Close similarities between string and expression... how to avoid non-determinism?
1 posts in topic
Flat View  Flat View
TOPIC ACTIONS:
 

Posted By:   yong_ming
Posted On:   Wednesday, March 20, 2002 12:44 AM

There are 3 rules in the lexer I like to write: IDENT : ( 'A'..'Z' | 'a'..'z' ) 'A'..'Z' | 'a'..'z' | '0'..'9' )*; STRING : '""' ( ('A'..'Z' | 'a'..'z') | '0'..'9' )* '""'; EXP : (UNDER)? '""' IDENT ( ('+' | '-' | '*' ) IDENT )* '""' (UNDER)? Example: EXP can be _""x+y"" or _""m8+e3""_. I have been trying for many days and was really getting no where. I cannot get around the non-determinism error. Can someone experienced shower me with some enlightment? THanks    More>>

There are 3 rules in the lexer I like to write:

			
IDENT : ( 'A'..'Z' | 'a'..'z' ) 'A'..'Z' | 'a'..'z' | '0'..'9' )*;

STRING : '""' ( ('A'..'Z' | 'a'..'z') | '0'..'9' )* '""';

EXP : (UNDER)? '""' IDENT ( ('+' | '-' | '*' ) IDENT )* '""' (UNDER)?


Example:

EXP can be _""x+y"" or _""m8+e3""_.


I have been trying for many days and was really getting no where. I cannot get around the non-determinism error. Can someone experienced shower me with some enlightment?


THanks    <<Less

Re: Close similarities between string and expression... how to avoid non-determinism?

Posted By:   Monty_Zukowski  
Posted On:   Thursday, March 21, 2002 06:06 AM

ANTLR can't handle lexer rules which have the same prefix. You need to combine such rules into one rule. In this case the STRING and EXP rules need to be combined. The idea is to change the token type when you see something that really belongs to the other rule.

The simplest but not fastest way is this:


STRING : (EXP)=>(EXP {$setType(EXP);})
| '"' ( ('A'..'Z' | 'a'..'z') | '0'..'9' )* '"';

protected EXP : (UNDER)? '"' IDENT ( ('+' | '-' | '*' ) IDENT )* '"' (UNDER)?


You could, however, take the approach taken in the java.g number rule. Something like this:

STRING: (UNDER {$setType(EXP);})?
'"' IDENT
( (('+' | '-' | '*' ) {$setType(EXP);})
IDENT
)*
'"'
(UNDER {$setType(EXP);})?

Basically the approach is to take the union of both rules, pick the most restrictive rule as the default, and then when you see elements of the other rule set the type appropriately.


Please let us know how this works for you or if you have other problems.

About | Sitemap | Contact