The Impact Of Adding Semantic Predicates
0 posts in topic
Flat View  Flat View
TOPIC ACTIONS:
 

Posted By:   Michael_Sommer
Posted On:   Wednesday, May 13, 2009 03:20 PM

Dear Ladies and Gentlemen , to explain my question I use this little example of an imaginary programming language. grammar test; @members{ boolean hexMode=false; } @lexer::members{ boolean hexMode=false; } start : decl*; decl : exportHead | classHead | idDecl | hexDecl; idDecl : OPEN_BRACKET ID CLOSE_BRACKET; hexDecl : OPEN_BRACKET_FOLLOWING_HEX (INTEGER|EXTRAHEX)* CLOSE_BRACKET; exportHead : EXPORT; classHead : CLASS classAttr* dottedName; dottedName : ID (DOT ID)*; classAttr : PUBLIC | AUTO | AN   More>>
Dear Ladies and Gentlemen ,



to explain my question I use this little example of an imaginary programming language.

			
grammar test;

@members{
boolean hexMode=false;
}
@lexer::members{
boolean hexMode=false;
}

start : decl*;

decl : exportHead
| classHead
| idDecl
| hexDecl;

idDecl : OPEN_BRACKET ID CLOSE_BRACKET;

hexDecl : OPEN_BRACKET_FOLLOWING_HEX (INTEGER|EXTRAHEX)* CLOSE_BRACKET;

exportHead
: EXPORT;

classHead
: CLASS classAttr* dottedName;

dottedName
: ID (DOT ID)*;

classAttr
: PUBLIC
| AUTO
| ANSI;

DOT : {true}?=>'.';
ANSI : {true}?=>'ansi';
AUTO : {true}?=>'auto';
PUBLIC : {true}?=>'public';
CLASS : {true}?=>'.class';
EXPORT : {true}?=>'.export';

OPEN_BRACKET
: {true}?=>'{';
OPEN_BRACKET_FOLLOWING_HEX
: {true}?=>('= {'|'={') {hexMode=true;};
CLOSE_BRACKET
: '}' {hexMode=false;};
EXTRAHEX: {hexMode}?=>('a'..'f'|'A'..'F');

ID : {!hexMode}?=>(('a'..'z'|'A'..'Z'|'`'|'_'|'@'|'?') ('a'..'z'|'A'..'Z'|'`'|'_'|'@'|'?'|'0'..'9')*);
QSTRING : {true}?=>'"' .* '"';
SQSTRING: {true}?=>'\'' .* '\'';
INTEGER : {true}?=>('0'..'9')+;

ML_COMMENT
: {true}?=>'/*' .* '*//*' {$channel=HIDDEN;}; // ignore Comments
SL_COMMENT
: {true}?=>'//' .* ('
'|'
'|'
') {$channel=HIDDEN;}; // ignore SingleLineComments until Linebreak

WS : {true}?=>(' '|'
'|'
'|' ')+ {$channel=HIDDEN;}; //ignore Whitespaces


So as you see, I'm using semantic predicates to say how the lexer should differ beteen hexadecimal letters (EXTRAHEX) and normal identifiers (ID). As you maybe also notice, I've added {true}? in front of every lexer rule.
This was necessary since I introduce the semantic predicates for the hexadecimal bytes. Otherwise the lexer always matched only ID-rule, even for the keywords (e.g. 'public', 'auto').

And this is my question: Is it really necessary to add all these {true}? to convince the lexer to lex also keywords or is my version just a workaround for something which would solve my problem. I tried also to add backtrack=true which seems to have no effect.



Thanking you in anticipation,



Michael Sommer

   <<Less
About | Sitemap | Contact