Lexer: INT ID (ID may start with Digit)
0 posts in topic
Flat View  Flat View
TOPIC ACTIONS:
 

Posted By:   Stefan_Kuhn
Posted On:   Thursday, September 16, 2004 09:16 AM

Now I know why a variable in a programming language must not start with a digit but ok, here is my problem: An ID may start with a digit, so without my FLOATorID token, a Number like 12e12 is ok, but 12e12e12 results in a 12e12 float and an e12 ID what is wrong. I solved it, but I`m not happy with my solution, there has to be a better one. Here is my small lexer without tokens: options { k=2; charVocabulary='u0000'..'u007F'; //allow ACSII Chars testLiterals=false; } Tokens {...} Basic Types like '.' & '=' //Whitespace WS : ( ' ' | 'f' | ' ' ){_ttype = Token.SKIP;}; //Newline NL : ' ' ' ' {ne   More>>

Now I know why a variable in a programming language must not start with a digit but ok, here is my problem: An ID may start with a digit, so without my FLOATorID token, a Number like 12e12 is ok, but 12e12e12 results in a 12e12 float and an e12 ID what is wrong. I solved it, but I`m not happy with my solution, there has to be a better one. Here is my small lexer without tokens:

			
options {
k=2;
charVocabulary='u0000'..'u007F'; //allow ACSII Chars
testLiterals=false;
}

Tokens {...}
Basic Types like '.' & '='

//Whitespace
WS :
( ' '
| 'f'
| ' '
){_ttype = Token.SKIP;};

//Newline
NL
: '
' '
' {newline();} // Windows/Dos
| '
' {newline();} // UNIX
| '
' {newline();} // Mac

//Single Line Comment
SL_COMMENT : (options {warnWhenFollowAmbig=false;}
: "//" (~('
'|'
'))* {$setType(Token.SKIP);}

FLOATorID : (FLOAT ID)=> FLOAT ID {$setType(ID);}
| (INT '.')=> INT ('.' INT)? ('e' INT)? {$setType(FLOAT);}
| (INT 'e')=> INT ('e' INT)? {$setType(FLOAT);}
| INT {$setType(INT);};

// Floating Number
FLOAT :
(INT '.')=> INT ('.' INT)? ('e' INT)?
| (INT 'e')=> INT ('e' INT)?
| INT {$setType(INT);};

//Integer
INT : ('+'|'-')? ('0'..'9')+;

//String
ID options {testLiterals=true;} :
( 'a'..'z' | 'A'..'Z' | '0'..'9' | '_' | '-' | '+' | '/' | '?' | '%' | ':' | ' <' | '>' | '[' | ']')+;
   <<Less
About | Sitemap | Contact