Posted By:
Stefan_Kuhn
Posted On:
Thursday, September 16, 2004 09:16 AM
Now I know why a variable in a programming language must not start with a digit but ok, here is my problem: An ID may start with a digit, so without my FLOATorID token, a Number like 12e12 is ok, but 12e12e12 results in a 12e12 float and an e12 ID what is wrong. I solved it, but I`m not happy with my solution, there has to be a better one. Here is my small lexer without tokens: options { k=2; charVocabulary='u0000'..'u007F'; //allow ACSII Chars testLiterals=false; } Tokens {...} Basic Types like '.' & '=' //Whitespace WS : ( ' ' | 'f' | ' ' ){_ttype = Token.SKIP;}; //Newline NL : ' ' ' ' {ne
More>>
Now I know why a variable in a programming language must not start with a digit but ok, here is my problem: An ID may start with a digit, so without my FLOATorID token, a Number like 12e12 is ok, but 12e12e12 results in a 12e12 float and an e12 ID what is wrong. I solved it, but I`m not happy with my solution, there has to be a better one. Here is my small lexer without tokens:
options {
k=2;
charVocabulary='u0000'..'u007F'; //allow ACSII Chars
testLiterals=false;
}
Tokens {...}
Basic Types like '.' & '='
//Whitespace
WS :
( ' '
| 'f'
| ' '
){_ttype = Token.SKIP;};
//Newline
NL
: '
' '
' {newline();} // Windows/Dos
| '
' {newline();} // UNIX
| '
' {newline();} // Mac
//Single Line Comment
SL_COMMENT : (options {warnWhenFollowAmbig=false;}
: "//" (~('
'|'
'))* {$setType(Token.SKIP);}
FLOATorID : (FLOAT ID)=> FLOAT ID {$setType(ID);}
| (INT '.')=> INT ('.' INT)? ('e' INT)? {$setType(FLOAT);}
| (INT 'e')=> INT ('e' INT)? {$setType(FLOAT);}
| INT {$setType(INT);};
// Floating Number
FLOAT :
(INT '.')=> INT ('.' INT)? ('e' INT)?
| (INT 'e')=> INT ('e' INT)?
| INT {$setType(INT);};
//Integer
INT : ('+'|'-')? ('0'..'9')+;
//String
ID options {testLiterals=true;} :
( 'a'..'z' | 'A'..'Z' | '0'..'9' | '_' | '-' | '+' | '/' | '?' | '%' | ':' | '
<' | '>' | '[' | ']')+;
<<Less