jGuru Forums

0 posts in topic
Flat View
TOPIC ACTIONS:

Posted By:   Stefan_Kuhn
Posted On:   Thursday, September 16, 2004 09:16 AM

Now I know why a variable in a programming language must not start with a digit but ok, here is my problem: An ID may start with a digit, so without my FLOATorID token, a Number like 12e12 is ok, but 12e12e12 results in a 12e12 float and an e12 ID what is wrong. I solved it, but I`m not happy with my solution, there has to be a better one. Here is my small lexer without tokens: options { k=2; charVocabulary='u0000'..'u007F'; //allow ACSII Chars testLiterals=false; } Tokens {...} Basic Types like '.' & '=' //Whitespace WS : ( ' ' | 'f' | ' ' ){_ttype = Token.SKIP;}; //Newline NL : ' ' ' ' {ne   More>>

Now I know why a variable in a programming language must not start with a digit but ok, here is my problem: An ID may start with a digit, so without my FLOATorID token, a Number like 12e12 is ok, but 12e12e12 results in a 12e12 float and an e12 ID what is wrong. I solved it, but I`m not happy with my solution, there has to be a better one. Here is my small lexer without tokens:

```
options {

k=2;

charVocabulary='u0000'..'u007F'; //allow ACSII Chars

testLiterals=false;

}

Tokens {...}

Basic Types like '.' & '='

//Whitespace

WS :

( ' '

| 'f'

| ' '

){_ttype = Token.SKIP;};

//Newline

NL

: '

' '

'  {newline();}   // Windows/Dos

| '

'  {newline();}  // UNIX

| '

'  {newline();}  // Mac

//Single Line Comment

SL_COMMENT : (options {warnWhenFollowAmbig=false;}

: "//" (~('

'|'

'))* {\$setType(Token.SKIP);}

FLOATorID : (FLOAT ID)=> FLOAT ID {\$setType(ID);}

| (INT '.')=> INT ('.' INT)?  ('e' INT)?  {\$setType(FLOAT);}

| (INT 'e')=> INT ('e' INT)? {\$setType(FLOAT);}

|  INT {\$setType(INT);};

// Floating Number

FLOAT :

(INT '.')=> INT ('.' INT)?  ('e' INT)?

| (INT 'e')=> INT ('e' INT)?

|  INT {\$setType(INT);};

//Integer

INT : ('+'|'-')? ('0'..'9')+;

//String

ID options {testLiterals=true;} :

( 'a'..'z' | 'A'..'Z' | '0'..'9' | '_' | '-' | '+' | '/' | '?' | '%' | ':' | '
<' | '>' | '[' | ']')+;
```
<<Less