dcsimg
Underscore? What is going on?!
1 posts in topic
Flat View  Flat View
TOPIC ACTIONS:
 

Posted By:   Anonymous
Posted On:   Saturday, December 11, 2004 01:26 PM

I have this fairly simple thing that just won't work. Given an input file with this: class_of; hmm; And the following antlr definition: class SillyParser extends Parser; silly : "class_of" SEMI CLASSNAME SEMI ; class SillyLexer extends Lexer; CLASSNAME : ( 'a'..'z' )+; SEMI : ';' ; WS : ( ' ' | ' ' | ' ' ' ' { newline(); } | ' ' { newline(); } ) {$setType(Token.SKIP);} //ignore this token ; I get this: line 1:1: expecting "class_of", found 'clas   More>>

I have this fairly simple thing that just won't work. Given an input file with this:



class_of; hmm;


And the following antlr definition:


			
class SillyParser extends Parser;
silly
:
"class_of" SEMI CLASSNAME SEMI
;

class SillyLexer extends Lexer;

CLASSNAME : ( 'a'..'z' )+;

SEMI : ';' ;

WS : ( ' '
| ' '
| '
' '
' { newline(); }
| '
' { newline(); }
)
{$setType(Token.SKIP);} //ignore this token
;


I get this:


line 1:1: expecting "class_of", found 'class'
exception: line 1:6: unexpected char: '_'


If I remove the underscore and matching only "classof" (changing the input file too), it works. In general, whenever I put an undescore inside a literal it seems to fail. Same thing happens with the dash (-).


I've spent three hours on this. Can you believe that? I bet it's just something amazingly stupid I'm doing, but I just can't see it. What the hell is going on?!


Some help would be much appreciated.

   <<Less

Re: Underscore? What is going on?!

Posted By:   Anonymous  
Posted On:   Friday, December 17, 2004 02:28 AM

First, define a charvocabulary in the lexer (see ANTLR doc). This might solve the problem immediately.

If not, the following might go on: The lexer can only recognize all the things you tell it - i.e. SEMI, WS and sequences of characters. But no one told it to recognize things with underscores. The "class_of" will only create a lookup in some keyword table - but the lexer will never recognize it. Either you do:


CLASSNAME : ( 'a'..'z' | '_')+;

but then abc_def is now a valid classname, in contrast to the original definition. Or you add a little more stuff in the lexer (maybe with { ... } after the CLASSNAME to immediately separate classnames from "potential keywords" (things with underscores in them).

Regards
Harald M.

About | Sitemap | Contact