parser defined string literals don't match
1 posts in topic
Flat View  Flat View
TOPIC ACTIONS:
 

Posted By:   Christopher_Stephens
Posted On:   Tuesday, October 23, 2001 12:36 PM

I'm having a lot of trouble matching parser defined string literals when my grammar is separated into two files. I'd like to take advantage of ANTLR's vocabulary inheritance, but I need this to work. Can anybody tell me why ".accept" matches using this set of import/export rules: class TestingParser extends Parser; options { exportVocab=TestingParser; } command : ".accept" { cout < < "matched accept" < < endl; } ; class TestingLexer extends Lexer; options { charVocabulary='3'..'377'; exportVocab=TestingParser; caseSensitive=false; } ID : ( 'a' .. 'z' | '0' .. '9' | '.' )+ ;    More>>

I'm having a lot of trouble matching parser defined string literals when my grammar is separated into two files. I'd like to take advantage of ANTLR's vocabulary inheritance, but I need this to work.



Can anybody tell me why ".accept" matches using this set of import/export rules:



			
class TestingParser extends Parser;
options {
exportVocab=TestingParser;
}

command : ".accept" { cout < < "matched accept" < < endl; } ;


class TestingLexer extends Lexer;
options {
charVocabulary='3'..'377';
exportVocab=TestingParser;
caseSensitive=false;
}

ID : ( 'a' .. 'z' | '0' .. '9' | '.' )+ ;
WS : ( ' ' | ' ' | '
' { newline(); } | '
' | ' ' )+ { $setType(Token::SKIP); } ;


but ".accept" does not match using this set of import/export rules (regardless of whether or not the parser and lexer definitions are in the same file):



			
class TestingParser extends Parser;
options {
importVocab=TestingLexer;
exportVocab=TestingParser;
}

...

class TestingLexer extends Lexer;
options {
...
exportVocab=TestingLexer;
...
}


Any help would be great...

   <<Less

Oddities with exportVocab/importVocab

Posted By:   Ric_Klaren  
Posted On:   Wednesday, October 24, 2001 04:55 AM

Hmmm this should be FAQ-able. (the answer to your question
is in the last paragraph)

As a rule of thumb use this scheme
when using importvocab/exportvocab with grammars
in different files (the parser/lexer in one
file scheme is subtly different):


  1. First in the lexer you export the current
    vocabulary (say L) e.g. exportVocab = L.
  2. Then in the parser you have to get the definitions
    from the lexer e.g. importVocab = L,
    in the parser you can extend the vocabulary
    with for example imaginary tokens. These you
    might need in a treeparser so you do in the parser
    a exportVocab = P (don't use the same
    name as for L!!!).
  3. Last but not least in the treeparser you
    do a importVocab = P. If you transform
    the tree and add new nodetypes and want to use
    those in subsequent treewalkers, then you need
    to extend the chain of imports/exports similar
    to the way that is done with parser and lexer.

Also make sure you have your build dependencies set up right. It can be
very frustrating to debug something that comes from parser/lexer/walker
differing in opinions on tokensets because of a incomplete build...
(e.g. build in the order lexer, parser, treewalker)

You can check for errors in this stuff by looking
at the xxxTokenTypes files and looking for discrepancies.

To come back to your question: Your last scheme
is the right one.. So my guess is that you run your
parser through antlr before you run the
lexer through it. Best guess is to remove all
antlr generated stuff then build the lexer and the parser
in that order, then it should work. (if it doesn't get back to me =) )

About | Sitemap | Contact