Token positions from AST's
1 posts in topic
Flat View  Flat View
TOPIC ACTIONS:
 

Posted By:   Chris_Rae
Posted On:   Monday, September 2, 2002 02:38 AM

Hi, Sorry to ask this question yet again, but I can find a working solution after 2 days of searching. The problem is, I need to know the line and column number of an AST element. I know absolutely nothing about C++, parsers, lexers or grammers. I simply use the following code to create an AST that represents a Java source file: // Create InputStream FileInputStream fileInputStream = new FileInputStream( file ); // Create a Scanner that reads from an InputStream JavaLexer lexer = new JavaLexer( fileInputStream ); lexer.setFilename( file.getName() ); // Create a Parser that reads from a Scanner JavaRecognizer parser = new JavaRe   More>>

Hi,



Sorry to ask this question yet again, but I can find a working solution after 2 days of searching.



The problem is, I need to know the line and column number of an AST element. I know absolutely nothing about C++, parsers, lexers or grammers. I simply use the following code to create an AST that represents a Java source file:



			
// Create InputStream
FileInputStream fileInputStream = new FileInputStream( file );

// Create a Scanner that reads from an InputStream
JavaLexer lexer = new JavaLexer( fileInputStream );
lexer.setFilename( file.getName() );

// Create a Parser that reads from a Scanner
JavaRecognizer parser = new JavaRecognizer( lexer );
parser.setFilename( file.getName() );

// Parse
parser.compilationUnit();

ROOT = parser.getAST();
( (CommonAST) ROOT ).setVerboseStringConversion( true, parser.getTokenNames() );

JavaTreeParser treeParser = new JavaTreeParser();
treeParser.compilationUnit( ROOT );


Then I extract the tokens I'm interested in from the AST. I have no knowledge of C++ or ANTLR and cant't understand the solutions that have posted - I didn't even know what an AST was before using ANTLR!!



I have tried extending the CommonAST class and overiding the initialize( Token tok ) method with the following:



			
public void initialize( Token tok ) {
setText( tok.getText() );
setType( tok.getType() );

LINE = tok.getLine();
COLUMN = tok.getColumn();
}


However, I don't know how I am to incororate the new AST class to get it to work. What I need is a working example of Java code that incorporates the line and column numbers into an AST - telling me that I need to this, that & the next thing to the ANTLR package is no good - I have no patients or desire learn what parsing, lexing and grammers are to solve such a trivial issue.



Thanks,

Chris    <<Less

Re: Token positions from AST's

Posted By:   Ian_Warner  
Posted On:   Wednesday, November 27, 2002 07:56 AM


You need to subclass antlr.ASTFactory. This is the factory class used to create ASTs from Tokens.



An ASTFactory has a create(Token) method (plus a few overloads). The code generated by ANTLR will call astFactory.create(Token). You need to override this method and set the relevant members of your AST implementation.



At the top of your parser grammar place something like:




class MyParser extends Parser;

options {
// Options...
}

tokens {
// Tokens...
}

// This is the IMPORTANT bit!!!
{
astFactory = new MyASTFactoryImpl();
}

myFirstRule
:
;



This will re-initialize the protected variable astFactory which, is inherited from antlr.Parser.



That's how I do it anyway :)

About | Sitemap | Contact