I'd like to parse a complete file of the language I'm interested in, without a completed parser. This way I can test the parts I have done incrementally.


Where I come to a line I haven't yet made a completed rule for, I use a rule in the parser like:

dummy_rule: //scarf until semicolon is found

( options { greedy=false; } :

. //match any lexer rule

)* //non-greedy == break out of the loop if possible

SEMICOLON_TOKEN

;

So the parser should just ignore any tokens it matches until it finds a semicolon (defined in the lexer)


In the lexer, I have a rule defining an identifier

IDENTIFIER

// check tokens{} section (and any other parser string) before applying this rule

options {testLiterals=true;}

:

('a'..'z'|'A'..'Z')

//no two LOW_LINE_TOKENS in a row

( ('_')? ('a'..'z'|'A'..'Z'|'0'..'9') )*

;

However, when I parse textual input containing an apostrophe, also defined as a token

PERCENT_SIGN_TOKEN : '%';

AMPERSAND_TOKEN : '&';

APOSTROPHE_TOKEN : '\'';

LEFT_PARENTHESIS_TOKEN : '(';

RIGHT_PARENTHESIS_TOKEN : ')';

I get a run-time error

parser exception: antlr.TokenStreamRecognitionException: unexpected char: '

antlr.TokenStreamRecognitionException: unexpected char: '

at rp.RosettaLexer.nextToken(RosettaLexer.java:533)

at antlr.TokenBuffer.fill(TokenBuffer.java:61)

at antlr.TokenBuffer.LA(TokenBuffer.java:70)

at antlr.LLkParser.LA(LLkParser.java:50)

at rp.RosettaParser.facet_term(RosettaParser.java:565)

at rp.RosettaParser.facet_definition(RosettaParser.java:371)

at rp.RosettaParser.facet_declaration(RosettaParser.java:178)

at rp.RosettaParser.design_unit(RosettaParser.java:146)

at rp.RosettaParser.design_file(RosettaParser.java:122)

at rp.RosettaParser.parseFile(RosettaParser.java:81)

at rp.RosettaParser.doFile(RosettaParser.java:67)

at rp.RosettaParser.doFile(RosettaParser.java:59)

at rp.RosettaParser.main(RosettaParser.java:39)

I also have a rule defining an apostrophe delimited character, ('g') and if I alter the input stream to include apostrophes in only this format, there are no problems.

QUESTION: Why can't I skip single semicolons with the '.' wildcard?

Thanks


Tim