Thursday, June 7, 2001 06:40 AM
The lexer can't generate the proper token because, as you say, you need the proper syntax context. It will be the parser which will have to decide what the token should be.
When I did this in the R/BASIC parser, I returned IF and all the other keywords as keywords, then had a special rule keywordId which could be used where you would normally have used ID. Syntactic predicates were needed around the places I would use keywordId so that the ambiguities would be resolved properly.