Custom tokens
0 posts in topic
Flat View  Flat View
TOPIC ACTIONS:
 

Posted By:   valentin_tihomirov
Posted On:   Wednesday, January 26, 2005 02:43 AM

The lexer return token class objects. Actually, the lexer matches complex tokens, like floating point number has a INT '.' INT pattern which returned to the parser as a plain string will requare additional parsing translating this token into a number at the higher level. I wanna return my own token object extended by the preparsed data field; it would be the structure of two INT strings for the floating example. The manual has some insight to the issue in the "Heterogeneous Token Object Streams" section were the use of $setToken(t) method is demonstrated. However, no details on the custom token class are provided. Which ancestor class must it be derived from? What is the role of Lexer.setTokenObjectClass function?    More>>

The lexer return token class objects. Actually, the lexer matches complex tokens, like floating point number has a INT '.' INT pattern which returned to the parser as a plain string will requare additional parsing translating this token into a number at the higher level. I wanna return my own token object extended by the preparsed data field; it would be the structure of two INT strings for the floating example. The manual has some insight to the issue in the "Heterogeneous Token Object Streams" section were the use of $setToken(t) method is demonstrated. However, no details on the custom token class are provided. Which ancestor class must it be derived from? What is the role of Lexer.setTokenObjectClass function?

   <<Less
About | Sitemap | Contact