dcsimg
Need help in creating Lexer
1 posts in topic
Flat View  Flat View
TOPIC ACTIONS:
 

Posted By:   Mehul_Shah
Posted On:   Tuesday, July 31, 2001 12:52 PM

I am trying to make a lexer which should take the input and recognize certain forms of input as tokens rest of the input I want to break by white spaces as words (where words can be made of any characters...). Say I want to recognize two tokens LX_AND = "and" // in any case LX_WN = W/99 // W followed by 2 digits. LX_WS = ' '|' '|' ' // White spaces ignore Rest of the input I want to recognize as token LX_WORD. For example: Input = Tom 2.123 w/12 34 and good It should generate following token seq... LX_WORD = Tom LX_WORD = 2.123 LX_WN = w/12 LX_WORD = 34 LX_AND = and    More>>

I am trying to make a lexer which should take the input and recognize certain forms of input as tokens rest of the input I want to break by white spaces as words (where words can be made of any characters...).


Say I want to recognize two tokens

LX_AND = "and" // in any case

LX_WN = W/99 // W followed by 2 digits.

LX_WS = ' '|' '|'
' // White spaces ignore

Rest of the input I want to recognize as token LX_WORD.



For example:

Input = Tom 2.123 w/12 34 and good

It should generate following token seq...

LX_WORD = Tom

LX_WORD = 2.123

LX_WN = w/12

LX_WORD = 34

LX_AND = and

LX_WORD = good




Thanks,

Mehul.

   <<Less

Re: Need help in creating Lexer

Posted By:   Terence_Parr  
Posted On:   Friday, August 3, 2001 11:32 PM

Take a look at the token filter stuff and the articles/faq entries on making ANTLR behave like SED and AWK.
About | Sitemap | Contact