I am trying to make a lexer which should take the input and recognize certain forms of input as tokens rest of the input I want to break by white spaces as words (where words can be made of any characters...). Say I want to recognize two tokens LX_AND = "and" // in any case LX_WN = W/99 // W followed by 2 digits. LX_WS = ' '|' '|' ' // White spaces ignore Rest of the input I want to recognize as token LX_WORD. For example: Input = Tom 2.123 w/12 34 and good It should generate following token seq... LX_WORD = Tom LX_WORD = 2.123 LX_WN = w/12 LX_WORD = 34 LX_AND = and    More>>