dcsimg
I need help writing a program and am not too conversant with JAVA
1 posts in topic
Flat View  Flat View
TOPIC ACTIONS:
 

Posted By:   Michelle_Nersesian
Posted On:   Thursday, November 15, 2001 02:04 PM

I have a very simple (well, very being a relative term) question for the JAVA geeks out there.


I am working on a lexical frequency program that takes a text, breaks it down into separate word tokens, then sorts and counts how many times each word occurs. Easy enough.


However, I am also doing this in sign language gloss, which uses something called Classifiers. Everything that happens [from there to here, ie, in the brackets] needs to be counted as ONE token. Any ideas?


Thanks,
Michelle Nersesian

Re: I need help writing a program and am not too conversant with JAVA

Posted By:   pravus_angelus  
Posted On:   Thursday, November 15, 2001 06:31 PM

It sounds like you need a StringTokenizer. Example usage:


package java.util.StringTokenizer;

//...


String myString;
StringTokenizer tokens;


myString = "here are some|words, you can|see they have these| bars in| them";
tokens = new StringTokenizer(myString, "|");


while(tokens.hasMoreTokens()) {
System.out.println(tokens.nextToken());
}



would print something along the lines of:

here are some
words, you can
see they have these
bars in
them



so you could tokenize your input based on some token separators, and operate on each token independently (which is what it sounds like you want)
About | Sitemap | Contact