Topic: I need help writing a program and am not too conversant with JAVA

I need help writing a program and am not too conversant with JAVA

1 posts in topic

Flat View

Thread View

Older Post First | Reply to Topic

TOPIC ACTIONS:

Print Topic

Posted By: Michelle_Nersesian
Posted On: Thursday, November 15, 2001 02:04 PM

I have a very simple (well, very being a relative term) question for the JAVA geeks out there.

I am working on a lexical frequency program that takes a text, breaks it down into separate word tokens, then sorts and counts how many times each word occurs. Easy enough.

However, I am also doing this in sign language gloss, which uses something called Classifiers. Everything that happens [from there to here, ie, in the brackets] needs to be counted as ONE token. Any ideas?

Thanks,
Michelle Nersesian

Report | Quote This | Send private message | Reply | Print

Re: I need help writing a program and am not too conversant with JAVA

Posted By: pravus_angelus
Posted On: Thursday, November 15, 2001 06:31 PM

It sounds like you need a StringTokenizer. Example usage:


package java.util.StringTokenizer;

//...


String myString;
StringTokenizer tokens;


myString = "here are some|words, you can|see they have these| bars in| them";
tokens = new StringTokenizer(myString, "|");


while(tokens.hasMoreTokens()) {
System.out.println(tokens.nextToken());
}

would print something along the lines of:


here are some
words, you can
see they have these
 bars in
 them

so you could tokenize your input based on some token separators, and operate on each token independently (which is what it sounds like you want)

Report | Quote This | Send private message | Reply | Print