Back to Study Material Java

StringTokenizer in Java

StringTokenizer is a utility class in java.util that breaks a string into smaller pieces called tokens, separated by delimiter characters. It was the stand...

StringTokenizer in Java

Splitting Strings the Legacy Way

StringTokenizer is a utility class in java.util that breaks a string into smaller pieces called tokens, separated by delimiter characters. It was the standard way to parse text before Java 1.4 introduced String.split() and before more modern parsing libraries became common. Although StringTokenizer is now considered a legacy class and its use is discouraged in new code, it still appears in older codebases and in exam questions, so understanding it is valuable.

Creating a StringTokenizer The simplest constructor takes just the string to parse: new StringTokenizer("one two three"). By default, whitespace—spaces, tabs, and newlines—serves as the delimiter. The second constructor lets you specify custom delimiters: new StringTokenizer("a,b,c", ",") treats comma as the only delimiter. A third constructor adds a boolean flag; when set to true, the delimiters themselves are returned as tokens in addition to the words between them.

Iterating Over Tokens StringTokenizer implements Enumeration. You use hasMoreTokens() to check whether any tokens remain and nextToken() to retrieve the next one. There is no way to reset or restart a StringTokenizer once tokens have been consumed; you must create a new instance. The countTokens() method returns how many tokens are left without consuming them, which is occasionally useful for pre-allocating arrays.

StringTokenizer vs String.split() String.split() accepts a regular expression, making it far more flexible. It returns a String array, which is easier to work with in modern code using enhanced for loops or streams. The main advantage StringTokenizer once had was speed and lower memory use, but that gap is negligible today. Prefer String.split() or Scanner for new code, and reserve StringTokenizer knowledge for reading or maintaining legacy systems.

Code Example: StringTokenizer st = new StringTokenizer("red,green,blue", ","); while (st.hasMoreTokens()) { System.out.println(st.nextToken()); } // red // green // blue

Interview Tip: If an interviewer asks about StringTokenizer, mention that it is a legacy class and that String.split() or Scanner is preferred. Also note that StringTokenizer does not support regular expression delimiters, which is its key limitation compared to split(). Knowing this distinction shows awareness of Java's evolution.