Tokenize String Disregarding Trailing/Leading Whitespace
Tokenize String Disregarding Trailing/Leading Whitespace
I have to create a getToken function that will return one token at a time from the input buffer. I also need to implement an isWhiteSpace function that returns true if the character passed to it is white space (a blank, a tab, a line feed) and false if it is a CRLF or EOF.
My problem is that when I enter a string, it will only go to the first whitespace character and stop. If I start with a whitespace, it will just print a blank string. How can I fix this?
public class Lab1
public static void main(String args)
Scanner sc = new Scanner(System.in);
System.out.println("Welcome to the Tokenizer!");
while (true)
System.out.print("Command: ");
String s = sc.nextLine();
String tk = getToken(s);
if (tk.equals("quit"))
break;
else
System.out.println(tk);
static String getToken(String w)
String b = "";
for (int i = 0; i < w.length(); i++)
char c = w.charAt(i);
if (!isWhite(c))
b = b + c;
else
b = b + "";
break;
return b;
static boolean isWhite(char ch) ch == 'n');
String.trim
Hint:
break; will stop any surrounding for-loop– user180100
Sep 5 '17 at 17:24
break;
You might not want to take a break; from this exercise until you've read through all of the characters...
– Sticks
Sep 5 '17 at 17:29
This is a non-trivial logic problem from your side - if you don't understand why the code is doing what it's doing, now would probably be as good a time as any to learn to debug. To figure out how to get the code to do what you want, you should probably sit down and figure it out in your head / on paper before writing any code.
– Dukeling
Sep 5 '17 at 17:55
@RC. Ah I see! But when I remove the break it ends up combining the tokens from the whole string. How would I change it to where it prints each token on a separate line?
– lukas109
Sep 5 '17 at 17:57
1 Answer
1
If you have more than one token per line, your getToken method will have to be renamed to getTokens and return an ArrayList of Strings.
Then you can loop through the tokens on the main method.
Here's a sample code with some suggested changes:
static ArrayList<String> getTokens(String w)
ArrayList<String> tokens = new ArrayList<>(0);
StringBuilder lastWord = new StringBuilder();
for (int i = 0; i < w.length(); i++)
char c = w.charAt(i);
if (isWhiteSpace(c)) // rename to isWhiteSpace, it's a more specific name
tokens.add(lastWord); // if it's a white space add the last word to the list
lastWord.clear(); // clear the buffer
else
lastWord.append(c); // append the char to the buffer
// handle the last word
if (lastWord.length() > 0)
tokens.add(lastWord);
return tokens;
By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.
What's wrong with
String.trim?– user180100
Sep 5 '17 at 17:23