The Artima Developer Community
Sponsored Link

Java Answers Forum
Stream Tokeneizer

1 reply on 1 page. Most recent reply: Oct 21, 2003 8:44 AM by al

Welcome Guest
  Sign In

Go back to the topic listing  Back to Topic List Click to reply to this topic  Reply to this Topic Click to search messages in this forum  Search Forum Click for a threaded view of the topic  Threaded View   
Previous Topic   Next Topic
Flat View: This topic has 1 reply on 1 page
Anis

Posts: 3
Nickname: anis
Registered: Oct, 2003

Stream Tokeneizer Posted: Oct 20, 2003 1:36 AM
Reply to this message Reply
Advertisement
StreamTokenizer stok = new StreamTokenizer(new FileInputStream(FileName));
stok.wordChars((char)0,(char)255);
stok.whitespaceC hars('(',')');

why this code dosn't work with number characters
example when we have this
((AAA)(BBB)CCC) this code give the right answer but
when we have ((AAA)(BBB)3CCC) the code ignore the number 3 char
it give AAA as token 1
BBB as token 2
CCC as token 3
it ignore the number 3
why
thank you
please help


al

Posts: 11
Nickname: allelopath
Registered: Jun, 2003

Re: Stream Tokeneizer Posted: Oct 21, 2003 8:44 AM
Reply to this message Reply
Note that StreamTokenizer(InputStream is) is deprecated. As of JDK version 1.1, the preferred way to tokenize an input stream is to convert it into a character stream, for example:

Reader r = new BufferedReader(new InputStreamReader(is));
StreamTokenizer st = new StreamTokenizer(r);


This code does what you want:

import java.io.*;
import java.util.StringTokenizer;

public class STTest
{
private String line = "";
private String[] lineOfTokens = null;
private StringTokenizer stringTokens;

public STTest()
{
try
{
BufferedReader br = new BufferedReader(new FileReader("testfile.dat"));

while ( (line = br.readLine()) != null )
{
stringTokens = new StringTokenizer(line, "()");
lineOfTokens = new String[stringTokens.countTokens()];
int x = 0;
}
catch (FileNotFoundException e)
{
e.printStackTrace();
}
catch (IOException e)
{
e.printStackTrace();
}

for (int i = 0; i < lineOfTokens.length; i++)
{
lineOfTokens[i] = stringTokens.nextToken();
}

for (int i = 0; i < lineOfTokens.length; i++)
{
System.out.println (lineOfTokens[i] + "\t");
}
}

public static void main(String[] args)
{
STTest sttest = new STTest();
}

}

Flat View: This topic has 1 reply on 1 page
Topic: STream tokeneizer Previous Topic   Next Topic Topic: substrings

Sponsored Links



Google
  Web Artima.com   

Copyright © 1996-2019 Artima, Inc. All Rights Reserved. - Privacy Policy - Terms of Use