uk.me.parabola.mkgmap.scan
Class TokenScanner

java.lang.Object
  extended by uk.me.parabola.mkgmap.scan.TokenScanner

public class TokenScanner
extends java.lang.Object

Read a file in terms of word and symbol tokens.

Author:
Steve Ratcliffe

Constructor Summary
TokenScanner(java.lang.String filename, java.io.Reader reader)
           
 
Method Summary
 boolean checkToken(java.lang.String val)
           
 java.lang.String getFileName()
           
 int getLinenumber()
           
 boolean isEndOfFile()
           
 int nextInt()
          Convenience routine to get an integer.
 Token nextToken()
          Get the next token tht is not a space or newline.
 java.lang.String nextValue()
          Get the value of the next token and consume the token.
 java.lang.String nextWord()
          As nextWordWithInfo() but just the string is returned.
 WordInfo nextWordWithInfo()
          Read a string that can be quoted.
 Token peekToken()
          Peek and return the first token.
 java.lang.String readLine()
          Read the tokens up until the end of the line and combine then into one string.
 java.lang.String readUntil(TokType type, java.lang.String value)
           
 void setExtraWordChars(java.lang.String extraWordChars)
           
 void skipLine()
          Skip everything up to a new line token.
 void skipSpace()
          Skip any white space.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

TokenScanner

public TokenScanner(java.lang.String filename,
                    java.io.Reader reader)
Method Detail

peekToken

public Token peekToken()
Peek and return the first token. It is not consumed.


nextToken

public Token nextToken()
Get the next token tht is not a space or newline.

Returns:
The first valid text or symbol token.

nextValue

public java.lang.String nextValue()
Get the value of the next token and consume the token. You'd probably only call this after having peeked the type earlier.


isEndOfFile

public boolean isEndOfFile()

skipSpace

public void skipSpace()
Skip any white space. After calling this the next token will be end of file or something other than SPACE or EOL.


skipLine

public void skipLine()
Skip everything up to a new line token. The new line token will be consumed, so the next token will the the first on a new line (or at EOF).


readLine

public java.lang.String readLine()
Read the tokens up until the end of the line and combine then into one string.

Returns:
A single string, not including the newline terminator. Never returns null, returns an empty string if there is nothing there. The end of line is consumed.

readUntil

public java.lang.String readUntil(TokType type,
                                  java.lang.String value)

nextInt

public int nextInt()
            throws java.lang.NumberFormatException
Convenience routine to get an integer. Skips space and reads a token. This token is converted to an integer if possible.

Returns:
An integer as read from the next non space token.
Throws:
java.lang.NumberFormatException - When the next symbol isn't a valid integer.

nextWord

public java.lang.String nextWord()
As nextWordWithInfo() but just the string is returned.

Returns:
The next word as a string. A quoted entity is regarded as a word for the purposes of this scanner.

nextWordWithInfo

public WordInfo nextWordWithInfo()
Read a string that can be quoted. If it is quoted, then everything until the closing quotes is part of the string. Both single and double quotes can be used. If there are no quotes then it behaves like nextToken apart from skipping space. Initial and final space is skipped. The word string is returned along with a flag to indicate whether it was quoted or not.


checkToken

public boolean checkToken(java.lang.String val)

getLinenumber

public int getLinenumber()

getFileName

public java.lang.String getFileName()

setExtraWordChars

public void setExtraWordChars(java.lang.String extraWordChars)