marf.nlp.Parsing
Class TokenType

java.lang.Object
  |
  +--marf.nlp.Parsing.TokenType
All Implemented Interfaces:
java.io.Serializable
Direct Known Subclasses:
TokenSubType

public class TokenType
extends java.lang.Object
implements java.io.Serializable

Generic Token Type. Can be subclassed for more specific token types. $Id: TokenType.java,v 1.6 2005/06/01 05:35:09 mokhov Exp $

Since:
0.3.0
Version:
$Revision: 1.6 $
Author:
Serguei Mokhov
See Also:
Serialized Form

Field Summary
static int BACKTRACK
          This indicates we have to backtrack one token and don't do any extra processing.
static int BRACKET
          Bracket or parenthesis token type.
static int EOF
          End of File.
static int ERROR
          This indicates an error in the token.
static int ID
          Identifier token type.
static int KEYWORD
          Identified keyword token type.
static int NUM
          Numerical token type.
static int OPERATOR
          Operator token type.
static int PUNCT
          Punctuation token type.
static java.util.Hashtable TokenTypes
          Hash representing literal values of the above tokens (for output, for example).
 int type
          Current token type.
static int UNKNOWN
          Unknown token type.
 
Constructor Summary
TokenType()
          Default Constructor.
 
Method Summary
static java.lang.String getRevision()
          Retrieves class' revision.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

ID

public static final int ID
Identifier token type.

See Also:
Constant Field Values

NUM

public static final int NUM
Numerical token type.

See Also:
Constant Field Values

OPERATOR

public static final int OPERATOR
Operator token type.

See Also:
Constant Field Values

PUNCT

public static final int PUNCT
Punctuation token type.

See Also:
Constant Field Values

BRACKET

public static final int BRACKET
Bracket or parenthesis token type.

See Also:
Constant Field Values

KEYWORD

public static final int KEYWORD
Identified keyword token type. A more preciuse set of ID.

See Also:
Constant Field Values

UNKNOWN

public static final int UNKNOWN
Unknown token type.

See Also:
Constant Field Values

EOF

public static final int EOF
End of File. Not really a token type, but and idicator of the fact the the end of the source file has been reached

See Also:
Constant Field Values

ERROR

public static final int ERROR
This indicates an error in the token.

See Also:
Constant Field Values

BACKTRACK

public static final int BACKTRACK
This indicates we have to backtrack one token and don't do any extra processing.

See Also:
Constant Field Values

type

public int type
Current token type.


TokenTypes

public static java.util.Hashtable TokenTypes
Hash representing literal values of the above tokens (for output, for example).

Constructor Detail

TokenType

public TokenType()
Default Constructor.

Method Detail

getRevision

public static java.lang.String getRevision()
Retrieves class' revision.

Returns:
revision string