marf.nlp.Parsing
Class TokenType

java.lang.Object
  marf.nlp.Parsing.TokenType

All Implemented Interfaces:

java.io.Serializable

Direct Known Subclasses:

TokenSubType

public class TokenType
extends java.lang.Object
implements java.io.Serializable

Generic Token Type. Can be subclassed for more specific token types.

$Id: TokenType.java,v 1.16 2008/01/03 03:21:57 mokhov Exp $

Since:

0.3.0.2

Version:

$Revision: 1.16 $

Author:

Serguei Mokhov

See Also:

Serialized Form

Field Summary

static int	BACKTRACK This indicates we have to backtrack one token and don't do any extra processing.
static int	BRACKET Bracket or parenthesis token type.
static int	EOF End of File.
static int	ERROR This indicates an error in the token.
static int	ID Identifier token type.
protected int	iType Current token type.
static int	KEYWORD Identified keyword token type.
static int	NUM Numerical token type.
static int	OPERATOR Operator token type.
static int	PUNCT Punctuation token type.
private static long	serialVersionUID For serialization versioning.
static java.util.Hashtable	soTokenTypes Hash representing literal values of the above tokens (for output, for example).
static int	UNKNOWN Unknown token type.

Constructor Summary

TokenType()
Default Constructor.

Method Summary

static java.lang.String	getMARFSourceCodeRevision() Retrieves class' revision.
int	getType() Allows querying for the current token type.
void	setType(int piType) Allows setting the current token type.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ID

public static final int ID

Identifier token type.

See Also:

Constant Field Values

NUM

public static final int NUM

Numerical token type.

See Also:

Constant Field Values

OPERATOR

public static final int OPERATOR

Operator token type.

See Also:

Constant Field Values

PUNCT

public static final int PUNCT

Punctuation token type.

See Also:

Constant Field Values

BRACKET

public static final int BRACKET

Bracket or parenthesis token type.

See Also:

Constant Field Values

KEYWORD

public static final int KEYWORD

Identified keyword token type. A more precise set of IDs.

See Also:

Constant Field Values

UNKNOWN

public static final int UNKNOWN

Unknown token type.

See Also:

Constant Field Values

EOF

public static final int EOF

End of File. Not really a token type, but and indicator of the fact the the end of the source file has been reached

See Also:

Constant Field Values

ERROR

public static final int ERROR

This indicates an error in the token.

See Also:

Constant Field Values

BACKTRACK

public static final int BACKTRACK

This indicates we have to backtrack one token and don't do any extra processing.

See Also:

Constant Field Values

iType

protected int iType

Current token type.

soTokenTypes

public static java.util.Hashtable soTokenTypes

Hash representing literal values of the above tokens (for output, for example).

serialVersionUID

private static final long serialVersionUID

For serialization versioning. When adding new members or make other structural changes regenerate this number with the serialver tool that comes with JDK.

Since:

0.3.0.4

See Also:

Constant Field Values

Constructor Detail

TokenType

public TokenType()

Default Constructor.

Method Detail

getType

public int getType()

Allows querying for the current token type.

Returns:

the current token type

Since:

0.3.0.5

setType

public void setType(int piType)

Allows setting the current token type.

Parameters:

piType - the piType token type

Since:

0.3.0.5

getMARFSourceCodeRevision

public static java.lang.String getMARFSourceCodeRevision()

Retrieves class' revision.

Returns:

revision string