marf.nlp.Parsing
Class Token

java.lang.Object
  marf.nlp.Parsing.Token

All Implemented Interfaces:

java.io.Serializable

public class Token
extends java.lang.Object
implements java.io.Serializable

This class denotes a Token data structure.

$Id: Token.java,v 1.22 2008/01/03 03:21:57 mokhov Exp $

Since:

0.3.0.2

Version:

$Revision: 1.22 $

Author:

Serguei Mokhov

See Also:

Serialized Form

Field Summary

protected double	dNumericalValue Numerical value associated with the token to be mapped to StreamTokenizer.nval
protected java.awt.Point	oPosition Line and column where the token occurred.
protected TokenSubType	oTokenType Token type.
private static long	serialVersionUID For serialization versioning.
protected java.lang.String	strLexeme Spelling of the token.

Constructor Summary

Token(java.lang.String pstrLexeme, int piLineNumber, int piTokenType, double pdNumericalValue)
Constructor that maps values produced by StreamTokenizer.

Token(java.lang.String pstrLexeme, java.awt.Point poPosition, TokenSubType poTokenType)
Constructor with full parameters.

Token(Token poToken)
Copy-constructor.

Method Summary

java.lang.String	getLexeme() Allows accessing the lexeme property.
int	getLineNumber() Gets the line number component of the token's position.
static java.lang.String	getMARFSourceCodeRevision() Retrieves class' revision.
double	getNumericalValue() Gets the numerical representation of the token.
java.awt.Point	getPosition() Allows accessing position property.
TokenSubType	getTokenType() Allows accessing the token type.
boolean	serialize(int piOperation, java.io.FileWriter poFileWriter) Loads/saves the token to a text file.
void	setLexeme(java.lang.String pstrLexeme) Sets the value of the lexeme property.
void	setPosition(java.awt.Point poPosition) Sets the value of the new position of the token.
void	setTokenType(TokenSubType poTokenType) Sets the value of the token type.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

strLexeme

protected java.lang.String strLexeme

Spelling of the token.

oPosition

protected java.awt.Point oPosition

Line and column where the token occurred.

oTokenType

protected TokenSubType oTokenType

Token type.

dNumericalValue

protected double dNumericalValue

Numerical value associated with the token to be mapped to StreamTokenizer.nval

See Also:

StreamTokenizer.nval

serialVersionUID

private static final long serialVersionUID

For serialization versioning. When adding new members or make other structural changes regenerate this number with the serialver tool that comes with JDK.

Since:

0.3.0.4

See Also:

Constant Field Values

Constructor Detail

Token

public Token(java.lang.String pstrLexeme,
             java.awt.Point poPosition,
             TokenSubType poTokenType)

Constructor with full parameters.

Parameters:

pstrLexeme - token's lexeme

poPosition - position in the source

poTokenType - type of a token

Token

public Token(java.lang.String pstrLexeme,
             int piLineNumber,
             int piTokenType,
             double pdNumericalValue)

Constructor that maps values produced by StreamTokenizer.

Parameters:

pstrLexeme - token's lexeme

piLineNumber - line number of the token

piTokenType - type of a token

pdNumericalValue - numerical value of the token

Since:

0.3.0.5

See Also:

StreamTokenizer

Token

public Token(Token poToken)

Copy-constructor.

Parameters:

poToken - token to clone data members from.

Method Detail

serialize

public boolean serialize(int piOperation,
                         java.io.FileWriter poFileWriter)

Loads/saves the token to a text file. TODO: migrate to MARF's dump/store machinery.

Parameters:

piOperation - load or save

poFileWriter - file writer object to use

Returns:

true of the write is successful

getLexeme

public java.lang.String getLexeme()

Allows accessing the lexeme property.

Returns:

the current value of the lexeme property

setLexeme

public void setLexeme(java.lang.String pstrLexeme)

Sets the value of the lexeme property.

Parameters:

pstrLexeme - the new value of the lexeme property

getPosition

public java.awt.Point getPosition()

Allows accessing position property.

Returns:

the current value of the position

setPosition

public void setPosition(java.awt.Point poPosition)

Sets the value of the new position of the token.

Parameters:

poPosition - the new value of the position property

getLineNumber

public int getLineNumber()

Gets the line number component of the token's position.

Returns:

the current value of line number

Since:

0.3.0.5

getNumericalValue

public double getNumericalValue()

Gets the numerical representation of the token.

Returns:

the current numerical value of the token

Since:

0.3.0.5

getTokenType

public TokenSubType getTokenType()

Allows accessing the token type.

Returns:

the current value of the TokenType property

setTokenType

public void setTokenType(TokenSubType poTokenType)

Sets the value of the token type.

Parameters:

poTokenType - the new value of the TokenType

getMARFSourceCodeRevision

public static java.lang.String getMARFSourceCodeRevision()

Retrieves class' revision.

Returns:

revision string