Uses of Class
marf.nlp.Parsing.Token

Packages that use Token

marf.nlp.Parsing
marf.nlp.Parsing.GrammarCompiler
marf.nlp.util

Uses of Token in marf.nlp.Parsing

Fields in marf.nlp.Parsing declared as Token

protected Token	LexicalError.oFaultingToken Token information at which Lexer encountered the error.
protected Token	SemanticError.oFaultingToken Token information at which Lexer encountered the error.
protected Token	SyntaxError.oFaultingToken Token information at which Parser encountered the error.
protected Token	GenericLexicalAnalyzer.oToken Current token being processed.
protected Token	SymTabEntry.oUID Entry symbol, encapsulated as Token.

Methods in marf.nlp.Parsing that return Token

Token	GenericLexicalAnalyzer.createToken(java.lang.String pstrLexeme, TokenSubType poTokenSubType) Creates an instance of a Token data structure provided its type and lexeme, and location is calculated dynamically by the StreamTokenizer.
abstract Token	GenericLexicalAnalyzer.getNextToken() Core method of the LexicalAnalyzer.
Token	LexicalAnalyzer.getNextToken() Core method of the MARF LexicalAnalyzer.
Token	SymTabEntry.getUID() Returns UID.
Token	Parser.popBracket() Pops a bracket-type token from the stack.

Methods in marf.nlp.Parsing with parameters of type Token

void	SymbolTable.addEntry(int piEntryType, Token poToken) Not implemented.
int	SymbolTable.addSymbol(Token poToken) Adds a symbol token to the table.
int	SymbolTable.addSymbol(Token poToken, boolean pbResolved) Not implemented.
int	SymbolTable.addSymbol(Token oToken, int piSymDataType) Not implemented.
java.lang.Object	TransitionTable.getEntryAt(NonTerminal poNonTerminal, Token poToken) Gets a table entry at a given position (NonTerminal, Token).
void	Parser.pushBracket(Token poToken) Pushes bracket-type token onto the bracket stack.
void	SymTabEntry.setUID(Token poToken) Sets UID.
void	SyntaxError.updateTokenInfo(Token poFaultingToken) Allows updating the state of an error, since the info might not be initially available (as in the TT, for example).

Constructors in marf.nlp.Parsing with parameters of type Token

LexicalError(int piErrorCode, Token poFaultingToken)
Constructor with an error code and a faulting token.

LexicalError(Token poFaultingToken, java.lang.String pstrCustomErrorMessage)
Constructor alternative to Unknown error given token and a custom message.

SemanticError(int piErrorCode, Token poFaultingToken)
Constructor with an error code and a faulting token.

SemanticError(Token poFaultingToken, java.lang.String pstrCustomErrorMessage)
Constructor alternative to Unknown error with the custom error message.

SymTabEntry(Token poUID)
Constructs object with the entry symbol, encapsulated as Token.

SymTabEntry(Token poUID, boolean pbDeclared)
Constructs object with the entry symbol, encapsulated as Token, and a flag to indicate whether the symbol was resolved.

SymTabEntry(Token poUID, SymDataType poSymDataType)
Constructs object with the entry symbol, encapsulated as Token, and entry symbol's data type; set the symbol as resolved.

SyntaxError(int piErrorCode, Token poFaultingToken)
Constructor with the error code and its cause.

SyntaxError(Token poFaultingToken, java.lang.String pstrCustomErrorMessage)
Constructor 2 - an alternative to Unknown Error.

Token(Token poToken)
Copy-constructor.

Uses of Token in marf.nlp.Parsing.GrammarCompiler

Fields in marf.nlp.Parsing.GrammarCompiler declared as Token

protected Token	GrammarCompiler.oToken Current lexical token.
protected Token	GrammarElement.oToken A bit extra info in the encapsulated token.

Methods in marf.nlp.Parsing.GrammarCompiler that return Token

Token	GrammarAnalyzer.getNextToken() Returns next grammar token.
Token	GrammarElement.getToken() Allows querying this element's enclosed token.

Methods in marf.nlp.Parsing.GrammarCompiler with parameters of type Token

boolean

Terminal.isToken(Token poToken) Checks is the parameter token is also token by having the same name.

Constructors in marf.nlp.Parsing.GrammarCompiler with parameters of type Token

GrammarElement(Token poToken, int piID)
Preferred Constructor.

NonTerminal(Token poToken, int piID)
Creates a non-terminal from a token and the ID.

SemanticToken(Token poToken)
Token-based constructor.

SemanticToken(Token poToken, int piID)
Token- and ID-based constructor.

Terminal(Token poToken, int piID)
Constructor with the token and its ID.

Uses of Token in marf.nlp.util

Fields in marf.nlp.util declared as Token

protected Token

NLPStreamTokenizer.oTopToken Reference to the token on the top of our own stack.