marf.nlp.Parsing.GrammarCompiler
Class GrammarAnalyzer

java.lang.Object
  marf.nlp.Parsing.GenericLexicalAnalyzer
      marf.nlp.Parsing.GrammarCompiler.GrammarAnalyzer

Direct Known Subclasses:

ProbabilisticGrammarAnalyzer

public class GrammarAnalyzer
extends GenericLexicalAnalyzer

GrammarAnalyzer class encapsulates functionality required for lexical analysis of a the grammar definition file.

$Id: GrammarAnalyzer.java,v 1.24 2008/01/03 03:21:57 mokhov Exp $

Since:

0.3.0.2

Version:

$Revision: 1.24 $

Author:

Serguei Mokhov

Field Summary

static java.lang.String	DEFAULT_ERROR_LOG_FILE Default filename to log lexical errors.
static java.lang.String	DEFAULT_GRAMMAR_FILE Default filename for a grammar file.
static java.lang.String	DEFAULT_SCAN_LOG_FILE Default filename to log scan output.
protected GrammarTokenType	oTokenType Grammar token type.

Fields inherited from class marf.nlp.Parsing.GenericLexicalAnalyzer

bErrorsPresent, DEFAULT_ERROR_FILE, DEFAULT_OUTPUT_FILE, oFileReader, oLexicalErrors, oStreamTokenizer, oSymTab, oToken, oTokenList, strErrorLogFilename, strOutputFilename, strSourceFilename

Constructor Summary

GrammarAnalyzer()
Constructs a grammar analyzer with the default filename.

GrammarAnalyzer(java.lang.String pstrGrammarFileName)
Constructs a grammar analyzer with the specified grammar filename.

GrammarAnalyzer(SymbolTable poSymTab)
Constructs a grammar analyzer using a predefined symbol table.

Method Summary

protected void	addToTokenList() Adds the current token to the token list if it is not null and not of backtrack, EOF, or error type.
protected void	createEOFToken() Creates an EOF token as the current token.
protected void	createNextToken() Creates next grammar token from the stream of characters.
protected void	createOrdinaryToken() Creates an ordinary keyword, punctuation, operator, bracket, or lexical error tokens.
protected void	createWordToken() Creates a general word token for keywords, errors, semantic tokens, etc.
java.util.Vector	getLexicalGrammarErrors() Allows querying for the collection of lexical errors.
static java.lang.String	getMARFSourceCodeRevision() Retrieves class' revision.
Token	getNextToken() Returns next grammar token.
boolean	init() Customizes this analyzer for grammar files after calling parent's init().
boolean	serialize(int piOperation) Load/Save the contents of lists such as Token list and Error list.

Methods inherited from class marf.nlp.Parsing.GenericLexicalAnalyzer

createToken, getErrorLogFilename, getErrorsPresent, getLexicalErrors, getOutputFilename, getSourceFilename, getSymTab, getTokenList, scan, setErrorLogFilename, setOutputFilename, setSourceFilename

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_GRAMMAR_FILE

public static final java.lang.String DEFAULT_GRAMMAR_FILE

Default filename for a grammar file.

See Also:

Constant Field Values

DEFAULT_ERROR_LOG_FILE

public static final java.lang.String DEFAULT_ERROR_LOG_FILE

Default filename to log lexical errors.

Since:

0.3.0.5

See Also:

Constant Field Values

DEFAULT_SCAN_LOG_FILE

public static final java.lang.String DEFAULT_SCAN_LOG_FILE

Default filename to log scan output.

Since:

0.3.0.5

See Also:

Constant Field Values

oTokenType

protected GrammarTokenType oTokenType

Grammar token type.

Constructor Detail

GrammarAnalyzer

public GrammarAnalyzer()

Constructs a grammar analyzer with the default filename.

See Also:

DEFAULT_GRAMMAR_FILE

GrammarAnalyzer

public GrammarAnalyzer(java.lang.String pstrGrammarFileName)

Constructs a grammar analyzer with the specified grammar filename.

Parameters:

pstrGrammarFileName - the filename of the grammar file to read from

GrammarAnalyzer

public GrammarAnalyzer(SymbolTable poSymTab)

Constructs a grammar analyzer using a predefined symbol table. Uses the default filename for grammar.

Parameters:

poSymTab - the symbol table reference

See Also:

DEFAULT_GRAMMAR_FILE

Method Detail

init

public boolean init()

Customizes this analyzer for grammar files after calling parent's init(). Allows C++, C, and script (#) style comments; case-sensitive; ignores EOL; and treats some punctuation a special. Sets default lex out filenames for standard and error lexical output.

Overrides:

init in class GenericLexicalAnalyzer

Returns:

true of initialization is successful

See Also:

GenericLexicalAnalyzer.init()

serialize

public boolean serialize(int piOperation)

Load/Save the contents of lists such as Token list and Error list. 0 means LOAD, 1 means SAVE. Load currently is not implemented.

Specified by:

serialize in class GenericLexicalAnalyzer

Parameters:

piOperation - 0 means load, 1 means save

Returns:

true if the serialization was successful

See Also:

GenericLexicalAnalyzer.serialize(int)

createEOFToken

protected void createEOFToken()

Creates an EOF token as the current token. Its default lexeme is `$'.

createWordToken

protected void createWordToken()
                        throws NotImplementedException

Creates a general word token for keywords, errors, semantic tokens, etc.

Throws:

NotImplementedException - for dictionary words

createOrdinaryToken

protected void createOrdinaryToken()

Creates an ordinary keyword, punctuation, operator, bracket, or lexical error tokens.

createNextToken

protected void createNextToken()
                        throws LexicalError

Creates next grammar token from the stream of characters.

Throws:

LexicalError - if invalid characters encountered or there was an I/O problem

getNextToken

public Token getNextToken()
                   throws LexicalError

Returns next grammar token.

Specified by:

getNextToken in class GenericLexicalAnalyzer

Returns:

newly recognized lexical token

Throws:

LexicalError - in case of invalid character stream (alphabet) entries found

See Also:

GenericLexicalAnalyzer.getNextToken()

addToTokenList

protected void addToTokenList()

Adds the current token to the token list if it is not null and not of backtrack, EOF, or error type.

See Also:

TokenType.BACKTRACK, TokenType.EOF, TokenType.ERROR

getLexicalGrammarErrors

public java.util.Vector getLexicalGrammarErrors()

Allows querying for the collection of lexical errors.

Returns:

the collection of lexical errors

getMARFSourceCodeRevision

public static java.lang.String getMARFSourceCodeRevision()

Retrieves class' revision.

Returns:

revision string