marf.nlp.Parsing.GrammarCompiler
Class GrammarAnalyzer

java.lang.Object
  extended bymarf.nlp.Parsing.GenericLexicalAnalyzer
      extended bymarf.nlp.Parsing.GrammarCompiler.GrammarAnalyzer
Direct Known Subclasses:
ProbabilisticGrammarAnalyzer

public class GrammarAnalyzer
extends GenericLexicalAnalyzer

GrammarAnalyzer class encapsulates functionality required for lexical analysis of a the grammar definition file.

$Id: GrammarAnalyzer.java,v 1.21 2006/01/19 04:13:17 mokhov Exp $

Since:
0.3.0.2
Version:
$Revision: 1.21 $
Author:
Serguei Mokhov

Field Summary
static java.lang.String DEFAULT_ERROR_LOG_FILE
          Default filename to log lexical errors.
static java.lang.String DEFAULT_GRAMMAR_FILE
          Default filename for a grammar file.
static java.lang.String DEFAULT_SCAN_LOG_FILE
          Default filename to log scan output.
protected  GrammarTokenType oTokenType
          Grammar token type.
 
Fields inherited from class marf.nlp.Parsing.GenericLexicalAnalyzer
bErrorsPresent, DEFAULT_ERROR_FILE, DEFAULT_OUTPUT_FILE, oFileReader, oLexicalErrors, oStreamTokenizer, oSymTab, oToken, oTokenList, strErrorLogFilename, strOutputFilename, strSourceFilename
 
Constructor Summary
GrammarAnalyzer()
          Constructs a grammar analyzer with the default filename.
GrammarAnalyzer(java.lang.String pstrGrammarFileName)
          Constructs a grammar analyzer with the specified grammar filename.
GrammarAnalyzer(SymbolTable poSymTab)
          Constructs a grammar analyzer using a predefined symbol table.
 
Method Summary
protected  void addToTokenList()
          Adds the current token to the token list if it is not null and not of backtrack, EOF, or error type.
protected  void createEOFToken()
          Creates an EOF token as the current token.
protected  void createNextToken()
          Creates next grammar token from the stream of characters.
protected  void createOrdinaryToken()
          Creates an ordinary keyword, punctuation, operator, bracket, or lexical error tokens.
protected  void createWordToken()
          Creates a general word token for keywords, errors, semantic tokens, etc.
 java.util.Vector getLexicalGrammarErrors()
          Allows querying for the collection of lexical errors.
static java.lang.String getMARFSourceCodeRevision()
          Retrieves class' revision.
 Token getNextToken()
          Returns next grammar token.
 boolean init()
          Customizes this analyzer for grammar files after calling parent's init().
 boolean serialize(int piOperation)
          Load/Save the contents of lists such as Token list and Error list.
 
Methods inherited from class marf.nlp.Parsing.GenericLexicalAnalyzer
createToken, getErrorLogFilename, getErrorsPresent, getLexicalErrors, getOutputFilename, getSourceFilename, getSymTab, getTokenList, scan, setErrorLogFilename, setOutputFilename, setSourceFilename
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

DEFAULT_GRAMMAR_FILE

public static final java.lang.String DEFAULT_GRAMMAR_FILE
Default filename for a grammar file.

See Also:
Constant Field Values

DEFAULT_ERROR_LOG_FILE

public static final java.lang.String DEFAULT_ERROR_LOG_FILE
Default filename to log lexical errors.

Since:
0.3.0.5
See Also:
Constant Field Values

DEFAULT_SCAN_LOG_FILE

public static final java.lang.String DEFAULT_SCAN_LOG_FILE
Default filename to log scan output.

Since:
0.3.0.5
See Also:
Constant Field Values

oTokenType

protected GrammarTokenType oTokenType
Grammar token type.

Constructor Detail

GrammarAnalyzer

public GrammarAnalyzer()
Constructs a grammar analyzer with the default filename.

See Also:
DEFAULT_GRAMMAR_FILE

GrammarAnalyzer

public GrammarAnalyzer(java.lang.String pstrGrammarFileName)
Constructs a grammar analyzer with the specified grammar filename.

Parameters:
pstrGrammarFileName - the filename of the grammar file to read from

GrammarAnalyzer

public GrammarAnalyzer(SymbolTable poSymTab)
Constructs a grammar analyzer using a predefined symbol table. Uses the default filename for grammar.

Parameters:
poSymTab - the symbol table reference
See Also:
DEFAULT_GRAMMAR_FILE
Method Detail

init

public boolean init()
Customizes this analyzer for grammar files after calling parent's init(). Allows C++, C, and script (#) style comments; case-sensitive; ignores EOL; and treats some puntuation a special. Sets default lex out filenames for standard and error lexical output.

Overrides:
init in class GenericLexicalAnalyzer
Returns:
true of initialization is successful
See Also:
GenericLexicalAnalyzer.init()

serialize

public boolean serialize(int piOperation)
Load/Save the contents of lists such as Token list and Error list. 0 means LOAD, 1 means SAVE. Load currently is not implemented.

Specified by:
serialize in class GenericLexicalAnalyzer
Parameters:
piOperation - 0 means load, 1 means save
Returns:
true if the serialization was successful
See Also:
GenericLexicalAnalyzer.serialize(int)

createEOFToken

protected void createEOFToken()
Creates an EOF token as the current token. Its default lexeme is `$'.


createWordToken

protected void createWordToken()
                        throws NotImplementedException
Creates a general word token for keywords, errors, semantic tokens, etc.

Throws:
NotImplementedException - for dictionary words

createOrdinaryToken

protected void createOrdinaryToken()
Creates an ordinary keyword, punctuation, operator, bracket, or lexical error tokens.


createNextToken

protected void createNextToken()
                        throws LexicalError
Creates next grammar token from the stream of characters.

Throws:
LexicalError - if invalid characters encountered or there was an I/O problem

getNextToken

public Token getNextToken()
                   throws LexicalError
Returns next grammar token.

Specified by:
getNextToken in class GenericLexicalAnalyzer
Returns:
newly recognized lexical token
Throws:
LexicalError - in case of invalid character stream (alphabet) entries found
See Also:
GenericLexicalAnalyzer.getNextToken()

addToTokenList

protected void addToTokenList()
Adds the current token to the token list if it is not null and not of backtrack, EOF, or error type.

See Also:
TokenType.BACKTRACK, TokenType.EOF, TokenType.ERROR

getLexicalGrammarErrors

public java.util.Vector getLexicalGrammarErrors()
Allows querying for the collection of lexical errors.

Returns:
the collection of lexical errors

getMARFSourceCodeRevision

public static java.lang.String getMARFSourceCodeRevision()
Retrieves class' revision.

Returns:
revision string


SourceForge Logo