marf.nlp.Parsing.GrammarCompiler
Class ProbabilisticGrammarAnalyzer

java.lang.Object
  marf.nlp.Parsing.GenericLexicalAnalyzer
      marf.nlp.Parsing.GrammarCompiler.GrammarAnalyzer
          marf.nlp.Parsing.GrammarCompiler.ProbabilisticGrammarAnalyzer

public class ProbabilisticGrammarAnalyzer
extends GrammarAnalyzer

Probabilistic Grammar Analyzer -- an lexical analyzer designed for probabilistic grammars out of a stream of characters.

$Id: ProbabilisticGrammarAnalyzer.java,v 1.16 2007/12/18 21:37:57 mokhov Exp $

Since:

0.3.0.2

Version:

$Revision: 1.16 $

Author:

Serguei Mokhov

Field Summary

Fields inherited from class marf.nlp.Parsing.GrammarCompiler.GrammarAnalyzer

DEFAULT_ERROR_LOG_FILE, DEFAULT_GRAMMAR_FILE, DEFAULT_SCAN_LOG_FILE, oTokenType

Fields inherited from class marf.nlp.Parsing.GenericLexicalAnalyzer

bErrorsPresent, DEFAULT_ERROR_FILE, DEFAULT_OUTPUT_FILE, oFileReader, oLexicalErrors, oStreamTokenizer, oSymTab, oToken, oTokenList, strErrorLogFilename, strOutputFilename, strSourceFilename

Constructor Summary

ProbabilisticGrammarAnalyzer(java.lang.String pstrGrammarFilename)
Constructs probabilistic grammar analyzer from a given grammar file.

Method Summary

protected void	createNextToken() Creates an instance of the next token in the grammar token stream given integer token type.
protected void	createProbabilityToken() Creates a probabilistic token type as a part of the grammar.
protected void	createWordToken() Creates word tokens for keywords, identifiers, non-terminals and dictionary words.
static java.lang.String	getMARFSourceCodeRevision() Retrieves class' revision.
boolean	init() Initializes this grammar analyzer and sets the stream tokenizer to recognize numerical values specially and don't fold anything to lowercase.

Methods inherited from class marf.nlp.Parsing.GrammarCompiler.GrammarAnalyzer

addToTokenList, createEOFToken, createOrdinaryToken, getLexicalGrammarErrors, getNextToken, serialize

Methods inherited from class marf.nlp.Parsing.GenericLexicalAnalyzer

createToken, getErrorLogFilename, getErrorsPresent, getLexicalErrors, getOutputFilename, getSourceFilename, getSymTab, getTokenList, scan, setErrorLogFilename, setOutputFilename, setSourceFilename

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ProbabilisticGrammarAnalyzer

public ProbabilisticGrammarAnalyzer(java.lang.String pstrGrammarFilename)

Constructs probabilistic grammar analyzer from a given grammar file.

Parameters:

pstrGrammarFilename - the filename of a probabilistic grammar

Method Detail

init

public boolean init()

Initializes this grammar analyzer and sets the stream tokenizer to recognize numerical values specially and don't fold anything to lowercase.

Overrides:

init in class GrammarAnalyzer

Returns:

true if parent's initialization was successful

See Also:

GenericLexicalAnalyzer.init()

createProbabilityToken

protected void createProbabilityToken()

Creates a probabilistic token type as a part of the grammar.

createNextToken

protected void createNextToken()
                        throws LexicalError

Creates an instance of the next token in the grammar token stream given integer token type. The token type is determined from the underlying stream tokenizer.

Overrides:

createNextToken in class GrammarAnalyzer

Throws:

LexicalError - if there was an error reading off the token

See Also:

GrammarAnalyzer.createNextToken()

createWordToken

protected void createWordToken()

Creates word tokens for keywords, identifiers, non-terminals and dictionary words.

Overrides:

createWordToken in class GrammarAnalyzer

See Also:

GrammarAnalyzer.createWordToken()

getMARFSourceCodeRevision

public static java.lang.String getMARFSourceCodeRevision()

Retrieves class' revision.

Returns:

revision string