marf.Preprocessing
Class Preprocessing

java.lang.Object
  marf.Storage.StorageManager
      marf.Preprocessing.Preprocessing

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable, IPreprocessing, IStorageManager

Direct Known Subclasses:

Dummy, Endpoint, Filter

public abstract class Preprocessing
extends StorageManager
implements IPreprocessing

Abstract Preprocessing Module.

$Id: Preprocessing.java,v 1.50 2007/12/16 06:41:22 mokhov Exp $

Since:

0.0.1

Version:

$Revision: 1.50 $

Author:

Serguei Mokhov

See Also:

Serialized Form

Field Summary

protected boolean	bRemoveNoise By default we do not remove noise.
protected boolean	bRemoveSilence By default we do not remove silence.
static double	DEFAULT_SILENCE_THRESHOLD Default amplitude value and below to what consider as a silence.
protected double	dSilenceThreshold Current silence removal threshold.
protected Sample	oSample Sample reference.
private static long	serialVersionUID For serialization versioning.

Fields inherited from class marf.Storage.StorageManager

bDumpOnNotFound, iCurrentDumpMode, oObjectToSerialize, strFilename

Fields inherited from interface marf.Preprocessing.IPreprocessing

MARF_INTERFACE_CODE_REVISION

Fields inherited from interface marf.Storage.IStorageManager

DUMP_BINARY, DUMP_CSV_TEXT, DUMP_GZIP_BINARY, DUMP_HTML, DUMP_SQL, DUMP_XML, MARF_INTERFACE_CODE_REVISION, STORAGE_FILE_EXTENSIONS

Constructor Summary

protected	Preprocessing() Default constructor for reflective creation of Preprocessing clones.
protected	Preprocessing(IPreprocessing poPreprocessing) Allows chaining of preprocessing modules.
protected	Preprocessing(Sample poSample) Main Preprocessing constructor that performs normalization as a part of construction process.

Method Summary

void	backSynchronizeObject() Implementation of back-synchronization of Sample loaded object.
java.lang.Object	clone() Implements Cloneable interface for the Preprocessing object.
boolean	compress() Compresses the instance of the stored sample by removing duplicates.
static double[]	compress(double[] padDataVector) Compresses input data by removing duplicate values.
boolean	cropAudio(double pdStartingFrequency, double pdEndFrequency) Derivatives implement this method to crop arbitrary part of the audio sample.
protected void	extractParameters() Performs general preprocessing module parameters extraction.
static java.lang.String	getMARFSourceCodeRevision() Returns source code revision information.
Sample	getSample() Returns enclosed sample.
boolean	normalize() Normalization of entire incoming samples by amplitude.
boolean	normalize(int piIndexFrom) Normalization of incoming samples by amplitude starting from certain index.
boolean	normalize(int piIndexFrom, int piIndexTo) Normalization of incoming samples by amplitude between specified indexes.
boolean	preprocess() Default implementation of preprocess() includes normalization of the sample, and optionally removal of noise and silence.
boolean	removeNoise() Implements the noise in a relatively general default way by invoking the low-pass FFT filter with the default settings.
boolean	removeSilence() Remove silence (amplitudes below certain threshold) from the sample thereby making it smaller and more unique compared to other samples.
static double[]	removeSilence(double[] padDataVector, double pdSilenceThreshold) Compresses input data by removing values below silence threshold.
void	setSample(Sample poSample) Allows setting a sample object reference.

Methods inherited from class marf.Storage.StorageManager

dump, dumpBinary, dumpCSV, dumpGzipBinary, dumpHTML, dumpSQL, dumpXML, enableDumpOnNotFound, equals, getDefaultExtension, getDefaultExtension, getDumpMode, getFilename, getObjectToSerialize, hashCode, restore, restoreBinary, restoreCSV, restoreGzipBinary, restoreHTML, restoreSQL, restoreXML, setDumpMode, setFilename, toString

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

DEFAULT_SILENCE_THRESHOLD

public static final double DEFAULT_SILENCE_THRESHOLD

Default amplitude value and below to what consider as a silence. Should be applied after normalization.

Since:

0.3.0.6

See Also:

Constant Field Values

oSample

protected Sample oSample

Sample reference.

dSilenceThreshold

protected double dSilenceThreshold

Current silence removal threshold.

Since:

0.3.0.6

See Also:

DEFAULT_SILENCE_THRESHOLD

bRemoveNoise

protected boolean bRemoveNoise

By default we do not remove noise.

bRemoveSilence

protected boolean bRemoveSilence

By default we do not remove silence.

serialVersionUID

private static final long serialVersionUID

For serialization versioning. When adding new members or make other structural changes regenerate this number with the serialver tool that comes with JDK.

Since:

0.3.0.5

See Also:

Constant Field Values

Constructor Detail

Preprocessing

protected Preprocessing()

Default constructor for reflective creation of Preprocessing clones. Typically should not be used unless really necessary for the frameworked modules.

Since:

0.3.0.5

Preprocessing

protected Preprocessing(Sample poSample)
                 throws PreprocessingException

Main Preprocessing constructor that performs normalization as a part of construction process.

Parameters:

poSample - loaded sample by a concrete implementation of SampleLoader

Throws:

PreprocessingException - if normalize() fails

See Also:

normalize()

Preprocessing

protected Preprocessing(IPreprocessing poPreprocessing)
                 throws PreprocessingException

Allows chaining of preprocessing modules. Makes it possible to apply several preprocessing modules on the same incoming sample, in the form of new FooBarBaz(new HighFrequencyBoost(new HighPassFilter(poSample))). Notice, it calls preprocess() for all inner preprocessing, but not the outer. The outer is supposed to be called by either MARF or an application as a part of defined API.

Parameters:

poPreprocessing - follow up preprocessing module

Throws:

PreprocessingException - if underlying preprocess() fails or the parameter is null.

Since:

0.3.0.3

See Also:

preprocess(), MARF, HighFrequencyBoost, HighPassFilter

Method Detail

preprocess

public boolean preprocess()
                   throws PreprocessingException

Default implementation of preprocess() includes normalization of the sample, and optionally removal of noise and silence.

Specified by:

preprocess in interface IPreprocessing

Returns:

true if there are any changes made

Throws:

PreprocessingException - in case of any error

Since:

0.3.0.6

See Also:

IPreprocessing.removeNoise(), IPreprocessing.removeSilence()

removeNoise

public boolean removeNoise()
                    throws PreprocessingException

Implements the noise in a relatively general default way by invoking the low-pass FFT filter with the default settings. Derivatives may override this default implementation the way they deem necessary. This default implementation of the noise removal appeared in 0.3.0.6, December 2007 and may be a subject for further parametrization.

Specified by:

removeNoise in interface IPreprocessing

Returns:

boolean that sample has changed (noise removed)

Throws:

PreprocessingException - declared but never thrown

See Also:

LowPassFilter

removeSilence

public boolean removeSilence()
                      throws PreprocessingException

Remove silence (amplitudes below certain threshold) from the sample thereby making it smaller and more unique compared to other samples.

Specified by:

removeSilence in interface IPreprocessing

Returns:

boolean that sample has changed (silence removed)

Throws:

PreprocessingException - in case of any error

See Also:

removeSilence(double[], double), dSilenceThreshold

normalize

public final boolean normalize()
                        throws PreprocessingException

Normalization of entire incoming samples by amplitude. Equivalent to normalize(0).

Specified by:

normalize in interface IPreprocessing

Returns:

true if the sample has been successfully normalized; false otherwise

Throws:

PreprocessingException - if internal sample reference is null

See Also:

normalize(int)

normalize

public final boolean normalize(int piIndexFrom)
                        throws PreprocessingException

Normalization of incoming samples by amplitude starting from certain index. Useful in case where only the last portion of a sample needs to be normalized, like in HighFrequencyBoost. Equivalent to normalize(piIndexFrom, sample array length - 1).

Specified by:

normalize in interface IPreprocessing

Parameters:

piIndexFrom - sample array index to start normalization from

Returns:

true if the sample has been successfully normalized; false otherwise

Throws:

PreprocessingException - if internal sample reference is null

Since:

0.3.0

See Also:

normalize(int, int), HighFrequencyBoost

normalize

public final boolean normalize(int piIndexFrom,
                               int piIndexTo)
                        throws PreprocessingException

Normalization of incoming samples by amplitude between specified indexes. Useful in case where only a portion of a sample needs to be normalized.

Specified by:

normalize in interface IPreprocessing

Parameters:

piIndexFrom - sample array index to start normalization from

piIndexTo - sample array index to end normalization at

Returns:

true if the sample has been successfully normalized; false otherwise

Throws:

PreprocessingException - if internal sample reference is null or one or both indexes are out of range

Since:

0.3.0

compress

public static double[] compress(double[] padDataVector)

Compresses input data by removing duplicate values.

Parameters:

padDataVector - source data vector to apply compression to

Returns:

a newly allocated compressed array of doubles; if the parameter is empty, it is returned instead

Since:

0.3.0.5

removeSilence

public static double[] removeSilence(double[] padDataVector,
                                     double pdSilenceThreshold)

Compresses input data by removing values below silence threshold.

Parameters:

padDataVector - source data vector to apply compression to

pdSilenceThreshold - the silence threshold below or equal to which amplitude values are ignored

Returns:

a newly allocated compressed array of doubles; may be empty if all input elements are below the threshold or input is empty

Since:

0.3.0.6

compress

public boolean compress()

Compresses the instance of the stored sample by removing duplicates.

Returns:

true, if the compression was successful.

Since:

0.3.0.5

See Also:

compress(double[])

cropAudio

public boolean cropAudio(double pdStartingFrequency,
                         double pdEndFrequency)
                  throws PreprocessingException

Derivatives implement this method to crop arbitrary part of the audio sample.

Specified by:

cropAudio in interface IPreprocessing

Parameters:

pdStartingFrequency - double Frequency to start to crop from

pdEndFrequency - double Frequency to crop the sample to

Returns:

boolean true - cropped, false - not

Throws:

NotImplementedException

PreprocessingException - declared, but is never thrown

getSample

public final Sample getSample()

Returns enclosed sample.

Specified by:

getSample in interface IPreprocessing

Returns:

Sample object

setSample

public void setSample(Sample poSample)

Allows setting a sample object reference.

Specified by:

setSample in interface IPreprocessing

Parameters:

poSample - new sample object

Since:

0.3.0.4

backSynchronizeObject

public void backSynchronizeObject()

Implementation of back-synchronization of Sample loaded object.

Overrides:

backSynchronizeObject in class StorageManager

Since:

0.3.0.2

See Also:

StorageManager.restore()

clone

public java.lang.Object clone()

Implements Cloneable interface for the Preprocessing object. Performs "deep" copy, including the contained sample.

Specified by:

clone in interface IPreprocessing

Overrides:

clone in class StorageManager

Returns:

a copy of the object implementing this interface

Since:

0.3.0.5

See Also:

Object.clone()

extractParameters

protected void extractParameters()

Performs general preprocessing module parameters extraction. E.g. parameters include the fact whether to remove noise and silence and the silence threshold. Typically invoked from constructors and is not meant for general public.

Since:

0.3.0.6

getMARFSourceCodeRevision

public static java.lang.String getMARFSourceCodeRevision()

Returns source code revision information.

Returns:

revision string

Since:

0.3.0.2