Class TokenPatternFeatureGenerator

Namespace

NOpenNLP.Tools.Util.Featuregen

Assembly

NOpenNLP.Tools.dll

Partitions tokens into sub-tokens based on character classes and generates class features for each of the sub-tokens and combinations of those sub-tokens.

public class TokenPatternFeatureGenerator : IAdaptiveFeatureGenerator

Inheritance

object

TokenPatternFeatureGenerator

Implements

IAdaptiveFeatureGenerator

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Constructors

TokenPatternFeatureGenerator()

Initializes a new instance. For tokinization the SimpleTokenizer is used.

public TokenPatternFeatureGenerator()

TokenPatternFeatureGenerator(ITokenizer)

Initializes a new instance.

public TokenPatternFeatureGenerator(ITokenizer supportTokenizer)

Parameters

supportTokenizer ITokenizer

Methods

ClearAdaptiveData()

Informs the feature generator that the context of the adaptive data (typically a document) is no longer valid.

public virtual void ClearAdaptiveData()

CreateFeatures(IList<string>, string[], int, string[])

Adds the appropriate features for the token at the specified index with the specified array of previous outcomes to the specified list of features.

public virtual void CreateFeatures(IList<string> feats, string[] toks, int index, string[] preds)

Parameters

feats IList<string>

toks string[]

index int

The index of the token which is currently being processed.

preds string[]

UpdateAdaptiveData(string[], string[])

Informs the feature generator that the specified tokens have been classified with the corresponding set of specified outcomes.

public virtual void UpdateAdaptiveData(string[] tokens, string[] outcomes)

Parameters

tokens string[]

The tokens of the sentence or other text unit which has been processed.

outcomes string[]

The outcomes associated with the specified tokens.