Namespace NOpenNLP.Tools.Util.Featuregen
This package contains classes for generating sequence features.
Classes
- AdditionalContextFeatureGenerator
The AdditionalContextFeatureGenerator generates the context from the passed in additional context.
- AggregatedFeatureGenerator
The AggregatedFeatureGenerator aggregates a set of IAdaptiveFeatureGenerators and calls them to generate the features.
- BrownBigramFeatureGenerator
Generates Brown cluster features for token bigrams.
- BrownCluster
Class to load a Brown cluster document: word\tword_class\tprob http://metaoptimize.com/projects/wordreprs/
The file containing the clustering lexicon has to be passed as the value of the dict attribute of each BrownCluster feature generator.
- BrownClusterBigramFeatureGeneratorFactory
Generates Brown clustering features for token bigrams.
- BrownClusterTokenClassFeatureGeneratorFactory
Generates Brown clustering features for token classes.
- BrownClusterTokenFeatureGeneratorFactory
Generates Brown clustering features for current token.
- BrownTokenClassFeatureGenerator
Generates Brown cluster features for current token and token class.
- BrownTokenClasses
Obtain the paths listed in the pathLengths array from the Brown class. This class is not to be instantiated.
- BrownTokenFeatureGenerator
Generates Brown cluster features for current token.
- CachedFeatureGenerator
Caches features of the aggregated IAdaptiveFeatureGenerators.
- CharacterNgramFeatureGenerator
The CharacterNgramFeatureGenerator uses character ngrams to generate features about each token. The minimum and maximum length can be specified.
- DictionaryFeatureGenerator
The DictionaryFeatureGenerator uses the DictionaryNameFinder to generated features for detected names based on the InSpanGenerator.
- FeatureGeneratorUtil
This class provide common utilities for feature generation.
- GeneratorFactory
Creates a set of feature generators based on a provided XML descriptor.
Example of an XML descriptor:
<featureGenerators name="namefind"> <generator class="opennlp.tools.util.featuregen.CachedFeatureGeneratorFactory"> <generator class="opennlp.tools.util.featuregen.WindowFeatureGeneratorFactory"> <int name="prevLength">2</int> <int name="nextLength">2</int> <generator class="opennlp.tools.util.featuregen.TokenClassFeatureGeneratorFactory"/> </generator> <generator class="opennlp.tools.util.featuregen.WindowFeatureGeneratorFactory"> <int name="prevLength">2</int> <int name="nextLength">2</int> <generator class="opennlp.tools.util.featuregen.TokenFeatureGeneratorFactory"/> </generator> <generator class="opennlp.tools.util.featuregen.DefinitionFeatureGeneratorFactory"/> <generator class="opennlp.tools.util.featuregen.PreviousMapFeatureGeneratorFactory"/> <generator class="opennlp.tools.util.featuregen.BigramNameFeatureGeneratorFactory"/> <generator class="opennlp.tools.util.featuregen.SentenceFeatureGeneratorFactory"> <bool name="begin">true</bool> <bool name="end">false</bool> </generator> </generator> </featureGenerators>Each XML element is mapped to a GeneratorFactory.IXmlFeatureGeneratorFactory which is responsible to process the element and create the specified IAdaptiveFeatureGenerator. Elements can contain other elements in this case it is the responsibility of the mapped factory to process the child elements correctly. In some factories this leads to recursive calls the
GeneratorFactory.IXmlFeatureGeneratorFactory.Createmethod.In the example above the generators element is mapped to the AggregatedFeatureGeneratorFactory which then creates all the aggregated IAdaptiveFeatureGenerators to accomplish this it evaluates the mapping with the same mechanism and gives the child element to the corresponding factories. All created generators are added to a new instance of the AggregatedFeatureGenerator which is then returned.
- InSpanGenerator
Generates features if the tokens are recognized by the provided ITokenNameFinder.
- OutcomePriorFeatureGenerator
The definition feature maps the underlying distribution of outcomes.
- POSTaggerNameFeatureGenerator
Adds the token POS Tag as feature. Requires a POS Tag model.
- PreviousMapFeatureGenerator
This IAdaptiveFeatureGenerator generates features indicating the outcome associated with a previously occuring word.
- PreviousTwoMapFeatureGenerator
This IAdaptiveFeatureGenerator generates features indicating the outcome associated with two previously occuring words.
- SentenceFeatureGenerator
This feature generator creates sentence begin and end features.
- StringPattern
Recognizes predefined patterns in strings.
- TokenClassFeatureGenerator
Generates features for different for the class of the token.
- TokenFeatureGenerator
Generates a feature which contains the token itself.
- TokenPatternFeatureGenerator
Partitions tokens into sub-tokens based on character classes and generates class features for each of the sub-tokens and combinations of those sub-tokens.
- TrigramNameFeatureGenerator
Adds trigram features based on tokens and token classes.
- WindowFeatureGenerator
Generates previous and next features for a given IAdaptiveFeatureGenerator. The window size can be specified.
Features: Current token is always included unchanged Previous tokens are prefixed with p distance Next tokens are prefix with n distance
- WordClusterFeatureGeneratorFactory
Defines a word cluster generator factory; it reads an element containing 'w2vwordcluster' as a tag name; these clusters are typically produced by word2vec or clark pos induction systems.
Interfaces
- IAdaptiveFeatureGenerator
An interface for generating features for name entity identification and for updating document level contexts.
Note:
Feature generation is not thread safe and a instance of a feature generator must only be called from one thread. The resources used by a feature generator are typically shared between man instances of features generators which are called from many threads and have to be thread safe.
Delegates
- FeatureGeneratorResourceProvider
The FeatureGeneratorResourceProvider provides access to the resources provided in the model. Inside the model resources are identified by a name.
Note:
This class is not be intended to be implemented by users.
All implementing classes must be thread safe.