Class TwoPassDataIndexer

Namespace

NOpenNLP.Tools.Ml.Model

Assembly

NOpenNLP.Tools.dll

Collecting event and context counts by making two passes over the events. The first pass determines which contexts will be used by the model, and the second pass creates the events in memory containing only the contexts which will be used. This greatly reduces the amount of memory required for storing the events. During the first pass a temporary event file is created which is read during the second pass.

public class TwoPassDataIndexer : AbstractDataIndexer, IDataIndexer

Inheritance

object

AbstractDataIndexer

TwoPassDataIndexer

Implements

IDataIndexer

Inherited Members

AbstractDataIndexer.CUTOFF_PARAM

AbstractDataIndexer.CUTOFF_DEFAULT

AbstractDataIndexer.SORT_PARAM

AbstractDataIndexer.SORT_DEFAULT

AbstractDataIndexer.trainingParameters

AbstractDataIndexer.reportMap

AbstractDataIndexer.printMessages

AbstractDataIndexer.Init(TrainingParameters, IDictionary<string, string>)

AbstractDataIndexer.contexts

AbstractDataIndexer.outcomeList

AbstractDataIndexer.numTimesEventsSeen

AbstractDataIndexer.predLabels

AbstractDataIndexer.outcomeLabels

AbstractDataIndexer.predCounts

AbstractDataIndexer.Contexts

AbstractDataIndexer.NumTimesEventsSeen

AbstractDataIndexer.OutcomeList

AbstractDataIndexer.PredLabels

AbstractDataIndexer.OutcomeLabels

AbstractDataIndexer.PredCounts

AbstractDataIndexer.SortAndMerge(IList<ComparableEvent>, bool)

AbstractDataIndexer.Index(IObjectStream<Event>, IDictionary<string, int>)

AbstractDataIndexer.NumEvents

AbstractDataIndexer.Update(string[], IDictionary<string, int>)

AbstractDataIndexer.ToIndexedStringArray(IDictionary<string, int>)

AbstractDataIndexer.Values

AbstractDataIndexer.Display(string)

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Methods

Index(IObjectStream<Event?>)

Performs the data indexing. Make sure the Init(TrainingParameters, IDictionary<string, string>?) method is called first.

public override void Index(IObjectStream<Event?> eventStream)

Parameters

eventStream IObjectStream<Event>

a stream of events

Exceptions

IOException

if there is an error during reading