Class NGramUtils

Namespace: NOpenNLP.Tools.Ngram

Assembly: NOpenNLP.Tools.dll

Utility class for ngrams. Some methods apply specifically to certain 'n' values, for e.g. tri/bi/uni-grams.

public static class NGramUtils

Inheritance: object

NGramUtils

Inherited Members: object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Methods

CalculateBigramMLProbability(string, string, ICollection<StringList>)

Calculate the probability of a bigram in a vocabulary using maximum likelihood estimation.

public static double CalculateBigramMLProbability(string x0, string x1, ICollection<StringList> set)

Parameters

x0 string: first word in the bigram
x1 string: second word in the bigram
set ICollection<StringList>: the vocabulary

Returns

double: the maximum likelihood probability

CalculateBigramPriorSmoothingProbability(string, string, ICollection<StringList>, double)

Calculate the probability of a bigram in a vocabulary using prior Laplace smoothing algorithm.

public static double CalculateBigramPriorSmoothingProbability(string x0, string x1, ICollection<StringList> set, double k)

Parameters

x0 string: the first word in the bigram
x1 string: the second word in the bigram
set ICollection<StringList>: the vocabulary
k double: the smoothing factor

Returns

double: the prior Laplace smoothing probability

CalculateLaplaceSmoothingProbability(StringList, IEnumerable<StringList>, double)

Calculate the probability of a ngram in a vocabulary using Laplace smoothing algorithm.

public static double CalculateLaplaceSmoothingProbability(StringList ngram, IEnumerable<StringList> set, double k)

Parameters

ngram StringList: the ngram to get the probability for
set IEnumerable<StringList>: the vocabulary
k double: the smoothing factor

Returns

double: the Laplace smoothing probability

See Also: Additive Smoothing

CalculateMissingNgramProbabilityMass(StringList, double, IEnumerable<StringList>)

Calculate the probability of a ngram in a vocabulary using the missing probability mass algorithm.

public static double CalculateMissingNgramProbabilityMass(StringList ngram, double discount, IEnumerable<StringList> set)

Parameters

ngram StringList: the ngram
discount double: discount factor
set IEnumerable<StringList>: the vocabulary

Returns

double: the probability

CalculateNgramMLProbability(StringList, IEnumerable<StringList>)

Calculate the probability of a ngram in a vocabulary using maximum likelihood estimation.

public static double CalculateNgramMLProbability(StringList ngram, IEnumerable<StringList> set)

Parameters

ngram StringList: a ngram
set IEnumerable<StringList>: the vocabulary

Returns

double: the maximum likelihood probability

CalculateTrigramLinearInterpolationProbability(string, string, string, ICollection<StringList>, double, double, double)

Calculate the probability of a trigram in a vocabulary using a linear interpolation algorithm.

public static double CalculateTrigramLinearInterpolationProbability(string x0, string x1, string x2, ICollection<StringList> set, double lambda1, double lambda2, double lambda3)

Parameters

x0 string: the first word in the trigram
x1 string: the second word in the trigram
x2 string: the third word in the trigram
set ICollection<StringList>: the vocabulary
lambda1 double: trigram interpolation factor
lambda2 double: bigram interpolation factor
lambda3 double: unigram interpolation factor

Returns

double: the linear interpolation probability

CalculateTrigramMLProbability(string, string, string, IEnumerable<StringList>)

Calculate the probability of a trigram in a vocabulary using maximum likelihood estimation.

public static double CalculateTrigramMLProbability(string x0, string x1, string x2, IEnumerable<StringList> set)

Parameters

x0 string: first word in the trigram
x1 string: second word in the trigram
x2 string: third word in the trigram
set IEnumerable<StringList>: the vocabulary

Returns

double: the maximum likelihood probability

CalculateUnigramMLProbability(string, ICollection<StringList>)

Calculate the probability of a unigram in a vocabulary using maximum likelihood estimation.

public static double CalculateUnigramMLProbability(string word, ICollection<StringList> set)

Parameters

word string: the only word in the unigram
set ICollection<StringList>: the vocabulary

Returns

double: the maximum likelihood probability

GetNGrams(StringList, int)

Get the ngrams of dimension n of a certain input sequence of tokens.

public static ICollection<StringList> GetNGrams(StringList sequence, int size)

Parameters

sequence StringList: a sequence of tokens
size int: the size of the resulting ngrams

Returns

ICollection<StringList>: all the possible ngrams of the given size derivable from the input sequence

GetNGrams(string[], int)

Get the ngrams of dimension n of a certain input sequence of tokens.

public static ICollection<string[]> GetNGrams(string[] sequence, int size)

Parameters

sequence string[]: a sequence of tokens
size int: the size of the resulting ngrams

Returns

ICollection<string[]>: all the possible ngrams of the given size derivable from the input sequence

GetNMinusOneTokenFirst(StringList)

Get the (n-1)th ngram of a given ngram, that is the same ngram except the last word in the ngram.

public static StringList? GetNMinusOneTokenFirst(StringList ngram)

Parameters

ngram StringList: a ngram

Returns

StringList: a ngram, or null if the given ngram holds a single token

GetNMinusOneTokenLast(StringList)

Get the (n-1)th ngram of a given ngram, that is the same ngram except the first word in the ngram.

public static StringList? GetNMinusOneTokenLast(StringList ngram)

Parameters

ngram StringList: a ngram

Returns

StringList: a ngram, or null if the given ngram holds a single token