Class LanguageDetectorME

Namespace

NOpenNLP.Tools.Langdetect

Assembly

NOpenNLP.Tools.dll

Implements learnable Language Detector.

This will process the entire string when called with PredictLanguage(string) or PredictLanguages(string).

If you want this to stop early, use ProbingPredictLanguages(string) or ProbingPredictLanguages(string, LanguageDetectorConfig). When run in probing mode, this starts at the beginning of the charsequence and runs language detection on chunks of text. If the end of the string is reached or there are MinConsecImprovements consecutive predictions for the best language and the confidence increases over those last predictions and if the difference in confidence between the highest confidence language and the second highest confidence language is greater than MinDiff, the language detector will stop and report the results.

The authors wish to thank Ken Krugler and Yalder for the inspiration for many of the design components of this detector.

public class LanguageDetectorME : ILanguageDetector

Inheritance

object

LanguageDetectorME

Implements

ILanguageDetector

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Constructors

LanguageDetectorME(LanguageDetectorModel)

Initializes the current instance with a language detector model. Default feature generation is used.

public LanguageDetectorME(LanguageDetectorModel model)

Parameters

model LanguageDetectorModel

the language detector model

Fields

model

protected readonly LanguageDetectorModel model

Field Value

LanguageDetectorModel

Properties

SupportedLanguages

public virtual string[] SupportedLanguages { get; }

Property Value

string[]

Methods

PredictLanguage(string)

This will process the full content length.

public virtual Language PredictLanguage(string content)

Parameters

content string

content to be processed

Returns

Language

the language with the highest confidence

PredictLanguages(string)

This will process the full content length.

public virtual Language[] PredictLanguages(string content)

Parameters

content string

content to be processed

Returns

Language[]

the predicted languages

ProbingPredictLanguages(string)

This will stop processing early if the stopping criteria specified in DEFAULT_LANGUAGE_DETECTOR_CONFIG are met.

public virtual ProbingLanguageDetectionResult ProbingPredictLanguages(string content)

Parameters

content string

content to be processed

Returns

ProbingLanguageDetectionResult

result

ProbingPredictLanguages(string, LanguageDetectorConfig)

This will stop processing early if the stopping criteria specified in config are met.

public virtual ProbingLanguageDetectionResult ProbingPredictLanguages(string content, LanguageDetectorConfig config)

Parameters

content string

content to process

config LanguageDetectorConfig

config to customize detection

Returns

ProbingLanguageDetectionResult

result

Train(IObjectStream<LanguageSample?>, TrainingParameters, LanguageDetectorFactory)

public static LanguageDetectorModel Train(IObjectStream<LanguageSample?> samples, TrainingParameters mlParams, LanguageDetectorFactory factory)

Parameters

samples IObjectStream<LanguageSample>

mlParams TrainingParameters

factory LanguageDetectorFactory

Returns

LanguageDetectorModel