Namespace NOpenNLP.Tools.Langdetect
Classes
- DefaultLanguageDetectorContextGenerator
A context generator for language detector.
- Language
Class for holding the document language and its confidence.
- LanguageDetectorCrossValidator
Cross validator for the language detector.
- LanguageDetectorEvaluator
The LanguageDetectorEvaluator measures the performance of the given ILanguageDetector with the provided reference LanguageSamples.
- LanguageDetectorEventStream
Iterator-like class for modeling language detector events.
- LanguageDetectorFactory
Default factory used by Language Detector. Extend this class to change the Language Detector behaviour, such as the ILanguageDetectorContextGenerator. The default DefaultLanguageDetectorContextGenerator will use char n-grams of size 1 to 3 and the following normalizers:
- LanguageDetectorME
Implements learnable Language Detector.
This will process the entire string when called with PredictLanguage(string) or PredictLanguages(string).
If you want this to stop early, use ProbingPredictLanguages(string) or ProbingPredictLanguages(string, LanguageDetectorConfig). When run in probing mode, this starts at the beginning of the charsequence and runs language detection on chunks of text. If the end of the string is reached or there are MinConsecImprovements consecutive predictions for the best language and the confidence increases over those last predictions and if the difference in confidence between the highest confidence language and the second highest confidence language is greater than MinDiff, the language detector will stop and report the results.
The authors wish to thank Ken Krugler and Yalder for the inspiration for many of the design components of this detector.
- LanguageDetectorModel
A model for language detection.
- LanguageDetectorSampleStream
This class reads in string encoded training samples, parses them and outputs LanguageSample objects.
Format:
Each line contains one sample document.
The language is the first string in the line followed by a tab and the document content.
Sample line: category-string tab-char document line-break-char(s)
- LanguageSample
Class which holds a classified document and its Language.
Interfaces
- ILanguageDetector
The interface for LanguageDetector which provide the Language according to the context.
- ILanguageDetectorContextGenerator
A context generator interface for language detector.
- ILanguageDetectorEvaluationMonitor
IEvaluationMonitor<T> for the language detector.