Namespace NOpenNLP.Tools.Tokenize.Lang.En

Classes

TokenSampleStream

Class which produces an IEnumerable<T> of TokenSample from a file of space delimited token. This class uses a number of English-specific heuristics to un-separate tokens which are typically found together in text.