Class TokenSampleStream

Namespace

NOpenNLP.Tools.Tokenize.Lang.En

Assembly

NOpenNLP.Tools.dll

Class which produces an IEnumerable<T> of TokenSample from a file of space delimited token. This class uses a number of English-specific heuristics to un-separate tokens which are typically found together in text.

public class TokenSampleStream : IEnumerable<TokenSample>, IEnumerable

Inheritance

object

TokenSampleStream

Implements

IEnumerable<TokenSample>

IEnumerable

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

NOpenNLP: upstream implements Iterator<TokenSample>. Per the port's conventions this exposes IEnumerable<T> instead, so callers can foreach over it without driving an enumerator by hand. The reader is consumed once, exactly as upstream's iterator is, so a second enumeration yields nothing.

Constructors

TokenSampleStream(Stream)

public TokenSampleStream(Stream @is)

Parameters

is Stream

Exceptions

IOException

if reading from the stream fails

Properties

HasNext

public bool HasNext { get; }

Property Value

bool

Methods

GetEnumerator()

Returns an enumerator that iterates through the collection.

public IEnumerator<TokenSample> GetEnumerator()

Returns

IEnumerator<TokenSample>

An enumerator that can be used to iterate through the collection.

Next()

public TokenSample Next()

Returns

TokenSample