Class WhitespaceTokenStream

Namespace
NOpenNLP.Tools.Tokenize
Assembly
NOpenNLP.Tools.dll

This stream formats a TokenSamples into whitespace separated token strings.

public class WhitespaceTokenStream : FilterObjectStream<TokenSample?, string?>, IObjectStream<string?>, IDisposable
Inheritance
WhitespaceTokenStream
Implements
Inherited Members

Constructors

WhitespaceTokenStream(IObjectStream<TokenSample?>)

This stream formats a TokenSamples into whitespace separated token strings.

public WhitespaceTokenStream(IObjectStream<TokenSample?> tokens)

Parameters

tokens IObjectStream<TokenSample>

Methods

Read()

Returns the next object. Calling this method repeatedly until it returns null will return each object from the underlying source exactly once.

public override string? Read()

Returns

string

the next object or null to signal that the stream is exhausted

Exceptions

IOException

if there is an error during reading

IOException

if there is an error during reading