Class EmptyLinePreprocessorStream

Namespace

NOpenNLP.Tools.Sentdetect

Assembly

NOpenNLP.Tools.dll

Stream to to clean up empty lines for empty line separated document streams.

• Skips empty line at training data start
  
• Transforms multiple empty lines in a row into one
  
• Replaces white space lines with empty lines
  
• TODO: Terminates last document with empty line if it is missing
  
  This stream should be used by the components that mark empty lines to mark document boundaries.

Note: This class is not thread safe.
Do not use this class, internal use only!

public class EmptyLinePreprocessorStream : FilterObjectStream<string?, string?>, IObjectStream<string?>, IDisposable

Inheritance

object

ObjectStreamBase<string>

FilterObjectStream<string, string>

EmptyLinePreprocessorStream

Implements

IObjectStream<string>

IDisposable

Inherited Members

FilterObjectStream<string, string>.samples

FilterObjectStream<string, string>.Reset()

FilterObjectStream<string, string>.Dispose(bool)

ObjectStreamBase<string>.Reset()

ObjectStreamBase<string>.Dispose(bool)

ObjectStreamBase<string>.Dispose()

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Constructors

EmptyLinePreprocessorStream(IObjectStream<string?>)

Stream to to clean up empty lines for empty line separated document streams.

• Skips empty line at training data start
  
• Transforms multiple empty lines in a row into one
  
• Replaces white space lines with empty lines
  
• TODO: Terminates last document with empty line if it is missing
  
  This stream should be used by the components that mark empty lines to mark document boundaries.

Note: This class is not thread safe.
Do not use this class, internal use only!

public EmptyLinePreprocessorStream(IObjectStream<string?> @in)

Parameters

in IObjectStream<string>

Methods

Read()

Returns the next object. Calling this method repeatedly until it returns null will return each object from the underlying source exactly once.

public override string? Read()

Returns

string

the next object or null to signal that the stream is exhausted

Exceptions

IOException

if there is an error during reading