Namespace NOpenNLP.Tools.Formats
Experimental package related to converting various corpora to OpenNLP Format.
Namespaces
Classes
- AbstractSampleStreamFactory<T>
Base class for sample stream factories.
- BioNLP2004NameSampleStream
Parser for the training files of the BioNLP/NLPBA 2004 shared task.
The data contains five named entity types: DNA, RNA, protein, cell_type and cell_line.
Data can be found on this web site:
Note: Do not use this class, internal use only!
http://www-tsujii.is.s.u-tokyo.ac.jp/GENIA/ERtask/report.html
- ChunkerSampleStreamFactory
Factory producing OpenNLP ChunkSampleStreams.
- Conll02NameSampleStream
Parser for the dutch and spanish ner training files of the CONLL 2002 shared task.
The dutch data has a -DOCSTART- tag to mark article boundaries, adaptive data in the feature generators will be cleared before every article.
The spanish data does not contain article boundaries, adaptive data will be cleared for every sentence.The data contains four named entity types: Person, Organization, Location and Misc.
Data can be found on this web site:
Note: Do not use this class, internal use only!
http://www.cnts.ua.ac.be/conll2002/ner/
- Conll02NameSampleStreamFactory
Note: Do not use this class, internal use only!
- Conll03NameSampleStream
An import stream which can parse the CONLL03 data.
- ConllXPOSSampleStream
Parses the data from the CONLL 06 shared task into POS Samples.
More information about the data format can be found here:
Note: Do not use this class, internal use only!
http://www.cnts.ua.ac.be/conll2006/
- ConllXPOSSampleStreamFactory
Note: Do not use this class, internal use only!
- ConllXSentenceSampleStreamFactory
Note: Do not use this class, internal use only!
- ConllXTokenSampleStreamFactory
Note: Do not use this class, internal use only!
- DetokenizerSampleStreamFactory<T>
Base class for factories which need detokenizer.
- DirectorySampleStream
The directory sample stream allows for creating a stream from a directory listing of files.
- DocumentSampleStreamFactory
Factory producing OpenNLP DocumentSampleStreams.
- EvalitaNameSampleStream
Parser for the Italian NER training files of the Evalita 2007 and 2009 NER shared tasks.
The data does not contain article boundaries, adaptive data will be cleared for every sentence.
Named Entities are annotated in the IOB2 format (as used in CoNLL 2002 shared task)
The Named Entity tag consists of two parts:
- The IOB2 tag: 'B' (for 'begin') denotes the first token of a Named Entity, I (for 'inside') is used for all other tokens in a Named Entity, and 'O' (for 'outside') is used for all other words;
- The Entity type tag: PER (for Person), ORG (for Organization), GPE (for Geo-Political Entity), or LOC (for Location).
Each file consists of four columns separated by a blank, containing respectively the token, the Elsnet PoS-tag, the Adige news story to which the token belongs, and the Named Entity tag.
Data can be found on this web site:
Note: Do not use this class, internal use only!
http://www.evalita.it
- EvalitaNameSampleStreamFactory
Note: Do not use this class, internal use only!
- FormatParameter<T>
A command line parameter of a corpus format.
- FormatParameters
The command line parameters shared by the corpus formats.
- LanguageDetectorSampleStreamFactory
Factory producing OpenNLP LanguageDetectorSampleStreams.
- LanguageSampleStreamFactory<T>
Stream factory for those streams which carry language.
- LemmatizerSampleStreamFactory
Factory producing OpenNLP LemmaSampleStreams.
- NameFinderCensus90NameStream
This class helps to read the US Census data from the files to build a StringList for each dictionary entry in the name-finder dictionary. The entries in the source file are as follows:
SMITH 1.006 1.006 1- The first field is the name (in ALL CAPS).
- The next field is a frequency in percent.
- The next is a cumulative frequency in percent.
- The last is a ranking.
- NameSampleDataStreamFactory
Factory producing OpenNLP NameSampleDataStreams.
- ParseSampleStreamFactory
Factory producing OpenNLP ParseSampleStreams.
- SentenceSampleStreamFactory
Factory producing OpenNLP SentenceSampleStreams.
- StreamFactoryRegistry
Registry for object stream factories.
- TerminateToolException
Exception to terminate the execution of a command line tool.
The exception should be thrown to indicate that the VM should be terminated with the specified error code, instead of just calling
Environment.Exitsomewhere in the code.The return code convention is the following:
0in case of graceful termination-1in case of runtime errors, such as IO errors1in case of invalid parameters
- TokenSampleStreamFactory
Factory producing OpenNLP TokenSampleStreams.
- WordTagSampleStreamFactory
Note: Do not use this class, internal use only!
Interfaces
- IFormatParameter
Describes one command line parameter a corpus format accepts.
- IFormatParameterValues
Supplies the parsed value of each IFormatParameter a factory declared.
- IObjectStreamFactory<T>
Creates an IObjectStream<T> of samples read from a particular corpus format, and describes the command line parameters that format accepts.