Package com.norconex.importer.handler.transformer.impl

Class TextStatisticsTransformer

java.lang.Object

com.norconex.importer.handler.transformer.impl.TextStatisticsTransformer

All Implemented Interfaces:

Configurable<TextStatisticsTransformerConfig>, ConfigurableDocHandler<TextStatisticsTransformerConfig>, DocHandler

public class TextStatisticsTransformer
extends Object
implements ConfigurableDocHandler<TextStatisticsTransformerConfig>

Analyzes the content of the supplied document and adds statistical information about its content or field as metadata fields. Default behavior provide the statistics about the content. Refer to the following for the new metadata fields to be created along with their description.

Statistic fields

Field name	Description
document.stat.characterCount	Total number of characters (excluding carriage returns/line feed).
document.stat.wordCount	Total number of words.
document.stat.sentenceCount	Total number of sentences.
document.stat.paragraphCount	Total number of paragraph.
document.stat.averageWordCharacterCount	Average number of character in every words.
document.stat.averageSentenceCharacterCount	Average number of character in sentences (including non-word characters, such as spaces, or slashes).
document.stat.averageSentenceWordCount	Average number of words per sentences.
document.stat.averageParagraphCharacterCount	Average number of characters in paragraphs (including non-word characters, such as spaces, or slashes).
document.stat.averageParagraphSentenceCount	Average number of sentences per paragraphs.
document.stat.averageParagraphWordCount	Average number of words per paragraphs.

You can specify a field matcher to obtain statistics about matching fields instead. When you do so, the field name will be inserted in the above names, right after "document.stat.". E.g.: document.stat.myfield.characterCount

Can be used both as a pre-parse (text-only) or post-parse handler.

See Also:

• TextStatisticsTransformer configuration reference

Field Summary

Fields inherited from interface com.norconex.commons.lang.config.Configurable

PROPERTY

Fields inherited from interface com.norconex.importer.handler.DocHandler

NAME

Constructor Summary

Constructors

Constructor	Description
TextStatisticsTransformer()

Method Summary

Modifier and Type	Method	Description
protected void	analyze(Reader input, Properties metadata, String field)
protected boolean	canEqual(Object other)
boolean	equals(Object o)
TextStatisticsTransformerConfig	getConfiguration()
boolean	handle(DocHandlerContext docCtx)	Perform any operations on a document.
int	hashCode()
String	toString()

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface com.norconex.importer.handler.DocHandler

close, init

Constructor Details

TextStatisticsTransformer

public TextStatisticsTransformer()

Method Details

handle

public boolean handle(DocHandlerContext docCtx)
               throws IOException

Description copied from interface: DocHandler

Perform any operations on a document.

Specified by:

handle in interface DocHandler

Parameters:

docCtx - the document handling context

Returns:

false to stop processing and reject a document or true to proceed to the next configured handler.

Throws:

IOException - An I/O exception of some sort has occurred

analyze

protected void analyze(Reader input,
 Properties metadata,
 String field)

getConfiguration

public TextStatisticsTransformerConfig getConfiguration()

Specified by:

getConfiguration in interface Configurable<TextStatisticsTransformerConfig>

equals

public boolean equals(Object o)

Overrides:

equals in class Object

canEqual

protected boolean canEqual(Object other)

hashCode

public int hashCode()

Overrides:

hashCode in class Object

toString

public String toString()

Overrides:

toString in class Object