Package com.norconex.importer.handler.transformer.impl

Class UrlExtractorTransformer

java.lang.Object

com.norconex.importer.handler.transformer.impl.UrlExtractorTransformer

All Implemented Interfaces:

Configurable<UrlExtractorTransformerConfig>, ConfigurableDocHandler<UrlExtractorTransformerConfig>, DocHandler

public class UrlExtractorTransformer
extends Object
implements ConfigurableDocHandler<UrlExtractorTransformerConfig>

Extracts unique URLs matching specific patterns in plain text content and store them in a given field.

URL-matching patterns used are relatively simple. It looks for strings starting with http://, https://, or www.. The later is prefixed with https:// when encountered (to make it absolute).

The matching is case-insensitive. If you need alternate ways to detect URLs, you can use a combination of RegexTransformer, ReplaceTransformer, or create your own implementation.

Storing values in an existing field

If a target field with the same name already exists for a document, values will be added to the end of the existing value list. It is possible to change this default behavior by supplying a PropertySetter.

If no URLs are found, the target field values (if any) are left intact.

Content source

It is possible to specify a fromField as the source of the text to use instead of using the document content.

This class is typically e used as a post-parsing handler only (to ensure we are dealing with text).

See Also:

• UrlExtractorTransformer configuration reference

Field Summary

Fields inherited from interface com.norconex.commons.lang.config.Configurable

PROPERTY

Fields inherited from interface com.norconex.importer.handler.DocHandler

NAME

Constructor Summary

Constructors

Constructor	Description
UrlExtractorTransformer()

Method Summary

Modifier and Type	Method	Description
protected boolean	canEqual(Object other)
boolean	equals(Object o)
UrlExtractorTransformerConfig	getConfiguration()
boolean	handle(DocHandlerContext docCtx)	Perform any operations on a document.
int	hashCode()
String	toString()

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface com.norconex.importer.handler.DocHandler

close, init

Constructor Details

UrlExtractorTransformer

public UrlExtractorTransformer()

Method Details

handle

public boolean handle(DocHandlerContext docCtx)
               throws IOException

Description copied from interface: DocHandler

Perform any operations on a document.

Specified by:

handle in interface DocHandler

Parameters:

docCtx - the document handling context

Returns:

false to stop processing and reject a document or true to proceed to the next configured handler.

Throws:

IOException - An I/O exception of some sort has occurred

getConfiguration

public UrlExtractorTransformerConfig getConfiguration()

Specified by:

getConfiguration in interface Configurable<UrlExtractorTransformerConfig>

equals

public boolean equals(Object o)

Overrides:

equals in class Object

canEqual

protected boolean canEqual(Object other)

hashCode

public int hashCode()

Overrides:

hashCode in class Object

toString

public String toString()

Overrides:

toString in class Object