Package com.norconex.importer.handler.transformer.impl

Class UrlExtractorTransformerConfig

java.lang.Object

com.norconex.importer.handler.BaseLabelableConfig

com.norconex.importer.handler.BaseDocHandlerConfig

com.norconex.importer.handler.transformer.impl.UrlExtractorTransformerConfig

All Implemented Interfaces:

Labelable, ChunkedTextSupport

public class UrlExtractorTransformerConfig
extends BaseDocHandlerConfig
implements ChunkedTextSupport

Extracts unique URLs matching specific patterns in plain text content and store them in a given field.

URL-matching patterns used are relatively simple. It looks for strings starting with http://, https://, or www.. The later is prefixed with https:// when encountered (to make it absolute).

The matching is case-insensitive. If you need alternate ways to detect URLs, you can use a combination of RegexTransformer, ReplaceTransformer, or create your own implementation.

Storing values in an existing field

If a target field with the same name already exists for a document, values will be added to the end of the existing value list. It is possible to change this default behavior by supplying a PropertySetter.

If no URLs are found, the target field values (if any) are left intact.

Content source

It is possible to specify a fromField as the source of the text to use instead of using the document content.

This class is typically e used as a post-parsing handler only (to ensure we are dealing with text).

See Also:

• UrlExtractorTransformer configuration reference

Constructor Summary

Constructors

Constructor	Description
UrlExtractorTransformerConfig()

Method Summary

Modifier and Type	Method	Description
protected boolean	canEqual(Object other)
boolean	equals(Object o)
TextMatcher	getFieldMatcher()	Gets field matcher for fields containing text.
int	getMaxReadSize()	The maximum number of characters to read from content for tagging at once.
PropertySetter	getOnSet()	The property setter to use when a value is set.
Charset	getSourceCharset()
String	getToField()
int	hashCode()
UrlExtractorTransformerConfig	setFieldMatcher(TextMatcher fieldMatcher)	Sets the field matcher for fields containing text.
UrlExtractorTransformerConfig	setMaxReadSize(int maxReadSize)	The maximum number of characters to read from content for tagging at once.
UrlExtractorTransformerConfig	setOnSet(PropertySetter onSet)	The property setter to use when a value is set.
UrlExtractorTransformerConfig	setSourceCharset(Charset sourceCharset)
UrlExtractorTransformerConfig	setToField(String toField)
String	toString()

Methods inherited from class com.norconex.importer.handler.BaseLabelableConfig

getLabel, setLabel

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Details

UrlExtractorTransformerConfig

public UrlExtractorTransformerConfig()

Method Details

getFieldMatcher

public TextMatcher getFieldMatcher()

Gets field matcher for fields containing text.

Specified by:

getFieldMatcher in interface ChunkedTextSupport

Returns:

field matcher

setFieldMatcher

public UrlExtractorTransformerConfig setFieldMatcher(TextMatcher fieldMatcher)

Sets the field matcher for fields containing text.

Parameters:

fieldMatcher - field matcher

getToField

public String getToField()

getOnSet

public PropertySetter getOnSet()

The property setter to use when a value is set.

Returns:

property setter

getMaxReadSize

public int getMaxReadSize()

The maximum number of characters to read from content for tagging at once. Default is TextReader.DEFAULT_MAX_READ_SIZE.

Specified by:

getMaxReadSize in interface ChunkedTextSupport

Returns:

maximum read size

getSourceCharset

public Charset getSourceCharset()

Specified by:

getSourceCharset in interface ChunkedTextSupport

setToField

public UrlExtractorTransformerConfig setToField(String toField)

Returns:

this.

setOnSet

public UrlExtractorTransformerConfig setOnSet(PropertySetter onSet)

The property setter to use when a value is set.

Parameters:

onSet - property setter

Returns:

this.

setMaxReadSize

public UrlExtractorTransformerConfig setMaxReadSize(int maxReadSize)

The maximum number of characters to read from content for tagging at once. Default is TextReader.DEFAULT_MAX_READ_SIZE.

Parameters:

maxReadSize - maximum read size

Returns:

this.

setSourceCharset

public UrlExtractorTransformerConfig setSourceCharset(Charset sourceCharset)

Returns:

this.

equals

public boolean equals(Object o)

Overrides:

equals in class BaseLabelableConfig

canEqual

protected boolean canEqual(Object other)

Overrides:

canEqual in class BaseLabelableConfig

hashCode

public int hashCode()

Overrides:

hashCode in class BaseLabelableConfig

toString

public String toString()

Overrides:

toString in class BaseLabelableConfig