Package com.norconex.crawler.web.doc.operations.recrawl.impl

Class GenericRecrawlableResolver

java.lang.Object

com.norconex.crawler.web.doc.operations.recrawl.impl.GenericRecrawlableResolver

All Implemented Interfaces:

Configurable<GenericRecrawlableResolverConfig>, RecrawlableResolver

public class GenericRecrawlableResolver
extends Object
implements RecrawlableResolver, Configurable<GenericRecrawlableResolverConfig>

Relies on both sitemap directives and custom instructions for establishing the minimum frequency between each document recrawl.

Sitemap support:

Provided crawler support for sitemaps has not been disabled, this class tries to honor last modified and frequency directives found in sitemap files.

By default, existing sitemap directives take precedence over custom ones. You chose to have sitemap directives be considered last or even disable sitemap directives using the GenericRecrawlableResolverConfig.setSitemapSupport(SitemapSupport) method.

Custom recrawl frequencies:

You can chose to have some of your crawled documents be re-crawled less frequently than others by specifying custom minimum frequencies with (GenericRecrawlableResolverConfig.setMinFrequencies(Collection)). Minimum frequencies are processed in the order specified.

Since:

2.5.0

Field Summary

Fields inherited from interface com.norconex.commons.lang.config.Configurable

PROPERTY

Constructor Summary

Constructors

Constructor	Description
GenericRecrawlableResolver()

Method Summary

Modifier and Type	Method	Description
protected boolean	canEqual(Object other)
boolean	equals(Object o)
GenericRecrawlableResolverConfig	getConfiguration()
int	hashCode()
boolean	isRecrawlable(WebCrawlerEntry prevData)	Whether a document recrawlable or not.
String	toString()

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Details

GenericRecrawlableResolver

public GenericRecrawlableResolver()

Method Details

isRecrawlable

public boolean isRecrawlable(WebCrawlerEntry prevData)

Description copied from interface: RecrawlableResolver

Whether a document recrawlable or not.

Specified by:

isRecrawlable in interface RecrawlableResolver

Parameters:

prevData - data about previously crawled document (if any)

Returns:

true if recrawlable

equals

public boolean equals(Object o)

Overrides:

equals in class Object

canEqual

protected boolean canEqual(Object other)

hashCode

public int hashCode()

Overrides:

hashCode in class Object

toString

public String toString()

Overrides:

toString in class Object

getConfiguration

public GenericRecrawlableResolverConfig getConfiguration()

Specified by:

getConfiguration in interface Configurable<GenericRecrawlableResolverConfig>