Package com.norconex.crawler.core.fetch

Class MultiFetcher

java.lang.Object

com.norconex.crawler.core.fetch.MultiFetcher

All Implemented Interfaces:

Fetcher

public class MultiFetcher
extends Object
implements Fetcher

Implementation of Fetcher that groups multiple other fetcher to treat them as a single fetcher. It will try them all until a supporting fetcher succeeds in fetching a resource.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	MultiFetcher.Builder
static interface	MultiFetcher.ResponseAggregator
static interface	MultiFetcher.UnsuccessfulResponseFactory

Method Summary

Modifier and Type	Method	Description
boolean	accept(FetchRequest fetchRequest)
static MultiFetcher.Builder	builder()
FetchResponse	fetch(FetchRequest fetchRequest)	Fetches a document.
List<Fetcher>	getFetchers()
int	getMaxRetries()
Duration	getRetryDelay()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

getFetchers

public List<Fetcher> getFetchers()

accept

public boolean accept(FetchRequest fetchRequest)

Specified by:

accept in interface Fetcher

fetch

public FetchResponse fetch(FetchRequest fetchRequest)

Fetches a document.

Specified by:

fetch in interface Fetcher

Parameters:

fetchRequest - fetch request

Returns:

fetch response

builder

public static MultiFetcher.Builder builder()

getMaxRetries

public int getMaxRetries()

getRetryDelay

public Duration getRetryDelay()