Package com.norconex.crawler.core

Class CrawlerDriver

java.lang.Object

com.norconex.crawler.core.CrawlerDriver

@NonNull
public class CrawlerDriver
extends Object

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	CrawlerDriver.Builder
static class	CrawlerDriver.FetchDriver

Method Summary

Modifier and Type	Method	Description
BeanMapper	beanMapper()
List<CrawlerBootstrapper>	bootstrappers()
static CrawlerDriver.Builder	builder()
Map<String,Class<?>>	cacheTypes()	Optional map of Hazelcast map-config-name (or wildcard pattern as declared in the YAML, e.g.
CrawlerCallbacks	callbacks()
Class<? extends CrawlerEntry>	crawlEntryType()	The exact type of CrawlerEntry if your crawler is subclassing it.
Class<? extends CrawlerConfig>	crawlerConfigClass()
CrawlerPipelineFactory	crawlPipelineFactory()
CrawlerDocPipelines	docPipelines()
EventManager	eventManager()
CrawlerDriver.FetchDriver	fetchDriver()	Provides a required fetcher implementation, responsible for obtaining resources being crawled.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

builder

public static CrawlerDriver.Builder builder()

crawlPipelineFactory

public CrawlerPipelineFactory crawlPipelineFactory()

crawlerConfigClass

public Class<? extends CrawlerConfig> crawlerConfigClass()

bootstrappers

public List<CrawlerBootstrapper> bootstrappers()

docPipelines

public CrawlerDocPipelines docPipelines()

callbacks

public CrawlerCallbacks callbacks()

beanMapper

public BeanMapper beanMapper()

eventManager

public EventManager eventManager()

fetchDriver

public CrawlerDriver.FetchDriver fetchDriver()

Provides a required fetcher implementation, responsible for obtaining resources being crawled.

crawlEntryType

public Class<? extends CrawlerEntry> crawlEntryType()

The exact type of CrawlerEntry if your crawler is subclassing it. Defaults to CrawlerEntry class.

cacheTypes

public Map<String,Class<?>> cacheTypes()

Optional map of Hazelcast map-config-name (or wildcard pattern as declared in the YAML, e.g. "ledger_*") to the concrete Java type that should be (de)serialized for values in that map. The framework automatically adds the ledger_* mapping using crawlEntryType. Extensions that persist their own types should add their entries here.