Package com.norconex.crawler.core.context

Class CrawlerContext

java.lang.Object

com.norconex.crawler.core.context.CrawlerContext

All Implemented Interfaces:

Closeable, AutoCloseable

@NonNull
public class CrawlerContext
extends Object
implements Closeable

Crawler class holding state properties required for running commands and tasks.

JMX Support

JMX support is disabled by default. To enable it, set the system property "enableJMX" to true. You can do so by adding this to your Java launch command:

     -DenableJMX=true
 

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	CrawlerContext.Builder

Field Summary

Fields

Modifier and Type	Field	Description
static final String	NAME

Method Summary

Modifier and Type	Method	Description
static CrawlerContext.Builder	builder()
protected boolean	canEqual(Object other)
void	close()
CrawlerEntry	createCrawlEntry(@NonNull String reference)
CrawlerEntry	createDocContext(@NonNull CrawlerEntry parentEntry)
boolean	equals(Object o)
BeanMapper	getBeanMapper()
List<CrawlerBootstrapper>	getBootstrappers()
Map<String,Class<?>>	getCacheTypes()	Map of map-config name/pattern to concrete value type, used to bake the correct (de)serialization class into the config before startup.
CrawlerCallbacks	getCallbacks()
CommitterService<Doc>	getCommitterService()
CrawlerConfig	getCrawlConfig()
CrawlerEntryLedger	getCrawlEntryLedger()
Class<? extends CrawlerEntry>	getCrawlEntryType()
DedupService	getDedupService()
CrawlerDocPipelines	getDocPipelines()
EventManager	getEventManager()
Fetcher	getFetcher()
String	getId()	Gets the crawler id.
Importer	getImporter()
CrawlerMetrics	getMetrics()
CachedStreamFactory	getStreamFactory()
Path	getTempDir()
ScopedThreadFactoryCreator	getThreadFactoryCreator()
Path	getWorkDir()
int	hashCode()
void	init(CrawlerSession session)
CrawlerContext.Builder	toBuilder()
String	toString()

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Details

NAME

public static final String NAME

See Also:

• Constant Field Values

Method Details

getId

public String getId()

Gets the crawler id.

Returns:

crawler id

toString

public String toString()

Overrides:

toString in class Object

createCrawlEntry

public CrawlerEntry createCrawlEntry(@NonNull
 @NonNull String reference)

createDocContext

public CrawlerEntry createDocContext(@NonNull
 @NonNull CrawlerEntry parentEntry)

init

public void init(CrawlerSession session)

close

public void close()

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

builder

public static CrawlerContext.Builder builder()

toBuilder

public CrawlerContext.Builder toBuilder()

equals

public boolean equals(Object o)

Overrides:

equals in class Object

canEqual

protected boolean canEqual(Object other)

hashCode

public int hashCode()

Overrides:

hashCode in class Object

getDedupService

public DedupService getDedupService()

getCrawlEntryLedger

public CrawlerEntryLedger getCrawlEntryLedger()

getMetrics

public CrawlerMetrics getMetrics()

getCrawlConfig

public CrawlerConfig getCrawlConfig()

getBeanMapper

public BeanMapper getBeanMapper()

getEventManager

public EventManager getEventManager()

getCallbacks

public CrawlerCallbacks getCallbacks()

getCommitterService

public CommitterService<Doc> getCommitterService()

getImporter

public Importer getImporter()

getBootstrappers

public List<CrawlerBootstrapper> getBootstrappers()

getDocPipelines

public CrawlerDocPipelines getDocPipelines()

getFetcher

public Fetcher getFetcher()

getWorkDir

public Path getWorkDir()

getTempDir

public Path getTempDir()

getStreamFactory

public CachedStreamFactory getStreamFactory()

getCrawlEntryType

public Class<? extends CrawlerEntry> getCrawlEntryType()

getCacheTypes

public Map<String,Class<?>> getCacheTypes()

Map of map-config name/pattern to concrete value type, used to bake the correct (de)serialization class into the config before startup. Populated from CrawlerDriver.cacheTypes(); the framework always adds ledger_* -> crawlEntryType automatically.

getThreadFactoryCreator

public ScopedThreadFactoryCreator getThreadFactoryCreator()