Package com.norconex.crawler.core.cluster.pipeline

Interface Step

All Known Implementing Classes:

BaseStep, CrawlerProcessStep, RequeueOrphansForDeletionStep, RequeueOrphansForProcessingStep, StartRefsQueueStep

public interface Step

Nested Class Summary

Nested Classes

Modifier and Type	Interface	Description
static class	Step.StepProgress

Method Summary

Modifier and Type	Method	Description
void	execute(CrawlerSession session)
String	getId()
default Step.StepProgress	getProgress()	Optional method returning the current progress of this step.
boolean	isDistributed()
default boolean	isStopRequested()	Indicates whether this step has been asked to stop.
PipelineStatus	reduce(CrawlerSession session, org.apache.commons.collections4.Bag<PipelineStatus> statuses)
void	stop(CrawlerSession session)	Requests for the step to stop.

Method Details

getId

String getId()

isDistributed

boolean isDistributed()

execute

void execute(CrawlerSession session)

stop

void stop(CrawlerSession session)

Requests for the step to stop. The method may return before the step is actually stopped (up to implementation).

Parameters:

session - crawl session

reduce

PipelineStatus reduce(CrawlerSession session,
 org.apache.commons.collections4.Bag<PipelineStatus> statuses)

isStopRequested

default boolean isStopRequested()

Indicates whether this step has been asked to stop.

Default implementation returns false so non BaseStep implementations are not forced to expose internal state.

Returns:

true if a stop was requested

getProgress

default Step.StepProgress getProgress()

Optional method returning the current progress of this step. Returns null if this task does not track its progress.

Returns:

progress or null