Package com.norconex.crawler.core.session

Class CrawlerSession

java.lang.Object
com.norconex.crawler.core.session.CrawlerSession
All Implemented Interfaces:
Closeable, AutoCloseable
public class CrawlerSession extends Object implements Closeable
A crawl session, representing a crawl from begin to finish, which may include zero to multiple "resumes" from stops, or failures. There is one crawl session per cluster, each node having their own instance of this class, exposing the same information.

  • Method Details

    • getCrawlerId

      public String getCrawlerId()
      The crawler unique identifier, for a specific crawler setup and does not change across sessions. Same as configured in the crawler configuration.
      Returns:
      crawler id

    • oncePerSession

      public void oncePerSession(String taskId, Runnable runnable)

    • oncePerSessionAndGet

      public <T> T oncePerSessionAndGet(String taskId, Supplier<T> supplier)

    • oncePerRun

      public void oncePerRun(String taskId, Runnable runnable)

    • oncePerRunAndGet

      public <T> T oncePerRunAndGet(String taskId, Supplier<T> supplier)

    • getCrawlSessionId

      public String getCrawlSessionId()
      A crawl session unique identifier, which represents a begin-to-end crawl, regardless how many times it may have been resumed due to stops or failures. Resuming crawls will keep the same session id.
      Returns:
      crawl session id

    • getCrawlRunId

      public String getCrawlRunId()
      A crawl run unique identifier, which is renewed every time the crawler is launched, regardless whether we are resuming an existing session or creating a new one.
      Returns:
      crawl run id

    • isIncremental

      public boolean isIncremental()

    • isResumed

      public boolean isResumed()

    • getCrawlState

      public CrawlerState getCrawlState()

    • isClosed

      public boolean isClosed()

    • getSessionAttributes

      public CrawlerAttributes getSessionAttributes()

    • getCrawlRunAttributes

      public CrawlerAttributes getCrawlRunAttributes()

    • getCrawlerAttributes

      public CrawlerAttributes getCrawlerAttributes()

    • close

      public void close()
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable

    • setPostCloseCleanup

      public void setPostCloseCleanup(Runnable cleanup)
      Registers a cleanup action to run AFTER the session has closed all its resources. This is useful for components like loggers that need to capture all session cleanup events before shutting down.
      Parameters:
      cleanup - the cleanup action to run after session close

    • getAdminPort

      public int getAdminPort()

    • isStartRefsQueueingComplete

      public boolean isStartRefsQueueingComplete()

    • setStartRefsQueueingComplete

      public void setStartRefsQueueingComplete(boolean isComplete)

    • updateCrawlState

      public void updateCrawlState(CrawlerState state)

    • fire

      public void fire(Event event)
      Shortcut method for firing an event equivalent to: session.getCrawlContext().getEventManager().fire(event).
      Parameters:
      event - the event to fire

    • fire

      public void fire(String eventName, Object source)
      Shortcut method for firing a CrawlerEvent with the supplied source and with the crawlSession field already set. Equivalent to: session.getCrawlContext().getEventManager().fire(crawlerEvent) .
      Parameters:
      eventName - name of the event to fire
      source - the event source

    • loadState

      public CrawlerSession.State loadState()

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • getCluster

      public Cluster getCluster()

    • getCrawlContext

      public CrawlerContext getCrawlContext()

    • getRunCoordinator

      public CrawlerRunCoordinator getRunCoordinator()
      Coordinates task deduplication across session and run scopes.

    • getStateStore

      public CrawlerStateStore getStateStore()
      Manages persistent crawl state and session-scoped attributes.

    • getEventBus

      public CrawlerEventBus getEventBus()
      Fires events through the underlying event manager.