Package com.norconex.crawler.web.fetch.impl.playwright

Class PlaywrightFetcherConfig

java.lang.Object
com.norconex.crawler.core.fetch.BaseFetcherConfig
com.norconex.crawler.web.fetch.impl.playwright.PlaywrightFetcherConfig
public class PlaywrightFetcherConfig extends BaseFetcherConfig

Configuration for PlaywrightFetcher.

By default Playwright uses its own downloaded browser binaries (~/.cache/ms-playwright). To use a system-installed browser instead (e.g., Chrome, Edge, Firefox already present on the machine or CI runner), set the channel field (e.g., "chrome", "msedge", "firefox") — no download is then required. No external WebDriver binary is ever needed.

Since:
4.0.0
See Also:

  • Constructor Details

    • PlaywrightFetcherConfig

      public PlaywrightFetcherConfig()

  • Method Details

    • getArgs

      public List<String> getArgs()
      Returns an unmodifiable view of the browser launch arguments. Use setArgs(List) to replace the list.
      Returns:
      unmodifiable list of launch arguments

    • setArgs

      public PlaywrightFetcherConfig setArgs(List<String> args)
      Sets the browser launch arguments.
      Parameters:
      args - launch arguments
      Returns:
      this config

    • getBrowser

      public PlaywrightBrowser getBrowser()
      The browser engine used for crawling. Default is PlaywrightBrowser.CHROMIUM.

    • isHeadless

      public boolean isHeadless()
      Whether to run the browser in headless mode. Default is true.

    • getChannel

      public String getChannel()
      Optional browser channel that instructs Playwright to use a system-installed browser instead of its own downloaded binaries. Supported values: "chrome", "msedge", "firefox". Leave blank (the default) to use Playwright's bundled browser binaries.

    • getExecutablePath

      public Path getExecutablePath()
      Optional path to a custom browser executable. When null, Playwright uses the bundled browser binary it downloaded.

    • isIgnoreHttpsErrors

      public boolean isIgnoreHttpsErrors()
      When true, HTTPS certificate errors are ignored. Default is false.

    • getSlowMo

      public Duration getSlowMo()
      Optional slow-motion delay applied between Playwright operations. Useful for debugging. Default is null (no delay).

    • getPageLoadTimeout

      public Duration getPageLoadTimeout()
      Maximum time to wait for the page load event. When null, the Playwright default (30 s) applies.

    • getEarlyPageScript

      public String getEarlyPageScript()
      Optional JavaScript code executed immediately after page navigation, before any wait conditions are applied.

    • getLatePageScript

      public String getLatePageScript()
      Optional JavaScript code executed after all wait conditions have been satisfied.

    • getWaitForElementType

      public PlaywrightFetcherConfig.WaitElementType getWaitForElementType()
      The type of reference used when waiting for an element to appear. Used together with getWaitForElementSelector(). Default is PlaywrightFetcherConfig.WaitElementType.CSSSELECTOR.

    • getWaitForElementSelector

      public String getWaitForElementSelector()
      Selector expression used to wait for a specific element to appear in the page before returning. The kind of expression is defined by getWaitForElementType(). When null, no element wait is performed.

    • getWaitForElementTimeout

      public Duration getWaitForElementTimeout()
      Maximum time to wait for the element identified by getWaitForElementSelector() to appear. When null, the Playwright default timeout applies.

    • getWindowSize

      public Dimension getWindowSize()
      Optional browser viewport size. When null, the Playwright default viewport is used.

    • getProxySettings

      public ProxySettings getProxySettings()
      Proxy settings used when launching the browser. When not set (default), no proxy is configured.

    • getBrowserMaxNavigations

      public int getBrowserMaxNavigations()
      A maximum number of navigations after which each per-thread browser instance is restarted. Browsers can accumulate memory over time; restarting periodically can improve stability. Default is -1 (unlimited – no restart).

    • getBrowserMaxAge

      public Duration getBrowserMaxAge()
      Maximum age of a per-thread browser instance before it is restarted. Browsers can accumulate memory over time; restarting periodically can improve stability. Default is null (no age-based restart).

    • getCleanupInterval

      @NonNull public @NonNull Duration getCleanupInterval()
      Interval between checks for idle (unused) browser instances to close. Cannot be null. Defaults to 10 seconds.

    • getScreenshotHandler

      public PlaywrightScreenshotHandler getScreenshotHandler()
      When configured, takes a screenshot of each fetched page and stores it according to the handler's configuration (document metadata field or local directory).

    • setBrowser

      public PlaywrightFetcherConfig setBrowser(PlaywrightBrowser browser)
      The browser engine used for crawling. Default is PlaywrightBrowser.CHROMIUM.
      Returns:
      this.

    • setHeadless

      public PlaywrightFetcherConfig setHeadless(boolean headless)
      Whether to run the browser in headless mode. Default is true.
      Returns:
      this.

    • setChannel

      public PlaywrightFetcherConfig setChannel(String channel)
      Optional browser channel that instructs Playwright to use a system-installed browser instead of its own downloaded binaries. Supported values: "chrome", "msedge", "firefox". Leave blank (the default) to use Playwright's bundled browser binaries.
      Returns:
      this.

    • setExecutablePath

      public PlaywrightFetcherConfig setExecutablePath(Path executablePath)
      Optional path to a custom browser executable. When null, Playwright uses the bundled browser binary it downloaded.
      Returns:
      this.

    • setIgnoreHttpsErrors

      public PlaywrightFetcherConfig setIgnoreHttpsErrors(boolean ignoreHttpsErrors)
      When true, HTTPS certificate errors are ignored. Default is false.
      Returns:
      this.

    • setSlowMo

      public PlaywrightFetcherConfig setSlowMo(Duration slowMo)
      Optional slow-motion delay applied between Playwright operations. Useful for debugging. Default is null (no delay).
      Returns:
      this.

    • setPageLoadTimeout

      public PlaywrightFetcherConfig setPageLoadTimeout(Duration pageLoadTimeout)
      Maximum time to wait for the page load event. When null, the Playwright default (30 s) applies.
      Returns:
      this.

    • setEarlyPageScript

      public PlaywrightFetcherConfig setEarlyPageScript(String earlyPageScript)
      Optional JavaScript code executed immediately after page navigation, before any wait conditions are applied.
      Returns:
      this.

    • setLatePageScript

      public PlaywrightFetcherConfig setLatePageScript(String latePageScript)
      Optional JavaScript code executed after all wait conditions have been satisfied.
      Returns:
      this.

    • setWaitForElementType

      public PlaywrightFetcherConfig setWaitForElementType(PlaywrightFetcherConfig.WaitElementType waitForElementType)
      The type of reference used when waiting for an element to appear. Used together with getWaitForElementSelector(). Default is PlaywrightFetcherConfig.WaitElementType.CSSSELECTOR.
      Returns:
      this.

    • setWaitForElementSelector

      public PlaywrightFetcherConfig setWaitForElementSelector(String waitForElementSelector)
      Selector expression used to wait for a specific element to appear in the page before returning. The kind of expression is defined by getWaitForElementType(). When null, no element wait is performed.
      Returns:
      this.

    • setWaitForElementTimeout

      public PlaywrightFetcherConfig setWaitForElementTimeout(Duration waitForElementTimeout)
      Maximum time to wait for the element identified by getWaitForElementSelector() to appear. When null, the Playwright default timeout applies.
      Returns:
      this.

    • setWindowSize

      public PlaywrightFetcherConfig setWindowSize(Dimension windowSize)
      Optional browser viewport size. When null, the Playwright default viewport is used.
      Returns:
      this.

    • setProxySettings

      public PlaywrightFetcherConfig setProxySettings(ProxySettings proxySettings)
      Proxy settings used when launching the browser. When not set (default), no proxy is configured.
      Returns:
      this.

    • setBrowserMaxNavigations

      public PlaywrightFetcherConfig setBrowserMaxNavigations(int browserMaxNavigations)
      A maximum number of navigations after which each per-thread browser instance is restarted. Browsers can accumulate memory over time; restarting periodically can improve stability. Default is -1 (unlimited – no restart).
      Returns:
      this.

    • setBrowserMaxAge

      public PlaywrightFetcherConfig setBrowserMaxAge(Duration browserMaxAge)
      Maximum age of a per-thread browser instance before it is restarted. Browsers can accumulate memory over time; restarting periodically can improve stability. Default is null (no age-based restart).
      Returns:
      this.

    • setCleanupInterval

      public PlaywrightFetcherConfig setCleanupInterval(@NonNull @NonNull Duration cleanupInterval)
      Interval between checks for idle (unused) browser instances to close. Cannot be null. Defaults to 10 seconds.
      Returns:
      this.

    • setScreenshotHandler

      public PlaywrightFetcherConfig setScreenshotHandler(PlaywrightScreenshotHandler screenshotHandler)
      When configured, takes a screenshot of each fetched page and stores it according to the handler's configuration (document metadata field or local directory).
      Returns:
      this.

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class BaseFetcherConfig

    • canEqual

      protected boolean canEqual(Object other)
      Overrides:
      canEqual in class BaseFetcherConfig

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class BaseFetcherConfig

    • toString

      public String toString()
      Overrides:
      toString in class BaseFetcherConfig