Package com.norconex.crawler.web.fetch.impl.playwright

Class PlaywrightFetcher

java.lang.Object
com.norconex.crawler.core.fetch.AbstractFetcher<PlaywrightFetcherConfig>
com.norconex.crawler.web.fetch.impl.playwright.PlaywrightFetcher
All Implemented Interfaces:
Configurable<PlaywrightFetcherConfig>, EventListener<Event>, Fetcher, EventListener, Consumer<Event>
public class PlaywrightFetcher extends AbstractFetcher<PlaywrightFetcherConfig>

Uses Microsoft Playwright for browser-based crawling. Playwright drives real browsers (Chromium, Firefox, WebKit) across Windows, macOS, and Linux without requiring an external WebDriver binary — browser binaries are downloaded automatically by Playwright on first use.

Key advantages over WebDriverFetcher

  • No external driver binary needed.
  • HTTP response headers (including status codes) are captured natively via Playwright's response interception — no proxy sniffer is required.
  • Supports three cross-platform engines: Chromium, Firefox, and WebKit.

Thread safety

Playwright is not thread-safe. Each crawl thread gets its own Playwright and Browser instance managed via ThreadLocal. All instances are tracked for clean shutdown.

Supported HTTP method

This fetcher only supports HTTP GET method.

HTTP Headers

HTTP response headers (including the real status code and reason phrase) are captured via Playwright's page.onResponse() listener and stored in the document metadata.

Since:
4.0.0
See Also: