Class WebCrawler

java.lang.Object
com.norconex.crawler.web.WebCrawler

public final class WebCrawler extends Object
Facade for launching or obtaining a Web Crawler.
  • Method Details

    • main

      public static void main(String[] args)
      Invokes the Web Crawler from the command line. You can invoke it without any arguments to get a list of command-line options.
      Parameters:
      args - command-line options
    • launch

      public static int launch(String... args)
      Launches the Web Crawler. Similar to main(String[]), but do not call System.exit(int) and returns the execution status code instead. It will throw a runtime exception upon failure (typically a CrawlerException).
      Parameters:
      args - command line arguments
      Returns:
      execution status code
    • create

      public static Crawler create(WebCrawlerConfig crawlerConfig)
      Creates a Web Crawler instance.
      Parameters:
      crawlerConfig - Web Crawler configuration
      Returns:
      crawler