UrlStatusCrawlerEventListener

Records URLs and their HTTP status codes to a CSV report during a crawl.

crawler/webJavadoc: UrlStatusCrawlerEventListener

Notes

Listens for crawler events and records URLs along with their HTTP response status codes to a CSV report file. Useful for generating broken-link reports or auditing status codes returned during a crawl session.

Examples

The following example will generate a broken links report by recording 404 status codes (from HTTP responses).

YAML

class: UrlStatusCrawlerEventListener
statusCodes: "404"
outputDir: /report/path
fileNamePrefix: brokenLinks

JSON

{
  "class": "UrlStatusCrawlerEventListener",
  "statusCodes": "404",
  "outputDir": "/report/path",
  "fileNamePrefix": "brokenLinks"
}

XML

<class>UrlStatusCrawlerEventListener</class>
<statusCodes>404</statusCodes>
<outputDir>/report/path</outputDir>
<fileNamePrefix>brokenLinks</fileNamePrefix>

Usage

Full configuration skeleton, with every option and its default

YAML

class: UrlStatusCrawlerEventListener
fileNamePrefix: string
outputDir: string
statusCodes: string
timestamped: false

JSON

{
  "class": "UrlStatusCrawlerEventListener",
  "fileNamePrefix": "string",
  "outputDir": "string",
  "statusCodes": "string",
  "timestamped": false
}

XML

<urlStatusCrawlerEventListener>
  <class>UrlStatusCrawlerEventListener</class>
  <fileNamePrefix>string</fileNamePrefix>
  <outputDir>string</outputDir>
  <statusCodes>string</statusCodes>
  <timestamped>false</timestamped>
</urlStatusCrawlerEventListener>

Properties

Property	Type	Required	Default
fileNamePrefix	string	No	urlstatuses-
outputDir	string	No	-
statusCodes	string	No	-
timestamped	boolean	No	false

Property Details

fileNamePrefix

A string prepended to generated file names to distinguish them from other output files.

outputDir

The local directory where the URL status report file will be written. Defaults to the crawler working directory.

statusCodes

A comma-separated list of HTTP status codes to record. Supports ranges (e.g., 400-499). Default is null, which records all status codes.

timestamped

When enabled, a timestamp is appended to the report file name to ensure a new file is created on each crawl run rather than overwriting the previous one.