DeleteRejectedEventListener

Sends deletion requests to the committer when documents are rejected or not found.

crawler/coreJavadoc: DeleteRejectedEventListener

Notes

Listens for crawler rejection events (such as "not found" or "filtered") and sends a deletion request to the committer for the affected document. Useful for keeping the index clean when documents are removed from or filtered out of the source.

Examples

The following example will send deletion requests whenever a reference is not found (e.g., a 404 response from a web server) or if it was filtered out by the crawler.

YAML

class: DeleteRejectedEventListener
eventMatcher:
  method: CSV
  pattern: REJECTED_NOTFOUND,REJECTED_FILTER

JSON

{
  "class": "DeleteRejectedEventListener",
  "eventMatcher": {
    "method": "CSV",
    "pattern": "REJECTED_NOTFOUND,REJECTED_FILTER"
  }
}

XML

<class>DeleteRejectedEventListener</class>
<eventMatcher>
  <method>CSV</method>
  <pattern>REJECTED_NOTFOUND,REJECTED_FILTER</pattern>
</eventMatcher>

Usage

Full configuration skeleton, with every option and its default

YAML

class: DeleteRejectedEventListener
eventMatcher:
  ignoreCase: false
  ignoreDiacritic: false
  matchEmpty: false
  method: BASIC
  negateMatches: false
  partial: false
  pattern: string
  replaceAll: false
  trim: false

JSON

{
  "class": "DeleteRejectedEventListener",
  "eventMatcher": {
    "ignoreCase": false,
    "ignoreDiacritic": false,
    "matchEmpty": false,
    "method": "BASIC",
    "negateMatches": false,
    "partial": false,
    "pattern": "string",
    "replaceAll": false,
    "trim": false
  }
}

XML

<deleteRejectedEventListener>
  <class>DeleteRejectedEventListener</class>
  <eventMatcher>
    <ignoreCase>false</ignoreCase>
    <ignoreDiacritic>false</ignoreDiacritic>
    <matchEmpty>false</matchEmpty>
    <method>BASIC</method>
    <negateMatches>false</negateMatches>
    <partial>false</partial>
    <pattern>string</pattern>
    <replaceAll>false</replaceAll>
    <trim>false</trim>
  </eventMatcher>
</deleteRejectedEventListener>

Properties

Property	Type	Required	Default
eventMatcher	TextMatcher	No	TextMatcher

Property Details

eventMatcher

A text matcher identifying which crawler events trigger a deletion request to the committer. Default matches any event matching the regular expression REJECTED_.*.