IdolCommitter

Commit documents to OpenText IDOL search engine.

committer/idolJavadoc: IdolCommitter

Notes

Commits crawled documents to an OpenText IDOL search engine. Supports Connector Framework Server (CFS) integration, optional database name configuration, and custom DREADDDATA and DREDELETEREF request parameters.

Examples

The following example uses the minimum required settings to configure IDOL Committer.

YAML

class: IdolCommitter
url: http://some_host:9100
databaseName: some_database

JSON

{
  "class": "IdolCommitter",
  "url": "http://some_host:9100",
  "databaseName": "some_database"
}

XML

<committer>
  <class>IdolCommitter</class>
  <url>http://some_host:9100</url>
  <databaseName>some_database</databaseName>
</committer>

Usage

Full configuration skeleton, with every option and its default

YAML

class: IdolCommitter
cfs: false
databaseName: string
dreAddDataParams: {}
dreDeleteRefParams: {}
fieldMappings: {}
queue:
  class: FsQueue
  batchSize: 0
  commitLeftoversOnInit: false
  maxPerFolder: 0
  onCommitFailure:
    ignoreErrors: false
    maxRetries: 0
    retryDelay: 0
    splitBatch: OFF
restrictions:
  - fieldMatcher: {}
    valueMatcher: {}
sourceContentField: string
sourceReferenceField: string
url: string

JSON

{
  "class": "IdolCommitter",
  "cfs": false,
  "databaseName": "string",
  "dreAddDataParams": {},
  "dreDeleteRefParams": {},
  "fieldMappings": {},
  "queue": {
    "class": "FsQueue",
    "batchSize": 0,
    "commitLeftoversOnInit": false,
    "maxPerFolder": 0,
    "onCommitFailure": {
      "ignoreErrors": false,
      "maxRetries": 0,
      "retryDelay": 0,
      "splitBatch": "OFF"
    }
  },
  "restrictions": [
    {
      "fieldMatcher": {},
      "valueMatcher": {}
    }
  ],
  "sourceContentField": "string",
  "sourceReferenceField": "string",
  "url": "string"
}

XML

<idolCommitter>
  <class>IdolCommitter</class>
  <cfs>false</cfs>
  <databaseName>string</databaseName>
  <dreAddDataParams/>
  <dreDeleteRefParams/>
  <fieldMappings/>
  <queue>
    <class>FsQueue</class>
    <batchSize>0</batchSize>
    <commitLeftoversOnInit>false</commitLeftoversOnInit>
    <maxPerFolder>0</maxPerFolder>
    <onCommitFailure/>
  </queue>
  <restrictions>
    <restriction>
      <fieldMatcher/>
      <valueMatcher/>
    </restriction>
  </restrictions>
  <sourceContentField>string</sourceContentField>
  <sourceReferenceField>string</sourceReferenceField>
  <url>string</url>
</idolCommitter>

Properties

Property	Type	Required	Default
cfs	boolean	No	false
databaseName	string	No	-
dreAddDataParams	Record<string, string>	No	-
dreDeleteRefParams	Record<string, string>	No	-
fieldMappings	Record<string, string>	No	-
queue	CommitterQueue	No	FsQueue
restrictions	PropertyMatcher[]	No	-
sourceContentField	string	No	-
sourceReferenceField	string	No	-
url	string	No	http://localhost:9001

Property Details

cfs

The URL points to a Connector Framework Server (CFS).

databaseName

Optional name of IDOL database where documents will be stored.

dreAddDataParams

Additional parameters sent with requests for new and updated documents.

dreDeleteRefParams

Additional parameters sent with requests for deleted documents.

fieldMappings

Mapping of source fields to target ones (IDOL fields).

queue

Configures the committer queue used to batch documents before sending them to the target system. Adjust queue settings to tune throughput and memory usage.

restrictions

Defines filter conditions that restrict which documents this committer processes. Documents not matching the restrictions are ignored by this committer.

sourceContentField

The document field name containing the value to be stored in IDOL DRECONTENT field. Leave blank to use the document content instead of a field (default).

sourceReferenceField

The document field name containing the value to be stored in IDOL DREREFERENCE field. Leave blank to use the document reference instead of a field (default).

url

The IDOL index URL. Default is http://localhost:9001)