ClusterConfig

Cluster configuration for distributed crawling.

crawler/coreJavadoc: ClusterConfig

Notes

Configures distributed crawler cluster settings, including whether clustering is enabled, the port for administrative access, and the cluster connector implementation to use.

Usage

YAML

adminDisabled: false
adminPort: 0
clustered: false
connector:
  class: HazelcastClusterConnector
  clusterName: string
  configurer:
    class: JdbcHazelcastConfigurer
    autoDiscoveryEnabled: false
    backupCount: 0
    columnKeyType: string
    columnValueType: string
    hazelcastProperties: {}
    initialLoadMode: LAZY
    jdbcDriver: string
    jdbcPassword: string
    jdbcUrl: string
    jdbcUsername: string
    jetEnabled: false
    maxPoolSize: 0
    queueStoreEnabled: false
    sqlMerge: string
    tcpMembers: string
    writeBatchSize: 0
    writeCoalescing: false
    writeDelaySeconds: 0
  instanceName: string
  nodeExpiryTimeout: 0
  workerHeartbeatInterval: 0

JSON

{
  "adminDisabled": false,
  "adminPort": 0,
  "clustered": false,
  "connector": {
    "class": "HazelcastClusterConnector",
    "clusterName": "string",
    "configurer": {
      "class": "JdbcHazelcastConfigurer",
      "autoDiscoveryEnabled": false,
      "backupCount": 0,
      "columnKeyType": "string",
      "columnValueType": "string",
      "hazelcastProperties": {},
      "initialLoadMode": "LAZY",
      "jdbcDriver": "string",
      "jdbcPassword": "string",
      "jdbcUrl": "string",
      "jdbcUsername": "string",
      "jetEnabled": false,
      "maxPoolSize": 0,
      "queueStoreEnabled": false,
      "sqlMerge": "string",
      "tcpMembers": "string",
      "writeBatchSize": 0,
      "writeCoalescing": false,
      "writeDelaySeconds": 0
    },
    "instanceName": "string",
    "nodeExpiryTimeout": 0,
    "workerHeartbeatInterval": 0
  }
}

XML

<cluster>
  <adminDisabled>false</adminDisabled>
  <adminPort>0</adminPort>
  <clustered>false</clustered>
  <connector>
    <class>HazelcastClusterConnector</class>
    <clusterName>string</clusterName>
    <configurer/>
    <instanceName>string</instanceName>
    <nodeExpiryTimeout>0</nodeExpiryTimeout>
    <workerHeartbeatInterval>0</workerHeartbeatInterval>
  </connector>
</cluster>

Properties

Property	Type	Required	Default
adminDisabled	boolean	No	false
adminPort	integer	No	27295
clustered	boolean	No	false
connector	ClusterConnector	No	MVStoreClusterConnector

Property Details

adminDisabled

When enabled, the crawler administrative server endpoints are not launched.

adminPort

The port each cluster node listens on for administrative commands. If the specified port is in use, the next available port is used instead. Default is 27295.

clustered

When enabled, the crawler operates in cluster mode with multiple nodes. When disabled (default), the crawler runs as a standalone instance optimized for single-node performance.

connector

The cluster connector implementation used to run the crawler. Defaults to MVStoreClusterConnector (file-backed, no external infrastructure needed). Use HazelcastClusterConnector for multi-node clustered mode, or MemoryClusterConnector for a non-persistent in-memory alternative.