TruncateTransformer

Truncates field values

importerJavadoc: TruncateTransformer

Notes

This feature Shortens field values, and optionally replace truncated portion by a hash value to help ensure uniqueness (not 100% guaranteed to be collision-free). If the field to truncate has multiple values, all values will be subject to truncation. You can store the value(s), truncated or not, in another target field.

Storing values in an existing field

If a target field with the same name already exists for a document, values will be added to the end of the existing value list. It is possible to change this default behavior by supplying a PropertySetter.

The maxLength is guaranteed to be respected. This means any appended hash code and suffix will fit within the maxLength.

Truncating (or clearing) the document content

When no fieldMatcher is specified, this handler truncates the document content (body) instead of a field. Setting maxLength to 0 with no fieldMatcher effectively clears the document content entirely, while leaving its metadata untouched — a common substitute for keeping a document's metadata while discarding its body. Pair it with toField to first copy the content elsewhere (subject to the same maxLength) before it is cleared — use a large maxLength if the copy should not be truncated.

Can be used both as a pre-parse or post-parse handler.

Examples

Assuming this "myField" value...

• Please truncate me before you start thinking I am too long.

The following example will truncate the content to

• Please truncate me before you start thi!0996700004

YAML

handler:
  class: TruncateTransformer
  maxLength: 50
  appendHash: true
  suffix: "!"
  fieldMatcher:
    pattern: myField

JSON

{
  "handler": {
    "class": "TruncateTransformer",
    "maxLength": 50,
    "appendHash": true,
    "suffix": "!",
    "fieldMatcher": {
      "pattern": "myField"
    }
  }
}

XML

<handler>
  <class>TruncateTransformer</class>
  <maxLength>50</maxLength>
  <appendHash>true</appendHash>
  <suffix>!</suffix>
  <fieldMatcher>
    <pattern>myField</pattern>
  </fieldMatcher>
</handler>

Clearing a document's content while keeping its metadata (no fieldMatcher, maxLength of 0):

YAML

handler:
  class: TruncateTransformer
  maxLength: 0

JSON

{
  "handler": {
    "class": "TruncateTransformer",
    "maxLength": 0
  }
}

XML

<handler>
  <class>TruncateTransformer</class>
  <maxLength>0</maxLength>
</handler>

Usage

Full configuration skeleton, with every option and its default

YAML

class: TruncateTransformer
appendHash: false
fieldMatcher:
  ignoreCase: false
  ignoreDiacritic: false
  matchEmpty: false
  method: BASIC
  negateMatches: false
  partial: false
  pattern: string
  replaceAll: false
  trim: false
label: string
maxLength: 0
onSet: APPEND
suffix: string
toField: string

JSON

{
  "class": "TruncateTransformer",
  "appendHash": false,
  "fieldMatcher": {
    "ignoreCase": false,
    "ignoreDiacritic": false,
    "matchEmpty": false,
    "method": "BASIC",
    "negateMatches": false,
    "partial": false,
    "pattern": "string",
    "replaceAll": false,
    "trim": false
  },
  "label": "string",
  "maxLength": 0,
  "onSet": "APPEND",
  "suffix": "string",
  "toField": "string"
}

XML

<handler>
  <class>TruncateTransformer</class>
  <appendHash>false</appendHash>
  <fieldMatcher>
    <ignoreCase>false</ignoreCase>
    <ignoreDiacritic>false</ignoreDiacritic>
    <matchEmpty>false</matchEmpty>
    <method>BASIC</method>
    <negateMatches>false</negateMatches>
    <partial>false</partial>
    <pattern>string</pattern>
    <replaceAll>false</replaceAll>
    <trim>false</trim>
  </fieldMatcher>
  <label>string</label>
  <maxLength>0</maxLength>
  <onSet>APPEND</onSet>
  <suffix>string</suffix>
  <toField>string</toField>
</handler>

Properties

Property	Type	Required	Default
appendHash	boolean	No	false
fieldMatcher	TextMatcher	No	TextMatcher
label	string	No	-
maxLength	integer	No	0
onSet	enum	No	-
suffix	string	No	-
toField	string	No	-

Property Details

appendHash

Appends a hash to the end of the truncated value.

fieldMatcher

Matches document fields based on specified patterns and methods. When unset, the document content (body) is truncated instead.

label

An optional, user-supplied label for this step in the importer pipeline. It has no effect on processing — it exists purely to help identify this step, for example in large configurations or in the visual configurator.

maxLength

The maximum length of the value.

onSet

What to do if a target field of the same name already exist.

Allowed Values

• APPEND
• PREPEND
• REPLACE
• OPTIONAL

suffix

The suffix to append to the truncated value.

toField

A target field name used to store the value.