SubstringTransformer

Keep a substring of the content matching a begin and end character indexes.

importerJavadoc: SubstringTransformer

Notes

Useful when you have to truncate long content, or when you know precisely where is located the text to extract in some files.

The begin value is inclusive, while the end value is exclusive. Both are optional. When not specified (or a negative value), the index is assumed to be the beginning and end of the content, respectively.

This class can be used as a pre-parsing (text content-types only) or post-parsing handlers.

Examples

The following example truncates long text to be 10,000 characters maximum.

YAML

handler:
  class: SubstringTransformer
  end: 10000

JSON

{
  "handler": {
    "class": "SubstringTransformer",
    "end": 10000
  }
}

XML

<handler>
  <class>SubstringTransformer</class>
  <end>10000</end>
</handler>

Usage

Full configuration skeleton, with every option and its default

YAML

class: SubstringTransformer
begin: 0
end: 0
fieldMatcher:
  ignoreCase: false
  ignoreDiacritic: false
  matchEmpty: false
  method: BASIC
  negateMatches: false
  partial: false
  pattern: string
  replaceAll: false
  trim: false
label: string
sourceCharset: string

JSON

{
  "class": "SubstringTransformer",
  "begin": 0,
  "end": 0,
  "fieldMatcher": {
    "ignoreCase": false,
    "ignoreDiacritic": false,
    "matchEmpty": false,
    "method": "BASIC",
    "negateMatches": false,
    "partial": false,
    "pattern": "string",
    "replaceAll": false,
    "trim": false
  },
  "label": "string",
  "sourceCharset": "string"
}

XML

<handler>
  <class>SubstringTransformer</class>
  <begin>0</begin>
  <end>0</end>
  <fieldMatcher>
    <ignoreCase>false</ignoreCase>
    <ignoreDiacritic>false</ignoreDiacritic>
    <matchEmpty>false</matchEmpty>
    <method>BASIC</method>
    <negateMatches>false</negateMatches>
    <partial>false</partial>
    <pattern>string</pattern>
    <replaceAll>false</replaceAll>
    <trim>false</trim>
  </fieldMatcher>
  <label>string</label>
  <sourceCharset>string</sourceCharset>
</handler>

Properties

Property	Type	Required	Default
begin	integer	No	0
end	integer	No	-1
fieldMatcher	TextMatcher	No	TextMatcher
label	string	No	-
sourceCharset	string	No	-

Property Details

begin

The index of the beginning of the substring.

end

The index of the end of the substring.

fieldMatcher

Matches document fields based on specified patterns and methods

label

An optional, user-supplied label for this step in the importer pipeline. It has no effect on processing — it exists purely to help identify this step, for example in large configurations or in the visual configurator.

sourceCharset

Sets the character encoding of the source document to ensure proper reading and processing.