ExternalTransformer

Transforms a document using an external application to do so

importerJavadoc: ExternalTransformer

Notes

Executing an external application to extract data from and/or manipulate a document.

Command-line arguments:

When constructing the command to launch the external application, it will look for specific tokens to be replaced by file paths arguments (in addition to other arguments you may have). The path arguments are created by this class. They are case-sensitive and the file they represent are temporary (will be deleted after they have been dealt with). It is possible to omit one or more tokens to use standard streams instead where applicable.

Tokens supported by:

${INPUT}

Path to document to be handled by the external application. When omitted, the document content is sent to the external application using the standard input stream (STDIN).

${INPUT_META}

Path to file containing metadata information available so far for the document to be handled by the external application. By default in JSON format. When omitted, no metadata will be made available to the external application.

${REFERENCE}

Unique reference to the document being handled (URL, original file system location, etc.). When omitted, the document reference will not be made available to the external application.

There is no ${OUTPUT} token since transformers do not modify document content (only metadata).

To parse/extract raw text from files, it is recommended to use ExternalParser instead.

Parser Handler Compatibility

This functionality can be applied as either a pre-parse or post-parse handler.

Examples

The following example invokes an external application that accepts a document to transform and outputs a file containing the new metadata information.

YAML

handler:
  class: ExternalTransformer
  command: /path/tag/app ${INPUT} ${OUTPUT_META}

JSON

{
  "handler": {
    "class": "ExternalTransformer",
    "command": "/path/tag/app ${INPUT} ${OUTPUT_META}"
  }
}

XML

<handler>
  <class>ExternalTransformer</class>
  <command>/path/tag/app ${INPUT} ${OUTPUT_META}</command>
</handler>

Usage

Full configuration skeleton, with every option and its default

YAML

class: ExternalTransformer
command: string
environmentVariables: {}
extractionPatterns:
  - fieldGroup: 0
    onSet: APPEND
    regex: {}
    toField: string
    valueGroup: 0
label: string
metadataInputFormat: string
metadataOutputFormat: string
onSet: APPEND
tempDir: string

JSON

{
  "class": "ExternalTransformer",
  "command": "string",
  "environmentVariables": {},
  "extractionPatterns": [
    {
      "fieldGroup": 0,
      "onSet": "APPEND",
      "regex": {},
      "toField": "string",
      "valueGroup": 0
    }
  ],
  "label": "string",
  "metadataInputFormat": "string",
  "metadataOutputFormat": "string",
  "onSet": "APPEND",
  "tempDir": "string"
}

XML

<handler>
  <class>ExternalTransformer</class>
  <command>string</command>
  <environmentVariables/>
  <extractionPatterns>
    <extractionPattern>
      <fieldGroup>0</fieldGroup>
      <onSet>APPEND</onSet>
      <regex/>
      <toField>string</toField>
      <valueGroup>0</valueGroup>
    </extractionPattern>
  </extractionPatterns>
  <label>string</label>
  <metadataInputFormat>string</metadataInputFormat>
  <metadataOutputFormat>string</metadataOutputFormat>
  <onSet>APPEND</onSet>
  <tempDir>string</tempDir>
</handler>

Properties

Property	Type	Required	Default
command	string	No	-
environmentVariables	Record<string, string>	No	-
extractionPatterns	RegexFieldValueExtractor[]	No	-
label	string	No	-
metadataInputFormat	string	No	-
metadataOutputFormat	string	No	-
onSet	enum	No	-
tempDir	string	No	-

Property Details

command

The command line to execute, with ${INPUT} and ${OUTPUT} placeholders for the document going in and coming out. Escape spaces in the executable path and its arguments.

Sample command line

c:\Apps\myapp.exe ${INPUT} ${INPUT_META} ${OUTPUT_META} ${REFERENCE}

environmentVariables

Defines environment variables to set when executing the external command.

extractionPatterns

Pattern only used when metadataOutputFormat is not specified

label

An optional, user-supplied label for this step in the importer pipeline. It has no effect on processing — it exists purely to help identify this step, for example in large configurations or in the visual configurator.

metadataInputFormat

Sets the format of the metadata input file sent to the external application. One of json (default), xml, or properties is expected. Only applicable when the ${INPUT_META} token is part of the command.

metadataOutputFormat

Sets the format of the metadata output file from the external application. One of json (default), xml, or properties is expected. Set to null for relying on metadata extraction patterns instead. Only applicable when the ${OUTPUT_META} token is part of the command.

onSet

What to do if a target field of the same name already exist.

Allowed Values

• APPEND
• PREPEND
• REPLACE
• OPTIONAL

tempDir

Sets the directory where to store temporary files used for the process.