CsvSplitter

Split files with Coma-Separated values

importerJavadoc: CsvSplitter

Notes

Processes files with comma-separated values (or other delimiters like tabs) by dividing them into individual documents, one per line.

Parser Handler Compatibility

This functionality can be applied as either a pre-parse or post-parse handler.

Examples

The following example demonstrates how to use the CsvSplitter to split a CSV file into multiple documents.

Sample CSV File contentTypes:

'clientId','clientName','clientOrg','orgDesc'
'123','Joe Dalton','ACME Inc.','Organization\'s description'
'345','Avrel Dalton','Daisy Town','Another one'

Result:

The file will be split into two documents, one for each row after the header row:

Document 1

• Reference: 123
• Content: ACME Inc Organization's description

Document 2

• Reference: 345
• Content: 'Daisy Town Another one

YAML

handler:
  class: CsvSplitter
  separatorCharacter: ","
  quoteCharacter: "'"
  escapeCharacter: \
  useFirstRowAsFields: true
  linesToSkip: 0
  referenceColumn: clientId
  contentColumns:
    - clientOrg
    - orgDesc

JSON

{
  "handler": {
    "class": "CsvSplitter",
    "separatorCharacter": ",",
    "quoteCharacter": "'",
    "escapeCharacter": "\\",
    "useFirstRowAsFields": true,
    "linesToSkip": 0,
    "referenceColumn": "clientId",
    "contentColumns": [
      "clientOrg",
      "orgDesc"
    ]
  }
}

XML

<handler>
  <class>CsvSplitter</class>
  <separatorCharacter>,</separatorCharacter>
  <quoteCharacter>'</quoteCharacter>
  <escapeCharacter></escapeCharacter>
  <useFirstRowAsFields>true</useFirstRowAsFields>
  <linesToSkip>0</linesToSkip>
  <referenceColumn>clientId</referenceColumn>
  <contentColumns>
    <contentColumn>clientOrg</contentColumn>
    <contentColumn>orgDesc</contentColumn>
  </contentColumns>
</handler>

Usage

Full configuration skeleton, with every option and its default

YAML

class: CsvSplitter
contentColumns:
  - string
discardOriginal: false
escapeCharacter: string
fieldMatcher:
  ignoreCase: false
  ignoreDiacritic: false
  matchEmpty: false
  method: BASIC
  negateMatches: false
  partial: false
  pattern: string
  replaceAll: false
  trim: false
label: string
linesToSkip: 0
quoteCharacter: string
referenceColumn: string
separatorCharacter: string
useFirstRowAsFields: false

JSON

{
  "class": "CsvSplitter",
  "contentColumns": [
    "string"
  ],
  "discardOriginal": false,
  "escapeCharacter": "string",
  "fieldMatcher": {
    "ignoreCase": false,
    "ignoreDiacritic": false,
    "matchEmpty": false,
    "method": "BASIC",
    "negateMatches": false,
    "partial": false,
    "pattern": "string",
    "replaceAll": false,
    "trim": false
  },
  "label": "string",
  "linesToSkip": 0,
  "quoteCharacter": "string",
  "referenceColumn": "string",
  "separatorCharacter": "string",
  "useFirstRowAsFields": false
}

XML

<handler>
  <class>CsvSplitter</class>
  <contentColumns>
    <contentColumn>string</contentColumn>
  </contentColumns>
  <discardOriginal>false</discardOriginal>
  <escapeCharacter>string</escapeCharacter>
  <fieldMatcher>
    <ignoreCase>false</ignoreCase>
    <ignoreDiacritic>false</ignoreDiacritic>
    <matchEmpty>false</matchEmpty>
    <method>BASIC</method>
    <negateMatches>false</negateMatches>
    <partial>false</partial>
    <pattern>string</pattern>
    <replaceAll>false</replaceAll>
    <trim>false</trim>
  </fieldMatcher>
  <label>string</label>
  <linesToSkip>0</linesToSkip>
  <quoteCharacter>string</quoteCharacter>
  <referenceColumn>string</referenceColumn>
  <separatorCharacter>string</separatorCharacter>
  <useFirstRowAsFields>false</useFirstRowAsFields>
</handler>

Properties

Property	Type	Required	Default
contentColumns	string[]	No	-
discardOriginal	boolean	No	false
escapeCharacter	string	No	\
fieldMatcher	TextMatcher	No	TextMatcher
label	string	No	-
linesToSkip	integer	No	0
quoteCharacter	string	No	"
referenceColumn	string	No	-
separatorCharacter	string	No	,
useFirstRowAsFields	boolean	No	false

Property Details

contentColumns

Lists the columns whose values will form the document content.

discardOriginal

Discard the original document after the split is performed.

escapeCharacter

Sets the character used to escape special characters within field values.

fieldMatcher

Matches document fields based on specified patterns and methods

label

An optional, user-supplied label for this step in the importer pipeline. It has no effect on processing — it exists purely to help identify this step, for example in large configurations or in the visual configurator.

linesToSkip

Specifies the number of initial lines in the CSV file to skip during processing

quoteCharacter

Defines the character used to enclose fields containing special characters, like quotes or separators.

referenceColumn

Identifies the column whose values will be used as document references.

separatorCharacter

Specifies the character used to separate fields in the CSV file, such as a comma or tab.

useFirstRowAsFields

Indicates whether the first row of the CSV file should be treated as field names.