DomSplitter

Splits HTML, XHTML, or XML document on elements matching a given selector.

importerJavadoc: DomSplitter

Notes

This option builds a full DOM tree from the document content in memory. For large files, be cautious—this can use a lot of memory. For better performance with big files, consider a streaming approach like (e.g., XmlStreamSplitter).

The jsoup parser library is used to load a document content into a DOM tree. Elements are referenced using a CSS or JQuery-like syntax.

Content-types

By default, this filter applies only to documents with content types specified by the CommonRestrictions.domContentTypes(String) method. You can also define your own content types if they represent files containing HTML or XML-like markup tags.

Common content-types:

• application/atom+xml
• application/mathml+xml
• application/rss+xml
• application/vnd.wap.xhtml+xml
• application/x-asp
• application/xhtml+xml
• application/xml
• application/xslt+xml
• image/svg+xml
• text/html
• text/xml

Parser Handler Compatibility

This functionality can be applied as either a pre-parse or post-parse handler.

Examples

The following example splits contant found in an HTML document, each one being stored within a div with a class named "contact".

YAML

handler:
  class: DomSplitter
  selector: div.contact

JSON

{
  "handler": {
    "class": "DomSplitter",
    "selector": "div.contact"
  }
}

XML

<handler>
  <class>DomSplitter</class>
  <selector>div.contact</selector>
</handler>

content Type

The following example splits content from a field named embedded_file, but only when the content type is text/html. It targets HTML elements matching the div.contact selector within that field

YAML

handler:
  class: DomSplitter
  selector: div.contact
  parser: HTML
  contentTypeMatcher:
    method: BASIC
    pattern: text/html
  fieldMatcher:
    method: BASIC
    pattern: embedded_file

JSON

{
  "handler": {
    "class": "DomSplitter",
    "selector": "div.contact",
    "parser": "HTML",
    "contentTypeMatcher": {
      "method": "BASIC",
      "pattern": "text/html"
    },
    "fieldMatcher": {
      "method": "BASIC",
      "pattern": "embedded_file"
    }
  }
}

XML

<handler>
  <class>DomSplitter</class>
  <selector>div.contact</selector>
  <parser>HTML</parser>
  <contentTypeMatcher>
    <method>BASIC</method>
    <pattern>text/html</pattern>
  </contentTypeMatcher>
  <fieldMatcher>
    <method>BASIC</method>
    <pattern>embedded_file</pattern>
  </fieldMatcher>
</handler>

Usage

Full configuration skeleton, with every option and its default

YAML

class: DomSplitter
contentTypeMatcher:
  ignoreCase: false
  ignoreDiacritic: false
  matchEmpty: false
  method: BASIC
  negateMatches: false
  partial: false
  pattern: string
  replaceAll: false
  trim: false
discardOriginal: false
fieldMatcher:
  ignoreCase: false
  ignoreDiacritic: false
  matchEmpty: false
  method: BASIC
  negateMatches: false
  partial: false
  pattern: string
  replaceAll: false
  trim: false
label: string
parser: string
selector: string
sourceCharset: string

JSON

{
  "class": "DomSplitter",
  "contentTypeMatcher": {
    "ignoreCase": false,
    "ignoreDiacritic": false,
    "matchEmpty": false,
    "method": "BASIC",
    "negateMatches": false,
    "partial": false,
    "pattern": "string",
    "replaceAll": false,
    "trim": false
  },
  "discardOriginal": false,
  "fieldMatcher": {
    "ignoreCase": false,
    "ignoreDiacritic": false,
    "matchEmpty": false,
    "method": "BASIC",
    "negateMatches": false,
    "partial": false,
    "pattern": "string",
    "replaceAll": false,
    "trim": false
  },
  "label": "string",
  "parser": "string",
  "selector": "string",
  "sourceCharset": "string"
}

XML

<handler>
  <class>DomSplitter</class>
  <contentTypeMatcher>
    <ignoreCase>false</ignoreCase>
    <ignoreDiacritic>false</ignoreDiacritic>
    <matchEmpty>false</matchEmpty>
    <method>BASIC</method>
    <negateMatches>false</negateMatches>
    <partial>false</partial>
    <pattern>string</pattern>
    <replaceAll>false</replaceAll>
    <trim>false</trim>
  </contentTypeMatcher>
  <discardOriginal>false</discardOriginal>
  <fieldMatcher>
    <ignoreCase>false</ignoreCase>
    <ignoreDiacritic>false</ignoreDiacritic>
    <matchEmpty>false</matchEmpty>
    <method>BASIC</method>
    <negateMatches>false</negateMatches>
    <partial>false</partial>
    <pattern>string</pattern>
    <replaceAll>false</replaceAll>
    <trim>false</trim>
  </fieldMatcher>
  <label>string</label>
  <parser>string</parser>
  <selector>string</selector>
  <sourceCharset>string</sourceCharset>
</handler>

Properties

Property	Type	Required	Default
contentTypeMatcher	TextMatcher	No	TextMatcher
discardOriginal	boolean	No	false
fieldMatcher	TextMatcher	No	TextMatcher
label	string	No	-
parser	string	No	html
selector	string	No	-
sourceCharset	string	No	-

Property Details

contentTypeMatcher

By default, this filter applies only to documents with content types specified by the CommonRestrictions.domContentTypes(String) method. You can also define your own content types if they represent files containing HTML or XML-like markup tags.

Common content-types:

• application/atom+xml
• application/mathml+xml
• application/rss+xml
• application/vnd.wap.xhtml+xml
• application/x-asp
• application/xhtml+xml
• application/xml
• application/xslt+xml
• image/svg+xml
• text/html
• text/xml

discardOriginal

Discard the original document after the split is performed.

fieldMatcher

Matches document fields based on specified patterns and methods

label

An optional, user-supplied label for this step in the importer pipeline. It has no effect on processing — it exists purely to help identify this step, for example in large configurations or in the visual configurator.

parser

Defines using XML or HTML parser.

selector

XPath expression for splitting nodes.

sourceCharset

Sets the character encoding of the source document to ensure proper reading and processing.