Package com.norconex.importer.handler.condition.impl

Class DomCondition

java.lang.Object
com.norconex.importer.handler.condition.impl.DomCondition
All Implemented Interfaces:
Configurable<DomConditionConfig>, Condition, ConfigurableCondition<DomConditionConfig>, FailablePredicate<DocHandlerContext,IOException>
public class DomCondition extends Object implements ConfigurableCondition<DomConditionConfig>

A condition using a Document Object Model (DOM) representation of an HTML, XHTML, or XML document content to match an element, attribute or value.

In order to construct a DOM tree, text is loaded entirely into memory. It uses the document content to create the DOM by default, but it can also use metadata fields. If more than one metadata field values are identified as the source of DOM content, only one needs to match for this condition to be true. Use this condition with caution if you know you'll need to parse huge files. You can use TextFilter instead if this is a concern.

The jsoup parser library is used to load the content into a DOM tree. Elements are referenced using a CSS or JQuery-like syntax.

The use of a value matcher is optional. Without one, any element found by the provided DOM selector will constitute a match. If both a DOM selector and a value matcher are provided, the matching selector element value(s) will be retrieved and the value matcher will be applied against it (or them) for a match.

It is possible to control what gets extracted exactly for matching purposes thanks to the "extract" argument of the new method DomConditionConfig.setExtract(String). Possible values are:

DomUtil.getElementValue(org.jsoup.nodes.Element,java.lang.String)

Should be used as a pre-parse handler.

Content-types

If you are dealing with multiple document types and you are using this condition on the document content, it is important to restrict this condition to text-based XML-like content only to prevent DOM-parsing errors.

By default this condition only applies to documents matching the content types listed in CommonMatchers.DOM_CONTENT_TYPES. Other content types always make this condition false.

You can overwrite these default content types by providing your own content type matcher. Make sure the content types you use represent a file with HTML or XML-like markup tags.

Character encoding

When used as a pre-parse handler, this condition uses the detected character encoding unless the character encoding was specified using DomConditionConfig.setSourceCharset(java.nio.charset.Charset). Since document parsing should always converts content to UTF-8, UTF-8 is always assumed when used as a post-parse handler.

XML vs HTML

You can specify which DOM parser to use when reading documents. The default is "html" and will try to normalize/fix the content as HTML. This is generally a desired behavior, but this can sometimes have your selector fail. If you encounter this problem, try switching to "xml" parser, which does not attempt normalization on the content. The drawback with "xml" is you may not get all HTML-specific selector options to work. If you know you are dealing with XML to begin with, specifying "xml" is a good option.

See Also: