Package com.norconex.crawler.web.doc.operations.link.impl

Class DomLinkExtractorConfig

java.lang.Object
com.norconex.crawler.web.doc.operations.link.impl.DomLinkExtractorConfig
public class DomLinkExtractorConfig extends Object

Configuration for DomLinkExtractor.

Since:
3.0.0

  • Field Details

    • DEFAULT_SCHEMES

      public static final List<String> DEFAULT_SCHEMES

  • Constructor Details

    • DomLinkExtractorConfig

      public DomLinkExtractorConfig()

  • Method Details

    • setFieldMatcher

      public DomLinkExtractorConfig setFieldMatcher(TextMatcher fieldMatcher)

    • setContentTypeMatcher

      public DomLinkExtractorConfig setContentTypeMatcher(TextMatcher matcher)
      The matcher of content types to apply link extraction on. No attempt to extract links from any other content types will be made. Default is CommonMatchers.HTML_CONTENT_TYPES.
      Parameters:
      matcher - content type matcher
      Returns:
      this

    • addLinkSelector

      public DomLinkExtractorConfig addLinkSelector(String selector)
      Adds a new link selector extracting the "text" from matches.
      Parameters:
      selector - JSoup selector
      Returns:
      this

    • addLinkSelector

      public DomLinkExtractorConfig addLinkSelector(String selector, String extract)

    • removeLinkSelector

      public DomLinkExtractorConfig removeLinkSelector(String selector)

    • clearLinkSelectors

      public DomLinkExtractorConfig clearLinkSelectors()

    • getExtractSelectors

      public List<String> getExtractSelectors()

    • setExtractSelectors

      public DomLinkExtractorConfig setExtractSelectors(List<String> selectors)
      Only apply link selectors to portions of a document matching the supplied selectors.
      Parameters:
      selectors - the CSS selectors
      Returns:
      this

    • addExtractSelectors

      public DomLinkExtractorConfig addExtractSelectors(List<String> selectors)

    • getNoExtractSelectors

      public List<String> getNoExtractSelectors()

    • setNoExtractSelectors

      public DomLinkExtractorConfig setNoExtractSelectors(List<String> selectors)
      Do not apply link selectors to portions of a document matching the supplied selectors.
      Parameters:
      selectors - the CSS selectors
      Returns:
      this

    • addNoExtractSelectors

      public DomLinkExtractorConfig addNoExtractSelectors(List<String> selectors)

    • getSchemes

      public List<String> getSchemes()
      Gets the schemes to be extracted.
      Returns:
      schemes to be extracted

    • setSchemes

      public DomLinkExtractorConfig setSchemes(List<String> schemes)
      Sets the schemes to be extracted.
      Parameters:
      schemes - schemes to be extracted
      Returns:
      this

    • clearRestrictions

      public void clearRestrictions()
      Clears all restrictions.

    • getRestrictions

      public PropertyMatchers getRestrictions()
      Gets all restrictions
      Returns:
      the restrictions

    • getContentTypeMatcher

      public TextMatcher getContentTypeMatcher()
      The matcher of content types to apply link extraction on. No attempt to extract links from any other content types will be made. Default is CommonMatchers.DOM_CONTENT_TYPES.

    • getFieldMatcher

      public TextMatcher getFieldMatcher()
      Matcher of one or more fields to use as the source of content to extract links from, instead of the document content.

    • getLinkSelectors

      public List<DomLinkExtractorConfig.LinkSelector> getLinkSelectors()

    • getCharset

      public Charset getCharset()
      The assumed source character encoding.

    • getParser

      public String getParser()
      The parser to use when creating the DOM-tree.

    • isIgnoreNofollow

      public boolean isIgnoreNofollow()

    • isIgnoreLinkData

      public boolean isIgnoreLinkData()
      Whether to ignore extra data associated with a link.

    • setCharset

      public DomLinkExtractorConfig setCharset(Charset charset)
      The assumed source character encoding.
      Returns:
      this.

    • setParser

      public DomLinkExtractorConfig setParser(String parser)
      The parser to use when creating the DOM-tree.
      Returns:
      this.

    • setIgnoreNofollow

      public DomLinkExtractorConfig setIgnoreNofollow(boolean ignoreNofollow)
      Returns:
      this.

    • setIgnoreLinkData

      public DomLinkExtractorConfig setIgnoreLinkData(boolean ignoreLinkData)
      Whether to ignore extra data associated with a link.
      Returns:
      this.

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class Object

    • canEqual

      protected boolean canEqual(Object other)

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • toString

      public String toString()
      Overrides:
      toString in class Object