Package com.norconex.crawler.web.doc.operations.link.impl

Class RegexLinkExtractorConfig

java.lang.Object
com.norconex.crawler.web.doc.operations.link.impl.RegexLinkExtractorConfig
public class RegexLinkExtractorConfig extends Object

Configuration for RegexLinkExtractor.

Since:
2.7.0

  • Field Details

    • DEFAULT_CONTENT_TYPE_PATTERN

      public static final String DEFAULT_CONTENT_TYPE_PATTERN
      See Also:

    • DEFAULT_MAX_URL_LENGTH

      public static final int DEFAULT_MAX_URL_LENGTH
      Default maximum length a URL can have.
      See Also:

  • Constructor Details

    • RegexLinkExtractorConfig

      public RegexLinkExtractorConfig()

  • Method Details

    • getPatterns

      public List<RegexLinkExtractorConfig.ExtractionPattern> getPatterns()

    • setPatterns

      public RegexLinkExtractorConfig setPatterns(List<RegexLinkExtractorConfig.ExtractionPattern> patterns)

    • clearPatterns

      public RegexLinkExtractorConfig clearPatterns()

    • setContentTypeMatcher

      public RegexLinkExtractorConfig setContentTypeMatcher(TextMatcher matcher)
      The matcher of content types to apply link extraction on. No attempt to extract links from any other content types will be made. Default matches all content types.
      Parameters:
      matcher - content type matcher
      Returns:
      this

    • clearRestrictions

      public void clearRestrictions()
      Clears all restrictions.

    • getRestrictions

      public PropertyMatchers getRestrictions()
      Gets all restrictions
      Returns:
      the restrictions

    • getMaxUrlLength

      public int getMaxUrlLength()
      The maximum supported URL length. Default is 2048.

    • getCharset

      public Charset getCharset()
      Gets the character set of pages on which link extraction is performed. Default is null (charset detection will be attempted).

    • getContentTypeMatcher

      public TextMatcher getContentTypeMatcher()
      The matcher of content types to apply link extraction on. No attempt to extract links from any other content types will be made. Default matches all content types

    • getFieldMatcher

      public TextMatcher getFieldMatcher()
      Matcher of one or more fields to use as the source of content to extract links from, instead of the document content.

    • setMaxUrlLength

      public RegexLinkExtractorConfig setMaxUrlLength(int maxUrlLength)
      The maximum supported URL length. Default is 2048.
      Returns:
      this.

    • setCharset

      public RegexLinkExtractorConfig setCharset(Charset charset)
      Gets the character set of pages on which link extraction is performed. Default is null (charset detection will be attempted).
      Returns:
      this.

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class Object

    • canEqual

      protected boolean canEqual(Object other)

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • toString

      public String toString()
      Overrides:
      toString in class Object