com.norconex.crawler.web.doc.operations.canon.impl.GenericCanonicalLinkDetector

All Implemented Interfaces:: Configurable<GenericCanonicalLinkDetectorConfig>, CanonicalLinkDetector

public class GenericCanonicalLinkDetector extends Object implements CanonicalLinkDetector, Configurable<GenericCanonicalLinkDetectorConfig>

Generic canonical link detector. It detects links from the HTTP headers as well as HTML files. Good canonical reference documentation can be found on this Google Webmaster Tools help page.

HTTP Headers

This detector will look for a metadata field (normally obtained from the HTTP Headers) name called "Link" with a value following this pattern:

 <http://www.example.com/sample.pdf> rel="canonical"

All documents will be verified for a canonical link (not just HTML).

Document content

This detector will look within the HTML <head> tags for a <link> tag following this pattern:

 <link rel="canonical" href="https://www.example.com/sample" />

Only HTML documents will be verified for a canonical link. By default, these content-types are considered HTML:

 text/html, application/xhtml+xml, vnd.wap.xhtml+xml, x-asp

You can specify your own content types as long as they contain HTML text.

The above example ignores canonical link resolution.

Since:: 2.2.0

Field Summary

Fields inherited from interface com.norconex.commons.lang.config.Configurable
PROPERTY
Constructor Summary

Constructors

Constructor

Description

GenericCanonicalLinkDetector()
Method Summary

Modifier and Type

Method

Description

protected boolean

canEqual(Object other)

String

detectFromContent(String reference, InputStream is, ContentType contentType)

Detects from a document content the presence of a canonical URL.

String

detectFromMetadata(String reference, Properties metadata)

Detects from metadata gathered so far, which when invoked, is normally the HTTP header values.

boolean

equals(Object o)

GenericCanonicalLinkDetectorConfig

getConfiguration()

int

hashCode()

String

toString()

Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Details
- GenericCanonicalLinkDetector
  
  public GenericCanonicalLinkDetector()
Method Details
- getConfiguration
  
  public GenericCanonicalLinkDetectorConfig getConfiguration()
  
  Specified by:
  
  getConfiguration in interface Configurable<GenericCanonicalLinkDetectorConfig>
- detectFromMetadata
  
  public String detectFromMetadata(String reference, Properties metadata)
  
  Description copied from interface: CanonicalLinkDetector
  
  Detects from metadata gathered so far, which when invoked, is normally the HTTP header values.
  
  Specified by:
  
  detectFromMetadata in interface CanonicalLinkDetector
  
  Parameters:
  
  reference - document reference
  
  metadata - metadata object containing HTTP headers
  
  Returns:
  
  the detected canonical URL or null if none is found.
- detectFromContent
  
  public String detectFromContent(String reference, InputStream is, ContentType contentType) throws IOException
  
  Description copied from interface: CanonicalLinkDetector
  
  Detects from a document content the presence of a canonical URL. This occur before a document gets parsed and may apply to only a few content types.
  
  Specified by:
  
  detectFromContent in interface CanonicalLinkDetector
  
  Parameters:
  
  reference - document reference
  
  is - the document content input stream
  
  contentType - the document content type
  
  Returns:
  
  the detected canonical URL or null if none is found.
  
  Throws:
  
  IOException - problem reading content
- equals
  
  public boolean equals(Object o)
  
  Overrides:
  
  equals in class Object
- canEqual
  
  protected boolean canEqual(Object other)
- hashCode
  
  public int hashCode()
  
  Overrides:
  
  hashCode in class Object
- toString
  
  public String toString()
  
  Overrides:
  
  toString in class Object

Class GenericCanonicalLinkDetector

HTTP Headers

Document content

Field Summary

Fields inherited from interface com.norconex.commons.lang.config.Configurable

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Constructor Details

GenericCanonicalLinkDetector

Method Details

getConfiguration

detectFromMetadata

detectFromContent

equals

canEqual

hashCode

toString