CanonicalLinkDetector

Identifies the possible canonical URL of a document. Non-canonical documents are rejected in favor of canonical ones.

crawler/webJavadoc: CanonicalLinkDetector

Notes

Detects the canonical URL of a web page and skips non-canonical versions in favor of the canonical one. This prevents duplicate content from being indexed when the same page is accessible via multiple URLs. Select one of the available implementations to configure canonical link detection behavior.

Examples

The following disables canonical link resolution.

YAML

canonicalLinkDetector: null

JSON

{
  "canonicalLinkDetector": null
}

XML

<canonicalLinkDetector/>

Implementations

• GenericCanonicalLinkDetector Default