Package com.norconex.crawler.core.doc.operations.checksum

Interface DocumentChecksummer

All Known Implementing Classes:
AbstractDocumentChecksummer, Md5DocumentChecksummer
public interface DocumentChecksummer

Creates a checksum representing a a document. Checksums are used to quickly filter out documents that have already been processed or that have changed since a previous run.

Two or more Doc can hold different values, but be deemed logically the same. Such documents do not have to be equal, but they should return the same checksum. An example of this can be two different URLs pointing to the same document, where only a single instance should be kept.

There are no strict rules that define what is equivalent or not.

See Also:

  • Method Summary

    Modifier and Type
    Method
    Description
    String
    createDocumentChecksum(Doc document)
    Creates a document checksum.

  • Method Details

    • createDocumentChecksum

      String createDocumentChecksum(Doc document)
      Creates a document checksum.
      Parameters:
      document - an HTTP document
      Returns:
      a checksum value