Package com.norconex.crawler.core.cmd.crawl

Record Class RunProgressWriter.ProgressRecord

java.lang.Object
java.lang.Record
com.norconex.crawler.core.cmd.crawl.RunProgressWriter.ProgressRecord
Record Components:
type - progress or heartbeat
crawlerId - the crawler this concerns
at - when the sample was taken
elapsedMs - milliseconds since the crawl began
queued - references waiting to be processed. A reference is a ledger entry, not necessarily a document: a folder traversed for its children is counted here too
processing - references being processed right now
processed - references processed so far this run
baseline - references from the previous run not yet seen again
sessionEventCounts - what this crawl session has done, by event name, spanning the runs of a resumed crawl the same way the gauges above do. Every event the crawler fires is counted, so this is not a selection: ask it for COMMITTER_UPSERT_END to get documents committed, and a new event type appears as a new key with nothing to change here
runEventCounts - the share of those totals belonging to this run, which differs from the session's only when a crawl was stopped and resumed
Enclosing class:
RunProgressWriter
public static record RunProgressWriter.ProgressRecord(String type, String crawlerId, Instant at, long elapsedMs, long queued, long processing, long processed, long baseline, Map<String,Long> sessionEventCounts, Map<String,Long> runEventCounts) extends Record
One line of the progress file.

  • Constructor Details

    • ProgressRecord

      public ProgressRecord(String type, String crawlerId, Instant at, long elapsedMs, long queued, long processing, long processed, long baseline, Map<String,Long> sessionEventCounts, Map<String,Long> runEventCounts)
      Creates an instance of a ProgressRecord record class.
      Parameters:
      type - the value for the type record component
      crawlerId - the value for the crawlerId record component
      at - the value for the at record component
      elapsedMs - the value for the elapsedMs record component
      queued - the value for the queued record component
      processing - the value for the processing record component
      processed - the value for the processed record component
      baseline - the value for the baseline record component
      sessionEventCounts - the value for the sessionEventCounts record component
      runEventCounts - the value for the runEventCounts record component

  • Method Details

    • toString

      public final String toString()
      Returns a string representation of this record class. The representation contains the name of the class, followed by the name and value of each of the record components.
      Specified by:
      toString in class Record
      Returns:
      a string representation of this object

    • hashCode

      public final int hashCode()
      Returns a hash code value for this object. The value is derived from the hash code of each of the record components.
      Specified by:
      hashCode in class Record
      Returns:
      a hash code value for this object

    • equals

      public final boolean equals(Object o)
      Indicates whether some other object is "equal to" this one. The objects are equal if the other object is of the same class and if all the record components are equal. Reference components are compared with Objects::equals(Object,Object); primitive components are compared with '=='.
      Specified by:
      equals in class Record
      Parameters:
      o - the object with which to compare
      Returns:
      true if this object is the same as the o argument; false otherwise.

    • type

      public String type()
      Returns the value of the type record component.
      Returns:
      the value of the type record component

    • crawlerId

      public String crawlerId()
      Returns the value of the crawlerId record component.
      Returns:
      the value of the crawlerId record component

    • at

      public Instant at()
      Returns the value of the at record component.
      Returns:
      the value of the at record component

    • elapsedMs

      public long elapsedMs()
      Returns the value of the elapsedMs record component.
      Returns:
      the value of the elapsedMs record component

    • queued

      public long queued()
      Returns the value of the queued record component.
      Returns:
      the value of the queued record component

    • processing

      public long processing()
      Returns the value of the processing record component.
      Returns:
      the value of the processing record component

    • processed

      public long processed()
      Returns the value of the processed record component.
      Returns:
      the value of the processed record component

    • baseline

      public long baseline()
      Returns the value of the baseline record component.
      Returns:
      the value of the baseline record component

    • sessionEventCounts

      public Map<String,Long> sessionEventCounts()
      Returns the value of the sessionEventCounts record component.
      Returns:
      the value of the sessionEventCounts record component

    • runEventCounts

      public Map<String,Long> runEventCounts()
      Returns the value of the runEventCounts record component.
      Returns:
      the value of the runEventCounts record component