Package com.norconex.crawler.core.cluster

Interface CacheManager

All Known Implementing Classes:
HazelcastCacheManager, MemoryCacheManager, MVStoreCacheManager
public interface CacheManager

  • Method Details

    • getCacheMap

      <T> CacheMap<T> getCacheMap(String name, Class<T> valueType)
      Gets a named cache storing objects of the supplied type. By default the returned cache is persisted, but cache implementations may decide otherwise (based on cache name, configuration, etc.).
      Type Parameters:
      T - type of the value to store
      Parameters:
      name - cache name
      valueType - class of the value to store
      Returns:
      a named cache

    • getCacheSet

      CacheSet getCacheSet(String name)
      As string-based set, keeping only unique keys.
      Parameters:
      name - cache name
      Returns:
      a named cache set

    • cacheExists

      boolean cacheExists(String name)

    • exportCaches

      void exportCaches(Consumer<SerializedCache> c)
      Consumes every caches, where record value are serialized JSON String. Entries are lazy loaded.
      Parameters:
      c - serialized cache.

    • importCaches

      void importCaches(List<SerializedCache> caches)
      Import caches previously exported with exportCaches(Consumer).
      Parameters:
      caches - caches to import

    • clearCaches

      void clearCaches()
      Clears all caches.

    • getCrawlerCache

      CacheMap<String> getCrawlerCache()
      Gets a generic shared cache bound to a crawler that persists forever (never deleted).
      Returns:
      crawler-scoped persistent cache

    • getCrawlSessionCache

      CacheMap<String> getCrawlSessionCache()
      Gets a generic shared cache bound to a crawl session that persists until a new crawl session is started (for the same crawler).
      Returns:
      crawl session-scoped persistent cache

    • getCrawlRunCache

      CacheMap<String> getCrawlRunCache()
      Gets a generic shared cache tied to a specific crawler run and does not get persisted when the crawler run terminates.
      Returns:
      crawl run non-persistent cache

    • getCacheQueue

      <T> CacheQueue<T> getCacheQueue(String name, Class<T> valueType)
      Gets a FIFO queue.
      Type Parameters:
      T - the type of elements in the queue
      Parameters:
      name - the queue name
      valueType - the value type
      Returns:
      cache queue

    • getAdminCache

      CacheMap<String> getAdminCache()
      Gets the admin cache for storing administrative data.
      Returns:
      admin cache

    • getPipelineStepCache

      CacheMap<StepRecord> getPipelineStepCache()
      Gets the pipeline step cache for storing current pipeline steps.
      Returns:
      pipeline step cache

    • getPipelineWorkerStatusCache

      CacheMap<StepRecord> getPipelineWorkerStatusCache()
      Gets the pipeline worker status cache for storing worker statuses.
      Returns:
      pipeline worker status cache

    • addCacheEntryChangeListener

      <T> void addCacheEntryChangeListener(CacheEntryChangeListener<T> listener, String cacheName)
      Adds a cache entry change listener to the specified cache.
      Type Parameters:
      T - the type of values in the cache
      Parameters:
      listener - the listener to add
      cacheName - the name of the cache

    • removeCacheEntryChangeListener

      <T> void removeCacheEntryChangeListener(CacheEntryChangeListener<T> listener, String cacheName)
      Removes a cache entry change listener from the specified cache.
      Type Parameters:
      T - the type of values in the cache
      Parameters:
      listener - the listener to remove
      cacheName - the name of the cache

    • vendor

      Object vendor()
      Gets the vendor-specific implementation object.
      Returns:
      vendor implementation