Package com.norconex.crawler.fs.fetch.impl.s3

Class S3FetcherConfig

java.lang.Object
com.norconex.crawler.core.fetch.BaseFetcherConfig
com.norconex.crawler.fs.fetch.impl.BaseAuthNioFetcherConfig
com.norconex.crawler.fs.fetch.impl.s3.S3FetcherConfig
public class S3FetcherConfig extends BaseAuthNioFetcherConfig

Configuration for S3Fetcher.

Credentials are optional: when not set, the AWS SDK's default credential provider chain is used instead (environment variables, ~/.aws/credentials, an EC2/ECS/Lambda IAM role, etc.) - the normal way AWS-hosted workloads authenticate without embedding keys in configuration.

  • Constructor Details

    • S3FetcherConfig

      public S3FetcherConfig()

  • Method Details

    • getRegion

      public String getRegion()
      The AWS region the bucket lives in (e.g. us-east-1). If not set, the AWS SDK's default region provider chain is used.

    • getEndpoint

      public String getEndpoint()
      Custom endpoint URL, for S3-compatible services (e.g. MinIO, Ceph) rather than real AWS S3.

    • isForcePathStyle

      public boolean isForcePathStyle()
      Whether to use path-style bucket addressing (endpoint/bucket/key) instead of the AWS-default virtual-hosted style (bucket.endpoint/key). Most S3-compatible services (e.g. MinIO) require this to be enabled.

    • setRegion

      public S3FetcherConfig setRegion(String region)
      The AWS region the bucket lives in (e.g. us-east-1). If not set, the AWS SDK's default region provider chain is used.
      Returns:
      this.

    • setEndpoint

      public S3FetcherConfig setEndpoint(String endpoint)
      Custom endpoint URL, for S3-compatible services (e.g. MinIO, Ceph) rather than real AWS S3.
      Returns:
      this.

    • setForcePathStyle

      public S3FetcherConfig setForcePathStyle(boolean forcePathStyle)
      Whether to use path-style bucket addressing (endpoint/bucket/key) instead of the AWS-default virtual-hosted style (bucket.endpoint/key). Most S3-compatible services (e.g. MinIO) require this to be enabled.
      Returns:
      this.

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class BaseAuthNioFetcherConfig

    • canEqual

      protected boolean canEqual(Object other)
      Overrides:
      canEqual in class BaseAuthNioFetcherConfig

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class BaseAuthNioFetcherConfig

    • toString

      public String toString()
      Overrides:
      toString in class BaseAuthNioFetcherConfig