GenericRedirectUrlProvider

Resolves the target URL from HTTP redirect responses.

crawler/webJavadoc: GenericRedirectUrlProvider

Notes

Provides the resolved redirect URL when the crawler encounters HTTP redirect responses. Supports a configurable fallback character set for cases where the redirect location cannot be determined from response headers.

Usage

YAML

class: GenericRedirectUrlProvider
fallbackCharset: string

JSON

{
  "class": "GenericRedirectUrlProvider",
  "fallbackCharset": "string"
}

XML

<redirectUrlProvider>
  <class>GenericRedirectUrlProvider</class>
  <fallbackCharset>string</fallbackCharset>
</redirectUrlProvider>

Properties

Property	Type	Required	Default
fallbackCharset	string	No	UTF_8

Property Details

fallbackCharset

The character encoding to use when the redirect URL character set cannot be determined from the HTTP response headers. Default is UTF-8.