FeaturedImageResolver

Extracts and optionally scales a featured image from crawled web pages.

crawler/webJavadoc: FeaturedImageResolver

Notes

Resolves and optionally saves a featured image from crawled web pages. Supports filtering by minimum dimensions, scaling, format conversion, and storage as inline data, disk file, or URL reference in document metadata.

Examples

The following example extracts the first image being 300x400 or larger, scaling it down to be 50x50 and storing it as an inline JPEG in a document field, preserving aspect ratio and using the best quality possible.

YAML

class: FeaturedImageResolver
minDimensions:
  width: 300
  height: 400
scaleDimensions:
  width: 50
  height: 50
imageFormat: jpg
scaleQuality: MAX
storages:
  - INLINE

JSON

{
  "class": "FeaturedImageResolver",
  "minDimensions": {
    "width": 300,
    "height": 400
  },
  "scaleDimensions": {
    "width": 50,
    "height": 50
  },
  "imageFormat": "jpg",
  "scaleQuality": "MAX",
  "storages": [
    "INLINE"
  ]
}

XML

<class>FeaturedImageResolver</class>
<minDimensions>
  <width>300</width>
  <height>400</height>
</minDimensions>
<scaleDimensions>
  <width>50</width>
  <height>50</height>
</scaleDimensions>
<imageFormat>jpg</imageFormat>
<scaleQuality>MAX</scaleQuality>
<storages>
  <storage>INLINE</storage>
</storages>

Usage

Full configuration skeleton, with every option and its default

YAML

class: FeaturedImageResolver
domSelector: string
imageCacheSize: 0
imageFormat: string
largest: false
minDimensions: string
pageContentTypePattern: string
scaleDimensions: string
scaleQuality: AUTO
scaleStretch: false
storageDiskDir: string
storageDiskField: string
storageDiskStructure: URL2PATH
storageInlineField: string
storageUrlField: string
storages:
  - URL

JSON

{
  "class": "FeaturedImageResolver",
  "domSelector": "string",
  "imageCacheSize": 0,
  "imageFormat": "string",
  "largest": false,
  "minDimensions": "string",
  "pageContentTypePattern": "string",
  "scaleDimensions": "string",
  "scaleQuality": "AUTO",
  "scaleStretch": false,
  "storageDiskDir": "string",
  "storageDiskField": "string",
  "storageDiskStructure": "URL2PATH",
  "storageInlineField": "string",
  "storageUrlField": "string",
  "storages": [
    "URL"
  ]
}

XML

<featuredImageResolver>
  <class>FeaturedImageResolver</class>
  <domSelector>string</domSelector>
  <imageCacheSize>0</imageCacheSize>
  <imageFormat>string</imageFormat>
  <largest>false</largest>
  <minDimensions>string</minDimensions>
  <pageContentTypePattern>string</pageContentTypePattern>
  <scaleDimensions>string</scaleDimensions>
  <scaleQuality>AUTO</scaleQuality>
  <scaleStretch>false</scaleStretch>
  <storageDiskDir>string</storageDiskDir>
  <storageDiskField>string</storageDiskField>
  <storageDiskStructure>URL2PATH</storageDiskStructure>
  <storageInlineField>string</storageInlineField>
  <storageUrlField>string</storageUrlField>
  <storages>
    <storage>URL</storage>
  </storages>
</featuredImageResolver>

Properties

Property	Type	Required	Default
domSelector	string	No	-
imageCacheSize	integer	No	200
imageFormat	string	No	png
largest	boolean	No	false
minDimensions	string	No	400x400
pageContentTypePattern	string	No	text/html|application/(xhtml\+xml|vnd\.wap.xhtml\+xml|x-asp)
scaleDimensions	string	No	150x150
scaleQuality	enum	No	AUTO
scaleStretch	boolean	No	false
storageDiskDir	string	No	-
storageDiskField	string	No	crawler.featured-image-path
storageDiskStructure	enum	No	-
storageInlineField	string	No	crawler.featured-image-inline
storages	enum[]	No	[URL]
storageUrlField	string	No	crawler.featured-image-url

Property Details

domSelector

An optional CSS-like selector to target specific image elements within a page. When not set, all <img> elements in the page are considered.

imageCacheSize

The maximum number of images to keep in memory cache to avoid re-downloading and re-scaling the same images across multiple pages. Set to 0 to disable caching. Default is 200.

imageFormat

The target image format for stored images (e.g., jpg, png, gif, bmp). Default is png.

largest

When enabled and multiple qualifying images are found on a page, the largest image (by dimensions) is selected as the featured image. By default, the first qualifying image encountered is used.

minDimensions

The minimum pixel dimensions (width and height) an image must meet to be considered as the featured image. Default is 400x400.

pageContentTypePattern

A regular expression matching the content types of pages from which featured images will be extracted. Default matches HTML and XHTML content types.

scaleDimensions

The target pixel dimensions (width and height) to scale the featured image to before storing it. Default is 150x150.

scaleQuality

The image scaling quality. Options are AUTO (default, balances quality and speed), LOW, MEDIUM, HIGH, and MAX. Lower quality is faster but produces less sharp images.

Allowed Values

• AUTO
• LOW
• MEDIUM
• HIGH
• MAX

scaleStretch

When enabled, the image is stretched to exactly match the scaleDimensions, ignoring aspect ratio. By default, the image is scaled while preserving its aspect ratio.

storageDiskDir

The directory path where featured images are saved to disk when DISK is one of the configured storages. Defaults to a featuredImages subdirectory of the crawler working directory.

storageDiskField

The metadata field name where the local file path of the saved image is stored when using DISK storage. Default is the crawler's built-in featured-image-path field.

storageDiskStructure

The directory structure to use when saving images to disk with DISK storage. Options are URL2PATH (directories mirror URL segments), DATE (organized by date), and DATETIME (organized by date and time). Default is URL2PATH.

Allowed Values

• URL2PATH
• DATE
• DATETIME

storageInlineField

The metadata field name where the Base64-encoded inline image string is stored when using INLINE storage. The string is ready to use in an HTML <img src="..."> tag. Default is the crawler's built-in featured-image-inline field.

storages

One or more storage types for the featured image. Supported values are URL (default, stores the image URL as metadata), INLINE (stores a Base64-encoded image in metadata), and DISK (saves the image to the file system).

storageUrlField

The metadata field name where the absolute URL of the featured image is stored when using URL storage. Default is the crawler's built-in featured-image-url field.