Package com.norconex.committer.azurecognitivesearch

Class AzureSearchCommitter

java.lang.Object
com.norconex.committer.core.AbstractCommitter<AzureSearchCommitterConfig>
com.norconex.committer.core.batch.AbstractBatchCommitter<AzureSearchCommitterConfig>
com.norconex.committer.azurecognitivesearch.AzureSearchCommitter
All Implemented Interfaces:
BatchConsumer, Committer, Configurable<AzureSearchCommitterConfig>, AutoCloseable
public class AzureSearchCommitter extends AbstractBatchCommitter<AzureSearchCommitterConfig>

Commits documents to Microsoft Azure Search.

Document reference encoding

By default the document reference (Azure Search Document Key) is encoded using URL-safe Base64 encoding. This is Azure Search recommended approach when a document unique id can contain special characters (e.g. a URL). If you know your document references to be safe (e.g. a sequence number), you can set AzureSearchCommitterConfig.setDisableDocKeyEncoding(boolean) to true. To otherwise store a reference value un-encoded, you can additionally store it in a field other than your reference ("id") field.

Single vs multiple values

Fields with single value will be sent as such, while multi-value fields are sent as array. If you have a field defined as an array in Azure Search, sending a single value may cause an error.

It is possible for values to always be sent as arrays for specific fields. This is done using AzureSearchCommitterConfig.setArrayFields(String). It expects comma-separated-value list or a regular expression, depending of the value you set for AzureSearchCommitterConfig.setArrayFieldsRegex(boolean).

Field names and errors

Azure Search will produce an error if any of the documents in a submitted batch contains one or more fields with invalid characters. To prevent sending those in vain, the committer will validate your fields and throw an exception upon encountering an invalid one. To prevent exceptions from being thrown, you can set AzureSearchCommitterConfig.setIgnoreValidationErrors(boolean) to true to log those errors instead.

An exception will also be thrown for errors returned by Azure Search (e.g. a field is not defined in your Azure Search schema). To also log those errors instead of throwing an exception, you can set AzureSearchCommitterConfig.setIgnoreResponseErrors(boolean) to true.

Field naming rules

Those are the field naming rules mandated for Azure Search (in force for Azure Search version 2016-09-01): Search version

  • Document reference (ID): Letters, numbers, dashes ("-"), underscores ("_"), and equal signs ("="). First character cannot be an underscore.
  • Document field name: Letters, numbers, underscores ("_"). First character must be a letter. Cannot start with "azureSearch". Maximum length is 128 characters.
Author:
Pascal Essiembre