OpenSearch for On-Prem

Overview

IQ Server can optionally use OpenSearch as the backend for Advanced Search. If OpenSearch is not configured, IQ Server uses its built-in on-disk search index.

Prerequisites

IQ Server 204 or later
OpenSearch 2.x (tested with 2.19.2)
Network connectivity from IQ Server to the OpenSearch instance
An OpenSearch user with permissions to create and manage indices

Configuration

Add a search block to your IQ Server config.yml file.

Add a search block to your IQ Server config.yml file.

Self-managed OpenSearch (type: http)
Self-managed OpenSearch (type: http)

Self-Managed OpenSearch (type: http)

Use this for OpenSearch instances that use a URI and username/password authentication.

search:
  type: http
  mode: OPENSEARCH
  uri: https://opensearch.example.com:9200
  username: iq-user
  password: changeme

Amazon OpenSearch Service (`type: aws`)

Use this for Amazon OpenSearch Service deployments.

Authentication is handled through AWS credentials.

search:
  type: aws
  mode: OPENSEARCH
  domain: https://my-domain.us-east-1.es.amazonaws.com:443
  region: us-east-1

AWS credentials are resolved using the standard AWS SDK default credentials chain:

environment variables, system properties, ~/.aws/credentials, ECS container credentials, or EC2 instance profiles

Note

Changing the search index configuration requires a re-index.

Existing search indexes are not migrated automatically when switching between the built-in search index and OpenSearch.

Property Reference

Self-Managed OpenSearch (type: http)

Property	Required	Default	Description
mode	Yes	N/A	Must be set to `OPENSEARCH`
uri	Yes	N/A	OpenSearch connection URI. Both `http://` and `https://` are supported.
username	Yes	N/A	OpenSearch username used for authentication.
password	Yes	N/A	OpenSearch password used for authentication.
bulkBatchSize	No	10000	Number of documents per bulk indexing batch. Must be greater than or equal to `1`
bulkBatchDelayMs	No	0	Delay in milliseconds between bulk batches. Must be greater than or equal to `0`
bulkMaxRetries	No	0	Maximum retry attempts for failed bulk operations. Must be greater than or equal to `0`
bulkRetryBackoffSeconds	No	0	Initial backoff duration in seconds between retries. Must be greater than or equal to 0. Capped at `30` seconds.

Amazon OpenSearch Service (type: aws)

Property	Required	Default	Description
mode	Yes	N/A	Must be set to `OPENSEARCH`
domain	Yes	N/A	OpenSearch domain URI. Must use `https://`
region	Yes	N/A	AWS region (for example, `us-east-1` or `eu-west-1`)
maxConcurrency	No	50	Maximum concurrent connections to OpenSearch. Must be greater than or equal to `1`
connectionTimeout	No	30s	Timeout for establishing new connections.
connectionAcquisitionTimeout	No	10s	Timeout for acquiring a connection from the pool.
bulkBatchSize	No	5000	Number of documents per bulk indexing batch. Must be greater than or equal to `1`
bulkBatchDelayMs	No	250	Delay in milliseconds between bulk batches. Must be greater than or equal to `0`
bulkMaxRetries	No	15	Maximum retry attempts for failed bulk operations. Must be greater than or equal to `0`
bulkRetryBackoffSeconds	No	5	Initial backoff duration in seconds between retries. Must be greater than or equal to `0`. Capped at `600` seconds.

TLS / SSL

For self-managed OpenSearch deployments:

<https:// > enables TLS. The JVM trust store is used to validate the OpenSearch certificate. If OpenSearch uses a self-signed or internal CA certificate, add the certificate to the JVM trust store.
<http:// > disables TLS. Only use this in trusted network environments.
For Amazon OpenSearch Service deployments, HTTPS is required.

Index Naming

IQ Server creates an index alias named iq-index that points to a backing index named iq-index-<uuid>. These indices are managed automatically and should not be modified directly.

Removing OpenSearch

To stop using OpenSearch, remove the search block from config.yml and restart IQ Server.

IQ Server will use the built-in search index.

Bulk Indexing Tuning

The default bulk indexing settings are suitable for most deployments.

If you need to tune bulk indexing behavior:

bulkBatchSize - Larger batches are more efficient but use more memory. Reduce this value if OpenSearch is resource-constrained.
bulkBatchDelayMs - Adds a delay between batches to reduce load on OpenSearch. Increase this value if OpenSearch is under heavy load.
bulkMaxRetries and bulkRetryBackoffSeconds Control retry behavior for transient bulk indexing failures. Backoff uses exponential delay and is capped at the configured maximum.