Slow indexing
Indexing latency on one or more nodes stayed above your configured threshold for enough consecutive samples. Slow writes can fill the write queue and eventually cause indexing rejections.
For a complete list of insights, refer to AutoOps insights.
| Field | Value |
|---|---|
| Component | Elasticsearch |
| Severity | High |
| Scope | Node |
| Domains | performance, indexing |
You can customize these settings to adjust when AutoOps detects this event and presents the insight. Refer to AutoOps event settings for details.
The default customization settings are:
| Setting | Type | Default |
|---|---|---|
| Indexing latency threshold (ms) | Integer | 80 |
| Consecutive samples above threshold | Integer | 1 |
Raising these thresholds reduces noise but delays detection. Lowering them triggers the insight sooner but can increase alerts during minor blips.
The following is an example of what you might see when this insight is triggered. Real insights use live data and links from your deployment or cluster.
Indexing latency on es-data-01 and es-data-02 stayed above your configured threshold for enough consecutive samples. Peak latency over the evaluated window was 420 ms.
- Indices with high indexing activity:
logs-prod-000045
AutoOps shows different recommendations depending on how their conditions match your deployment or cluster.
Rollover indices
Condition: Shown when index has less primary shards than available data nodes.
If logs-prod-000045 are time-based, roll them over and set 1 primary shards on the new write index so you can avoid a heavy segment merge on the current indices.
Rollover indices by shard size
Condition: Shown when primary shard size > 30GB.
If logs-prod-000045 are time-based, roll them over when average shard size reaches your target.
Review index templates and mappings
Condition: Shown for indexes with highest indexing latency.
Review templates and mappings for logs-prod-000045; they strongly affect indexing performance. Fix oversized mappings, unnecessary fields, and inefficient index settings.
Review indexing slow logs
Condition: Always shown for this insight.
Enable indexing slow logs with the action below on logs-prod-000045, then review the logs to find slow index operations. On Elasticsearch 8.14+, set index.indexing.slowlog.include.user to true to record which user triggered a slow operation.
PUT logs-prod-000045/_settings
{
"index.indexing.slowlog.threshold.index.warn": "10s",
"index.indexing.slowlog.threshold.index.info": "5s",
"index.indexing.slowlog.threshold.index.debug": "2s",
"index.indexing.slowlog.threshold.index.trace": "500ms",
"index.indexing.slowlog.source": "1000"
}
Requires the manage index privilege. Requires Elasticsearch 8.0.0 or later. This action changes cluster or index configuration.
Add data node
Condition: Shown when index has more primary shards as available data nodes.
Add a data node to increase capacity and reduce pressure on the existing nodes.
Review templates and ILM
Condition: Shown when indexing is happening on non-hot data tiers.
Review index templates and ILM policies so new write indices stay on the hot tier until rollover is appropriate. Premature moves to colder tiers usually mean the ILM policy is misconfigured.
When each index operation takes too long, the write queue fills and indexing requests can be rejected. Common causes include large or complex documents, costly mappings or analyzers, a high ingest rate, disk I/O pressure, and competing background work on the same node.
If latency stays high, ingest falls behind and client retries can add more load.