Wazuh Indexer Setup Plugin — Development Guide

This document describes how to extend the Wazuh Indexer setup plugin to create new index templates and index management policies (ISM) for OpenSearch.

📦 Creating a New Index

1. Add a New Index Template

Create a new JSON file in the directory: /plugins/setup/src/main/resources

Follow the existing structure and naming convention. Example:

{
  "index_patterns": ["<pattern>"],
  "mappings": {
    "date_detection": false,
    "dynamic": "strict",
    "properties": {
      <custom mappings and fields>
    }
  },
  "order": 1,
  "settings": {
    "index": {
      "number_of_shards": 1,
      "number_of_replicas": 1
    }
  }
}

2. Register the Index in the Code

Edit the constructor of the SetupPlugin class located at: /plugins/setup/src/main/java/com/wazuh/setup/SetupPlugin.java

Add the template and index entry to the indices map. There are two kind of indices:

Stream index. Stream indices contain time-based events of any kind (alerts, statistics, logs…).
Stateful index. Stateful indices represent the most recent information of a subject (active vulnerabilities, installed packages, open ports, …). These indices are different of Stream indices as they do not contain timestamps. The information is not based on time, as they always represent the most recent state.

/**
* Main class of the Indexer Setup plugin. This plugin is responsible for the creation of the index
* templates and indices required by Wazuh to work properly.
*/
public class SetupPlugin extends Plugin implements ClusterPlugin {

  // ...

  // Stream indices
  this.indices.add(new StreamIndex("my-stream-index-000001", "my-index-template-1", "my-alias"));
  // State indices
  this.indices.add(new StateIndex("my-state-index", "my-index-template-2"));

  //...
}

✅ Verifying Template and Index Creation After building the plugin and deploying the Wazuh Indexer with it, you can verify the index templates and indices using the following commands:
curl -X GET <indexer-IP>:9200/_index_template/
curl -X GET <indexer-IP>:9200/_cat/indices?v

Alternatively, use the Developer Tools console from the Wazuh Dashboard, or your browser.

🔁 Creating a New ISM (Index State Management) Policy

1. Add Rollover Alias to the Index Template

Edit the existing index template JSON file and add the following setting:

"plugins.index_state_management.rollover_alias": "<index-name>"

2. Define the ISM Policy

Refer to the OpenSearch ISM Policies documentation for more details.

Here is an example ISM policy:

{
  "policy": {
    "policy_id": "<index-name>-rollover-policy",
    "description": "<policy-description>",
    "last_updated_time": <unix-timestamp-in-milliseconds>,
    "schema_version": 21,
    "error_notification": null,
    "default_state": "rollover",
    "states": [
      {
        "name": "rollover",
        "actions": [
          {
            "rollover": {
              "min_doc_count": 200000000,
              "min_index_age": "7d",
              "min_primary_shard_size": "25gb"
            }
          }
        ],
        "transitions": []
      }
    ],
    "ism_template": [
      {
        "index_patterns": [
          "wazuh-<pattern1>-*"
          // Optional additional patterns
          // "wazuh-<pattern2>-*"
        ],
        "priority": <priority-int>,
        "last_updated_time": <unix-timestamp-in-milliseconds>
      }
    ]
  }
}

3. Register the ISM Policy in the Plugin Code

Edit the IndexStateManagement class located at: /plugins/setup/src/main/java/com/wazuh/setup/index/IndexStateManagement.java

// ISM policy name constant (filename without .json extension)
static final String MY_POLICY = "my-policy-filename";

...

/**
 * Constructor
 *
 * @param index    Index name
 * @param template Index template name
 */
public IndexStateManagement(String index, String template) {
    super(index, template);
    this.policies = new ArrayList<>();

    // Register the ISM policy to be created
    this.policies.add(MY_POLICY);
}

📌 Additional Notes

Always follow existing naming conventions to maintain consistency.

Use epoch timestamps (in milliseconds) for last_updated_time fields.

ISM policies and templates must be properly deployed before the indices are created.

All event data streams share a single base template: templates/streams/events.json. At deployment time, the plugin generates one index template per event category by dynamically setting the index_patterns and rollover_alias fields from the base template. This means:

Source of truth: Only events.json exists in the repository.
At runtime: One index template is created for each category (e.g., wazuh-events-v5-cloud-services-template, wazuh-events-v5-security-template, etc.).

The StreamIndex class handles this: when constructed with only an index name (no explicit template path), it defaults to templates/streams/events and rewrites the index_patterns and rollover_alias to match the specific index.

How it works

// Single-arg constructor defaults to the shared events template
new StreamIndex("wazuh-events-v5-cloud-services")
// Equivalent to:
new StreamIndex("wazuh-events-v5-cloud-services", "templates/streams/events")

During createTemplate(), the plugin:

Reads events.json from the classpath
Overrides index_patterns to ["wazuh-events-v5-cloud-services*"]
Overrides rollover_alias to "wazuh-events-v5-cloud-services"
Creates the composable index template in OpenSearch

Verifying deployed templates

To list all event templates in a running cluster:

GET /_index_template/wazuh-events-*

Specialized stream templates

Some data streams use their own dedicated templates instead of the shared events.json:

Data Stream	Template	Notes
`wazuh-events-raw-v5`	`templates/streams/raw.json`	Stores original unprocessed events
`wazuh-events-v5-unclassified`	`templates/streams/unclassified.json`	Stores uncategorized events for investigation
`wazuh-active-responses`	`templates/streams/active-responses.json`	Active Response execution requests

These are registered with the two-arg constructor:

new StreamIndex("wazuh-events-raw-v5", "templates/streams/raw")
new StreamIndex("wazuh-events-v5-unclassified", "templates/streams/unclassified")
new StreamIndex("wazuh-active-responses", "templates/streams/active-responses")

🚀 Unclassified Events Data Stream (`wazuh-events-v5-unclassified`)

Overview

The wazuh-events-v5-unclassified data stream is a specialized stream designed to capture and store events that do not match any predefined event categories. This provides visibility into edge cases, parsing failures, and events that may require new categorization rules.

Purpose

Investigation and Troubleshooting: Analyze uncategorized events to identify patterns or issues
Rule Development: Identify events that need new categorization rules
System Monitoring: Track parsing failures and anomalies

Data Stream Configuration

Index Template

Location: plugins/setup/src/main/resources/templates/streams/unclassified.json
Index Pattern: wazuh-events-v5-unclassified*
Rollover Alias: wazuh-events-v5-unclassified
Priority: 1 (higher priority than standard event streams for proper template selection)

Fields Included

@timestamp: Event timestamp
event.original: Raw, unprocessed event data
wazuh.agent.*: Agent metadata (id, name, version, type)
wazuh.cluster.*: Cluster information (name, node)
wazuh.space.name: Wazuh space/tenant information
wazuh.schema.version: Schema version
wazuh.integration.*: Integration metadata (category, name, decoders, rules)

Storage Settings

Number of Shards: 3
Number of Replicas: 0
Auto-expand Replicas: 0-1
Refresh Interval: 5 seconds
Dynamic Mapping: Strict (prevents unintended field creation)

ISM Policy

Policy Details

Policy Name: stream-unclassified-events-policy
Location: plugins/setup/src/main/resources/policies/stream-unclassified-events-policy.json
Retention Period: 7 days
Priority: 100

Policy States

Hot State
- Actions: None (events are immediately indexed)
- Transition Condition: Transitions to delete after 7 days
Delete State
- Actions: Deletes the index
- Retry Policy: 3 attempts with exponential backoff (1-minute initial delay)

Use Cases

Event Classification Issues
- Events that failed to match any category
- Malformed or unusual event formats
Parsing Failures
- Events that couldn’t be decoded properly
- Invalid event structures
Rule Development
- Analyzing patterns that require new rules
- Edge cases not covered by existing rules
System Diagnostics
- Understanding integration performance
- Identifying missing integrations or decoders

Configuration

The data stream is created automatically during plugin initialization. Ensure:

The template file unclassified.json exists in templates/streams/
The ISM policy file stream-unclassified-events-policy.json exists in policies/
Both are registered in SetupPlugin.java and IndexStateManagement.java

Indexing Unclassified Events

To index events into this data stream, use:

POST /wazuh-events-v5-unclassified/_doc
{
  "@timestamp": "2024-02-19T10:00:00Z",
  "event": {
    "original": "raw uncategorized event data"
  },
  "wazuh": {
    "agent": {
      "id": "001",
      "name": "agent-name"
    },
    "space": {
      "name": "default"
    }
  }
}

Monitoring and Analysis

Query Unclassified Events

GET /wazuh-events-v5-unclassified/_search
{
  "query": {
    "match_all": {}
  }
}

Count Events by Agent

GET /wazuh-events-v5-unclassified/_search
{
  "size": 0,
  "aggs": {
    "events_by_agent": {
      "terms": {
        "field": "wazuh.agent.id",
        "size": 100
      }
    }
  }
}

Time-based Analysis

GET /wazuh-events-v5-unclassified/_search
{
  "size": 0,
  "aggs": {
    "events_over_time": {
      "date_histogram": {
        "field": "@timestamp",
        "interval": "1h"
      }
    }
  }
}

Testing

Integration tests for the unclassified data stream are located at: plugins/setup/src/test/java/com/wazuh/setup/UnclassifiedEventsIT.java

These tests verify:

Data stream creation
Template application
ISM policy creation and application
Document indexing capability
Correct field mappings

🚀 Active Responses Data Stream (`wazuh-active-responses`)

Overview

The wazuh-active-responses data stream stores Active Response execution requests generated when monitor triggers match their conditions. This is part of the Active Response 5.0 integration with Wazuh XDR, using the Indexer Alerting and Notifications plugins as the foundation.

Purpose

Active Response Pipeline: Structured and auditable execution pipeline for Active Response actions
Manager Retrieval: The Wazuh manager retrieves documents from this index to distribute and execute Active Responses on agents
Event Correlation: Each document references the source event (document ID and index) that triggered the response

Data Stream Configuration

Index Template

Location: plugins/setup/src/main/resources/templates/streams/active-responses.json
Index Pattern: wazuh-active-responses*
Rollover Alias: wazuh-active-responses
Priority: 1

Fields Included (WCS-compatible)

@timestamp: When the document was inserted into the wazuh-active-responses index (indexing time)
event.doc_id: Document ID of the matched alert that triggered the active response
event.index: Source index of the matched alert
wazuh.active_response.name: Name of the active response configured in the channel
wazuh.active_response.executable: Executable configured in the active response channel
wazuh.active_response.extra_arguments: Arguments configured in the channel
wazuh.active_response.location: Where to execute (local, defined-agent, all)
wazuh.active_response.agent_id: Agent configured in the channel
wazuh.active_response.type: Response type (stateless, stateful)
wazuh.active_response.stateful_timeout: Seconds configured in the channel (for stateful)
wazuh.agent.*: Agent metadata
wazuh.cluster.*: Cluster information
wazuh.space.name: Wazuh space/tenant information

ISM Policy

Policy Details

Policy Name: stream-active-responses-policy
Location: plugins/setup/src/main/resources/policies/stream-active-responses-policy.json
Retention Period: 3 days
Priority: 100

Configuration

The data stream is created automatically during plugin initialization. Ensure:

The template file active-responses.json exists in templates/streams/
The ISM policy file stream-active-responses-policy.json exists in policies/
Both are registered in SetupPlugin.java and IndexStateManagement.java

Testing

Integration tests for the active responses data stream are located at: plugins/setup/src/test/java/com/wazuh/setup/ActiveResponsesIT.java

Keyboard shortcuts

Wazuh Indexer technical documentation