What Is OpenSearch and How Does It Support SIEM Use Cases?

OpenSearch can serve as the core of a Security Information and Event Management (SIEM) system. OpenSearch is an open source search and analytics platform that is well-suited for ingesting, storing, and analyzing large volumes of security logs and events. 

Key aspects of using OpenSearch for SIEM include:

  • Log centralization: OpenSearch can centralize logs from diverse sources, including network devices, servers, applications, and cloud services.
  • Schema flexibility: It does not require a predefined schema, allowing for the direct ingestion of semi-structured data and accommodating new log types and schema changes without disrupting data ingestion.
  • Threat detection: OpenSearch enables the application of detection rules to identify suspicious activities and potential security incidents within the ingested data.
  • Alerting and visualization: It can generate alerts based on detected threats and provides visualization tools (like OpenSearch Dashboards) for exploring and analyzing security data.
  • Scalability and performance: OpenSearch is designed for scalability and offers low-latency queries, which are crucial for efficient investigative workflows in a SIEM context.
  • Cloud integration: It is particularly well-suited for multi-cloud environments, as it can effectively handle logs originating from various cloud platforms.

While OpenSearch provides the foundational search and analytics capabilities, a complete SIEM solution built upon it may also integrate with other tools for advanced features like incident response automation or compliance reporting.

In this article:

  • Understanding OpenSearch as a SIEM Solution
  • Tutorial: Getting Started with OpenSearch SIEM
  • Best Practices for Using OpenSearch Security Analytics (SIEM)

Understanding OpenSearch as a SIEM Solution

OpenSearch provides core capabilities needed to build a scalable, open source SIEM solution. Through its security analytics plugin, OpenSearch supports real-time threat detection, centralized log analysis, and compliance monitoring.

Threat detection

OpenSearch enables real-time detection of suspicious activity by ingesting logs, applying detection rules, and triggering alerts. The security analytics plugin includes prebuilt detectors for different log types, such as authentication, network, and application logs, mapped to adversary tactics defined by the MITRE ATT&CK framework.

Detection rules define conditions that indicate threats, such as repeated failed logins followed by administrative access. OpenSearch ships with a library of Sigma rules, and users can also create custom rules. When a rule matches incoming log data, the system generates a finding. Alerts can be configured to notify teams via email, Slack, or other channels.

Machine learning can also enhance detection. Using the ML Commons plugin, OpenSearch supports anomaly detection, such as identifying unusual login times or access patterns.

Event correlation

The platform supports multi-source event correlation, linking signals across different logs to form a broader picture of complex attacks. For example, a VPN login from a new location followed by server access and abnormal network traffic could be identified as a single incident. Correlation rules define these multi-step scenarios, and results can be visualized as graphs to aid investigation.

Log analysis

OpenSearch can aggregate, normalize, and index large volumes of log data from various sources. Analysts can perform searches across all logs (current and historical) from one interface. This allows efficient investigation, threat hunting, and troubleshooting.

Data normalization ensures consistent field names and formats across logs. Built-in mappings and ingest pipelines simplify this process, making it easier to write general-purpose detection rules and dashboards. Analysts can use OpenSearch Dashboards for ad hoc queries, drilldowns, and visualizations.

Compliance monitoring

OpenSearch helps organizations meet regulatory requirements by maintaining centralized audit trails and enabling real-time compliance alerts. Logs related to access, configuration changes, and user behavior are retained and searchable, making it easier to demonstrate adherence to standards like PCI DSS, HIPAA, GDPR, and ISO 27001.

Custom rules can trigger alerts for compliance violations, such as unauthorized access or after-hours activity. Audit logging features track access to the SIEM itself, ensuring that log integrity is maintained and that all access is monitored.

Dashboards can be customized to support reporting, and saved visualizations can be exported for audits. OpenSearch also supports log retention policies needed to meet data retention requirements.

Tutorial: Getting Started with OpenSearch SIEM

This tutorial shows how to set up OpenSearch for SIEM use cases. Instructions are adapted from the OpenSearch documentation.

Introduction: Working with Log Types

Log types define the data sources used in security analytics and determine which detection rules and field mappings are available when creating detectors. They help categorize incoming logs and ensure consistent processing across different sources.

Security analytics supports two categories of log types:

  • Standard log types: Automatically recognized by OpenSearch, with prebuilt mappings and rules. These cover common sources grouped into categories such as access management (AD/LDAP, Okta), applications (GitHub, Microsoft 365), cloud services (AWS CloudTrail, Azure), network activity (DNS, NetFlow, VPC Flow), security (WAF), and system activity (Linux, Windows).
  • Custom log types: Used when a data source does not match a standard type. Administrators can create a custom log type, assign it to a category, and define its mappings to make it usable for detection rules.

Each log type has a details page showing its ID, category, source, and associated detection rules. Custom types can be deleted, while standard types cannot.

Field mappings are tied to the selected log type. Security analytics attempts to map log fields to Sigma rule fields automatically, but mappings can also be adjusted manually. Accurate mappings improve the quality of findings and ensure detection rules work correctly.

In addition to the user interface, log type management can be automated with REST APIs, allowing creation, retrieval, and modification of custom log types through scripts or external systems.

1. Creating Detectors

Detectors define what threats to monitor and how to respond. Setting one up involves two main steps: defining the detector and configuring alerts.

Step 1. Define a detector

A detector starts with a name, optional description, and a data source. The data source can be an index, alias, or data stream, but all selected sources must be of the same log type. It is best practice to create separate detectors per log type.

Once a log type is chosen, OpenSearch may generate a default dashboard with visualizations for that source. Available detection rules are prepopulated for the log type, and users can enable, disable, or filter them by severity or source. Field mappings are then reviewed: OpenSearch attempts to align log fields with rule fields automatically, but adjustments can be made manually to improve accuracy. Threat intelligence feeds can also be enabled for standard log types.

The detector is scheduled to run at defined intervals (e.g., every few minutes). The schedule determines how often rules are evaluated against incoming logs.

Step 2. Set up alerts

Alerts define how findings trigger notifications. A trigger is created by selecting detection rules, severities, and tags as conditions. Each trigger can be linked to one or more notification channels, such as email, Slack, or Amazon Chime. Messages can be customized with rule details, severity, and tags. Multiple triggers can be added per detector.

By default, creating a detector also provisions a composite monitor in the alerting plugin, which executes rule queries on the chosen schedule. This integration can be enabled or disabled through cluster settings.

Field mapping

Field mappings connect log fields to Sigma rule fields. OpenSearch maps these automatically but allows manual adjustments. Mapping relies on standards such as the Elastic Common Schema (ECS) and, for AWS CloudTrail and DNS, the Open Cybersecurity Schema Framework (OCSF). Predefined mappings for supported log types are published in OpenSearch documentation and the security analytics GitHub repository.

Amazon Security Lake support

Logs ingested from Amazon Security Lake are converted to OCSF format. Security analytics can map OCSF fields to ECS automatically, simplifying detector creation for sources like AWS CloudTrail, Amazon Route 53 (DNS), and VPC Flow Logs.

2. Creating Correlation Rules

Correlation rules define multi-step threat scenarios across different log sources. They allow security analytics to detect patterns that span multiple systems, such as suspicious activity in authentication logs followed by abnormal network traffic.

Configuring rules

Each correlation rule requires at least two queries, each tied to a different data source. More queries can be added to model complex scenarios. To create a rule:

  1. In OpenSearch Dashboards, go to Security Analytics → Correlation Rules and select Create correlation rule.
  2. Enter a name for the rule.
  3. For each query:
    • Select an index or index pattern.
    • Assign a log type to the index.
    • Choose a field and define its value. Additional fields can be added to refine the query.
  4. Add at least a second query to cover another log source. Continue adding queries if the scenario requires more.
  5. Select Create correlation rule. The rule is saved and appears in the list, where it can be viewed or edited later.

Setting a time window
The correlation engine evaluates whether findings from different queries occur within a defined time window. By default, this window is five minutes, but it can be adjusted using the Cluster Settings API. For example:

 

In this example, correlations are only detected if the defined events happen within two minutes of each other.

Best Practices for Using OpenSearch Security Analytics (SIEM)

Here are some important practices to keep in mind when using OpenSearch for SIEM.

1. Normalize Log Data Before Ingestion

Normalizing log data simplifies security analytics in OpenSearch. Logs from different sources can vary widely in format, field names, and encoding, which complicates detection rule development and querying. Using ingest pipelines to map fields, convert timestamps, and standardize message structures at the point of ingestion creates a unified log schema. This makes it easier to apply generic rules and run cross-dataset queries.

Normalization also aids in compliance and reporting, as consistent log fields support structured output and easier aggregation. Tools like Data Prepper or Logstash can be integrated to perform transformations, enrichments, and mappings before data lands in OpenSearch. Organizations should document their normalization logic and regularly review it as new log sources are added.

2. Create Layered Detection Rules to Reduce False Positives

Layered detection is the practice of deploying multiple, complementary detection rules to increase resilience against false positives and missed threats. Individual rules may be too generic or sensitive, leading to alert fatigue, but layering several rules, such as basic event detection, anomaly thresholds, and correlation logic, enables more precise security outcomes. 

OpenSearch supports custom pipelines that can stage detections, with higher-priority alerts escalated only when multiple conditions are satisfied. When designing layered rules, security teams should analyze common attack patterns and simulate both benign and malicious scenarios to calibrate thresholds. 

Documenting the reasoning behind each rule layer and continuously testing their effectiveness ensures the system adapts to changes in the threat landscape.

3. Regularly Update Correlation Rules and Detectors

Adversaries constantly evolve their tactics, making it essential to update and refine your correlation rules and detectors. Regular reviews ensure your SIEM remains effective against new threats and adjusts to changes in your infrastructure. This includes adapting to new application deployments, cloud integrations, or regulatory requirements. 

Using automated rule updates or subscribing to community rule feeds can give your team a head start in responding to emerging techniques. Security operations teams should establish a cadence for reviewing detection logic, analyzing recent alerts for gaps or redundancies, and sunset outdated rules. Proper change management and documentation ensure updates are controlled and auditable.

4. Implement Fine-Grained Access Controls

Fine-grained access control is vital for protecting sensitive security data within OpenSearch. By defining granular roles and permissions, organizations can limit who can view, edit, or delete SIEM configurations, dashboards, and underlying log data. 

OpenSearch supports integration with identity providers, role-based access control (RBAC), and document-level security to enforce these restrictions. Properly implemented, this prevents unauthorized access and reduces risks from insider threats or accidental actions.

Access control should be regularly audited to reflect staffing changes, organizational restructuring, or changes in data sensitivity. Segregating duties between security, infrastructure, and development teams reduces the risk of privilege misuse. Assigning the minimum necessary permissions and monitoring access patterns help maintain compliance with internal policies and external regulations such as GDPR or HIPAA.

5. Monitor System Performance and Resource Utilization

Monitoring the health of the OpenSearch cluster is critical for maintaining reliable SIEM operations. Large log volumes, complex queries, and high alert throughput can strain resources and impact detection latencies. 

It’s essential to track cluster metrics such as CPU, memory, disk usage, and indexing rates using built-in monitoring features or external tools like Prometheus and Grafana. Regular performance reviews help identify bottlenecks and justify scaling out clusters or adjusting index strategies.

Resource allocation should be planned to accommodate data growth and changing analytics workloads. Implementing automated alerting for cluster health issues provides early warning for intervention before outages occur. Routine optimization of index mappings, shard allocation, and retention policies ensures efficient resource utilization.

Instaclustr for OpenSearch: Powering Your SIEM Strategy

Spotting threats in real-time requires more than just vigilance; it requires the right tools to turn massive data streams into actionable intelligence. For many security teams, OpenSearch has become the engine of choice for SIEM. Its ability to ingest, index, and search through vast amounts of log data at lightning speed makes it ideal for identifying anomalies before they become breaches. Whether you are aggregating logs from distributed applications, monitoring network traffic, or visualizing security postures through sophisticated dashboards, OpenSearch gives you the raw power needed to stay ahead of bad actors.

However, running a mission-critical SIEM platform shouldn’t mean getting bogged down in infrastructure management. This is where Instaclustr for OpenSearch transforms your security operations. By delivering OpenSearch as a fully managed service, we remove the operational burden of deployment, patching, and complex scaling. You get a production-ready environment that is secure by default, allowing your security analysts to focus on what matters most: interpreting data, hunting threats, and refining compliance reporting, rather than troubleshooting cluster health or worrying about node failures.

We understand that when it comes to security, downtime is not an option. That’s why our managed platform is built for extreme reliability and high availability. With Instaclustr, you can confidently scale your SIEM capabilities to handle petabytes of security logs without missing a beat. We ensure your data is always accessible for real-time analysis and historical auditing, giving you the peace of mind that comes from a robust, performant, and compliant security infrastructure. You bring the data and the expertise; we’ll make sure the engine never stops running.

For more information: