Instaclustr by NetApp is excited to announce the general availability release of dedicated ingest nodes for OpenSearch® on the Instaclustr Managed Platform.
With general availability you get all the benefits of the preview release of dedicated ingest nodes and the brand-new support for dedicated ingest load balancing! Clusters with load balancers can now route ingest API requests against dedicated ingest nodes to reduce request overheads.
What Are Dedicated Ingest Nodes?
Ingest nodes handle the initial processing of incoming data and serve as an entry point for documents to be ingested into an index, whether it’s for real-time streaming or batch processing. Typically, ingest nodes perform transformation functions and are designed to handle high volumes of data without impacting standard indexing or search performance.
Dedicated ingest nodes allow for the isolation of ingest workloads to specific dedicated hardware, reducing the impact on overall cluster performance:
Traffic for ingest APIs are routed to dedicated ingest nodes when the ingest URL is used on a load balanced cluster (Source: Instaclustr)
Are There Any Considerations To Take Into Account?
Yes, a few. When planning to use dedicated ingest nodes, some of the key points to consider are:
- Dedicated ingest nodes have the sole responsibility of performing ingest operations and will not store the final data, and as such will require far less disk space.
- Clusters with load balancers are encouraged to use the Ingest URL for ingest APIs as it will reduce inter-node communication overheads by routing requests directly to an ingest node.
- The use cases for ingest nodes vary depending on the processor that is used. If you are unsure about whether ingest nodes will suit your workloads, our helpful Support team is available for consultation.
Ready To Spin up an OpenSearch Cluster?
The Instaclustr Managed Platform makes it easy to spin up an OpenSearch cluster with dedicated ingest nodes through our easy-to-use portal with just a few clicks. With our expert Technical Operations staff continuously monitoring those clusters, we aim to give you the peace of mind knowing that your cluster is healthy and operating at expected capacity.
Whether you are an existing customer or have not used the Instaclustr Managed Platform before, we have you covered! If you would like to learn more about more about how dedicated ingest nodes may play a key role in your data pipelines, or how you can leverage our Managed OpenSearch service to empower your platform, we encourage you to reach out to our friendly Support team.