The OpenSearch 3.4 release is here, bringing groundbreaking improvements to AI and RAG pipelines. From performance boosts to new aggregation features, this update is a game-changer for developers and data scientists alike. For broader coverage of all the updates check out the project’s website.

Even just focusing on AI and RAG, there’s some major improvements and new features to cover:

  • Performance enhancements for search, especially at the Lucene API level
  • A lot on aggregations—from a dashboard feature to a real-time context signal for agentic/RAG pipelines

Performance enhancements for AI-powered search

The OpenSearch contributors have been working on a variety of performance enhancements for 3.4 that are very exciting; enhancements that make production-scale OpenSearch clusters faster and more responsive, and some really neat optimization of the way OpenSearch uses Lucene 10 APIs to speed up aggregation and certain searches.

The GitHub meta ticket mentions a few interesting updates, including skip-list-based optimizations for aggregations. We’ll cover aggregations in detail in a bit, but it’s safe to say they’re even more performant in 3.4. The meta ticket on GitHub explains that aggregations should be faster when:

  • Filters apply on fields other than the aggregation field
  • Sub-aggregations are involved
  • Segments contain deleted documents

This is a critical improvement for real-time analytics, dashboards, and more. And this isn’t a tweak or micro-optimization—according to the ticket, the skip-list approach “achieves up to 2x improvements across broader workloads” than the previous implementation.

Lucene 10.3 made a large set of improvements itself, including new APIs to handle bulk collection. These calls are being implemented in OpenSearch 3.4, and while the improvements are hard to measure, the estimated impact is largest on aggregations, which is the subject of the next part of this post!

From a dashboard feature to a real-time context signal for AI search: aggregations

Aggregations are a core feature for AI search, especially agentic and RAG pipelines: They allow you to compute summaries, statistics, and insights from your indexed data without retrieving every document, which in RAG pipelines means:

  • A reduction in compute load: instead of retrieving every document, RAG uses aggregations to more efficiently search for documents.
  • Context: RAG heavily depends on context, and aggregations, when readily available and easily searchable, gives a web of context for your OpenSearch data.
  • Data-based decisions: RAG pipelines and AI agents need to be able to easily make data-grounded decisions, and aggregations allow you to quickly and efficiently gain necessary insights.

Aggregations saw a major performance boost from several directions in OpenSearch 3.4:

  • The skip-list improvement mentioned earlier directly impacts aggregation performance in certain key AI use cases
  • Aggregations also saw a boost from the bulk collection APIs in Lucene 10.3, which are used in this update.

Aggregations also saw a major feature ship in 3.4: streaming aggregations. This is made possible through OpenSearch’s integration of Apache Flight and Arrow for streaming capabilities. Of the many things being put into streams this version, aggregations are one of the most exciting for AI, RAG pipelines, and agentic AI systems:

  • Context becomes real-time, data is pushed live instead of queued for a REST API call
  • Better hybrid search and re-ranking, as these both use aggregation data to be more efficient

You can learn more about the aggregations supported in 3.4 on the GitHub ticket for GRPC support.

These features and improvements take aggregations from a narrower use case of “dashboards, analytics” and expands it to cover a broad suite of RAG pipeline use cases. The bottom line is better, faster aggregations reduce query latency, increase scalability, and reduce cost by reducing compute needs, and provide richer structured signals for LLMs and re-rankers—key components of agentic and RAG pipelines.

In conclusion, between general performance improvements and large improvements to aggregations, OpenSearch 3.4 is really exciting in the AI and RAG pipeline space.

Explore the full capabilities of OpenSearch 3.4 and start building smarter RAG pipelines today. Learn more on our website.