OpenSearch version 3.0 GA is here! What’s new and shiny in this major release
OpenSearch 3.0 is officially GA! This is an exciting time for OpenSearch, as version 3.0 brings a lot of capabilities, while maintaining the scalability and flexibility of OpenSearch. I’ve covered some of the changes over on the OpenSearch blog, and this post will dive just a little bit deeper into the new functionality, performance gains, and more. Specifically, I’ll dig into the Lucene 10 upgrade, OpenSearch dashboard changes, changes to the core code of OpenSearch, performance improvements in OpenSearch 3.0, and finally, I’ll cover migrating from OpenSearch 2.X or Elasticsearch.
These changes will quickly and accurately let you search your data; the performance gains alone will ripple into things like infrastructure cost and newly revamped features will allow your developers and analysts faster insights into your data.
Lucene 10 upgrade
The reasons for updating to Apache Lucene 10 are predominantly related to performance. Lucene 10 is more performant in many categories than its predecessor in OpenSearch 2.x, meaning faster searches for users. There were also major changes to I/O performance and performance of parallel-executing tasks; and there’s even more Lucene 10 brings:
- Sparse indexing: Also known as primary-key indexing, this organizes data into blocks with recorded minimum and maximum sizes. This allows queries to more efficiently skip non-relevant blocks when querying, which increases CPU/storage efficiency. This was made possible through the I/O parallelism improvements mentioned earlier, allowing OpenSearch users to search faster and get more done in less time.
- KNN/Neural search improvements: Because of the improvements to parallel searches we covered previously, parallelized execution of KNN and neural searches has improved in Lucene 10 as well. Lucene 10 also adds improvements to indexing vectors; better parallelization of I/O allows for optimizations in terms of how vectors are stored, creating a much more scalable neural search engine. This coincides well with the rise of in-house LLM and AI building, as these improvements make OpenSearch a stronger candidate for searching AI-related vector data or other AI/ML applications.
You can learn more about Lucene 10 improvements from its changelog.
OpenSearch dashboard changes
The OpenSearch dashboard has seen a lot of changes in 2024, including releasing a new UI. The updates in OpenSearch 3.0 include OpenSearch Workspaces, allowing you to assign individual accounts different dashboard views, and a totally revamped discover tool. These changes allow users to tailor data visualizations to each person’s needs on the dashboard and write queries against their OpenSearch data easier and faster by improving UX and other functionality.
Core changes in OpenSearch code
The OpenSearch server code monolith has been broken up into modular pieces to increase the speed at which OpenSearch can be developed, making the code system easier to navigate and understand. For OpenSearch maintainers and committers, this means a streamlining of the PR process. Instead of having to navigate hundreds of changes to the server monolith, these changes can be more clearly tracked in their respective breakout classes.
This is a monumental undertaking by the OpenSearch team, and you can track their changes on this ticket.
Performance in OpenSearch 3.0
Performance in OpenSearch wasn’t just boosted by Lucene 10. The OpenSearch team has been hard at work fine-tuning and overhauling OpenSearch 3.0 to make it as performant as possible.
From increasing performance by 20% on the Big5 queries in 2.19, to being up to 8.4x more performant in many cases than OpenSearch 1.X, and even higher increases in other cases, OpenSearch 3.0 has brought its full sights on the goal of better performance.
You can learn more about these changes from this GitHub ticket.
Migrating from OpenSearch 2.X and Elasticsearch to version 3.0
Migrating from OpenSearch 2.19 to OpenSearch 3.0 is a one-step process, but the same cannot be said for Elasticsearch. For instance, migrating from Elasticsearch to OpenSearch 3.0 is a multi-step process (Elasticsearch to OpenSearch 2.19, then to OpenSearch 3.0). If you’re thinking about making the switch, the time to is now: there are tools and guides available, but the OpenSearch project has made several innovations since the initial fork, and the more time passes, the more that divergence will make migration harder. Our team can help you migrate—contact us!
I hope you’ve enjoyed this whirlwind tour of the new OpenSearch 3.0! To learn more about OpenSearch, check out these YouTube videos:
NetApp Instaclustr is committed to releasing OpenSearch 3.0 on our platform as soon as possible, but in the meantime, you can try out OpenSearch 2.19 with the free trial.
