OpenSearch 3.0 Enhances Vector Database Performance, Search Infrastructure and Scalability to Meet AI-driven Demand

Latest iteration bolsters open, scalable, community-driven search and analytics, enabling sustainable innovation

The OpenSearch Software Foundation, the vendor-neutral home for the OpenSearch Project, today announced the general availability of OpenSearch 3.0. This major release delivers a 9.5x performance improvement over OpenSearch 1.3, building on benchmarking data that showed earlier iterations of OpenSearch operating 1.6x faster than its closest industry competitor.

Today's AI applications — like generative AI, hybrid search, retrieval-augmented generation (RAG) and recommendation engines — rely heavily on vector databases to find patterns in massive, complex datasets, but as the number of vectors explodes into the billions, many organizations struggle with speed, spend and scalability. Forrester emphasizes that traditional databases are no longer able to meet the growing demands of generative AI due to limitations in supporting modern vector multidimensional data and performing similarity searches.

OpenSearch 3.0 addresses this challenge and enables users to increase efficiency, deliver superior performance, and accelerate AI application development via new data management, AI agent, and vector search capabilities. Simultaneously, enhancements such as GPU-supported capabilities can reduce costs by 3.75x.

"The enterprise search market is skyrocketing in tandem with the acceleration of AI, and it is projected to reach $8.9 billion by 2030," said Carl Meadows, Governing Board Chair at the OpenSearch Software Foundation and Director of Product Management at Amazon Web Services (AWS). "OpenSearch 3.0 is a powerful step forward in our mission to support the community with an open, scalable platform built for the future of search and analytics, and it reflects our commitment to open collaboration and innovation that drives real-world impact."

Vector engine innovations increase processing speed and efficiency

To support its large-scale search platform and manage a vast amount of vector data, OpenSearch introduced GPU-based acceleration, leveraging NVIDIA cuVS for indexing workflows. New vector engine features include:

1. GPU Acceleration for OpenSearch Vector Engine: Delivers superior performance for large-scale vector workloads while significantly lowering operational spend by reducing index building time. By enabling GPU deployment, this experimental feature heightens performance for data-intensive workloads and accelerates index builds by up to 9.3x.
   
   
2. Model Context Protocol (MCP) Support: Native MCP support allows AI agents to easily communicate with OpenSearch, enabling more comprehensive and customizable AI-powered solutions.
   
   
3. Derived Source: Reduces storage consumption by one-third by removing redundant vector data sources and utilizing primary data to recreate source documents as needed for reindexing or source call back.

Data management features optimize resources, enhance flexibility and drive scalability
OpenSearch 3.0 provides major advancements in how the platform ingests, transports and manages data including:

1. Support for gRPC: Enables faster and more efficient data transport and data processing for OpenSearch deployments. This experimental feature provides a new approach to data transport between clients, servers, and node-to-node communications in OpenSearch.
   
   
2. Pull-based Ingestion: Enhances ingestion efficiency and gives OpenSearch more control over the flow of data and when it's retrieved by decoupling data sources and data consumers. This experimental feature also allows users to pull data from streaming systems like Apache Kafka and Amazon Kinesis.
   
   
3. Reader and Writer Separation: Ensures consistent, high-quality performance for indexing and search workloads by configuring each in isolation, allowing both workloads to work at optimal speed and scale, rather than decreasing in efficiency when the other is taxed.
   
   
4. Apache Calcite Integration: Enables intuitive, iterative query building and exploration by integrating the query builder into OpenSearch SQL and PPL. Simplifies use cases for security, observability and log analysis.
   
   
5. Index Type Detection: Enhances productivity by automatically determining whether an OpenSearch index contains log-related data and speeding up log analysis feature selection.

Core upgrades help future-proof OpenSearch's search platform and analytics suite
Enhancements to the platform's search infrastructure – removing legacy code, adopting a modular architecture and aligning with the latest Java advancements – boosts maintainability, performance potential, and efficiency. Updates include:

1. Lucene 10 Upgrade: Modernizes the platform's search infrastructure to ensure long-term innovation, improve indexing and search capabilities, and increase performance of parallel task execution.
   
   
2. Java 21 Minimum Supported Runtime: Enables access to modern language features and performance improvements.
   
   
3. Java Platform Module System Support: Improves organization, eliminates top level split packages and creates a foundation for refactoring the monolithic server module into separable libraries.

OpenSearch 3.0 is now available. See the official release blog for more information and full release notes. To learn more about the OpenSearch Software Foundation, including how to get involved, become a member or contribute, please visit foundation.opensearch.org/.

About the OpenSearch Software Foundation

The OpenSearch Software Foundation is a vendor-neutral community for search, analytics, observability, and vector database software. Hosted by the Linux Foundation and supported by premier members such as AWS, SAP and Uber, the OpenSearch Software Foundation works with community maintainers, developers, and member organizations to drive the continued growth of the OpenSearch project. With more than 900 million software downloads since its inception and participation from thousands of contributors, the OpenSearch project and its community are transforming how information is managed and discovered. To learn more, please visit foundation.opensearch.org/.

The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see our trademark usage page. Linux is a registered trademark of Linus Torvalds.

Media Contact
Kristi Piechnik
The Linux Foundation
kpiechnik@linuxfoundation.org