Zilliz, a leading AI data infrastructure company and the creator of Milvus, recently announced Loon, the new storage engine that powers Zilliz Vector Lakebase and ships in Milvus 3.0. Loon is the lake-native foundation that lets a single copy of vector data serve real-time search, large-scale discovery, and batch analytics at once — the storage layer behind Zilliz Cloud's evolution from a vector database into a unified data platform for AI.
Zilliz launches Loon, lake-native storage engine for Milvus 3.0 and Vector Lakebase.
Single copy of vector data serves real-time search, discovery, and batch analytics.
Loon's Vortex-based layout reduces data pulled per record read by ~135x vs Parquet.
Supports hybrid file formats: Parquet for scalars, Vortex for vectors, object storage for media.
Powers more than 10,000 enterprises including MiniMax, OpenEvidence, Filevine, Exa, and Salesforce.
Available across 30+ regions on AWS, Google Cloud, and Azure.
To make that possible, Loon treats a vector dataset as what it actually is — physically heterogeneous — and is built on three ideas. Hybrid file formats store each kind of column in the format that fits it: scalar and filter fields use Parquet for efficient scans; dense and sparse vectors use the open Vortex format for fast, byte-precise row-level reads on object storage; and raw videos, PDFs, and images stay in object storage, referenced rather than copied into the database. Row ID alignment ensures columns split across different formats still behave as one logical table, so a new embedding model can be added as its own column without rewriting the captions, metadata, or vectors already stored. A versioned Manifest serves as a single source of truth defining the dataset's current version, so serving clusters, on-demand compute, and external engines such as Spark and Ray can all read and safely update the same dataset instead of maintaining separate copies.
In Zilliz's internal testing of object storage, Loon's Vortex-based layout reduced the data pulled per record read by about 135x compared to Parquet — the difference between practical and impractical low-latency serving on inexpensive object storage. And because the same data evolves in place, adding a new embedding model becomes a lightweight version update rather than a multi-hundred-gigabyte rewrite. This architecture lets one copy of data on object storage feed many engines at once.
Vector Lakebase is built on a demanding premise: one logical copy of vector data should serve every AI workload — production search, discovery, and batch analytics — without copying or moving data between systems. The hardest part is the storage layer, because the same dataset has to behave like two systems at once: fast, record-level lookups for serving and wide scans for analytics, all on inexpensive object storage. It also has to handle data that never stops changing, as teams re-embed, re-label, and re-index the same records while their models improve.
James Luan, Cofounder and CTO of Zilliz, stated: "Vector retrieval is no longer the whole problem; Vector Lakebase is our answer to what happens after vector databases succeed. The systems that win will make continuous serving and continuous discovery feel like part of the same machine — and that only works when the storage layer can serve a single copy of data to every workload. Loon is that storage layer."
About Zilliz
Zilliz is a leading AI data infrastructure company and the creator of Milvus, the world's most widely adopted open-source vector database, with 44,000+ GitHub stars and over 100 million Docker pulls. Zilliz helps enterprises and AI startups make their unstructured data searchable, analyzable, and governable — turning text, images, audio, video, and more into a strategic asset for production AI. Zilliz's technology centers on Milvus and Zilliz Cloud. Milvus is an open-source vector database purpose-built for 100-billion-scale vector search. Zilliz Cloud extends that foundation into a fully managed Vector Lakebase platform. Zilliz powers more than 10,000 enterprises and AI-native startups worldwide, including MiniMax, OpenEvidence, Filevine, Exa, Salesforce, and Read AI.
