Vector Search in Apache Cassandra for RAG Retrieval

Semantic retrieval using one Cassandra table to understand how it works, where it shines, and what to watch for.

Vector search in Apache Cassandra ® lets teams store embeddings, index them with Storage-Attached Indexing, and run approximate nearest neighbor queries directly in CQL without managing a separate vector database. For RAG retrieval, that means Cassandra can act as both the operational database and the semantic search layer when the retrieval problem is primarily meaning-based.

Search used to require the person asking to already know the answer’s vocabulary. A technician precisely asks, “Recall 24V-113: tailgate latch”; a truck owner asks, “Is there a recall on the tailgate?” Embeddings close that gap by mapping meaning into geometry, so similar ideas land near each other in vector space regardless of the words used. That geometry is the retrieval layer underneath most AI applications. Using retrieval-augmented generation (RAG), a pattern for answering questions from supplied data instead of the model training set alone, the chatbot can be instructed to only answer from the documents vector search hands it. As a result, the retrieval quality sets the ceiling on the answer.

The standard approach utilizes a dedicated vector store alongside the operational database. While Cassandra is not the first operational database to keep embeddings next to the data they describe, version 5.0 made it a native part of the data model. When the use case fits, querying embeddings on the same infrastructure that houses the data can cut both complexity and cost: one database to operate instead of two, and no application code responsible for keeping them in sync.

This is Part 1 of 2. Part 1 demonstrates semantic retrieval against a realistic knowledge base containing vehicle recalls, tow ratings, and warranty terms using Cassandra. Then, we show where Approximate Nearest Neighbor (ANN) alone is not the complete answer, which sets up a different approach, hybrid RAG, in Part 2.

What is vector search on Apache Cassandra?

Vector search on Apache Cassandra lets you store, index, and query high-dimensional embeddings natively in the database using the vector<float, N> column type and Storage-Attached Indexing. Introduced in Cassandra 5.0, it enables ANN search through an ORDER BY … ANN OF clause in CQL, with no separate vector store.

At write time, title + body is embedded and stored in a vector column on the same row as the text. At query time the question is embedded with that same model, and Cassandra is asked for the nearest stored vectors. Three pieces of CQL do the whole job: declare the column, attach an SAI index so ANN is possible, and query with ORDER BY … ANN OF plus an optional similarity score in the select list.

Each piece of that, in turn:

• vector<float, N> is a native CQL column type holding an embedding of N dimensions, sitting on the row it describes rather than in another system. It’s 384 here because all-MiniLM-L6-v2 (which is used for the demo application) produces 384-dimensional vectors; document and query vectors must always use the same model and dimension.
• SAI (Storage-Attached Indexing) attaches a searchable index to the table as rows are written. For vector columns it plugs in JVector, an ANN engine that walks a graph of nearby embeddings for fast top-k retrieval.
• similarity_function accepts COSINE (the default), DOT_PRODUCT, or EUCLIDEAN. Changing it later means dropping and recreating the index.
• ORDER BY embedding ANN OF ? returns approximate nearest neighbors ordered by similarity. This is fast, but not a guarantee of the true global nearest neighbor.
• similarity_cosine(embedding, ?) is optional and exists to make the ranking legible. It returns a 0-to-1 value rather than raw cosine, so 0.5 means a document unrelated and a score of 0.75 is not “three-quarters relevant.”

Writes don’t need special handling because the embedding binds as one more parameter on an ordinary prepared INSERT alongside the document, and SAI indexes it as the row is written, under the replication factor and consistency level you already use. In addition, vectors impose no special replication requirements. The demo application’s keyspace is SimpleStrategy with RF 1 only because it’s a single-node Docker cluster.

Setting the scene

Throughout this series, you own a fictional 2024 Summit 1500 and you’re asking a support bot a question. The knowledge base is fictional but realistic: 21 documents covering 12 recalls, tow ratings, payload figures, maintenance intervals, and warranty terms for Summit 1500 and 2500 pickups.

The recall documents matter more than their count suggests. They share one sentence skeleton, mimicking how a real recall notice would read, making them a clean test of something specific: whether an embedding can resolve an identifier when the surrounding prose is nearly identical.

Everything in this series runs on Apache Cassandra 5.0.9 in Docker, with embeddings generated locally from the all-MiniLM-L6-v2, an embedding model, so there’s no API key to supply.

Companion app: cassandra-vector-demo. Follow the instructions in the README to reproduce the output below. Running the application isn’t required to understand the concepts in this series.

How Apache Cassandra vector search fits into a RAG pipeline

Throughout this demo, you never directly query Cassandra. Rather, you’ve asked a question, expect an answer back, and never see the retrieved document’s similarity scores. Vector search is the middle step that chooses which rows the language model is allowed to read:

1. The application takes the question as text.
2. The same embedding model used at insert time turns that question into a vector.
3. Cassandra returns a small list of nearest documents (ORDER BY embedding ANN OF … LIMIT k).
4. The application copies those titles and bodies into a prompt, next to the original question. That block is the context.
5. The language model writes an answer using (in a well-behaved RAG setup) only that context. It doesn’t scan the knowledge base, only the handful of chunks retrieval picked.

Apache Cassandra vector search examples: Paraphrase, exact IDs, and vague queries

We run three Summit owner questions against the table above and read what Cassandra ranks. The demo doesn’t call an LLM after retrieving the results; the point is whether they are trustworthy before an LLM ever sees it. The first question is a paraphrase, which is what embeddings exist to solve. The other two are the limits you plan around: an exact identifier, and a question too vague to have one answer. Both are properties of similarity search rather than of Cassandra; the same model against Pinecone, pgvector, or OpenSearch kNN should produce the same or similar ordering and both are fixed with a query change rather than a database change. The output below is generated from src/demo.py against Cassandra 5.0.9 with all 21 documents seeded.

Outcome 1: Everyday paraphrase finds the right recall

“Is there a recall on the tailgate?”

Vector search ranks Recall 24V-113 (tailgate latch) first, and the margin is wide: 0.7767 against 0.7032 for the runner-up. Nobody wrote the word “tailgate” into the question expecting a latch recall, and the model bridged it anyway, against 11 other recall documents written in nearly identical prose. Everyday paraphrase is exactly what embeddings are for, and one CQL query against one table is the whole implementation.

Outcome 2: An exact recall ID lands nowhere near the top

“What is recall 24V-330?”

Now you have the recall number.

The document for 24V-330 sits at rank 9, scoring 0.7332, below eight notices that never mention it. The recall notices share one sentence skeleton, so the ID is a handful of characters inside an otherwise near-identical embedding, and ANN ranks by overall closeness. Cosine distance has no notion of an exact token match, which is why the same model produces this ranking in any vector store, and why the fix is a lexical path beside ANN rather than a better embedding model. Part 2 of the series works to address this limitation.

Outcome 3: A question retrieval cannot answer as asked

“How much can the Summit 1500 tow?”

The ranking looks reasonable until you read the bodies. The top four sit within 0.08 of each other and answer four different questions: 11,300 lb for the 2024 1500, 9,100 lb for the 2023 1500, 17,600 lb for the 2500, and 1,750 lb of payload, which isn’t a tow rating at all.

Retrieval did its job since all four are genuinely relevant to what was asked. The question just never supplied a model year, so nothing in the index can pick a winner. The fix is a WHERE model_year = ? alongside ANN, which is exactly why those columns are on the table, and will be utilized in Part 2.

When to use Cassandra vector search in production RAG applications

Vector search is enough when the question is about meaning and an incorrect response is recoverable: everyday paraphrase like “is there a recall on the tailgate,” similar tickets, or similar products already sitting in Cassandra. It isn’t enough when an identifier, a model year, or a customer record has to be exact, and no similarity threshold will separate those two cases for you. Those need a lexical path or a WHERE clause next to ANN, which is the work of Part 2 using additional SAI indexes on this same table, so the fix stays inside Cassandra rather than adding the second system you just avoided.

Neither of those statements is specific to Cassandra. They describe ANN retrieval wherever it runs, and teams building RAG on a dedicated vector store can reach the same two conclusions and apply the same two fixes. What Cassandra changes is the operational surface rather than the ranking: a column and an index on a table you already replicate, back up, and secure.

Native doesn’t mean equivalent. A dedicated search platform still wins on lexical ranking and relevance tuning, but it changes the starting assumption and depending on the use case, a second system doesn’t need to be a default to accept.

Apache Cassandra vector search version and operational notes

Vector search on Cassandra is continuing to mature, so treat patch releases as part of planning.

• Prefer 5.0.7 or later. Earlier 5.0 patches could return stale neighbors after overwrites and tombstones, and WHERE plus ANN queries were prone to timeouts. CASSANDRA-20086 reworked that path in 5.0.7 and 6.0. This project uses 5.0.9. Note that Instaclustr does not support 5.0.7 nor 5.0.8.
• Cassandra 6.0 isn’t GA yet. Alpha1 arrived in March 2026, alpha2 in August. The work since 5.0 has been correctness rather than new capability, so nothing above changes.
• Cassandra 5.0.6 is Generally Available on the NetApp Instaclustr Managed Platform. This works for the demo, which is read-only against a freshly seeded table. Track the platform version list before putting vector writes into production.

Try it yourself

Run the three questions against a live Cassandra 5 table on Docker. The demo application is from cassandra-vector-demo. Clone it, follow the README, and compare your rankings to the output in this series.

The same CQL runs on Instaclustr for Apache Cassandra 5.0. A new cluster takes a few clicks in the console; start a free trial and point the demo at the managed contact point instead of 127.0.0.1. You write the table; NetApp manages the cluster.

Further reading on native vectors: Apache Cassandra 5.0 vector search, similarity search with the vector type.

Part 2 of this series continues with the same Summit table and uses hybrid RAG to improve correctness.