Hybrid RAG retrieval in Apache Cassandra: SAI filters plus vector search

Introduction

In Part 1 of this two-part series, we built the retrieval layer for a Retrieval Augmented Generation (RAG) support bot on Apache Cassandra 5 vector search. The bot returns results that would be sent to an LLM to answer questions about fictional Summit pickup trucks. The application has access to a knowledge base of 21 documents: 12 recalls plus tow ratings, payload figures, maintenance intervals, and warranty terms for the Summit 1500 and 2500.

Part 1 showed how little it takes to enable vector search in Cassandra 5, and where it works well: questions phrased in everyday language. It also showed where vector search alone can fall short. When a question hinges on an exact identifier, or on a detail the user never states (such as the model year), ANN can return the wrong documents with high similarity scores and no warning.

In Part 2, we borrow an idea from hybrid RAG research: ground retrieval in hard facts first, then let vector search handle meaning. We apply that idea inside the same Cassandra table, with no new systems.

What is hybrid RAG?

Hybrid RAG combines two retrieval methods. The term comes from the 2024 paper HybridRAG: Integrating Knowledge Graphs and Vector Retrieval Augmented Generation for Efficient Information Extraction, by researchers at BlackRock and NVIDIA.

HybridRAG sends the context from both methods to the LLM together. The authors tested it on question answering over financial earnings call transcripts. HybridRAG scored 0.96 on both faithfulness (how well the answer is supported by the retrieved context) and answer relevance. VectorRAG scored 0.94 and 0.91, and GraphRAG scored 0.96 and 0.89. No single method won every metric. HybridRAG had the lowest context precision (0.79), because merging two sets of context also brings in some noise. Overall, though, it gave the best balance.

A note on terminology: in search engines, “hybrid search” usually means combining keyword ranking (BM25) with vector ranking. The approach in this post borrows from both ideas. Exact facts constrain the results, and vectors rank them.

Cassandra and hybrid RAG

Cassandra is not a graph database and has no native GraphRAG. The core idea, grounding the answer in facts and then adding meaning, doesn’t require a graph. Store facts such as model, model year, category, and recall ID as columns on the same row as the embedding. Cassandra can then filter on those facts and rank by vector similarity in a single query. This method keeps the same architecture from Part 1: one database and one table.

Storage-Attached Indexing (SAI) is what makes this work. In Part 1, we used SAI only to index the embedding column, where it uses JVector for approximate nearest neighbor (ANN) search. SAI can also index ordinary columns. SAI indexes are attached to the table’s own storage and are built as memtables and SSTables are written. Compared with Cassandra’s legacy secondary indexes, they are designed to reduce write-time overhead and disk usage. A single query can use several SAI indexes.

In this design, SAI is the filter and ANN is the ranking. One CQL statement can restrict candidates to rows that match exact facts (WHERE) and then order the remaining rows by vector similarity (ORDER BY … ANN OF). The facts decide which documents are eligible, and the vectors decide their order. SAI and vector search are complementary parts of the same query, not competing features.

Setup

We build on the schema from Part 1. The companion app is cassandra-hybrid-retrieval-demo. It uses the same 21 documents and still uses one table, with two changes: a new keywords column, and SAI indexes on the metadata columns.

Here’s what each addition is for:

• keywords is a set<text> of exact terms curated for each document, such as the recall ID (24v-330), model numbers, years, and component names. The app pulls exact tokens out of the question and matches them with keywords CONTAINS ?. Those tokens are identifiers like 24V-330, three- or four-digit numbers like 1500 or 2024, and a short list of domain terms like tow and recall.
• category, model, and model_year are the fact filters, our stand-in for the graph side of hybrid RAG. The app infers category from words in the question. Model and model year represent customer context, such as the truck on file in the customer’s account, rather than something parsed from the question. In the demo, you pass them with --model and --model-year.
• USING 'sai' is required on every index. In Cassandra 5.0, CREATE INDEX without it creates a legacy secondary index, unless the cluster’s default_secondary_index setting has been changed to sai.

Before you design filters, know which query operators SAI supports in Cassandra 5.0:

• Numerics: =, <, >, <=, >=
• Scalar strings: =
• Predicates can be combined with AND
• Collections: CONTAINS, CONTAINS KEY
• Not supported: LIKE, scalar-string substring search

Cassandra 5.0 has no negation operators in WHERE clauses. NOT IN, NOT CONTAINS and NOT CONTAINS KEY arrive in Cassandra 6.0 (CEP-29), usually with ALLOW FILTERING; NOT LIKE was proposed but not included. On 5.0, aside from reversing range comparisons, excluding specific values has to happen in application code.

Three ways to answer the same question

The demo answers each question three ways against the same table, so you can compare the context each approach would hand to an LLM:

1. Keyword: runs keywords CONTAINS ? for each exact token in the question. The app ranks results by how many tokens each document matched. SAI returns matches, not relevance scores, so documents with the same match count are in no meaningful order.
2. Vector ANN: runs ORDER BY embedding ANN OF ?, exactly as in Part 1.
3. Hybrid: runs SAI filters and ANN ordering in one statement. It filters on the most selective token in the question (an identifier such as 24v-330, if there is one), the inferred category, and the customer’s model and year when provided.

For example, this is the hybrid statement the app runs for “What is recall 24V-330?”:

Note: the index keywords is case sensitive while category is case insensitive when we created our indexes in the Setup section.

Revisiting the Part 1 questions

Let’s ask the same three questions from Part 1.

Question 1: “Is there a recall on the tailgate?”

All three approaches rank Recall 24V-113 (tailgate latch) first. Vector search already handled this in Part 1, scoring it 0.7767 against 0.7032 for the runner-up, because embeddings are good at matching meaning. The question contains no identifier, so the hybrid query filters only on category = 'recalls', and ANN still puts the tailgate recall on top.

This baseline is worth keeping. Before adding filters to fix the hard cases, confirm they don’t break the cases vector search already gets right.

Now for the questions where vector search struggled.

Question 2: “What is recall 24V-330?”

In Part 1, vector search ranked the matching document ninth, with a score of 0.7332. That put it below eight other recall notices that never mention 24V-330, even though the question contained the exact recall ID. The 12 recall documents follow one template, so the IDs differ by only a few characters and the embeddings sit very close together. Cosine similarity has no concept of an exact token match.

SAI fills that gap. The keyword lane ranks recall-24v-330 first, because it’s the only document that matches both 24v-330 and recall. Ranks 2 through 5 match only recall, so they aren’t really ranked, just matched. The hybrid query (keywords CONTAINS '24v-330' AND category = 'recalls') returns a single row, and it’s the right one. The identifier decides which documents qualify instead of competing with semantic similarity.

Question 3: “How much can the Summit 1500 tow?”

In Part 1, all four of the top vector results looked relevant, scoring within 0.08 of each other.

The keyword lane does no better. It ranks the 2023 truck first, on a two-way tie that the app breaks by the order results came back rather than by relevance. Without customer context, the hybrid query can filter only on category = 'towing'. That removes the payload document, but it still mixes model years and models.

The question never says which model year, and no ranking method can infer it. When the facts are available, use them to filter first. Here, the customer’s truck is on file. Passing --model 1500 --model-year 2024 narrows the hybrid query to a single row, summit-1500-2024-tow (11,300 lb), with a score of 0.8990. Here’s what the data says:

Retrieval still runs either way. If you don’t check what comes back, it’s easy to miss that the top results describe different trucks. When the application doesn’t know a fact the answer depends on, it should look it up from the customer record or ask the user, rather than letting the LLM choose among conflicting chunks.

Limitations to know before you build

• SAI is not BM25. It returns matches, not relevance scores. A dedicated search engine still wins on full-text relevance, facets, and highlighting.
• Cassandra is not a graph database. Following relationships, such as related recalls or citations, happens in application code.
• Filters only help if the facts are right. The metadata on each row must be accurate, and the application must be able to extract or supply the filter values.

Version note: Queries that combine WHERE filters with ANN had correctness and timeout issues before Cassandra 5.0.7 (CASSANDRA-20086). The demo runs on 5.0.9, but it will also work on Instaclustr’s latest supported version of Cassandra.

Conclusion

If your data already lives in Cassandra 5, combining SAI filters with vector search on the same table lets you do two things in one query: ground retrieval in exact facts and rank results by meaning. You don’t need a separate vector database, or a search cluster just to match identifiers. Embeddings sit on the same row as the data they describe, and SAI indexes each write synchronously, so new rows are searchable as soon as the write is acknowledged. However, be aware that an ANN (top-K) query does not keep the consistency level you asked for: Cassandra downgrades it to LOCAL_ONE when the request is datacenter-local, and to ONE otherwise.

Try it yourself

Run the examples yourself from the companion app on GitHub. Cassandra 5.0.9 runs in Docker, and the Python app (3.10–3.13) runs locally with the all-MiniLM-L6-v2 embedding model, so you don’t need an API key. The first run downloads the model.

You can also run the same CQL on the NetApp Instaclustr Managed Platform with a 30-day free trial, which covers one small cluster at a time. Point the app at your cluster by setting CASSANDRA_HOST to the cluster’s contact point. You will also need to modify your setup to accept authentication to the Instaclustr Cassandra cluster, shown here.