Understanding OpenSearch® vector field types—Part 1: knn_vector

Overview

In today’s AI-driven landscape, users expect applications to behave like intelligent assistants—understanding intent, context, and natural language rather than just matching exact terms. This requires combining keyword-based search, which excels at term-level relevance, with semantic search, which captures deeper meaning. Modern applications such as chatbots, recommendation systems, and Retrieval-Augmented Generation (RAG) pipelines all depend on this semantic layer, and this is exactly where OpenSearch’s knn_vector field type becomes essential.

knn_vector enables developers to store machine learning embeddings and perform similarity search based on meaning rather than exact words, laying the groundwork for scalable semantic search.

This is a developer-focused two-part series. Part 1 covers OpenSearch’s knn_vector field type, and Part 2 will explore the sparse_vector field type.

OpenSearch vector field types

OpenSearch provides two dedicated vector field types for building intelligent search:

• knn_vector: Enables semantic similarity search through dense embeddings
• sparse_vector: Enables neural sparse search through token-weighted sparse representations

Together, these two field types complement each other, making them essential for building production-ready AI applications that are designed for accuracy, scalability, and context-awareness.

Running OpenSearch on the NetApp Instaclustr Managed Platform helps reduce the operational burden of managing infrastructure and scaling, allowing teams to focus entirely on building smarter search experiences and will be used for the purposes of this blog series.

Now, let’s take a closer look at knn_vector, how to configure it, and how to use it in practice.

Getting started with knn_vector on Instaclustr

What is knn_vector?

knn_vector is OpenSearch’s dedicated field type for storing vector embeddings and performing semantic similarity search. It enables developers to represent content as dense numerical vectors and retrieve results based on meaning rather than exact keyword matches, making it ideal for building intelligent, context-aware applications.

When defining a knn_vector field in an index mapping, several parameters can be configured to tune search behavior. These include:

• Distance metric (space_type): Determines how similarity is measured. Common options include Euclidean distance (l2) and cosine similarity (cosinesimil).
• Engine: The underlying library powering the search. Supported engines are Facebook AI Similarity Search (FAISS is the default value) and Lucene. Note that the Non-Metric Space Library (NMSLIB) was deprecated as of OpenSearch 2.16 and removed in OpenSearch 3.0.
• Method name (algorithm): Defines how the vector index is built and searched. Common options include Hierarchical Navigable Small World (HNSW) supported by both Faiss and Lucene and Inverted File Index (IVF) supported via Faiss.
• Data type: Vector elements can be stored as float (default), byte, or binary depending on your precision and performance requirements.

This knn_vector vector field can be built in two ways:

• by providing a method definition, where you explicitly specify the engine, algorithm, and space_type parameters, or
• by specifying a model ID, where the field inherits its configuration (including the dimension) from a pre-trained model already registered in OpenSearch.

When using the method definition approach, the dimension parameter is required and must match the output size of your embedding model. Refer to the documentation for a list of supported dimensions by model.

Note: To use this field type, you must first enable k-NN search in the index settings by setting "index.knn": true and then define the knn_vector field parameters in the index mapping.

Practical example

First, create an OpenSearch cluster on Instaclustr by following the steps demonstrated in the video. When provisioning the cluster, make sure to enable either the k-NN Plugin (Similarity Search Engine) or the AI Search Plugin (enabling the AI Search Plugin will automatically enable the k-NN plugin as well).

The code snippets below are implemented using the Dev Tools console in OpenSearch Dashboards. To understand the process, the index is always created first because the knn_vector field mapping must be in place before any data is ingested, as it defines how OpenSearch stores and indexes the vector embeddings. Once the index is ready, documents are ingested with their corresponding vector embeddings. A k-NN query is then executed to retrieve results ranked by similarity score, where a higher score indicates a closer semantic match. Together, these three steps form the foundation of a working semantic search pipeline.

Step 1: Creating a vector index

The following code creates an instaclustr_knn_demo index with k-NN search enabled and defines an embedding field to store 4-dimensional vectors. For this example, we did not define the optional parameters described in the above section. When these parameters are not explicitly defined, OpenSearch applies the following default values:

This example is suitable for quick testing and experimentation.

Note: For production use, define these parameters explicitly to match your embedding model and performance requirements. Also, remember to update the dimension value to match your embedding model’s output size. Real-world models typically produce vectors of 384 to 1536 dimensions.

Expected output:

Figure 1. Index created successfully with k-NN enabled

Step 2: Indexing a document

The next step is to index a document by providing the text content and its corresponding vector embedding generated by your embedding model.

The embedding vector in the code below is a manually written placeholder used for demonstration. In practice, these embedding values are produced by an embedding model—a machine learning model that converts text into a fixed-size list of numbers representing its meaning. To index different texts, pass the text inputs through an embedding model to generate its corresponding vector. A popular option for generating embeddings is the sentence-transformers Python library.

Note: Use the same model for both indexing and querying. Using different models at each stage can produce meaningless similarity scores because the vectors cannot be meaningfully compared.

Expected output:

Figure 2. Document indexed successfully with vector embedding

Step 3: Running a semantic search query

With documents indexed, you can run a k-NN query by passing a query vector to retrieve the most semantically similar results.

Expected output:

Figure 3. Semantically similar results returned with similarity scores.

Conclusion and next steps

The knn_vector field type provides the foundation for building AI-powered applications by enabling semantic search at scale. However, dense semantic search alone does not tell the whole story. Neural sparse search, powered by sparse_vector field type, adds a complementary layer of precision that is particularly effective when term-level relevance matters—and that is exactly what Part 2 will cover.

If you are building AI-driven applications, now is the time to get started with semantic search—create a free OpenSearch cluster on Instaclustr, follow the code snippets in this article, and run your first semantic search query.

Stay tuned for Part 2 of this series, where we will focus on the sparse_vector field type and explore how it enables neural sparse approximate nearest neighbor (ANN) search for even better search efficiency and precision.