AI InfrastructureGuides & Tutorials

What Is a Vector Database? Embeddings and Search Explained

A vector database stores embeddings and finds the stored items nearest to a query by meaning. Here is how embeddings, distance metrics and HNSW indexes work, and a rule for choosing exact search, pgvector or a dedicated store.

Toolbit AI - Team
6 min read
What Is a Vector Database? Embeddings and Search Explained

Short answer: A vector database stores embeddings, which are lists of numbers that capture meaning, and finds the stored items nearest to a query. That lets search match ideas instead of exact words, which is the retrieval step behind most RAG systems.

You may not need a separate one. Exact search in memory or in Postgres covers many projects, and this guide ends with a rule for choosing.

How does a vector database work?

A vector database works in four steps: embed the data, store the vectors, compare them with a distance metric, and index them for speed.

  1. Embed. OpenAI defines an embedding as "a vector (list) of floating point numbers." Its text-embedding-3-small returns 1536 numbers by default, and text-embedding-3-large returns 3072. The dimensions parameter shortens them, which OpenAI calls "the suggested approach."
  2. Store. Each record holds the vector, an ID and metadata. Qdrant calls this a point, with a payload that is "a JSON object with additional data." Qdrant groups points into collections, "named sets of points that you search within", and you choose the distance metric when you create one. Pinecone allows 40 KB of metadata per record.
  3. Compare. A metric scores how close two vectors are. pgvector offers L2 distance, negative inner product and cosine distance, as the operators <->, <#> and <=>. OpenAI embeddings are normalized to length 1, so cosine and Euclidean distance "will result in the identical rankings."
  4. Index. Exact search compares the query with every stored vector. An approximate index such as HNSW checks far fewer and gives up some recall.
Similar ideas cluster together, so a query is answered by the nearest points around it

Each dot is one embedded item. The search returns the points closest to the query, because closeness stands for related meaning. If you are still choosing a model, see our guide to choosing an embedding model.

Why do results change after you add an index?

Results change because an approximate index trades some recall for speed, and pgvector documents this. By default, pgvector "performs exact nearest neighbor search, which provides perfect recall." Its README also warns that "you will see different results for queries after adding an approximate index."

HNSW is the common approximate index. The original paper compares its layered graph to a skip list and reports "logarithmic complexity scaling." Instead of scanning every vector, the search follows links between nearby points.

Exact scan checks every point, while HNSW follows a few graph hops to nearly the same answer

The exact scan finds every true neighbor but does the most work. The graph skips most comparisons and finds almost all of them. pgvector also offers IVFFlat, a second index type that divides vectors into lists and searches only the lists closest to the query. These are pgvector's defaults:

SettingWhat it controlspgvector default
mMax connections per layer16
ef_constructionCandidate list size while building the graph64
hnsw.ef_searchCandidate list size at query time40
IVFFlat listsNumber of partitionsStart at rows / 1000 up to 1M rows, sqrt(rows) above

If recall drops too far, raise hnsw.ef_search: pgvector says a higher value gives better recall at the cost of speed.

Filters are the common surprise. pgvector applies them "after the index is scanned." If 10% of rows match and ef_search is 40, you get about 4 matching rows on average. Iterative scans fix this: they "automatically scan more of the index until enough results are found." The strict_order mode keeps exact distance order, while relaxed_order allows "slightly out of order" results for better recall. Turn them on with SET hnsw.iterative_scan = strict_order; (or relaxed_order).

Which kind of vector store fits your stack?

Vector stores fall into three types, and the difference is who runs the index.

TypeExampleWho runs itWhat the docs say
Embedded libraryChromaYour programApache 2.0; the in-memory client loses data when your program ends; Chroma Cloud is managed and serverless
Dedicated serverPinecone, QdrantThe vendor (Pinecone; Qdrant Cloud)Pinecone indexes dense, sparse or both vector fields, plus BM25 full-text search; Qdrant's free tier is 0.5 vCPU, 1GB RAM and 4GB disk, without high availability
Postgres extensionpgvectorYour existing databasePostgres 13+; "Store your vectors with the rest of your data"

Qdrant also offers Qdrant Edge, which runs inside your application process. It is in beta as of August 2026, and Qdrant says the API may change. For paid plans on any of these, check the vendor page.

Try it yourself with Chroma's quickstart

Chroma's quickstart is the fastest way to see search by meaning.

  1. Run pip install chromadb.
  2. Create a client, then a collection.
  3. Call collection.add with five short documents.
  4. Call collection.query with a question that shares no words with the best answer.

Chroma "will store your text and handle embedding and indexing automatically." Check whether the top result matches by meaning rather than by shared words.

Do you need a vector database? A decision rule

Start with exact search, and add an index or a dedicated store only when you have a reason. This is Toolbit's rule, built from the vendor docs above.

  • Exact tokens matter: for SKUs, error messages and names, use full-text search. Pinecone's indexing docs describe BM25 as the fit for these.
  • Your data lives in Postgres: use pgvector with exact search first. For filtered queries, the README starts with an index on the filter column. For normalized vectors, it recommends inner product "for best performance."
  • A prototype in one process: an exact scan in memory, or Chroma, is enough.
  • Exact search is too slow, or you want it run for you: add HNSW, or move to Pinecone or Qdrant. OpenAI says: "For searching over many vectors quickly, we recommend using a vector database."

Retrieval is one part of the job. What you put in front of the model matters as much, which is the subject of context engineering.

Vector database vs similar terms: what differs

TermWhat it isHow it differs
Embedding modelTurns text into a vectorMakes vectors; does not store or search them
Vector databaseStores vectors with IDs and metadataFinds the nearest vectors to a query
Full-text searchToken matching, such as BM25Matches words, not meaning
pgvectorA Postgres extensionKeeps vectors next to your other columns
RAGRetrieve facts, then generate an answerUses vector search as one step

For the full pattern, read what RAG is.

Vector database FAQ

Do I need a vector database for RAG? Not always. RAG needs retrieval, and exact search can handle a small corpus. OpenAI recommends a vector database for "searching over many vectors quickly."

Is pgvector enough? Often, if your data is already in Postgres. It does exact search, HNSW and IVFFlat indexes, and hybrid search. The README says "You can use Reciprocal Rank Fusion or a cross-encoder to combine results."

Why did my results change after indexing? An approximate index trades some recall for speed. pgvector says you "will see different results" after adding one. If recall drops too far, raise hnsw.ef_search; a higher value gives better recall at the cost of speed.

Cosine or dot product for OpenAI embeddings? OpenAI embeddings have length 1, so cosine and dot product give the same score. pgvector suggests inner product for normalized vectors.

Share this article

Related articles

Continue exploring similar guides and insights