Vector databases solve the same problem as a traditional database index, but for geometry instead of sorted values. When you query a B-tree, you're asking "give me the row where id = 42." When you query a vector index, you're asking "give me the 5 stored vectors geometrically closest to this query vector." The dominant indexing algorithm today is HNSW (Hierarchical Navigable Small World), a graph structure where each node connects to a tunable number of neighbors across multiple layers. At query time, the algorithm starts at the top layer (coarse graph), greedily hops toward the query vector, then descends to finer layers, converging on approximate neighbors without scanning everything. The key parameters are m (edges per node, controls graph connectivity and memory) and ef_construction (beam width during index build, controls recall vs build time). At query time, ef controls the search width. Higher values mean better recall at the cost of latency. You will tune these, so understanding them matters.
Pinecone's selling point is that you never touch infrastructure. You create an index via API, upsert vectors, and query. It handles sharding, replication, and index rebuilds. The downside is cost opacity, no self-hosting option, and limited control over index parameters. It's the right choice when your team has zero ops bandwidth and the workload is cloud-native from day one. The serverless tier introduced in 2024 changes the economics significantly for bursty traffic patterns, but you need to benchmark it against your p99 latency requirements before committing.
Weaviate distinguishes itself with first-class hybrid search: it stores both a vector index and an inverted BM25 index for each object, and lets you blend them at query time with an alpha parameter (0 = pure BM25, 1 = pure vector, 0.5 = blended). This is practically significant. Internal benchmarks across real RAG workloads consistently show hybrid search beating pure semantic search by 5 to 15 percent on recall, especially when users include product names, SKUs, or technical jargon that embeddings don't encode reliably. Weaviate also has a built-in text2vec module that calls an embedding API on ingest automatically, which simplifies the pipeline but introduces coupling you may not want in production.
Qdrant is written in Rust, and the performance shows. It supports payload-level filtering that runs inside the HNSW traversal rather than post-filtering results, which matters a lot when you have high-selectivity filters (e.g., "only search vectors where tenant_id = 'acme' and document_type = 'invoice'"). Post-filtering requires over-fetching candidates and throwing away misses, which degrades both recall and latency. Qdrant's filtered HNSW avoids that. It supports named vectors per point (useful for multi-modal indexes where you store text and image embeddings on the same record), and its quantization options (scalar and product quantization) let you trade recall for a 4x to 32x memory reduction, which matters at the 10M+ vector scale. Qdrant also runs well on a single machine or a Docker container, making it the default for self-hosted setups.
Pgvector is a PostgreSQL extension that adds a vector column type and index types including HNSW (added in pgvector 0.5.0) and IVFFlat. Its value is not performance. It's data locality. If your application already stores documents, users, and metadata in Postgres, pgvector lets you do a semantic search join in a single SQL query, with full ACID guarantees and no separate service to operate. At small scale (under one million vectors, non-critical latency), this is genuinely the best option. The problems start when you need to scale horizontally (Postgres sharding is painful), when your HNSW index starts competing for shared memory with OLTP queries, or when you need features like named vectors or built-in quantization. Use pgvector to ship the first version, and treat migration to a dedicated store as a planned task, not a crisis.
At scale, the tradeoffs shift. At 10 users, any of these work; use whatever you can run locally. At 10,000 users with a production RAG system, you need a durable vector store with replication, monitoring, and defined SLAs. Qdrant or Weaviate self-hosted on Kubernetes, or Pinecone serverless, all work here. At 10 million users, you're thinking about multi-tenancy isolation (separate namespaces or collections per tenant to avoid cross-tenant leakage and enable per-tenant deletion), index sharding strategies, embedding cache layers to avoid re-embedding the same documents, and write amplification from frequent upserts. Pinecone handles sharding for you; with Weaviate or Qdrant you own it. Factor that into your team's capacity before choosing open-source at scale.
Key Takeaways
- Pick a vector DB based on ops burden, filtering complexity, and whether you already run Postgres.
- HNSW indexing trades memory for recall; tune ef_construction and m for your latency/recall target.
- Hybrid search (dense + sparse BM25) consistently outperforms pure vector search on keyword-heavy queries.
- pgvector is the right default for prototypes; migrate to a dedicated store before you hit 1M vectors.
Pro tips
- Always set ef at query time, not just ef_construction at index build time. Qdrant, Weaviate, and pgvector all support a per-query ef override, and bumping it from 128 to 256 can recover 2 to 5 percent recall when accuracy matters more than p50 latency.
- Store the raw text alongside the vector in the payload or a linked relational row. Retrieving context from a second database hop adds latency and a failure surface. The vector DB's payload store is not free, but the tradeoff is almost always worth it.
- In multi-tenant RAG systems, use per-tenant namespaces or collections, not a metadata filter alone. A filter still scans candidate vectors across all tenants before discarding them; a namespace confines the search to only that tenant's data, which is both faster and safer for data isolation.
- Quantize embeddings before going above two million vectors. Qdrant's scalar quantization (int8) cuts memory by 4x with less than 1 percent recall loss on most text workloads. Waiting until you're out of RAM is the wrong time to discover this option.
Common pitfalls
- Mistake: Using pgvector's IVFFlat index with a small
listsvalue at scale. Fix: Use HNSW (pgvector >= 0.5.0) for production; IVFFlat recall degrades sharply when the collection grows beyond its initial nlist tuning. - Mistake: Embedding the query with a different model than what indexed the documents. Fix: Enforce model name as a constant or config value shared across ingest and retrieval code; mismatched spaces produce garbage results with no error.
- Mistake: Post-filtering search results by metadata after retrieval. Fix: Use Qdrant or Weaviate filtered HNSW to apply payload filters inside the index traversal; post-filtering requires over-fetching by 5 to 10x to compensate for filtered-out results.
- Mistake: Calling upsert one vector at a time in a loop. Fix: Batch upserts in groups of 100 to 500; single-vector upserts incur per-request network overhead that slows bulk ingestion by 10x or more.
When to use Pinecone vs Weaviate vs Qdrant vs pgvector
| Option | Use when | Avoid when |
|---|---|---|
| Pinecone | Zero ops budget, cloud-only deployment, need managed scaling without owning infrastructure. | You need self-hosting, fine-grained index parameter control, or predictable per-query costs at high volume. |
| Weaviate | Hybrid BM25+vector search is required, or you want built-in modules to auto-embed at ingest time. | You need extremely low-latency filtered search on high-selectivity filters; its filtered HNSW is less mature than Qdrant's. |
| Qdrant | Self-hosted, high-performance filtered search, multi-tenant isolation, or memory-constrained environments needing quantization. | Your team has no appetite for operating a separate service and your vector count stays below 500k. |
| pgvector | You already run Postgres, need ACID joins between vectors and relational data, and have fewer than 1M vectors. | You need horizontal scaling, sub-10ms p99 at high concurrency, or features like named multi-vectors per record. |
Code Example
# openai==1.14.0, qdrant-client==1.8.0
from qdrant_client import QdrantClient
from qdrant_client.models import Distance, VectorParams, PointStruct
import openai
client = QdrantClient(":memory:") # in-memory for local dev
client.create_collection(
collection_name="docs",
vectors_config=VectorParams(size=1536, distance=Distance.COSINE),
)
text = "Qdrant uses HNSW for approximate nearest-neighbor search."
vector = openai.embeddings.create(input=text, model="text-embedding-3-small").data[0].embedding
client.upsert(
collection_name="docs",
points=[PointStruct(id=1, vector=vector, payload={"text": text})],
)
results = client.query_points(
collection_name="docs",
query=vector,
limit=3,
)
print(results.points[0].payload["text"])How this code works
This code demonstrates how to use Qdrant, a vector database, to store and search text embeddings generated by OpenAI. It sets up an in-memory Qdrant instance using QdrantClient(":memory:"), which is excellent for local development and learning as it avoids file setup, though it means any data stored won't persist once the script finishes. A new collection named "docs" is then created using create_collection, which specifies that it will store vectors of size 1536 (matching the output dimension of OpenAI's text-embedding-3-small model) and use COSINE distance to measure similarity between them.
A sample text string is then passed to openai.embeddings.create to generate its numerical representation, or vector. This vector is added to the "docs" collection using client.upsert, associating it with a unique id and storing the original text in a payload for later retrieval. Finally, client.query_points searches the collection for vectors most similar to the query vector, and the code prints the original text from the payload of the top search results, completing a basic cycle of embedding, storing, and searching.
Production-grade example
Adds retries on specific exceptions, token logging, structured logs, timeout, and tenant-scoped filtered search.
# qdrant-client==1.8.0, openai==1.14.0, tenacity==8.2.3
import os
import time
import logging
from tenacity import retry, stop_after_attempt, wait_exponential, retry_if_exception_type
from qdrant_client import QdrantClient, models
from qdrant_client.http.exceptions import UnexpectedResponse
import openai
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")
log = logging.getLogger(__name__)
qdrant = QdrantClient(
url=os.environ["QDRANT_URL"],
api_key=os.environ["QDRANT_API_KEY"],
timeout=10,
)
oai = openai.OpenAI(api_key=os.environ["OPENAI_API_KEY"])
@retry(
retry=retry_if_exception_type((UnexpectedResponse, openai.RateLimitError, openai.APITimeoutError)),
wait=wait_exponential(multiplier=1, min=1, max=30),
stop=stop_after_attempt(4),
)
def embed_and_upsert(doc_id: int, text: str, metadata: dict) -> None:
t0 = time.monotonic()
resp = oai.embeddings.create(input=text, model="text-embedding-3-small")
vector = resp.data[0].embedding
tokens_used = resp.usage.total_tokens
embed_ms = (time.monotonic() - t0) * 1000
log.info("embed", extra={"doc_id": doc_id, "tokens": tokens_used, "embed_ms": round(embed_ms, 1)})
qdrant.upsert(
collection_name="docs",
points=[models.PointStruct(id=doc_id, vector=vector, payload={"text": text, **metadata})],
)
log.info("upserted", extra={"doc_id": doc_id})
@retry(
retry=retry_if_exception_type((UnexpectedResponse, openai.RateLimitError)),
wait=wait_exponential(multiplier=1, min=1, max=20),
stop=stop_after_attempt(3),
)
def semantic_search(query: str, tenant_id: str, top_k: int = 5) -> list[dict]:
resp = oai.embeddings.create(input=query, model="text-embedding-3-small")
query_vector = resp.data[0].embedding
results = qdrant.query_points(
collection_name="docs",
query=query_vector,
query_filter=models.Filter(
must=[models.FieldCondition(key="tenant_id", match=models.MatchValue(value=tenant_id))]
),
limit=top_k,
with_payload=True,
)
hits = [{"text": p.payload["text"], "score": p.score} for p in results.points]
log.info("search", extra={"tenant_id": tenant_id, "hits": len(hits), "top_score": hits[0]["score"] if hits else None})
return hitsHow this code works
This Python code provides robust functions for interacting with OpenAI's embedding service and the Qdrant vector database, forming the backbone for a powerful semantic search system. Its primary job is to convert text into numerical vectors (embeddings), store them efficiently, and then retrieve relevant documents based on a query.
The embed_and_upsert function first uses openai.OpenAI to transform a given text into a vector embedding using the text-embedding-3-small model. This vector, along with a doc_id and metadata, is then stored in a Qdrant collection named "docs" using qdrant.upsert. The semantic_search function performs a similar embedding step for a query string. It then uses qdrant.query_points to find documents in Qdrant whose embeddings are most similar to the query's embedding. A key feature is the query_filter, which allows filtering results by tenant_id using models.FieldCondition, crucial for multi-tenant applications. Notice the @retry decorator on both functions; this automatically retries operations if transient errors like network issues or API rate limits occur, making the code much more resilient in a real-world setting.
Practice & master
Try the exercise, check your understanding, then mark this lesson mastered to track your path to pro.
Exercise
Build a minimal document search tool using Qdrant in-memory mode. Embed five short text snippets using the OpenAI embeddings API, upsert them into a Qdrant collection, then run a query and print the top-2 results with their similarity scores. Add a metadata field called source to each point and filter results to only include points where source is 'wiki'.
# qdrant-client==1.8.0, openai==1.14.0
from qdrant_client import QdrantClient
from qdrant_client.models import Distance, VectorParams, PointStruct, Filter, FieldCondition, MatchValue
import openai
oai = openai.OpenAI() # reads OPENAI_API_KEY from env
client = QdrantClient(":memory:")
DOCS = [
{"id": 1, "text": "Qdrant uses HNSW for fast approximate search.", "source": "wiki"},
{"id": 2, "text": "pgvector adds vector search to PostgreSQL.", "source": "blog"},
{"id": 3, "text": "Pinecone is a fully managed vector database.", "source": "wiki"},
{"id": 4, "text": "Cosine similarity measures angle between vectors.", "source": "wiki"},
{"id": 5, "text": "HNSW stands for Hierarchical Navigable Small World.", "source": "blog"},
]
# TODO: Create a Qdrant collection named 'search_demo' with cosine distance and size 1536
# TODO: Embed each doc's text and upsert as PointStructs with payload including 'text' and 'source'
QUERY = "What indexing algorithm does Qdrant use?"
# TODO: Embed the query, then query_points with a Filter restricting source == 'wiki', limit=2
# TODO: Print each result's payload['text'] and scoreQuick check
You have 50,000 documents in pgvector and need to add a metadata filter on
tenant_idto every query. What is the primary risk as your collection grows to 5M vectors?A RAG system returns relevant results at p50 latency but poor recall. Increasing which HNSW parameter at query time will improve recall without rebuilding the index?
Your team needs hybrid search combining keyword and semantic signals. Which vector database offers this natively without additional infrastructure?