How to Build a RAG System with Claude (Voyage Embeddings + pgvector, 2026)

What RAG actually is (and why Claude needs Voyage for it)

Retrieval-Augmented Generation (RAG) means retrieving the right passages from your own documents and handing them to the model as context so it answers from your data, not from its training set. Here is the direct answer: to build a RAG system with Claude, you chunk your docs, embed the chunks with Voyage (voyage-4, input_type="document"), store the vectors in a database like pgvector, then at query time embed the question (input_type="query"), retrieve the top-k nearest chunks, and feed them to Claude’s Messages API to generate a grounded, cited answer. Claude has no embedding model of its own, so Voyage supplies the vectors.

The Anthropic embeddings docs are explicit: “Anthropic does not offer its own embedding model,” and they recommend Voyage AI as the provider (while advising you to assess others too). So Claude handles the reasoning and generation; Voyage handles the vectors; a vector database handles storage and search. This post is the build tutorial. If you are still deciding whether RAG is even the right tool versus training the model, read RAG vs fine-tuning first — this post assumes you have decided to build.

The 8-step RAG pipeline, at a glance

Every RAG system, no matter the vendor, is the same eight steps. Four happen once at ingestion time (chunk, embed, store), and the rest happen on every query (embed the question, retrieve, optionally rerank, generate).

Keep this mental model as you read: steps 1-3 build the index once, steps 4-8 run per request. The only two moving parts you have to pick are the embedding model (Voyage) and the vector store (pgvector, Pinecone, Qdrant, or others). Everything else is glue.

Step 1: Pick an embedding model (Voyage voyage-4, -lite, -code-3)

Anthropic’s docs recommend the Voyage 4 generation. Match the model to your workload:

• voyage-4-large — best general-purpose and multilingual retrieval quality.
• voyage-4 — balanced quality and efficiency; a solid default for most RAG.
• voyage-4-lite — optimized for latency and cost.
• voyage-4-nano — open-weight (Apache 2.0), available on Hugging Face.
• voyage-code-3 — optimized for code retrieval, if your corpus is source code.

All of them share a 32,000-token context length and a default embedding dimension of 1024 (you can also request 256, 512, or 2048). For a Spanish-first or mixed-language corpus, voyage-4 and voyage-4-large are trained for multilingual retrieval, so you do not need a separate Spanish model. This tutorial uses voyage-4.

Step 2: Chunk your documents (256-512 tokens + overlap)

You do not embed whole documents — you embed chunks, because retrieval works best when each vector represents one coherent idea. The following is general industry guidance for 2026, not an Anthropic or Voyage spec: aim for roughly 256-512 tokens per chunk with about 10-15% overlap between neighbors. The overlap keeps a sentence that straddles a boundary from being cut in half.

Better still is “semantic chunking” — splitting where the meaning shifts (section headings, topic changes) rather than at a fixed token count. And if recall matters, hybrid retrieval (dense embeddings plus a BM25 keyword pass) typically catches passages that pure vectors miss. Treat all of these as starting points to tune, not fixed rules.

One thing that is not optional: clean your data before you embed it. If your documents contain Mexican PII like CURP, RFC, or CLABE, redact it before the LLM ever sees it — embedded vectors are hard to scrub after the fact.

Step 3: Embed the chunks with Voyage (input_type=“document”)

Install the client and set your key:

pip install -U voyageai
export VOYAGE_API_KEY="your-secret-key"

Now embed the corpus. The single most important flag here is input_type="document". Voyage prepends a retrieval prompt based on this value, and using it materially improves retrieval quality. Do not omit it or set it to None.

import voyageai

vo = voyageai.Client()  # reads VOYAGE_API_KEY from the environment

chunks = [
    "The Mediterranean diet emphasizes fish, olive oil, and vegetables.",
    "Photosynthesis converts light energy into glucose and produces oxygen.",
    "Apple's Q4 conference call is scheduled for Thursday, November 2, 2023.",
]

# input_type="document" is REQUIRED for the corpus side of RAG.
doc_embeddings = vo.embed(
    chunks, model="voyage-4", input_type="document"
).embeddings
# -> list of 1024-dimensional vectors, one per chunk

Batch your chunks per call, and each returned vector is 1024 floats by default.

Step 4: Store the vectors in pgvector (SQL, and the DB is swappable)

You need somewhere to keep vectors plus their source text and metadata, and to search them by nearest neighbor. pgvector is the pragmatic default because it lives inside Postgres you probably already run. Enable the extension and create a table with a vector(1024) column matching your embedding dimension:

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE chunks (
    id        bigserial PRIMARY KEY,
    content   text        NOT NULL,
    source    text,
    embedding vector(1024)   -- must match Voyage's output dimension
);

-- Cosine-distance index for fast approximate nearest-neighbor search.
CREATE INDEX ON chunks
    USING hnsw (embedding vector_cosine_ops);

Insert each chunk with its vector (from your app code, passing the embedding as the vector literal):

INSERT INTO chunks (content, source, embedding)
VALUES ($1, $2, $3);   -- $3 is the 1024-float embedding for that chunk

The database is genuinely swappable — the same pipeline works with Pinecone or Qdrant, only the store-and-query calls change. For which store fits your scale and budget, see the companion post vector databases compared: pgvector vs Pinecone vs Qdrant.

Step 5: Embed the question and retrieve the top-k (input_type=“query”)

At query time, embed the user’s question with input_type="query" — the query counterpart to the document flag. Voyage embeddings are normalized to length 1, which means dot-product equals cosine similarity (and is faster to compute), and cosine versus Euclidean give identical rankings. So “top-k” is just sorting by dot product.

question = "When is Apple's conference call scheduled?"

# input_type="query" for the question side.
query_embedding = vo.embed(
    [question], model="voyage-4", input_type="query"
).embeddings[0]

With pgvector you push that search into SQL. The <=> operator is cosine distance, so the nearest chunks are those with the smallest distance — order ascending and take the first k:

SELECT content, source
FROM chunks
ORDER BY embedding <=> $1   -- $1 is the query embedding
LIMIT 8;                    -- top-k = 8

Step 6 (optional): Rerank to the best 5-10 with rerank-2.5

Vector search is fast but coarse: it can return a chunk that is topically close yet not actually the best answer. A reranker fixes this. Retrieve a wider net (say top-25), then let Voyage’s rerank-2.5 score each candidate against the query and keep the best 5-10. You call it with rerank(), not embed():

# Pull a wider candidate set first (e.g. top-25 from pgvector),
# then rerank down to the strongest few.
reranked = vo.rerank(
    query=question,
    documents=candidate_chunks,   # the ~25 texts you retrieved
    model="rerank-2.5",
    top_k=8,
)
best_chunks = [r.document for r in reranked.results]

Reranking costs an extra call, so treat it as an accuracy dial you turn up when retrieval quality matters more than latency. rerank-2.5-lite trades some accuracy for lower latency and cost.

Step 7-8: Generate a grounded, cited answer with Claude’s Messages API

Now Claude earns its keep. Inject the retrieved chunks as context in the Messages API and instruct the model to answer only from that context and cite its sources. Grounding the answer to the retrieved text is what keeps a RAG system from hallucinating. For the full Messages API walkthrough, see how to use the Claude API; for the grounding discipline, see AI chatbot without hallucinations.

import anthropic

client = anthropic.Anthropic()

context = "\n\n".join(
    f"[{i+1}] {c}" for i, c in enumerate(best_chunks)
)

message = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=1024,
    system=(
        "Answer the question using ONLY the numbered context below. "
        "Cite sources as [1], [2]. If the answer is not in the context, "
        "say you don't know.\n\n" + context
    ),
    messages=[{"role": "user", "content": question}],
)

print(message.content[0].text)

That system-plus-context prompt is the whole trick: the model reasons over your passages and returns a cited answer, and the instruction to admit “I don’t know” stops it from inventing facts when retrieval comes up empty.

Make it cheap and reliable: prompt caching + grounding

Two production concerns turn a demo into something you can afford to run.

Cost. If your system prompt or a stable context block repeats across requests, cache it. Claude’s prompt caching lets you mark a prefix as cacheable so repeated tokens are not re-billed at full price — see Claude prompt caching to cut token costs. This matters most when you keep a large, slowly-changing context block (policies, product docs) in front of every query.

Reliability. Grounding is not just a prompt nicety — it is the mechanism that keeps answers truthful. Always number your chunks, always ask for citations, and always give the model an escape hatch to say it does not know. Combined, caching and grounding are the difference between a RAG toy and a RAG feature.

Full runnable example, end to end

Here is the whole pipeline as one script — embed a tiny corpus, retrieve with in-memory dot product (swap in your pgvector query for real data), and generate a grounded answer with Claude:

import numpy as np
import voyageai
import anthropic

vo = voyageai.Client()
claude = anthropic.Anthropic()

# 1-3. Chunk + embed the corpus (input_type="document")
chunks = [
    "The Mediterranean diet emphasizes fish, olive oil, and vegetables.",
    "Photosynthesis converts light energy into glucose and produces oxygen.",
    "Apple's Q4 conference call is scheduled for Thursday, November 2, 2023.",
]
doc_embds = vo.embed(chunks, model="voyage-4", input_type="document").embeddings

# 4-5. Embed the question (input_type="query") and retrieve top-k
question = "When is Apple's conference call?"
q_embd = vo.embed([question], model="voyage-4", input_type="query").embeddings[0]

# Voyage vectors are normalized, so dot-product == cosine similarity.
scores = np.dot(doc_embds, q_embd)
top_k = np.argsort(scores)[::-1][:3]
retrieved = [chunks[i] for i in top_k]

# 7-8. Generate a grounded, cited answer with Claude
context = "\n\n".join(f"[{i+1}] {c}" for i, c in enumerate(retrieved))
resp = claude.messages.create(
    model="claude-sonnet-5",
    max_tokens=512,
    system=(
        "Answer using ONLY the numbered context. Cite sources as [n]. "
        "If it's not there, say you don't know.\n\n" + context
    ),
    messages=[{"role": "user", "content": question}],
)
print(resp.content[0].text)

For production, replace the NumPy retrieval with the pgvector cosine-distance query from Step 5, and add reranking if you need the accuracy. Anthropic also publishes a RAG cookbook with a full Pinecone example if you want a hosted-store variant.

Where to go next: which vector store, and when RAG isn’t the answer

You now have a working RAG system. Two decisions shape where you take it:

• Which vector store? pgvector is the low-friction default, but Pinecone and Qdrant win at certain scales and operational profiles. The vector databases compared post is the companion to this one — same pipeline, different storage layer.
• Is RAG even the right pattern? For live or structured data — your CRM, an order database, an API — retrieval over stale embeddings is the wrong tool. Connect Claude directly instead; see connect an AI agent to your data with MCP. And if you are wiring RAG into a real product surface, a WhatsApp AI agent for your business shows retrieval working inside a shipped channel.

The pattern generalizes: chunk, embed with Voyage, store, retrieve, ground, and let Claude generate. Everything else — the store, the reranker, the caching — is tuning. For more on the building blocks, the ai-agents hub collects the related guides.