Types of RAG: A Builder's Guide to RAG Architectures

Types of RAG: What Pipeline Builders Actually Need to Know

Searching "types of RAG" returns long lists of variants with little guidance on which ones matter if you're building agent memory or a production pipeline. Most articles are taxonomy dumps. This one isn't. Retrieval-augmented generation, RAG for short, is a pattern where a retriever pulls relevant chunks from a knowledge base and a generator uses those chunks to ground its answer. That's the whole idea. The types of RAG are just different ways to fix what breaks when you try it at scale.

Here's the problem: you don't need to know all fourteen variants. You need to know which three or four solve your specific failure mode. This guide categorizes types of RAG by architectural role, retrieval-focused, generation-focused, and orchestration-focused, and pairs each with explicit failure modes. GigaRAG is built for exactly this audience, so the examples reflect what actually happens in production. You'll get a decision framework mapping latency, cost, and accuracy constraints to specific RAG types, plus honest notes on what each one can't do.

In This Guide

• What Is RAG? A Quick Refresher for Pipeline Builders
• Naive RAG vs Advanced RAG: Which Should You Build?
• Why There Are So Many Types of RAG
• Types Of RAG: A Step-by-Step Guide
• A Better Way to Categorize Types of RAG
• Retrieval-Focused RAG Types
• Generation-Focused RAG Types
• Orchestration-Focused RAG Types
• RAG with Memory: What Agent Builders Need to Know
• How to Choose the Right Type of RAG
• RAG vs. Fine-Tuning: When Not to Use RAG at All
• Common Mistakes When Building Types of RAG Pipelines
• Final Thoughts: Types of RAG Worth Your Time

What Is RAG? A Quick Refresher for Pipeline Builders

RAG (retrieval-augmented generation) is a pattern that gives an LLM access to external data at inference time, so it can ground its answers in sources it wasn't trained on. Instead of relying only on the model's weights, the pipeline retrieves relevant chunks from a knowledge base and feeds them into the prompt.

Here's what happens behind the scenes. You take your documents, split them into chunks, and embed each chunk into a vector. When a query comes in, you embed it too, find the nearest chunks in vector space, and stuff those into the LLM's context window along with the question. The model then generates an answer using that retrieved text as evidence.

The reason this exists is simple: LLMs hallucinate when they don't know something, and they can't know your private data unless you show it to them. RAG fixes both problems without retraining the model. You update the knowledge base, and the answers update with it.

The main catch is that retrieval quality sets the ceiling on answer quality. If the right chunk doesn't come back, the best generator in the world can't save you. That's why the rest of this article focuses on the retrieval side as much as the generation side.

[!note] RAG is not a single algorithm but a family of architectures. The terms 'Naive', 'Advanced', and 'Modular' are common categorizations, but they are not official standards—they are descriptive labels used by practitioners.

Naive RAG vs Advanced RAG: Which Should You Build?

Why There Are So Many Types of RAG

The naive RAG pipeline I just described works fine for a demo. It breaks in production for reasons that are boring and specific.

Retrieval quality is the first failure point. Vector similarity alone returns chunks that are semantically close but not actually relevant. A query about "Python memory management" pulls back articles about Python's memory usage in data science, not garbage collection internals. The generator then answers confidently from the wrong context.

Latency is the second. Embedding a query, searching a vector index, and stuffing a dozen chunks into a prompt adds real milliseconds. For agent loops that make five or ten retrieval calls per turn, that latency compounds fast.

Cost is the third. Every retrieved chunk is a token you pay for. Naive pipelines over-retrieve because they can't tell which chunks matter, so they stuff the context window with noise and burn budget on tokens that don't improve the answer.

Context limits are the fourth. Some questions need more evidence than fits in a single prompt. Multi-hop questions, comparisons across documents, long technical histories. Naive RAG truncates or drops chunks, and the answer degrades.

Multi-step reasoning is the fifth. A single retrieve-then-generate pass can't handle questions that require finding one fact, using it to search for another, and only then answering. Agents need retrieval as a tool inside a loop, not a one-shot step.

Every RAG variant exists because someone hit one of these walls and built a workaround. The taxonomy isn't academic. It's a pile of production scars.

[!tip] For agent memory systems, start with Modular RAG because it lets you swap retrieval and generation components independently, making it easier to adapt as your agent's memory needs evolve.

Types Of RAG: A Step-by-Step Guide

1. Define your data type and scale (e.g., documents, tables, mixed).
2. List constraints: latency, cost, and accuracy targets.
3. Map constraints to RAG categories: retrieval, generation, or orchestration.
4. Prototype with a simple baseline (e.g., Naive RAG) to measure gaps.
5. Add advanced retrieval if accuracy is insufficient (e.g., hybrid search).
6. Test modular RAG if you need dynamic control or agent integration.
7. Evaluate failure modes (e.g., hallucinations, retrieval misses) before scaling.

A Better Way to Categorize Types of RAG

Most articles on types of RAG hand you a numbered list. Fourteen variants, five variants, each with a paragraph and a diagram. You read them all and still don't know which one to build.

That's because a flat list tells you what exists, not what each thing does for your pipeline. The question that matters when you're shipping is simpler: which part of the pipeline does this variant change?

Every RAG type modifies one of three things. It changes what gets retrieved, how the generator uses what was retrieved, or how retrieval and generation are orchestrated across steps. That's the whole map.

Retrieval-focused types

These variants change the retriever. They improve what comes back from the vector index before the generator ever sees it. Naive RAG, advanced RAG with reranking, HyDE, and multimodal RAG all live here.

You reach for these when retrieval quality is the bottleneck. When the right chunk exists in your index but the search keeps missing it.

Generation-focused types

These variants change the generator. They alter how the model reads, critiques, or drafts from the retrieved context. Self-RAG, Corrective RAG, and Speculative RAG fit this bucket.

You reach for these when retrieval works but the model uses the context badly. When it answers from the wrong chunk or takes too long to draft.

Orchestration-focused types

These variants change the control flow. They decide when to retrieve, how many times, and what to do with each result. Agentic RAG, GraphRAG, modular RAG, and adaptive RAG belong here.

You reach for these when the question needs more than one retrieval step. When the answer depends on routing, fallback, or reasoning across multiple hops.

The honest answer is that most production pipelines mix categories. You might use advanced RAG for retrieval quality and agentic orchestration for multi-step questions. The categories aren't boxes. They're a way to read a variant and immediately know which failure mode it addresses.

Retrieval-Focused RAG Types

Retrieval-focused variants all fix the same problem: the right chunk is in your index, but the search misses it. The generator never gets a chance to be wrong because it never sees the right context. These four types attack that failure from different angles.

Naive RAG: the baseline

Naive RAG is the simplest possible pipeline. You chunk documents, embed the chunks, store them in a vector index, and retrieve the top-k results for each query. The generator gets those chunks and answers.

It works for small, homogeneous datasets where the query language matches the document language. Internal docs, FAQs, product manuals. When someone asks the question in roughly the words the doc uses, naive RAG returns the right chunk.

It fails when the query and document use different vocabulary. It fails when the answer spans multiple chunks. It fails when the top-k results are all near-duplicates of each other. You get five chunks that say the same thing and none that answer the question.

Advanced RAG: reranking and query rewriting

Advanced RAG adds a second pass over the retrieved chunks. The vector search returns a larger candidate set, say 50 chunks. A reranker then scores those candidates against the query and keeps the top 5 or 10.

The reranker is usually a cross-encoder: a model that reads the query and chunk together and outputs a relevance score. It's slower than vector search but far more accurate. Query rewriting runs before retrieval: an LLM expands or rephrases the user's question to improve recall.

This is the most common production upgrade from naive RAG. It fixes vocabulary mismatch and duplicate results. The cost is latency: you add a model call before and after retrieval. For most pipelines, that's worth it.

HyDE: generating hypothetical documents

HyDE flips the retrieval direction. Instead of embedding the user's query, you ask an LLM to write a hypothetical document that would answer the question. You embed that document and search for similar chunks.

It helps when queries are short and documents are long. A three-word query embeds poorly against a 500-word chunk. A generated paragraph embeds much closer to the real answer.

The catch is that HyDE adds a full generation step before retrieval. If the LLM hallucinates a plausible-sounding but wrong hypothetical document, you retrieve chunks that match the hallucination. HyDE is best when your embedding model struggles with short queries and you can tolerate the extra latency.

Multimodal RAG: retrieving across modalities

Multimodal RAG retrieves from more than text. Images, audio, video, tables, charts. The retrieval index stores embeddings from multiple modalities, and the generator receives whichever modality matches the query.

It helps when the answer lives in a diagram or a screenshot. A user asks "what does the error screen look like" and the retriever returns an image, not a text description.

It fails when modalities are poorly aligned. Text embeddings and image embeddings live in different vector spaces. You need a shared embedding model or a mapping layer, and that mapping is where most multimodal pipelines break. The honest answer is that multimodal RAG is still harder to build than text-only RAG, and the gains only materialize when your knowledge base is genuinely multimodal.

Generation-Focused RAG Types

Retrieval-focused types fix what gets pulled from the index. Generation-focused types fix what the generator does with it. The retriever returns chunks, but the generator can still ignore them, over-trust them, or drown in them. These three variants change the generator's relationship to retrieved context.

Self-RAG: the model critiques its own retrieval

Self-RAG trains the generator to decide, per token or per chunk, whether retrieved context is worth using. The model outputs special reflection tokens: retrieve, relevant, supported, useful. If a chunk scores low on relevance, the model skips it and either retrieves again or answers from its own weights.

The mechanism is a fine-tuned LLM that learns to emit these judgments during generation. You don't need a separate reranker because the generator is the reranker. The tradeoff is that you need a model trained specifically for this. Off-the-shelf LLMs don't emit reflection tokens, and prompting them to pretend usually produces inconsistent results.

In practice, Self-RAG helps when retrieval quality is uneven and you can't afford a separate reranking step. It fails when the fine-tuned model's reflection judgments are wrong, which happens more often on out-of-domain queries. You've added a training dependency to fix a retrieval problem.

Corrective RAG: fixing bad retrieval before generation

Corrective RAG adds a retrieval evaluator between the retriever and the generator. The evaluator scores each retrieved chunk for relevance. If scores are high, generation proceeds. If scores are low, the pipeline triggers a fallback: query rewriting, web search, or a different index.

The evaluator is usually a lightweight model or heuristic, not the full generator. That keeps latency down compared to Self-RAG. The pipeline looks like: retrieve, evaluate, correct if needed, generate.

The main catch is that the evaluator is another component to tune. A bad evaluator either lets garbage through or triggers unnecessary fallbacks. Corrective RAG also adds a conditional branch, which makes latency unpredictable. Sometimes the query takes one pass, sometimes three. If you need consistent response times, that variance hurts.

Speculative RAG: parallel drafting for lower latency

Speculative RAG splits generation across multiple smaller models. A generalist model drafts an answer from retrieved context while specialist models each draft from a subset of the chunks. A final verifier picks the best draft or merges them.

The goal is latency, not accuracy. Smaller models draft faster than one large model, and parallel drafts hide the cost of generation. The verifier is the only large model in the loop.

The honest tradeoff is complexity. You're running three to five models per query instead of one. Orchestration overhead eats into the latency gains, and the verifier can pick a plausible but wrong draft. Speculative RAG makes sense when generation latency is your bottleneck and you have the infrastructure to run parallel inference. For most pipeline builders, it's over-engineering.

Orchestration-Focused RAG Types

Retrieval-focused types fix what gets pulled. Generation-focused types fix what the generator does with it. Orchestration-focused types change who calls what, in what order, and how many times. These are the variants that matter most for agent memory builders, because they turn retrieval from a single step into a loop.

Agentic RAG: retrieval as a tool in a reasoning loop

Agentic RAG gives the LLM control over retrieval. Instead of one retrieve-then-generate pass, the model decides when to search, what to search for, and whether the results are good enough. It can retrieve multiple times, refine its query, or stop early.

The mechanism is tool calling. The retriever is exposed as a function the model can invoke mid-reasoning. The model might retrieve, read the chunks, decide they don't answer the question, and retrieve again with a different query. That loop continues until the model judges it has enough context.

The main catch is cost and latency. Each retrieval round adds tokens and time. A query that naive RAG answers in one pass can take four or five under an agentic loop. You also need a model that's reliable at tool calling. Weaker models call the retriever too often or not at all.

GraphRAG: structured knowledge for multi-hop questions

GraphRAG retrieves from a knowledge graph instead of, or alongside, a vector index. Entities and relationships are extracted from your documents and stored as nodes and edges. A query about "who worked with whom on which project" traverses the graph rather than matching embeddings.

This helps when the answer requires connecting facts that don't appear in the same chunk. Vector search finds similar text. Graph traversal finds related entities. For multi-hop questions, that's the difference between an answer and a guess.

The honest tradeoff is setup cost. Building and maintaining a knowledge graph is real work. Extraction quality determines retrieval quality, and graphs drift as your source documents change. GraphRAG is worth it when your questions are inherently relational. For most document Q&A, a vector index is enough.

Modular RAG: composable pipeline components

Modular RAG treats the pipeline as a set of swappable parts: retriever, reranker, query rewriter, memory store, generator. You pick the components that fit your use case and wire them together.

The benefit is flexibility. You can add a reranker without changing your retriever, or swap generators without touching retrieval. The cost is integration work. Every component boundary is a place where latency, serialization, and failure modes creep in. Modular RAG is a design philosophy more than a specific architecture. It's how most production pipelines end up anyway, whether you planned it or not.

Branched RAG and Adaptive RAG: routing and fallback

Branched RAG sends a query down multiple retrieval paths in parallel, then merges the results. Adaptive RAG routes each query to a different strategy based on its type: simple questions get naive retrieval, complex questions get agentic loops or graph traversal.

RAG with Memory: What Agent Builders Need to Know

Most RAG articles stop at retrieval and generation. Agent builders can't. An agent that answers each query from scratch forgets everything between turns. Memory is what turns a stateless pipeline into something that can hold a conversation, track a task, or build on prior context.

Short-term vs. long-term memory in RAG pipelines

Short-term memory is the conversation itself. It lives in the context window: the current query, recent turns, and any retrieved chunks from this exchange. It's fast and automatic, but it dies when the window fills or the session ends.

Long-term memory is a store that persists across sessions. It can be a vector index of past interactions, a key-value store of user facts, or a knowledge graph of entities the agent has encountered. The retriever queries it like any other source, but the writes matter as much as the reads. The agent must decide what's worth remembering, store it, and retrieve it later without polluting the current context.

The honest catch: long-term memory adds a write step to every turn. That's extra latency and a new failure mode. Store too much and retrieval quality drops. Store too little and the agent repeats itself.

Which RAG types work best with persistent memory

Agentic RAG is the natural fit. The model already decides when to retrieve. Adding a memory store just gives it one more tool to call. The same reasoning loop that refines a query can decide whether to write a fact to long-term memory.

Modular RAG also works well. Memory becomes one more swappable component: a retriever that points at the memory store, a writer that persists important turns, a reranker that prioritizes recent over stale memories.

Naive RAG doesn't. It has no mechanism for writing memory, only reading it. You can bolt a memory store onto the side, but the pipeline won't know when to use it. GraphRAG can serve as long-term memory if your agent's knowledge is relational, but the setup cost applies here too.

The decision isn't which RAG type supports memory. It's whether your agent needs memory at all. A single-turn Q&A bot doesn't. A task-tracking assistant does.

How to Choose the Right Type of RAG

There is no best RAG model. The honest answer is that the right type depends on your constraints: latency budget, cost sensitivity, accuracy requirements, data modality, and whether your queries need multi-hop reasoning. A support bot answering one-off questions has different needs than an agent that plans across a knowledge graph.

Decision factors: latency, cost, accuracy, data type

Start with latency. If you need answers in under 500 milliseconds, naive RAG or advanced RAG with a single retrieval pass is your ceiling. Self-RAG and Corrective RAG add critique and repair loops that can double or triple response time. Agentic RAG is slower still, since the model may call retrieval multiple times before generating.

Cost follows the same pattern. Every extra LLM call costs tokens. HyDE adds a generation step before retrieval. Self-RAG adds reflection tokens. Agentic RAG adds an entire reasoning loop. If you're serving thousands of queries per day, those costs compound fast.

Accuracy requirements pull in the opposite direction. If hallucinations are expensive, Corrective RAG or Self-RAG earn their overhead. If your queries need relationships between entities, GraphRAG beats vector search alone. If your data spans images and text, multimodal RAG is non-negotiable.

Data type is the easiest filter. Text-only, single-hop questions: naive or advanced RAG. Structured, relational data with multi-hop queries: GraphRAG. Mixed modalities: multimodal RAG. Agent memory: agentic or modular RAG.

A simple decision table for common use cases

If you're still unsure, start with advanced RAG. Add reranking and query rewriting before you reach for anything more complex. Most production failures come from over-engineering, not under-engineering.

RAG vs. Fine-Tuning: When Not to Use RAG at All

RAG is not a replacement for fine-tuning. They solve different problems, and using one where the other belongs wastes weeks.

RAG changes what the model knows at query time. It pulls facts from a store and stuffs them into the context window. The model's weights never change. That's the whole mechanism, and it means RAG is good at one thing: giving a model access to facts it didn't have during training.

Fine-tuning changes how the model behaves. It updates the weights on a dataset, which teaches the model new patterns: a domain's writing style, a specific output format, a reasoning procedure. RAG cannot do any of that. You can retrieve a thousand examples of legal briefs, but the model will still write like a general-purpose LLM unless you fine-tune it on legal prose.

Here's the test. If your problem is "the model doesn't know this fact," use RAG. If your problem is "the model doesn't think or write the way I need," fine-tune. If both, you need both.

RAG also fails when the knowledge is procedural rather than declarative. A model can retrieve the steps of a debugging workflow, but it won't internalize the judgment of when to apply each step. That judgment lives in weights, not in retrieved text.

The main catch with fine-tuning is maintenance. Every time your domain knowledge changes, you retrain. RAG updates instantly: swap the documents in the store and you're done. That's why most production systems use RAG for facts and fine-tuning only for style or behavior that changes rarely.

Don't reach for RAG to teach a model a new reasoning pattern. It won't work.

Common Mistakes When Building Types of RAG Pipelines

Most RAG failures aren't architectural. They're operational. You can pick the right type of RAG and still ship a pipeline that returns garbage, because the mistakes happen in the details nobody writes blog posts about.

Poor chunking and embedding choices

Chunking is where pipelines quietly die. If you split documents at fixed character counts, you'll cut sentences in half and bury answers across chunk boundaries. The retriever then returns fragments that look relevant but aren't. You need chunks that respect document structure: headings, paragraphs, code blocks, table rows.

Embedding choice matters just as much. A general-purpose embedding model won't know that "cell" means biology in your corpus and spreadsheet in another. If your domain has specialized vocabulary, test embeddings on your actual data before committing. Don't assume the default works.

Skipping evaluation

You can't improve what you don't measure. Most teams ship a RAG pipeline, eyeball five answers, and call it done. That's not evaluation. You need a held-out set of question-answer pairs, and you need to score retrieval quality separately from generation quality. Retrieval metrics like recall@k tell you whether the right chunk made it into context. Generation metrics tell you whether the answer was correct given that context. If you only measure the final answer, you won't know which layer is broken.

Over-engineering the orchestration layer

Agentic RAG is the most over-applied pattern in production right now. Teams reach for multi-step reasoning loops, tool calling, and self-critique before they've verified that naive RAG fails on their data. Every orchestration layer adds latency, token cost, and failure modes. If a single retrieval pass answers 90% of your queries correctly, you don't need an agent. You need better chunking.

The honest answer is that most types of RAG exist to fix specific failures. If you haven't measured the failure, you're guessing at the fix.

Final Thoughts: Types of RAG Worth Your Time

Most types of RAG are variations on three moves: get better context, use context more carefully, or orchestrate more steps. Once you see that, the taxonomy stops mattering. What matters is your constraint.

If latency is your bottleneck, naive RAG with good chunking beats agentic RAG every time. If accuracy on multi-hop questions is the problem, GraphRAG or Self-RAG earns its complexity. If you're building agent memory, orchestration-focused types are where you'll live, but only after you've measured that simpler retrieval fails.

The honest answer is that you don't need to master every type of RAG. You need one that fits your data, your latency budget, and your evaluation results. Pick the simplest thing that works, then stop.

GigaRAG is built for exactly that: agent memory and pipeline builders who want to ship faster without rebuilding retrieval infrastructure from scratch.

Frequently Asked Questions

Is ChatGPT a RAG LLM?

ChatGPT itself is not a RAG LLM by default; it is a generative model trained on static data. However, OpenAI has integrated retrieval capabilities into some versions (like ChatGPT with browsing) to pull in up-to-date information. So, it can use RAG-like mechanisms, but it is not inherently a RAG system.

What is RAG vs MCP?

RAG (Retrieval-Augmented Generation) is a technique that combines a retrieval system with a language model to generate answers grounded in external data. MCP (Model Context Protocol) is a protocol for connecting AI models to external tools and data sources. While RAG is about how to retrieve and use data, MCP is about standardizing how models access those tools and data.

Which is the best RAG model?

There is no single 'best' RAG model because RAG is an architecture, not a model. The choice depends on your data, latency, and accuracy needs. For example, using a strong embedding model like OpenAI's text-embedding-3 or an open-source alternative like BGE can improve retrieval, while the generation model (e.g., GPT-4, Llama 3) affects answer quality. Test combinations to find what works for your use case.

What are the different types of RAG?

RAG types are commonly categorized into three main groups: Naive RAG (basic retrieve-then-generate), Advanced RAG (adds pre-retrieval and post-retrieval optimizations like query rewriting and re-ranking), and Modular RAG (flexible, component-based design that allows swapping or adding modules like memory or routing). Each serves different production needs.

How many types of RAG are there?

The number varies by source, but the most widely cited taxonomy includes three core types: Naive, Advanced, and Modular. Some articles list up to 25 variants by breaking down specific techniques (e.g., hybrid search, graph RAG, agentic RAG), but these are often subcategories or extensions of the main three.

What is Hybrid RAG?

Hybrid RAG combines multiple retrieval methods, typically mixing keyword-based (BM25) and vector-based (semantic) search. This approach improves coverage by capturing both exact matches and semantic relevance, reducing the risk of missing relevant documents. It is a popular choice for production systems because it balances precision and recall.

What is the difference between RAG and fine-tuning?

RAG augments a pre-trained model with external retrieval at inference time, allowing it to access up-to-date or domain-specific information without changing weights. Fine-tuning updates the model's weights on a specific dataset to improve its behavior or style. RAG is more dynamic and easier to update, while fine-tuning can be more resource-intensive but may improve intrinsic knowledge.

About GigaRAG

GigaRAG is for agent memory and RAG pipeline builders. get this right. Whether you are working through types of RAG or something adjacent, we publish what we have actually tested, including where it falls short.