What Reranking Does and Why It Exists in RAG

What Reranking Does and Why It Exists

What reranking does and why it exists is easiest to see when a RAG pipeline fails quietly. Your retriever pulls 20 chunks. The right one sits at position 8. Your context window holds five. The LLM answers from the wrong chunks, and you don't find out until a user complains. Reranking is the fix: a second scoring pass that reorders those 20 candidates so the relevant chunk moves into the top five before the LLM ever sees them.

The honest answer is that reranking is a precision tool, not a magic one. It can't rescue a retriever that never found the right chunk in the first place. It costs latency and money. And in agent memory pipelines, where GigaRAG treats reranking as a first-class concern, the rules change again.

In This Guide

• What Is Reranking?
• Embedding Model vs Reranker: What's the Difference?
• Why Reranking Exists: Precision vs. Recall and the Context Window
• What Reranking Does And Why It Exists: A Step-by-Step Guide
• How Reranking Works in a Two-Stage Retrieval Pipeline
• Cross-Encoders vs. Bi-Encoders: The Core Trade-Off
• Types of Reranking Models
• Reranking in Agent Memory and Stateful RAG Pipelines
• What Reranking Cannot Do
• When to Skip Reranking (and What to Do Instead)
• Evaluating Reranking: Metrics That Matter
• Common Mistakes When Implementing Reranking
• Final Thoughts on What Reranking Does and Why It Exists

What Is Reranking?

Reranking is a second scoring pass that reorders already-retrieved candidates so the most relevant ones sit at the top before they reach the LLM. First-stage retrieval grabs a wide net of possibly relevant chunks. Reranking then reads each candidate against the query more carefully and pushes the best matches up.

Here's why it exists. Your vector database retrieves 20 chunks. The one that actually answers the user's question sits at position 8. If you stuff all 20 into the context window, you waste tokens and dilute attention. If you take only the top 5, you miss the answer entirely. Reranking fixes that gap by reordering the 20 so the right chunk lands in the top 5.

Reranking vs. retrieval: the two-stage model

Retrieval and reranking solve different problems. Retrieval optimizes for recall: find everything that might be relevant, fast. It uses embeddings or BM25 to score thousands of documents in milliseconds. The trade-off is precision. Embedding similarity is approximate, and it misses nuance like negation, word order, or exact phrasing.

Reranking optimizes for precision. It takes the top 20 to 100 candidates from retrieval and scores them again with a model that reads the query and document together. That joint reading catches what embedding similarity misses. The cost is speed: reranking 100 pairs takes longer than embedding search over a million vectors. So you don't rerank everything. You rerank only what retrieval already surfaced.

What a reranker actually outputs

A reranker outputs a relevance score for each query-document pair, then sorts the list by that score. The scores themselves are usually not meaningful in absolute terms. A score of 0.8 doesn't mean "80% relevant." It means "more relevant than the candidate that scored 0.7." What matters is the order.

In practice, you set a cutoff. Rerank 50 candidates, keep the top 5 or 10, and pass those to the LLM. The reranker's job is done once the order is set. It doesn't generate text, doesn't summarize, doesn't add context. It just decides what deserves the limited space in your context window.

[!note] Reranking reorders the candidates your retriever already returned; it cannot surface a document that was never retrieved in the first place.

Embedding Model vs Reranker: What's the Difference?

Why Reranking Exists: Precision vs. Recall and the Context Window

First-stage retrieval is built for recall, not precision. It casts a wide net because missing a relevant chunk is worse than returning a few irrelevant ones. The problem is that the net comes back full of noise.

The recall problem in first-stage retrieval

Embedding search and BM25 both optimize for speed over accuracy. An embedding model compresses a document into a vector, and that compression loses information. Two chunks can have similar vectors but mean different things. BM25 matches on exact terms, so it misses synonyms and paraphrases entirely. Both approaches return candidates that are loosely related to the query, not precisely relevant.

Here's what happens in practice. You retrieve 50 chunks for a query about "how to configure connection pooling in Postgres." The top results include a chunk about connection pooling in MySQL, one about Postgres authentication, and one about generic database performance tuning. Only one chunk actually answers the question. The rest are noise that will compete for space in the context window.

Retrieval doesn't know any better. It wasn't designed to read carefully. It was designed to be fast over millions of documents. That's the trade-off you accept when you build the first stage.

Context window limits force a trade-off

You can't send all 50 retrieved chunks to the LLM. Context windows are finite, and even large ones come with real costs. Every token you spend on irrelevant context is a token you don't spend on the actual answer. Worse, irrelevant context dilutes the LLM's attention. Models perform better when the relevant information sits close to the top of the prompt, not buried under noise.

So you face a choice. Take the top 5 chunks and risk missing the answer, or take all 50 and waste tokens while degrading output quality. Neither option works well. The correct chunk might sit at position 12. If you cut at 10, you lose it. If you keep everything, you pay for 40 chunks of noise.

This is the gap reranking exists to close. It gives you a way to keep the wide net from retrieval but still send only the best candidates to the LLM.

How reranking improves precision without sacrificing recall

Reranking doesn't change what retrieval found. It changes the order. The recall is already fixed by the first stage: if the correct chunk is in the candidate set, it stays in the candidate set. Reranking just moves it up.

That's the key insight. You retrieve 50 candidates to preserve recall. You rerank those 50 to restore precision. Then you cut to the top 5 or 10, confident that the most relevant chunks are the ones that survived. You get the best of both stages: broad coverage from retrieval, careful ordering from reranking.

The cost is latency and compute. Reranking 50 pairs with a cross-encoder takes longer than embedding search over a million vectors. But it's a one-time cost per query, and it buys you a context window filled with signal instead of noise. For most RAG pipelines, that trade is worth it.

[!tip] For agent memory and stateful pipelines, rerank against the current turn's query rather than the original session query, and keep a small recency or importance signal in the mix so long-lived memory does not get buried by a single strong lexical match.

What Reranking Does And Why It Exists: A Step-by-Step Guide

1. Retrieve a wider candidate set than you plan to pass to the LLM.
2. Choose a reranker that fits your latency and cost budget.
3. Score each query-document pair with the reranker.
4. Reorder candidates by reranker score and truncate to your context budget.
5. Assemble the final context from the reordered top results.
6. Measure answer quality and latency before and after to confirm the gain.

How Reranking Works in a Two-Stage Retrieval Pipeline

A two-stage pipeline splits retrieval into a coarse pass and a fine pass. Stage one finds candidates. Stage two orders them. Reranking is stage two.

Stage one: fast, approximate retrieval

You run the query against your vector index or BM25 index. This stage has to be fast because it touches a large portion of your corpus. An embedding search over a million vectors takes milliseconds. BM25 over the same corpus is similar.

The trade-off is accuracy. The first stage returns N candidates, where N is typically 20 to 100. Most of those candidates are noise. A few are gold. The first stage can't tell the difference reliably because it never reads the query and document together. It compares compressed representations, not the actual text.

That's fine. Stage one isn't supposed to be precise. It's supposed to be fast and have high recall. If the correct chunk is anywhere in those N candidates, stage one did its job.

Stage two: precise, expensive reranking

The reranker takes the query and each of the N candidates, pairs them up, and scores every pair individually. A cross-encoder reads the full query text and the full chunk text together, token by token, and outputs a relevance score. This is a much slower operation than embedding search. Scoring 50 pairs with a cross-encoder can take hundreds of milliseconds, depending on the model and hardware.

But the scoring is precise. The reranker sees the actual words, not compressed vectors. It can tell that "connection pooling in Postgres" is more relevant to a chunk about Postgres pooling than a chunk about MySQL pooling, even if the embeddings for both chunks are close.

The output is a reordered list. Same candidates, new order. The most relevant chunks move to the top. Then you cut to the top K and send those to the LLM.

Choosing N and K: the practical knobs

N is how many candidates you retrieve in stage one. K is how many you keep after reranking. Both are knobs you control.

N sets the ceiling on recall. If the correct chunk isn't in the top N from retrieval, reranking can't save it. The reranker only reorders what it's given. So N needs to be large enough to catch the right chunk most of the time. In practice, N between 20 and 50 works for most RAG pipelines. Going higher improves recall but adds reranking cost linearly.

K sets how much context you send to the LLM. K is usually 3 to 10, driven by your context window and token budget. The gap between N and K is where reranking earns its keep. You retrieve 50, rerank them, and keep the top 5. The reranker's job is to make sure the top 5 are the right 5.

The honest catch: reranking adds latency to every query. You pay for it whether or not it changes the final ordering. If your first-stage retrieval already returns the correct chunk at position 1 most of the time, reranking is overhead. If the correct chunk sits at position 8 or 12, reranking is the difference between the LLM seeing the answer and never knowing it existed.

Cross-Encoders vs. Bi-Encoders: The Core Trade-Off

The two-stage pipeline works because stage one and stage two use different architectures. Bi-encoders power retrieval. Cross-encoders power reranking. They're both transformer models, but they process text differently, and that difference is the whole game.

How bi-encoders work and why they're fast

A bi-encoder takes two inputs: the query and the document. It encodes each one separately, producing a vector for the query and a vector for the document. The two encoders never see each other's work. The relevance score is just the cosine similarity between those two vectors.

This separation is what makes bi-encoders fast. You can precompute document vectors once and store them in a vector index. At query time, you encode the query once, then run a nearest-neighbor search. The document encoding cost is paid upfront, not per query.

The trade-off is that the query and document never interact during encoding. The model can't learn that "bank" in the query means "river bank" and not "financial institution" until after both vectors are already computed. By then it's too late. The vectors are fixed.

How cross-encoders work and why they're accurate

A cross-encoder takes the query and document, concatenates them into a single input sequence, and runs them through the transformer together. Every token in the query attends to every token in the document. The model sees the full interaction.

This joint processing is why cross-encoders are more accurate. The model can resolve ambiguity in context. It can tell that "how to train a model" is about machine learning, not about training a person, because it reads the full query and full document side by side.

The cost is that nothing is precomputable. You can't store a vector for a document and reuse it. Every query-document pair has to be run through the full transformer from scratch. Scoring 50 candidates means 50 forward passes. That's why cross-encoders are used for reranking, not retrieval.

Why you can't just use a cross-encoder for everything

The honest answer is latency and cost. A cross-encoder over a million documents would take minutes per query. A bi-encoder over the same corpus takes milliseconds. The cross-encoder is more accurate, but it's not a million times more accurate. It's maybe 10 to 20 percent better at ranking the top candidates.

So you use both. The bi-encoder narrows a million documents down to 50. The cross-encoder orders those 50 precisely. Each model does what it's good at.

The main catch is that cross-encoder quality depends on the bi-encoder's recall. If the right document isn't in the top 50, the cross-encoder never sees it. You can't fix a retrieval miss with a better reranker.

Types of Reranking Models

Cross-encoders are the default, but they're not the only option. The field splits into four families, each with a different scoring mechanism and a different latency profile.

Cross-encoder rerankers

A cross-encoder scores one query-document pair at a time. It concatenates both into a single sequence, runs the full transformer, and outputs a relevance score. This is the most accurate family because every token attends to every other token. The cost is one forward pass per candidate. Scoring 50 candidates means 50 passes.

Listwise and pairwise rerankers

Pairwise models compare two documents against the same query and decide which is more relevant. Listwise models score an entire candidate list at once, optimizing the full ordering rather than individual pairs. Listwise is closer to what you actually want: a ranked list, not a set of independent scores. But listwise models are harder to train and less common in open-source form. Pairwise models sit in between, more accurate than pointwise cross-encoders on some benchmarks, but slower because they need multiple comparisons per candidate.

LLM-based rerankers

You can use a general-purpose LLM to rerank by prompting it: "Given this query, rank these 10 passages from most to least relevant." The model reads everything and returns an ordering. This works surprisingly well for small candidate sets, and it handles nuanced relevance signals like tone, recency, and user intent that a trained cross-encoder might miss. The catch is cost and latency. An LLM call for reranking is slower than a cross-encoder forward pass, and you pay per token. It's practical for reranking 10 to 20 candidates, not 100.

Hybrid and learned fusion approaches

Hybrid reranking combines multiple signals: a cross-encoder score plus BM25 lexical overlap plus metadata filters like recency or source authority. You weight the signals and produce a final score. Learned fusion goes further, training a small model to combine the signals optimally. This is useful when your relevance depends on factors a single reranker can't see, like "prefer documents from the last 30 days" or "penalize paywalled sources." The trade-off is complexity. You now have multiple models and a weighting scheme to tune and maintain.

Reranking in Agent Memory and Stateful RAG Pipelines

Most reranking tutorials assume a single query in, a single ranked list out. That assumption breaks the moment your pipeline has memory. Agents don't ask isolated questions. They carry conversation history, user preferences, and previously retrieved context into every turn. A reranker that scores only against the current query is blind to half the signal.

Why single-turn reranking assumptions break in agents

In a stateless RAG pipeline, the query is the only relevance signal you have. "What's the capital of France?" retrieves documents about Paris. Simple. But an agent mid-conversation might ask "what about the weather there?" The word "there" carries no standalone meaning. A reranker scoring candidates against "what about the weather there?" alone will produce garbage, because the actual query is "what's the weather in Paris, given we were just discussing France."

The same problem appears with user preferences. If a user has spent three turns asking about vegetarian recipes, a reranker scoring "good dinner ideas" against the raw query might rank a steak recipe first. The conversation state says otherwise. Single-turn reranking throws away context that changes what "relevant" means.

Memory-aware reranking: scoring against conversation state

The fix is to expand what the reranker scores against. Instead of passing only the current query, you pass a constructed context: the current query plus a compressed summary of relevant conversation history, plus any user preferences stored in memory. The reranker then scores candidates against this enriched input.

In practice, this means your reranking step sits downstream of a memory retrieval step. First, pull the relevant memory: recent turns, stated preferences, entities the user has referenced. Second, build the scoring context. Third, rerank candidates against that context. The reranker itself doesn't change. What changes is the input you feed it.

Some builders go further and rerank memory entries themselves before injecting them into the scoring context. If your memory store holds 200 past interactions, you don't want all of them in the reranker's input. A lightweight first pass over memory, using embeddings or BM25, pulls the 10 most relevant past turns. Those get concatenated with the current query and passed to the reranker. Two-stage retrieval, applied twice: once over documents, once over memory.

Practical considerations for agent memory builders

The main catch is latency. Every reranking pass in an agent loop adds time to a turn that already includes memory retrieval, tool calls, and LLM generation. If your reranker takes 200ms and you run it on every turn, that's 200ms added to every user interaction. For a chat agent, that's noticeable. You need to decide whether reranking runs on every turn or only when retrieval confidence is low.

Memory drift is a second problem. Conversation state changes fast. A reranker tuned on static query-document pairs may not handle the noisy, overlapping, sometimes contradictory content of a multi-turn conversation. You'll need to test whether your reranker actually improves answer quality in a stateful setting, not just on a static retrieval benchmark.

The honest answer is that memory-aware reranking is still an open problem. The rerankers themselves are trained on single-turn data. You're adapting them to a task they weren't built for. It works, but it needs evaluation on your specific agent, with your specific memory store, before you trust it.

What Reranking Cannot Do

Reranking is a scoring pass over candidates you already retrieved. It cannot conjure what was never fetched, and it cannot make a bad first stage good. Builders who treat it as a fix for broken retrieval end up with the same wrong answers, just ranked more confidently.

Reranking cannot fix bad retrieval

If your first-stage retrieval never surfaces the right document, reranking has nothing to work with. It reorders what it's given. A reranker scoring 50 irrelevant chunks will produce 50 irrelevant chunks in a new order. The failure happens upstream, and no amount of cross-encoder precision downstream recovers a document that was never in the candidate set.

This matters because first-stage recall is where most RAG pipelines actually fail. Embedding models miss synonyms, BM25 misses semantic matches, and hybrid search still has blind spots. Reranking multiplies the quality of retrieval, but anything times zero is zero. Before you add a reranker, measure recall@50 or recall@100 on your retriever. If the right document isn't in the top 50, reranking won't help.

Reranking cannot add missing information

A reranker scores relevance between a query and a candidate. It does not generate new content, fill gaps in your knowledge base, or synthesize across documents. If the answer to a user's question simply isn't in your data, reranking will confidently rank the least irrelevant chunk first. The LLM then hallucinates from that chunk, and the reranker has made the hallucination look well-sourced.

The same applies to stale data. A reranker can't tell that a document is outdated, only that it matches the query. If your knowledge base holds last year's pricing and the user asks for current pricing, reranking will surface the old document with high confidence. Freshness is a retrieval and indexing problem, not a ranking problem.

Latency and cost: the hidden price

Cross-encoder rerankers run a full transformer forward pass for every query-candidate pair. Rerank 50 candidates and you've run 50 inference calls. That's not free. A typical cross-encoder adds 50 to 200 milliseconds per query, and that's before you account for batching overhead or cold starts. In a multi-turn agent loop, that cost compounds on every turn.

The cost isn't just latency. Running a reranker means either hosting a model yourself or paying per token to an API provider. For high-volume pipelines, reranking can cost more than the first-stage retrieval it's improving. You need to measure whether the precision gain justifies the added milliseconds and dollars. Sometimes it does. Often, for simple queries over a small corpus, it doesn't.

When to Skip Reranking (and What to Do Instead)

Reranking is not a default. It's a tool you reach for when first-stage retrieval is good but not precise enough, and when the cost of a wrong answer outweighs the cost of the extra inference. Plenty of pipelines run fine without it. The honest question isn't "should I add reranking?" but "what breaks if I don't?"

Signals that reranking is unnecessary

You can skip reranking when your corpus is small. If you're retrieving from a few hundred documents, the first stage already surfaces the right chunk most of the time. Reranking 20 candidates from a 300-document index is solving a problem you don't have.

You can also skip it when your queries are short and factual. "What's the capital of France?" doesn't need a cross-encoder. The embedding similarity between the query and the answer is unambiguous. Reranking earns its keep on ambiguous, multi-intent, or long-tail queries where the first stage returns several plausible candidates and the right one isn't obvious.

The clearest signal: measure precision@5 without a reranker. If your pipeline already puts the correct chunk in the top 5 for 90% or more of your test queries, reranking buys you almost nothing. You're adding latency to fix a problem that doesn't exist at your scale.

Alternatives: better embeddings, hybrid search, query rewriting

Before you add a reranker, fix the cheaper things first.

Better embeddings are the most direct lever. If your first-stage retriever uses an old or generic embedding model, switching to a stronger one (or fine-tuning on your domain) often lifts precision more than a reranker would, at zero added inference cost per query. The embedding is precomputed and stored. Retrieval stays fast.

Hybrid search combines dense embeddings with BM25 keyword matching. It catches exact terms and rare identifiers that embeddings blur. For code search, product catalogs, or any corpus with lots of proper nouns, hybrid search alone can close much of the gap that reranking would otherwise fill.

Query rewriting attacks the problem from the user's side. If queries are vague or under-specified, rewriting them into something more precise before retrieval improves the candidate set itself. A reranker can't fix a bad query. A rewrite can.

A simple decision checklist for builders

Run through this before you add a reranker:

• Is recall@50 below 80%? Fix retrieval first. Reranking won't help.
• Is precision@5 already above 90%? Skip reranking. You don't need it.
• Are your queries short and unambiguous? Skip it.
• Is your corpus under 1,000 documents? Skip it.
• Do you have a strict latency budget under 100ms? Skip it, or rerank fewer candidates.
• Are wrong answers expensive (medical, legal, financial)? Add it.
• Do you serve ambiguous, multi-turn, or long-tail queries? Add it.

If you answer "skip it" to most of these, spend your effort on embeddings, hybrid search, or query rewriting instead. Reranking is a precision tool for a specific failure mode: good recall, noisy ranking, high cost of error. Outside that, it's overhead.

Evaluating Reranking: Metrics That Matter

You can't tell if a reranker helps by eyeballing a few answers. You need numbers. The catch is that retrieval metrics and answer quality don't always move together. A reranker can lift nDCG while the final answer stays the same, or improve answers while nDCG barely shifts. Measure both, or you'll draw the wrong conclusion.

nDCG and MRR: ranking quality metrics

nDCG (normalized discounted cumulative gain) measures how well your ranked list matches an ideal ordering. It rewards putting relevant documents high and penalizes burying them. The "discounted" part means position matters: rank 1 counts more than rank 5. A reranker that moves the right chunk from position 8 to position 2 produces a visible nDCG jump.

MRR (mean reciprocal rank) is simpler. It asks one question: where does the first relevant result land? If it's at rank 1, you get 1.0. Rank 2 gives 0.5. Rank 10 gives 0.1. MRR is blunt but useful when your pipeline only cares about the single best hit, which is often the case in RAG.

Use nDCG when you want a full picture of ranking quality. Use MRR when only the top hit matters. Both are cheap to compute on a labeled test set.

Precision@k and recall@k: retrieval metrics

Precision@k answers: of the top k results, how many are relevant? If you retrieve 10 chunks and 6 are on-topic, precision@10 is 0.6. Recall@k answers: of all relevant chunks in the corpus, how many did you retrieve? If there are 4 relevant chunks and you got 3 in your top 10, recall@10 is 0.75.

Reranking changes precision@k more than recall@k. The candidate set is fixed before the reranker runs, so recall can't improve. What changes is which candidates land in the top k. Track precision@5 before and after adding a reranker. If it doesn't move, the reranker isn't earning its latency.

End-to-end evaluation: does the answer improve?

Ranking metrics are proxies. The metric that actually matters is whether the LLM produces better answers. Build a test set of 50 to 100 queries with expected answers, run the full pipeline with and without the reranker, and compare outputs.

Score answers on correctness, not fluency. A human or an LLM judge can do this. Track the percentage of queries where the answer improves, stays the same, or gets worse. If 10% improve and 5% get worse, the reranker is a net positive but a small one. If answers don't change at all, your context window was never the bottleneck.

The honest read: ranking metrics tell you the reranker is doing its job. End-to-end evaluation tells you whether that job matters.

Common Mistakes When Implementing Reranking

Most reranking failures aren't model problems. They're wiring problems. Builders bolt a cross-encoder onto a pipeline, watch latency climb, and rip it out a week later. The mistakes below account for most of that churn.

Reranking too few candidates

If you retrieve 50 chunks but only rerank the top 10, you've capped what the reranker can do. The whole point is rescuing relevant chunks that first-stage retrieval buried at position 15 or 20. Rerank the full candidate set you can afford, then truncate. A common budget: retrieve 50 to 100, rerank all of them, keep the top 5 to 10 for the LLM. Reranking 10 candidates to pick 5 is barely better than no reranking at all.

Ignoring latency and cost budgets

Cross-encoders are slow. Scoring 100 query-document pairs can add 200 to 500 milliseconds, depending on model size and hardware. That's fine for a batch job. It's not fine for a chat interface where users expect sub-second responses. Decide your latency ceiling before you pick a reranker, not after. If you need reranking under 50 milliseconds, you're looking at smaller distilled models or listwise approaches, not a full cross-encoder.

Skipping end-to-end evaluation

The most common mistake is measuring the reranker in isolation. nDCG goes up, so you ship it. But if the LLM's answers don't improve, you've added latency and cost for a metric that doesn't matter to users. Run the full pipeline with and without the reranker on a held-out query set. Compare answers, not just rankings. If answers don't change, cut the reranker.

Final Thoughts on What Reranking Does and Why It Exists

Reranking exists for one reason: first-stage retrieval is imprecise, and context windows are small. You can't fit 50 chunks into a prompt, so you keep the top 5. If the right chunk sits at position 8, it never reaches the LLM. A reranker reorders the candidates so the right one moves up before truncation happens.

That's the whole job. It doesn't retrieve anything new. It doesn't understand your domain better than your embeddings do. It just scores what's already there, more carefully than the first pass could afford to.

The honest answer is that reranking is a powerful fix for a specific failure mode, not a general upgrade. If your first-stage retrieval already puts the right chunk in the top 3, a reranker adds latency and cost without changing the answer. If your retrieval is bad, reranking can't rescue chunks that were never retrieved. It earns its place only when you're retrieving wide, truncating hard, and losing good candidates in the middle ranks.

For builders running agent memory and stateful RAG pipelines, this gets more complicated. Reranking against a single query is one thing. Reranking against conversation history, user preferences, and previously retrieved context is another. GigaRAG handles reranking and retrieval in one place for exactly that use case, but the decision framework still applies: measure the end-to-end answer quality before and after, and cut the reranker if nothing changes.

Frequently Asked Questions

What does reranking do?

Reranking takes the candidate documents returned by first-stage retrieval and reorders them by a more precise relevance score. It runs a heavier model over a small set, so the most relevant items end up at the top of the context window instead of being cut off.

What are the different types of reranking models?

The main families are cross-encoders, which score query and document jointly, and late-interaction or multi-vector models that keep token-level representations. Some pipelines also use LLM-based rerankers that prompt a model to rank or score candidates directly.

What is the difference between an embedding model and a reranker?

An embedding model encodes queries and documents independently so they can be compared with vector similarity at scale. A reranker scores query and document together, which is more accurate but too slow to run over an entire corpus.

What are the different reranking techniques used in RAG systems?

Common techniques include cross-encoder scoring, late-interaction scoring, and LLM-based listwise or pointwise ranking. Some systems combine lexical signals like BM25 with dense scores before reranking to improve recall.

Does reranking always improve RAG quality?

1. Reranking helps when the right document is already in the candidate set but ranked too low. If retrieval missed the document entirely, or if the bottleneck is generation rather than ordering, reranking adds latency without fixing the problem.

How does reranking behave in multi-turn or agent memory pipelines?

In stateful pipelines the query changes each turn, so reranking must be scoped to the current turn's intent rather than the original session query. Memory-backed retrieval also benefits from blending recency or importance signals with relevance scores.

When should you skip reranking entirely?

Skip it when retrieval recall is already high, when latency budgets are tight, when the corpus is small enough that top-k is reliable, or when the failure mode is generation rather than ranking. In those cases, invest in retrieval quality or prompt design first.

About GigaRAG

GigaRAG is for agent memory and RAG pipeline builders. get this right. Whether you are working through what reranking does and why it exists or something adjacent, we publish what we have actually tested, including where it falls short.