The Complete RAG Pipeline Guide — Eight Practical Techniques That Maximize Retrieval Quality

"Most of the reasons RAG fails come not from generation (Gen) but from the quality of chunking."

These days the phrase "RAG is Dead" is trendy, but most of the data handled in practice is not code but unstructured text (contracts, wikis, customer support records, and so on). For that kind of data, RAG is still essential. The question is how you implement it.

1. The Basic Structure of a RAG Pipeline

Document → Chunking → Embedding → Store in vector DB
                                                    ↓
User question → question embedding → similarity search → extract relevant chunks → pass to LLM → generate answer

1-1. Chunking: Cutting a 100-Page Document into Pieces

Why is it needed?

• Putting a 100-page document into an LLM as-is exceeds the context window
• It must be cut into an appropriate size (usually 300-1,000 tokens) to be retrievable

Practical pitfalls of chunking:

Most real-world RAG failures come from "chunking quality being garbage."

1-2. Embedding: Turning Text into Numbers

Analogy: It is like similar products in a mart (frozen dumplings and frozen gyoza) being placed on similar shelves. Words with similar meanings are placed close together in numerical space.

Key point: Even if the exact word is not included, it can find documents that match by "meaning." This is semantic search.

1-3. Vector DB: A Store Dedicated to Similarity Search

2. Eight Practical Techniques That Maximize Retrieval Quality

Technique 1: Hybrid Search

Problem: Semantic search alone cannot find technical terms.

Solution: Run both kinds of search at once and merge the results.

Why do you need both? In specialized domains such as finance and manufacturing (semiconductors), the exact match of special abbreviations like "CBD chamber" or "BWG" often matters more than semantic search.

In practice, the standard is to run both searches at once and fuse (hybridize) the results.

Technique 2: Reranker

Analogy: The first-stage search is the step that picks 10 candidates by "age, job, region." The reranker lays those 10 profiles side by side with the question and re-evaluates, like a second-round interview, "does this really contain the answer to this question?"

Practical trade-offs:

• Accuracy: definitely goes up
• Response speed: gets slower (extra computation)
• Personal tools that value reliability: essential
• Hundreds of concurrent commercial users: needs careful design

Technique 3: Contextual Retrieval

Problem: When a 100-page financial report is cut up, if some chunk is left with only "revenue grew 3% versus the previous quarter," the context of which company and when disappears.

Solution (announced by Anthropic, September 2024): Before cutting the chunks, use an LLM to automatically prepend a short context like "this chunk is part of company XX's Q2 2024 report" to each chunk.

Effect:

• Retrieval failure rate reduced by up to 67%

Cost-saving tip: Calling an LLM on every chunk is expensive, so running a binary filter first ("does this chunk contain meaningful information?") can cut costs dramatically.

Technique 4: Late Chunking

Existing approach: Cut the document first → embed each piece independently (surrounding context is lost)

Late Chunking:

1. Feed the entire document as-is into a long-context embedding model
2. Record the overall connectivity as numbers
3. Split into chunks at the end

Analogy: Rather than cutting a movie scene by scene and summarizing each, it is the approach of "watching the whole movie from start to finish once, then summarizing scene by scene." The surrounding context is cleanly preserved.

Technique 5: Tuning the Right Chunk Size

Practical recommendation: 500-1,000 tokens. Leaving 20-50% overlap between chunks reduces context breaks.

Technique 6: Metadata Filtering

Pre-filtering by metadata before retrieval narrows the search scope and improves both accuracy and speed at once.

Search query: "Q2 2024 revenue"
Filter: { "year": 2024, "quarter": "Q2", "type": "finance" }

Technique 7: Query Expansion

Have an LLM expand the user's short question into several forms and then search.

Original question: "What are RAG's downsides?"
Expanded questions:
1. "What are RAG's limitations and problems?"
2. "In what cases does RAG fail?"
3. "Downsides and cautions of Retrieval Augmented Generation"

Searching with 3-5 variant questions rather than a single question greatly improves search recall.

Technique 8: Debugging — Inspect Ranks 1-20 Yourself

The first thing to do when RAG is not working:

1. Collect 5-10 questions that are frequently used in the real setting
2. Visually inspect the chunks ranked 1st-20th returned by vector search
3. Figure out "why is this data ranked 3rd?" and "why is the needed data outside the ranking?"

The first step of RAG optimization (context engineering) is this debugging.

3. The Real Core of the "RAG is Dead" Debate

Why is RAG "dead" in the world of code?

• Source code is structured data (function names and file paths are exact)
• Finding a specific string directly with grep is more accurate and safer than vector search
• Coding agents like Cursor and Claude Code replace it with Agentic Search

Why is RAG alive in the business world?

Most of the data handled in practice:

• Customer support records
• Internal wikis
• Contracts
• Marketing reports

For such unstructured text, semantic search (= the domain of RAG) — which finds semantically connected words like "indemnity clause" or "penalty" even when the word "damages" is absent — is essential.

4. Practical RAG Building Checklist

□ Chunking: check whether documents with tables/images need manual preprocessing
□ Appropriate chunk size (500-1,000 tokens) + overlap (20-50%)
□ Enable hybrid search (Dense + Sparse/BM25)
□ Decide whether to apply a reranker (accuracy vs speed trade-off)
□ Configure metadata filtering (date, category, source)
□ Apply contextual retrieval (automatically prepend context to chunks)
□ Debug: visually inspect ranks 1-20 with 5-10 real questions
□ Monitor: continuously track retrieval failure rate and response quality

Summary: RAG Pipeline Maturity Check

RAG is not dead. Only badly implemented RAG died.

Related posts:

• The Truth About AI Agent Token Costs
• The Complete Web Search MCP Guide
• GPU VRAM Allocation Structure and the KV Cache Bible