Why Context Is Everything — The Single Variable That Decides Agent Performance

To get to the point, an agent's performance is decided not by the model but by the context. Give the same GPT or Claude different information and the answer changes. Here is why context is everything, and how to fill it.

1. The Model Is the Engine, Context Is the Fuel

The same car goes fast or slow depending on whether you put in premium gasoline or water. AI models are the same. A model is just an engine that picks the next word by probability; what it looks at to decide is determined by context. Without information it fills the gap with hallucination; with information it gets the answer right. When an agent gives a wrong answer, before blaming the model you should first look at what you showed it.

2. Why Context: Three Reasons

First, the model has no memory. When the conversation ends, it forgets. Past conversations, files, and tool results must be carried in context for it to continue working. Context is the agent's memory.

Second, the model does not know the world. It does not know what happened after training ended, your company's code, or today's news. You have to bring it to the model through RAG, search, and file reading. Context is the agent's eyes.

Third, instructions are not interpretation but material. Three related files and one example raise performance more than a verbose system prompt. Context is the agent's blueprint.

3. What Happens When Context Is Empty

The bill shock we covered in posts 43 and 44 also has its root in context leakage. When 70 irrelevant skills and the entire conversation history are sent every time, a single hello burns 20,000 tokens. Loading only what is needed captures money and performance at the same time.

4. How to Fill It: The 5 Principles of Context Engineering

First, load in order of relevance. Put files and records directly related to the question first and background later. The model weighs what comes earlier more heavily.

Second, give one example. Showing the desired output format directly eliminates parsing failures. One example beats ten lines of explanation.

Third, cut it up. Do not put a 100-page document in whole; search and give only the fragments related to the question. RAG and rerankers do this work. The hybrid search and Late Chunking from post 51 are the standard.

Fourth, discard. Summarize or throw away old conversations and the intermediate steps of tools already used. The context window is finite, so you must free space for new information.

Fifth, verify. Add a step where the agent asks itself whether the context it has is sufficient before answering. Asking a judgment model like Jev for pass/fail is the synergy from post 56.

5. Context Size by Model: As of September 2026

You should not believe advertised figures as they are, but they are usable for tier comparison. The following is based on each company's official documentation.

For local models, the spec sheet numbers are not the whole story. Serving infrastructure such as vLLM and Ollama often cuts them lower because of memory constraints, so check the endpoint documentation.

6. The Impact of Context: Bigger Is Not Better

First, effective context is 50-80% of the advertised figure. This is the consistent conclusion of independent benchmarks. A 1M model has a high-quality recall range up to 600K-700K, and accuracy drops noticeably beyond that. Design with 60% of the advertised figure as your working ceiling.

Second, it loses the middle. Content buried in the middle of a long context tends to be ignored by the model. So place the essentials at the front and back, and fill the middle with background that can be discarded. This is where the saying comes from that a well-polished 64K beats a lazy 256K.

Third, cost and speed are proportional to length. Putting a 1M document into Gemini costs 2 dollars, and into Opus 5 dollars. GPT bills at 2x once you cross the 272K boundary. An agent loop resends the whole thing every turn, so length is money. The bill shock of posts 43 and 44 is exactly this structure.

Fourth, there is a size that fits each purpose. Chat and consultation are 8K-32K, a single document 50K-200K, multi-document synthesis 200K-600K, whole-codebase reasoning 500K-1M, and multi-step agents 100K-500K. Work that needs 10M is a tiny minority, such as enterprise document search.

Why Big Is Bad: Five Causes

First, attention grows thin. Attention is a softmax, so as tokens increase the weights spread out. Choosing 1 out of a thousand and choosing 1 out of a million differ in difficulty. The longer the length, the more the score given to the essentials is shaved down.

Second, it loses the middle. Because of the structure of positional encoding, the model sees the front and back well and lets the middle slip. Long-context benchmarks such as RULER and LongBench have confirmed this repeatedly. It may pass a needle-in-a-haystack test yet omit the middle evidence in a real synthesis task.

Third, memory is money. The KV cache grows in proportion to length. The cache for a 1M context eats tens of GB, driving up serving cost and slowing the first token. The longer you put in, the later the answer comes.

Fourth, noise contaminates the answer. When irrelevant documents get mixed in, the model pulls their content in as material for hallucination. This is why reducing 100 search results to 10 with a reranker is more accurate.

Fifth, testing becomes meaningless. With a large context you cannot trace what influenced the answer. Debugging and reproducibility break, and evaluation is left to luck.

So the principle is one. Not as much as you can put in, but only as much as is needed. Selection comes before size.

7. Strategy by Capacity

A large window does not mean you should put everything in. As irrelevant information grows, the model misses the essentials. This is called getting lost. Selection comes before size.

8. One-Line Summary

Spend half the time you use choosing a model on designing context instead. The moment you decide what to show is the moment the agent's performance is decided.

9. Practical Rules: The Five Golden Rules of Code Context

Remember just one core rule. When giving code to an agent, do not shove the entire source in whole. That single line is the fork in the road that captures both the agent's quality and its cost.

Principle 1: No Whole Originals, Cut by Block

If you say "look at this project's entire code and fix it," the agent internally loads every line into context and reasons. Shove a 5,000-line codebase in whole and the agent mixes the logic of the front with the logic of the back. The result is code that turns into a mess, or new errors that did not exist before.

Cut it by block and ask. For example, if you say "look only at the login function part of this auth.py file and fix the session handling logic," the agent concentrates only on the context around that function. At this point the gap between models almost disappears. Even the strongest model is not much different from a cheap model at the level of a few lines of code. What shows the true value of an expensive model is handling large context all at once; in small blocks, value models win by a landslide.

Principle 2: For Errors, Only the Function, Never the Whole

When an error occurs, the most common mistake is putting the whole code back in. If you say "this code throws an error, look at the whole thing again," the agent rereads all the existing context, may revert parts already fixed, or add unnecessary changes.

The right approach is to pass only the error message and the relevant function's code. If you say "the following function throws a TypeError. params' name comes out None, tell me why," the agent pinpoints the exact cause within those 30 lines of the function. Putting the whole thing in every time makes tokens snowball into a bill shock.

Principle 3: 2,000 Lines Is the Safe Zone, Split Beyond It

The stable block size verified in practice is 2,000 lines or less. At this level, Claude, DeepSeek, and even Qwen 9B-class models all produce consistent-quality results. Beyond 2,000 lines, the model's context window fills up and it forgets the later code or generates code that contradicts the earlier part.

However, you must not unconditionally cut at 2,000 lines. The basis for splitting is the function or feature unit. For example, if the user authentication module is 1,800 lines, keep it as one block, and if the database connection module is separately 1,200 lines, split that into a different block. Pass the dependency between the two modules as a one-line summary. The single line "this auth module calls the db module's get_user function to fetch user information" is enough.

Principle 4: Verify Each Block, Always Confirm Before Moving to the Next Step

Once you have written or modified a block, always run that block alone to verify it. Moving to the next block before verification is done makes errors spread like dominoes and tangle the whole system. Verification is simple. Call the block's function alone and check whether you get the expected result. Run unit tests. Check whether there are compile errors. All three must pass before you proceed to the next block.

Principle 5: Classify by Logic, Do Not Split Indiscriminately

Just because there is a rule to split at 2,000 lines or less, you must not force-split code whose logic is interwoven. For example, making each of the five methods inside one class a separate block leads the agent to produce nonsense code like "it does not understand the class's self reference relationship, so it does not know which class this method belongs to."

The basis for splitting is autonomy. If one block can run independently and produce a meaningful result, it is a split candidate. If it must directly touch another block's internal state, do not split it. Follow the boundaries of the logic, but do not cut them out artificially.

Golden Rules Summary

The Number of Code Lines Is Unrelated to Execution Speed

In general, commercial programs easily run to millions of lines of code. Does that make them slow? Not at all. If the logic is written well, the number of code lines makes little difference. What decides execution speed is not the total volume of code but the efficiency of the algorithm. An O(n) logic of 100,000 lines is faster than an O(n squared) logic of 1,000 lines. Code does not get slow because it is long; it gets slow when the logic is bad. So when writing code, do not obsess over reducing the line count; focus on designing the flow of logic correctly. The basis for dividing blocks is also not the line count but the boundaries of the logic.

Make Responsibilities Clear When Blocking

Finally, when dividing blocks you must make each block's responsibilities clear. That way you can easily tell where an error comes from. One block takes only one responsibility. A database connection error must not come out of the user authentication block. Screen output logic must not come out of the database block. When responsibilities are mixed, you cannot tell which block to look at when an error occurs, and even if you ask the agent, it modifies the wrong block. When you make a block, define in one line what it is responsible for. If code inside the block cannot be explained by that one line, you have divided it wrong.