Context Management for AI Coding Assistants: Feed Your Tool, Get Better Output

The core answer: AI coding assistants produce better code when you deliberately control what enters their context window, meaning you curate relevant files, prune stale information, and structure prompts so the model sees the right code at the right time, rather than dumping entire repositories and hoping the model finds what matters.

Why Context Quality Beats Context Quantity

Every model you use through Cursor, GitHub Copilot, Claude Code, or a raw API call has a finite context window, and that window is not free. Anthropic's Claude models currently support up to 200,000 tokens in most consumer-facing products, with some enterprise tiers offering larger windows, but token count alone doesn't tell you how well the model uses that space. Research on long-context retrieval (often called "lost in the middle") has repeatedly shown that models attend unevenly across a large context, favoring information near the start and end. Dumping fifteen files into a prompt because the window technically fits them is a common mistake. The model will read all of it, but it won't weigh all of it equally, and irrelevant files dilute attention away from the code that actually matters.

The practical implication: a 20,000-token prompt containing only the three files relevant to a bug will usually outperform a 150,000-token prompt containing the entire service plus its test suite plus five unrelated modules, even though the second prompt technically gives the model "more information." I have watched this play out repeatedly in code review sessions where a well-scoped diff gets a sharp, correct suggestion, while a full-repo dump gets a generic, hedge-everything response that ignores project-specific patterns.

How Do You Decide What Belongs in Context?

Start with the failure you're trying to prevent, not the information you have available. Three buckets are usually worth including:

• The code under direct discussion. The function, class, or file you're editing, plus its immediate callers and callees if they affect correctness.
• The contract, not the implementation. For dependencies two or three layers removed, an interface, type definition, or docstring is often more useful than the full implementation, because it tells the model what it can assume without burning tokens on logic it doesn't need to reproduce.
• Project conventions that aren't inferable from the snippet alone. Naming schemes, error-handling patterns, or architectural rules that a model would otherwise guess wrong.

What you should generally leave out: generated code, vendored dependencies, build artifacts, large data fixtures, and files that are only tangentially related "just in case." If you find yourself thinking "the model might need this," that's usually a sign it doesn't belong in the initial prompt. You can always add it in a follow-up turn if the model asks a clarifying question or produces output that reveals a gap.

Persistent Context Files: CLAUDE.md, AGENTS.md, and .cursorrules

The biggest practical improvement in this space over the last two years has been the rise of standing context files that tools read automatically instead of requiring you to paste the same background every session.

• Claude Code looks for a CLAUDE.md file at the repository root (and supports nested versions in subdirectories) to pick up project-specific instructions automatically, as documented in Anthropic's Claude Code documentation.
• Cursor supports .cursorrules (and the newer project rules format) to encode house style, forbidden patterns, and architectural notes, per Cursor's documentation.
• GitHub Copilot supports custom instructions files that scope guidance to a repository, described in GitHub's Copilot documentation.

A good CLAUDE.md or equivalent is short and specific, not a restatement of your README. Example of what actually earns its place:

1# Project context for AI assistants
2
3- Use `Result<T, E>` for error handling, never throw exceptions in `src/core`.
4- Database access goes through `repositories/`, never call the ORM directly from handlers.
5- Tests use `pytest` fixtures defined in `conftest.py`; do not create ad hoc fixtures inline.
6- The `legacy/` directory is frozen; do not refactor it unless explicitly asked.

That file costs a few hundred tokens and every session and every teammate benefit from it. The failure mode I see most often is teams writing a 3,000-word onboarding document into CLAUDE.md, which then consumes a meaningful chunk of every context window before the actual task even starts. Keep it to the rules a new senior engineer would need on day one, and push anything else into linked docs the model can fetch on demand.

Managing Context Across a Long Session

Context degrades over a long conversation, not just because of window limits but because earlier mistakes, dead-end approaches, and reverted decisions stay in the transcript and can bias later responses. If you ask a model to try approach A, reject it, then ask for approach B, the rejected approach A is still sitting in context, and models will sometimes drift back toward it or hedge between the two.

Practical mitigations:

• Start a new session after a pivot, rather than continuing to correct course in the same thread. Carry forward only the decisions that stuck, summarized in two or three sentences, not the full back-and-forth.
• Use explicit checkpoint summaries. Every 20 to 30 exchanges, ask the model to summarize the current state of the task, then start the next segment from that summary instead of the raw history. This is functionally similar to how OpenAI's guidance on long conversations recommends compressing history rather than letting it grow unbounded.
• Watch for repeated tool calls or repeated file reads in agentic tools like Claude Code or Copilot's agent mode. If the assistant keeps re-reading the same file, it's often a sign that earlier context got evicted or that the task description is ambiguous enough that the model is second-guessing itself.

What About Retrieval and Embeddings?

For codebases too large to fit in any context window, semantic search over embeddings is the standard approach, and it's what powers the "codebase-aware" features in Cursor and Copilot's workspace indexing. The trade-off is real: embedding-based retrieval is fast and scales to large repos, but it retrieves based on semantic similarity, not on actual dependency relationships, so it can miss a critical file that uses different vocabulary than your query even though it's directly relevant.

I've seen this fail concretely: asking an assistant to update a payment retry policy, where the retrieval step surfaces every file with "retry" or "payment" in it, but misses a rate limiter in a differently named module that will break under the new policy. Static analysis (actual call graphs, import trees) catches that kind of dependency that embeddings miss. If your tool supports both, lean on symbol-based or import-based context gathering for correctness-critical changes, and reserve semantic search for exploratory questions like "where in this codebase do we handle authentication."

Cost and Latency Tradeoffs You Should Actually Track

Bigger context isn't just a quality risk, it's a cost and latency cost. API pricing for most frontier models scales with input tokens, and a 100,000-token prompt costs meaningfully more per call than a 10,000-token one, on top of taking longer to process before the model produces a single output token. If you're running an assistant in a CI pipeline or an automated review bot triggered on every pull request, this compounds fast: a team merging 50 PRs a day with a 60,000-token average context per review call is paying for and waiting on a lot of tokens that a tighter diff-scoped prompt would avoid entirely.

A concrete pattern that works well in practice: for automated PR review bots, send only the diff plus a fixed budget (say, 2,000 tokens) of surrounding context per changed file, rather than the full file contents. This keeps latency predictable and avoids the bot silently timing out or truncating on unusually large changesets.

When Context Management Is the Wrong Fix

Not every bad output is a context problem. If a model consistently gets a task wrong even with tight, correct context, the issue is more likely a model capability gap (asking a smaller or older model to do multi-step architectural reasoning it wasn't built for) or a prompt clarity problem (the task itself is ambiguous, and no amount of file curation fixes an ambiguous ask). Throwing more context at a capability gap wastes tokens and time. The tell is simple: if adding more relevant files doesn't change the quality of the output at all, stop tuning context and either switch models or rewrite the instruction.

A Workflow Worth Adopting

Treat context the way you'd treat a pull request: minimal, relevant, reviewed before you submit it. Before sending a prompt to an agentic tool, ask yourself what a competent teammate would need to see to make the same change, no more. Maintain a lean CLAUDE.md or .cursorrules file and revisit it quarterly, since stale conventions in that file actively mislead the model rather than just doing nothing. When a session goes long or pivots direction, summarize and restart rather than letting the transcript accumulate dead ends. And track your token spend per task type for at least a month; if you don't have a number for what a typical review or refactor costs in tokens, you can't tell whether a context strategy is actually saving you anything or just moving the cost around.