1.3 Context windows and why "just add more text" fails

What a context window is

A model’s context window is the amount of text (measured in tokens) it can consider at once when generating the next token. Everything you want the model to follow—requirements, examples, code, logs—must fit inside that window along with the model’s own output as the conversation grows.

In practice, think of context like a workbench with limited space. If you keep piling things on, important pieces get buried or pushed off the bench.

What counts toward the context window

Context isn’t just what you typed in the last message. It usually includes:

• System/developer instructions: “house rules,” safety constraints, and any fixed guidance set by the environment.
• Your messages: specs, examples, pasted code, logs, and clarifications.
• The model’s messages: all prior answers (often the largest chunk over time).
• Tool outputs (if used): results from file reads, searches, test runs, etc.

Two practical consequences:

• If you keep long model outputs in the thread, you’re spending your budget on past drafts instead of the current truth.
• If you paste huge chunks “just in case,” you often reduce quality because you dilute the signal and increase conflicts.

Why “just add more text” fails

There are multiple ways big context makes the model worse. The most common are:

1) Truncation (the oldest details fall off)

When the conversation exceeds the window, something has to go. Typically the earliest messages are dropped or summarized by the system. That can remove the constraints that made earlier outputs good.

Result: the model “forgets” requirements you thought were permanent.

2) Attention dilution (signal gets buried)

Even when everything technically fits, a large context can make it harder for the model to focus on what matters. Important constraints can be outnumbered by irrelevant details, older drafts, or noisy logs.

Result: the model follows the vibe of the thread, not the latest spec.

3) Conflicting instructions (you accidentally create contradictions)

In long threads, you often accumulate statements like:

• “Use TypeScript.”
• “Actually use JavaScript.”
• “Don’t use any libraries.”
• “Use this library.”

The model can’t reliably resolve contradictory constraints unless you explicitly reset and restate the current truth.

Result: it “randomly” follows an older instruction or merges incompatible ones.

4) Cognitive load (you ask for too many things at once)

Huge prompts often bundle multiple tasks: design, implement, refactor, migrate, document, and test—simultaneously. That’s hard for humans too. The model usually responds with a large, brittle output that looks complete but contains many small errors.

Result: a big rewrite that’s harder to review, harder to verify, and slower overall.

Symptoms you’ve overflowed or polluted context

These are strong signals your context strategy needs tightening:

• Instruction drift: it keeps ignoring constraints you know you stated.
• Inconsistent outputs: the same prompt yields radically different interpretations within the same thread.
• “Rewrite the world” behavior: it changes many files or rewrites code you didn’t ask to touch.
• Stale references: it refers to functions/files that no longer exist (from earlier drafts).
• Overconfident contradictions: it asserts two incompatible facts across adjacent messages.
• Verbose filler: long explanations that don’t move the implementation forward.

The context budget technique

Treat context like a budget you allocate intentionally. Before you paste anything, decide what you’re buying with that token spend.

A practical allocation

• Spec budget: what you want + constraints + acceptance criteria.
• Evidence budget: failing test output, logs, or repro steps.
• Working-set budget: only the files/functions needed for the next diff.
• History budget: a short state summary (not the whole conversation).

Example: a “budgeted” prompt

Goal (1 sentence):
Fix the auth redirect bug without changing the login API.

Constraints:
- Keep diff small; no rewrites.
- Do not change: src/auth/api.ts

Evidence (exact):
- Repro steps:
  1) ...
- Error log:
  ...

Working set:
- Relevant files: src/auth/router.ts, src/app/routes.ts

Definition of done:
- Redirect works for /login and /logout flows
- Existing tests pass + add one regression test

Structuring context so instructions stick

When you want reliable output, your prompt should separate stable rules from task-specific details.

Order matters

1. Stable rules: language/runtime, style, safety constraints, “diff-only,” “ask clarifying questions.”
2. Task spec: the smallest next outcome, with acceptance criteria.
3. Evidence: exact errors, failing tests, reproduction steps.
4. Working set: only the relevant code snippets/files.

This order helps because it anchors the model’s behavior before it sees the messy details.

Multi-file work without dumping the repo

Large projects require discipline. The best pattern is to create a map and then work in a working set.

Step 1: Build a repo map (once per project)

Create a short index of the codebase: top-level folders, key modules, entrypoints, and how data flows. Keep it small and update it occasionally.

Why: it reduces the model’s need to guess structure and reduces repeated paste-dumps.

Step 2: Work in a small “working set”

For each change, constrain scope:

• Which files are allowed to change.
• Which functions are relevant.
• What tests or checks will verify correctness.

Why: you keep context small, and diffs stay reviewable.

Summaries that preserve momentum

The goal of summarizing is not compression for its own sake—it’s preserving the current truth so you can keep iterating without dragging old drafts.

What to include in a good summary

• Current state: what exists now (features, behavior, known bugs).
• Decisions: what architecture choices are locked in and why.
• Constraints: what must not change.
• Open questions: what’s unresolved.
• Next step: the smallest next diff you want.

Example: a “state summary” block

State summary (current truth):
- App: Node + Express; auth via JWT in Authorization header.
- Known issue: refresh flow fails when token expires; users get stuck in a redirect loop.
- Constraints: do not change auth token format; keep routes stable.
- Working files: src/auth/middleware.ts, src/auth/refresh.ts
- Next step: add a regression test for redirect loop, then fix middleware ordering.

When to start a new session

Starting fresh is often faster than fighting a polluted thread. Consider a reset when:

• You’ve done multiple large changes and the thread contains many outdated drafts.
• The model repeatedly contradicts itself or ignores constraints.
• You’re switching tasks (e.g. from feature work to security review).
• You need a different “voice” (e.g. from brainstorming to precise diff work).

Copy-paste templates

Template: have the model ask for context

Before proposing changes, list exactly what additional context you need.
Be specific (file paths + function names). If you can proceed without it, say so.

Template: budgeted prompt for code changes

Goal:
[one sentence]

Constraints:
- Diff-only changes
- Files allowed to change: [...]
- Must not change: [...]

Current truth (summary):
- ...

Evidence:
- ...

Acceptance criteria:
- ...

Deliverable:
1) Plan (3–7 steps)
2) Minimal diff
3) What to run to verify

Template: end-of-session summary

Summarize the current state in 8–12 bullets:
- What works now
- What changed in this session
- Known issues
- Constraints
- Next step (smallest diff)

Where to go next

• 1.4 Temperature, randomness, and creativity knobs
• 1.5 Hallucinations: the predictable failure mode