Context window

What it is

A context window is the amount of text and code, plus the conversation so far, a model can hold at once when it's reasoning about an answer. Every model has a limit; feed it more than that and the earliest parts get dropped or summarized. Bigger context costs more to process, since the model is reading, and paying attention to, more of it on every step, whether or not that extra material actually helps the answer.

Why it matters to you

This is where a chunk of your model spend actually goes. A task that reads a huge file, or one carrying a long back-and-forth history, burns far more context, and far more cost, than a tightly scoped task with a clean, focused sandbox. Keeping tasks well-scoped is a direct lever on the bill, well beyond mere tidiness; it's usually the single cheapest change a team can make to what they're already spending.

How it works in CodeHerder

CodeHerder's own cost data shows model choice and context usage as the real lever on what a task costs, separate from which plan tier you're on. The reasoning behind treating the reasoning model, and how much context it's fed, as the cost lever worth watching, plus what that looked like in practice on a real batch of tasks, is laid out in the reasoning model is the cost lever post.

• The reasoning model is the cost lever →

What people get wrong

A bigger context window sounds like it should simply be better, since it means the model can "see more." It can, but every extra token in that window is something the model has to read and pay for on every turn, so a huge window fed with irrelevant material can make a task slower and pricier without making the answer any better. The fix usually isn't a smaller model, it's a smaller, better-scoped task with less to read in the first place, and a sandbox that only contains what the task actually needs.