Date: October 2026 | Author: Zixuan Wang | 10min

<aside> 💡

TL;DR. Tool observations fill most of a long-horizon agent's context (above 80% of an average SWE-agent turn). How harnesses should manage the agents’ context is a still a core design decision. This blog discuss:

• How methods differ in who decides (harness rules or the model) and when (at inference or through RL) of context management.
• How serving layer prices context management actions: an edit re-prefills all tokens after the first changed token. </aside>

1. Introduction

ReAct (Yao et al. 2022) set the default agent loop as: the model writes a reasoning trace before an action $a_t$, the environment returns an observation $o_t$, and both are appended:

$$ c_{t+1} = c_t \oplus a_t \oplus o_t $$

Context management is the action that replaces the append with an operator $g$ that builds the next context block:

$$ c_{t+1} = g(c_t, a_t, o_t) $$

Masking replaces old observations with placeholders, compaction swaps a prefix for a summary, and offloading keeps only a file path. This operator fires on a harness rule (a token threshold) or on the model's own action (call compact(), fold a branch). This problem is essential because:

• Horizons keep growing. METR's 50% time horizon has doubled about every seven months since 2019 (Kwa et al. 2025).
• Contexts are mostly tool observations. They make up about 84% of an average SWE-agent turn (Lindenbauer et al. 2025).
• The usable window is shorter than full context.

Fig. 1. Task length (in human professional time) that frontier agents complete with 50% reliability doubles roughly every 7 months. (Image source: Kwa et al. 2025)

2. The Serving Cost of Context Management

With prefix caching, the server reuses KV up to the first token that differs from a cached prompt and redo prefilling of the rest token. An append keeps the history cached and an edit near the front pays for nearly all of the context again. Context management is therefore an algorithm-infrastructure co-design problem.

Shao et al. (2026) measure the cost of context management in prefix-reuse FLOPs, which charge each turn only for what the cache cannot cover. Even a single-token difference invalidates the cache from that token onward (Ji 2025). Typical culprits are a timestamp in the system prompt, unstable JSON key order, and adding or removing tools. Each has an append-only fix: serialize deterministically, put time and state in the newest message, mask tools instead of removing them, and announce configuration changes in a new message, as Codex does (Bolin 2026).

Fig. 2. A sliding window keeps a cache hit only on the instruction; an append-only context hits on every prior step. (Image source: Ji 2025)

Suffix Cache Reuse (Shao et al. 2026) cuts BrowseComp-Plus compute from 10.98 to 7.14 PFLOPs per question at the same 60.2% accuracy (Qwen3.6-27B). Memento (Kontonis et al. 2026) shows that model will lose 15 points on downstream task, eg, AIME'24 when it re-prefills its summaries instead of keeping the stale KV.

3. Existing Practice of Harness Side Context Management

The simplest $g$ is fixed code that rewrites the message list between model calls. Even when $g$ calls an LLM to write a summary, the engineer fixes the trigger and the format. Here we introduce three operators cover most production systems: