Addressing Context Overflow in Long-Horizon Agents
Agents operating over extended periods, such as those running for an hour with 200 tool calls, frequently encounter challenges related to context overflow and goal loss. The core issue stems from the limitations of LLM attention mechanisms, where each added token consumes a finite ‘attention budget,’ leading to performance degradation as input length increases. This is exacerbated by the iterative nature of agent loops, where each observation is persistently added to the context.
Several harness systems are designed to address these challenges. The AWS Samples design guide highlights the role of the harness in managing aspects of the agent beyond the model itself. Mechanisms like context budgeting, memory strategy, and todo-state are implemented to control the flow of information and maintain agent focus. These strategies involve offloading less critical information to external storage and summarizing the context when it approaches capacity.
Key Mechanisms for Agent Stability
One core mechanism is context budgeting, implemented in systems like Claude Code and Deep Agents. Claude Code caps auto memory at 200 lines or 25KB, while Deep Agents uses hard numbers for tool response size and session context limits. Another is todo-state, where the agent creates and maintains a structured to-do list, ensuring that objectives are consistently reiterated within the context. Compaction, a process of summarizing and re-initiating the context with the summary, is also a critical element, though it carries the risk of losing crucial constraints.
Implementation Details and Thresholds
Different implementations utilize varying thresholds and strategies. LangChain Deep Agents offloads responses exceeding 20,000 tokens, while Claude Code limits compaction prompts to 5,000 tokens per skill. OpenAI’s Responses API offers server-side compaction via context_management, and the Claude Developer Platform exposes a compact_20260112 context-management edit. These mechanisms demonstrate a move towards engineering context as a resource, rather than simply treating it as an unbounded input.
