GitHub Copilot’s development has shifted focus from reducing individual tool call token counts to optimizing the overall task efficiency. The team recognized that a concise tool response could lead to increased costs if it lacked the necessary context for the agent to proceed correctly. This resulted in a multi-faceted approach to improve performance.
The initial efforts centered around a utility called RTK (Rust Token Killer), which aimed to shorten shell output before an agent processed it. However, experiments revealed that removing this information often forced the agent to reopen outputs and rerun commands, ultimately increasing token usage and task duration. This highlighted the importance of evaluating efficiency across the entire task workflow, not just individual tool calls.
To address this, the team implemented a selective output compressor. This compressor prioritized preserving source-like and arbitrary output, such as commands like git diff and git show, which contained valuable information. It also reorganized search results from tools like grep and compressed repetitive noise in outputs like install, build, and test commands, only when substantial savings were achieved. This strategy was validated through extensive agentic coding benchmarks and open-source repository testing.
The final shipped version incorporates three key policies: preserving source-like and arbitrary output, reorganizing search results without loss of content, and compressing repetitive noise selectively. A crucial element is a recovery path that allows the agent to retrieve the complete original output if the compressor removes valuable information. This recovery path also serves as an evaluation signal, tracking agent behavior to identify potential issues with the compression strategy. Online experiments demonstrated a slight cost decrease with no measurable impact on task quality, confirming the effectiveness of this approach.
Furthermore, the team addressed formatting issues, such as line-number prefixes used in the view tool. By removing these prefixes, they eliminated unnecessary tokens and improved efficiency. This demonstrates a commitment to optimizing not just the core functionality but also the subtle details that can accumulate and impact performance.
