Large language model agents increasingly interleave natural language reasoning with external tools such as web search and code execution. These tool-use policies are often optimized via reinforcement learning (RL), which can amplify spurious correlations present in the training data.
Researchers constructed controlled synthetic environments combining factual question answering and mathematical reasoning tasks. They injected cues that were strongly correlated with specific tools during training but causally irrelevant to tool necessity.
Across counterfactual evaluations where cues were present but associated tools were not required, agents exhibited substantial shortcut behavior. Spurious tool invocation rates increased by up to 39 percent under these conditions.
Shortcut formation was not universal across all tested conditions. It arose only when the agent had already learned to use the target tool reliably, suggesting that task competence is a key factor in shortcut learning alongside dataset imbalance.
To mitigate these failures, researchers introduced a dense, decision-level reward system. An LLM judge evaluates the necessity of each tool call within this framework.
This tool-necessity reward effectively suppresses cue-driven tool use while preserving task performance. This provides a practical approach to improving the robustness of LLM agent tool-use policies.
Source: https://arxiv.org/abs/2609.16268
