Skip to content
OpenSmartRoute

LLMs1 min read

Query Rewriting Enhances RAG Performance Through Complementarity

Research found that combining query rewriting strategies with a strong RAG baseline yields significant improvements in retrieval accuracy, primarily driven by complementarity rather than increased retrieval budget. A cost-aware router further optimizes this approach.

By OpenSmartRoute editorial · written through the router by writer-small

From arXiv cs.CL - “Better Together: Complementary Query Rewriting Under a Strong RAG Baseline”

The study investigated the impact of query rewriting on Retrieval-Augmented Generation (RAG) systems. Four rewriting strategies (S1-S4) were compared against two strong LLM baselines (HyDE, Query2Doc) across three datasets: HotpotQA, AmbigNQ, and the 512K-document EnterpriseRAG-Bench. The research found that rewriting alone was at best competitive with a strong baseline. However, combining methods resulted in outsized gains, particularly when a post-hoc union of four methods (S1+S3+S4+HyDE) improved HIT@10 by +12.5 points on enterprise data.

Analysis revealed that different rewriting strategies performed differently on various questions. The union of five methods reached 52.98 on the EnterpriseRAG-Bench dataset. Budget-matched controls captured only approximately 40% of this gain, indicating that complementarity was the primary driver of the improvement. On HotpotQA, the union added +1.6 to +1.8 points, saturating the all-method oracle, while on AmbigNQ, the same fusion resulted in a decrease (-2.4 below the best solo).

To mitigate the cost of rewriting, a confidence-gated router was evaluated. This router applied rewriting only when the baseline's top-1 score was low, capturing approximately half of the enterprise full-merge gain (+4.3 HIT@10) while reducing rewriting cost to less than 40% of queries. A downstream answer-quality evaluation confirmed the router improved F1 by +1.92 at roughly 40% of the expansion cost.

The findings suggest treating query rewriting as a complementary coverage source, applied through cost-aware routing, rather than a standalone replacement for a strong baseline.

Source: https://arxiv.org/abs/2609.05637

Published Sep 9, 2026 · updated Sep 9, 2026 · 229 words

Keep reading

Related posts

More in LLMs

LLMs1 min read

Document-Level MT Evaluation Shows Statistical Equivalence

Research found that document-level machine translation evaluation, presenting full documents to annotators, yields statistically equivalent scores and rankings compared to segment-level evaluations. This suggests current document-level systems and associated metrics may not be accurately measuring intended aspects of translation quality.

LLMs1 min read

CONDUIT: KV Cache Reuse Policy for VLMs

CONDUIT is a training-free refresh policy for KV caches in vision-language models, unifying single and multi-image reuse. It achieves high accuracy and speed improvements with a small refresh budget, outperforming budgeted methods on latency.

LLMs1 min read

Language Models Exhibit Layered Magnetic Organization

Research identifies 'magnetic vectors' within LLMs that organize surrounding tokens, influencing their length. This layer-wise geometric organization, observed across models and tasks, offers a new probe-free approach to understanding language model computation.

Query Rewriting Enhances RAG Performance Through Complementarity - OpenSmartRoute