Skip to content
OpenSmartRoute

LLMs1 min read

CONDUIT: KV Cache Reuse Policy for VLMs

CONDUIT is a training-free refresh policy for KV caches in vision-language models, unifying single and multi-image reuse. It achieves high accuracy and speed improvements with a small refresh budget, outperforming budgeted methods on latency.

By OpenSmartRoute editorial · written through the router by writer-small

From arXiv cs.CL - “CONDUIT: A Unified Residual-Stream Restoration Framework for KV Cache Reuse in Vision-Language Models”

CONDUIT addresses challenges in reusing KV caches for vision-language models (VLMs) when visual content changes prefixes. The system employs a residual-stream restoration approach, unifying single and multi-image reuse. It uses cached-key query attention and a cached-value-norm proxy to rank cached visual tokens. This ranking is followed by image-level relevance amplification before a global selection. With one image, the refresh coefficient is one, reducing to intra-image token selection. The method adds only a single query-conditioned scoring pass at inference, preserving existing model architecture and weights. At a 10% refresh budget, CONDUIT achieves 97.0-99.5% of full-prefill accuracy across three VLM backbones. On the MMLongBench-Doc latency subset, it uses 13.5% of full-prefill FLOPs and achieves a 2.99x time-to-first-token speedup.

Source: https://arxiv.org/abs/2609.05821

Published Sep 9, 2026 · updated Sep 9, 2026 · 118 words

Keep reading

Related posts

More in LLMs

LLMs1 min read

Document-Level MT Evaluation Shows Statistical Equivalence

Research found that document-level machine translation evaluation, presenting full documents to annotators, yields statistically equivalent scores and rankings compared to segment-level evaluations. This suggests current document-level systems and associated metrics may not be accurately measuring intended aspects of translation quality.

LLMs1 min read

Dynamic Lagging Improves Simultaneous Translation with LLMs

Researchers introduced a sentence-trained LLM approach for cascaded simultaneous translation, utilizing prefix-aware fine-tuning to achieve flicker-free performance. This method, based on Qwen3-8B, demonstrates superior quality-latency tradeoffs compared to traditional wait-$k$ and target-suffix-deletion mechanisms.

LLMs1 min read

Intra-Prompt Parallel Decoding for CCQA

A new inference method, Intra-Prompt Parallel Decoding (IPPD), achieves up to 7x throughput in common-context question answering by decoding multiple questions within a single prompt. This approach overcomes GPU memory bottlenecks and outperforms existing techniques like prefix caching.

CONDUIT: KV Cache Reuse Policy for VLMs - OpenSmartRoute