LLMs1 min read
Osprey: Target-Agnostic Drafter Pretraining
Osprey pretrains a small language model to accelerate speculative decoding, improving acceptance rates across diverse target models. The system reduces per-target adaptation work through a reusable backbone and lightweight adjustments.
From arXiv cs.CL