LLMs1 min read
Study reveals how attention and recurrence function in hybrid language models
Hybrid language models use attention and recurrent states with distinct roles; attention handles retrieval, recurrence influences output generation. Interventions clarify their functions.
From arXiv cs.CL