Research1 min read
Tail-Risk Aware Scheduling for Agentic LLM Workflows
A new scheduling method reduces tail latency in agentic LLM workflows by strategically releasing turns based on evolving tail risk. Evaluations using real execution traces show significant improvements in workflow flow time under contention.
From arXiv cs.AI