Current capabilities of large language model chatbots are primarily measured by static benchmarks such as MMLU, HumanEval, and MATH. As these metrics become saturated, there is growing concern that user experience does not improve proportionally to these scores.
Purposeful dialogue centers multi-round conversations around a specific goal or intention, ranging from general helpfulness to specialized roles like travel planning or customer service. This approach treats interactions as collaborative games where the AI helps humans achieve objectives rather than simply predicting the next token.
In code generation, for example, single-pass benchmarks often fail to capture the reality of solving GitHub issues. Effective automation requires back-and-forth communication to clarify requirements, request documentation, and verify understanding, similar to pair programming.
Building on this, long-term interactions with memory allow agents to update user profiles and adapt preferences. A personal assistant could automatically aggregate information from sources like Twitter or Slack to provide tailored summaries and draft emails based on learned edits.
Historical dialogue systems like Schank's "restaurant script" relied on pre-planned steps, whereas modern LLMs use sequence prediction. To enable structured dialogue, engineers must introduce specific formatting into the training process.
Pretraining utilizes mixed internet corpora including news, books, and code. Dialogue formatting is then applied using methods like Hugging Face's tokenizer.apply_chat_template to structure system prompts and past exchanges. Finally, Reinforcement Learning from Human Feedback (RLHF) rewards desired answers based on this structured format, marking the first time such dialogue formatting appears in the training data.
Source: https://thegradient.pub/dialog/
