SocialRL is a reinforcement learning framework designed to enhance the social intelligence of language models. The approach addresses limitations of existing methods that optimize single-turn utterances and rely on sparse outcome rewards. It utilizes multi-turn reinforcement learning with PPO, propagating delayed outcome rewards across interactions to facilitate long-horizon planning. The framework incorporates six process reward dimensions, including goal advancement, relational attunement, and contextual coherence, to capture the complexities of goal-relationship trade-offs. A reward model dynamically generates scoring criteria for each dimension, and a stage-aware weight schedule adjusts priorities throughout the dialogue – emphasizing relationship building early, goal advancement mid-way, and balanced closure later. Across multiple social-dialogue benchmarks, SocialRL achieved an average of 9.2 percentage points improvement in Goal Achievement compared to Base models. These results demonstrate effectiveness across synthetic and real social scenes, as well as standard and challenging social scenarios.
Source: https://arxiv.org/abs/2609.09764