The research introduces AIM, a framework designed to manage memory within multi-agent, multi-user large language model systems. Traditional LLMs limit knowledge to individual sessions, while existing agentic memory systems operate at the user level, restricting shared knowledge. AIM provides a unified framework that classifies information as private, user-scoped, or public, enforcing index-level access controls. This allows for retrieval of private memories only by their owner while enabling shared knowledge to improve coordination.
The team developed MUMBench, a multi-user interaction dataset containing private and shareable information across four domains. This dataset is designed to evaluate memory operations such as retrieval, creation, update, and deletion within a multi-user environment. The dataset consists of interactions across multiple users and agents.
Evaluation of AIM across three independent runs on MUMBench yielded specific results. The framework achieved 96.0% visibility classification accuracy, indicating a high degree of precision in categorizing memory data. Strict operation accuracy was 58.8%, measuring the correctness of memory operations. State-aware operation accuracy reached 70.5%, assessing the framework's ability to maintain context during memory interactions.
These results demonstrate the potential of AIM for building more sophisticated and privacy-conscious multi-agent systems. The framework’s architecture and performance metrics provide a foundation for further research and development in this area. The MUMBench dataset provides a valuable resource for evaluating and comparing different memory management approaches.
Source: https://arxiv.org/abs/2609.12320
