Document Type : Original/Review Paper

Authors

• Sajad Bastami ¹
• Mohammad Bagher Dowlatshahi ²
• Rojiar Pir Mohammadiani ³
• Seyedeh Zahra Mousavi ²

¹ Computer Engineering Department, Kurdistan University, Sanandaj City, Iran

² Computer Engineering Department, Lorestan University, Khorramabad City, Iran

³ Faculty of Engineering, University of Kurdistan

Abstract

Multi-agent reinforcement learning (MARL) is a key paradigm for coordination in robotics, autonomous systems, and distributed control. However, existing MARL methods face fundamental limitations in scalability, adaptability to dynamic environments, and stability under evolving interactions. To address these challenges, we propose Adaptive Graph-Transformer Reinforcement Learning (AGTRL), a framework integrating graph-based relational modelling with transformer attention for adaptive coordination in large-scale multi-agent systems. AGTRL unifies graph-based perception and attention-based coordination in an end-to-end pipeline, encoding role information and adaptively weighting interactions by context. This combination, missing in prior MARL methods, bridges scalability and robustness in dynamic environments. AGTRL constructs a dynamic graph of agent relationships and uses multi-head self-attention to prioritize relevant interactions in real-time, ensuring robust performance under perturbations. We evaluate robustness under communication dropout (up to 40% link removal) and dynamic edge removal, measuring performance via episode reward and win rate. The framework incorporates an adaptive stability-performance trade-off mechanism that maintains learning efficacy in the presence of communication constraints and environmental uncertainty. We introduce a graph-enhanced policy architecture that jointly optimizes individual agent policies and inter-agent coordination through attention-weighted message passing. Comprehensive evaluations on benchmark environments—including StarCraft II micromanagement scenarios, cooperative navigation (Spread), and adversarial tasks (Predator-Prey)—demonstrate that AGTRL achieves superior sample efficiency, scalability, and robustness compared to state-of-the-art MARL baselines. Experimental results show AGTRL improves convergence speed by 32% on average and maintains stable performance with up to 40% communication dropout, establishing its viability for real-world deployment in dynamic multi-agent domains.

Keywords

• Keywords: Multi-Agent Reinforcement Learning
• Graph Neural Networks
• Transformer Attention Mechanisms
• Adaptive Coordination
• Scalable Multi-Agent Systems

Main Subjects

• B.3. Communication/Networking and Information Technology