Quantifying the effects of environment and population diversity in multi-agent reinforcement learning
Problem
Reinforcement learning agents often perform poorly when placed in novel environments or with unfamiliar co-players; the study quantified how training diversity affects this generalization problem.
Why it matters
Agents that cannot generalize cannot be reliably deployed in real-world, previously unseen situations.
What the AI did
Trained multi-agent RL agents using procedurally generated training levels and diverse co-player populations, introducing a new environment-agnostic measure of behavioral diversity.
Results
Procedurally generated training levels significantly improved performance on held-out levels; population size and intrinsic motivation increased diversity; diverse co-players strengthened performance in some but not all cases.
Impact
Shows procedural generation and population diversity can improve MARL generalization, with stated trade-offs on training levels.
Record
| Venue | Autonomous Agents and Multi-Agent Systems |
|---|---|
| Published | 2022 |
| Work type | article |
| Authors | Kevin R. McKee, Joel Z. Leibo, Charlie Beattie, Richard Everett |
| Citations | 18 (OpenAlex, at import time) |
| References | 43 |
| Model or method | not stated in source |
| Organization | not stated in source |
| Primary source | https://doi.org/10.1007/s10458-022-09548-8 |
| Open access | hybrid |
| Keywords | multi-agent reinforcement learning, generalization, procedural generation, population diversity, co-play |
Importance rating
real but narrower advance
Generalization is called a major MARL challenge, but abstract gives no scale or application domain; contribution is methods insight with mixed results.