Multi-Agent-Reinforcement-Learning - GitLite

KO | EN

gitlite — search

#python #hacktoberfest #ai #typescript #go #android #docker #macos #nodejs #java #awesome #cli #security #ios

Multi-Agent-Reinforcement-Learning

(★ 253)

PyTorch implements multi-agent reinforcement learning algorithms, including QMIX, Independent PPO, Centralized PPO, Grid Wise Control, Grid Wise Control+PPO, Grid Wise Control+DDPG.

File Explorer

Download Latest Version (.zip)

- - agents.cpython-36.pyc
- agents.py
- - pettingzoo_environment.cpython-36.pyc
  - reply_buffer.cpython-36.pyc
- pettingzoo_environment.py
- reply_buffer.py
- - grid_net_actor.cpython-36.pyc
  - grid_net_critic.cpython-36.pyc
  - ppo_net.cpython-36.pyc
  - qmix_net.cpython-36.pyc
- grid_net_actor.py
- grid_net_critic.py
- ppo_net.py
- qmix_net.py
- - base_policy.cpython-36.pyc
  - centralized_ppo.cpython-36.pyc
  - grid_wise_control.cpython-36.pyc
  - grid_wise_control_ddpg.cpython-36.pyc
  - grid_wise_control_ppo.cpython-36.pyc
  - independent_ppo.cpython-36.pyc
  - qmix.cpython-36.pyc
- base_policy.py
- centralized_ppo.py
- grid_wise_control.py
- grid_wise_control_ddpg.py
- grid_wise_control_ppo.py
- independent_ppo.py
- qmix.py
- mpe_simple_spread.gif
- - config_objects.cpython-36.pyc
  - config_utils.cpython-36.pyc
  - env_utils.cpython-36.pyc
  - train_utils.cpython-36.pyc
- config_objects.py
- config_utils.py
- env_utils.py
- train_utils.py
.gitignore
config.yaml
LICENSE
main.py
README.md
requirements.txt
runner.py

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Abstract

The implementation of multi-agent reinforcement learning algorithm in Pytorch, including: Grid-Wise Control, Qmix, Centralized PPO. Different learning strategies can be specified during training, and model and experimental data can be saved.

**Quick Start:** Run the **main.py** script to start training. Please specify all parameters in the **config.yaml** file (The parameters used in this project are not optimal parameters, please adjust them according to the actual requirement).

# Petting Zoo

**MPE:** Multi Particle Environments (MPE) are a set of communication oriented environment where particle agents can (sometimes) move, communicate, see each other, push each other around, and interact with fixed landmarks.

These environments are from [OpenAI’s MPE](https://github.com/openai/multiagent-particle-envs) codebase, with several minor fixes, mostly related to making the action space discrete by default, making the rewards consistent and cleaning up the observation space of certain environments.

The environment applied in this project is **Simple Spread** (I'm also considering adding other environments in future releases).

<img src="README.assets/mpe_simple_spread.gif" alt="Env image" style="zoom:67%;" />

# Requirement

Note: The following are suggested versions only, and do not mean the program will not work with other versions.

| Name       | Version     |
| ---------- | ----------- |
| Python     | 3.10.9      |
| gymnasium  | 0.28.1      |
| numpy      | 1.23.5      |
| PettingZoo | 1.23.0      |
| Pytorch    | 1.12.1      |

Update on 4.10.2023: [Pytorch 2.0.0+cu118](https://pytorch.org/get-started/previous-versions/) on python 3.9.16 works. Please notice that python >3.9 won't work because PettingZoo 1.12.0 is not available.

# Corresponding Papers

- [Grid-Wise Control for Multi-Agent Reinforcement Learning in Video Game AI]([proceedings.mlr.press/v97/han19a/han19a.pdf](http://proceedings.mlr.press/v97/han19a/han19a.pdf))
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://arxiv.org/abs/1803.11485)

- [The Surprising Effectiveness of PPOin Cooperative Multi-Agent Games](https://arxiv.org/abs/2103.01955)

# Reference

- **petting zoo:**

```
@article{terry2020pettingzoo,
  Title = {PettingZoo: Gym for Multi-Agent Reinforcement Learning},
  Author = {Terry, J. K and Black, Benjamin and Grammel, Nathaniel and Jayakumar, Mario and Hari, Ananth and Sulivan, Ryan and Santos, Luis and Perez, Rodrigo and Horsch, Caroline and Dieffendahl, Clemens and Williams, Niall L and Lokesh, Yashas and Sullivan, Ryan and Ravi, Praveen},
  journal={arXiv preprint arXiv:2009.14471},
  year={2020}
}
```

- **Qmix:**  [starry-sky6688/StarCraft: Implementations of IQL, QMIX, VDN, COMA, QTRAN, MAVEN, CommNet, DyMA-CL, and G2ANet on SMAC, the decentralised micromanagement scenario of StarCraft II (github.com)](https://github.com/starry-sky6688/StarCraft)