alpha-zero-general - GitLite

KO | EN

gitlite — search

#javascript #hacktoberfest #go #golang #android #machine-learning #rust #deep-learning #macos #nodejs #cli #kubernetes #security #openai

alpha-zero-general

(★ 4,502)

A clean implementation based on AlphaZero for any game in any framework + tutorial + Othello/Gobang/TicTacToe/Connect4 and more

File Explorer

Download Latest Version (.zip)

- - __init__.py
  - Connect4NNet.py
  - NNet.py
- __init__.py
- Connect4Game.py
- Connect4Logic.py
- Connect4Players.py
- README.md
- test_connect4.py
- Dockerfile.pytorch
- jupyter_notebook_config.py
- requirements.txt
- run_jupyter.sh
- - __init__.py
  - DotsAndBoxesNNet.py
  - NNet.py
- __init__.py
- Convert Keras Model for use with Tensorflow.js.ipynb
- DotsAndBoxesGame.py
- DotsAndBoxesLogic.py
- DotsAndBoxesPlayers.py
- DotsAndBoxesServer.py
- DotsAndBoxesTest.py
- main-dotsandboxes.py
- pit-dotsandboxes.py
- Play Dots and Boxes using AlphaZero General.ipynb
- README.md
- Train Dots and Boxes using AlphaZero General.ipynb
- - GobangNNet.py
  - NNet.py
- __init__.py
- GobangGame.py
- GobangLogic.py
- GobangPlayers.py
- - __init__.py
  - NNet.py
  - OthelloNNet.py
- - __init__.py
  - NNet.py
  - OthelloNNet.py
- __init__.py
- OthelloGame.py
- OthelloLogic.py
- OthelloPlayers.py
- README.md
- 6x6.png
- writeup.pdf
- - NNet.py
  - RTSNNet.py
- - README.md
- - Board.py
  - config.py
  - config_class.py
  - encoders.py
- - rts_pygame.py
  - rts_ue4.py
- install.sh
- learn.py
- pit.py
- README.md
- RTSGame.py
- RTSPlayers.py
- SANTORINI_README.md
- SantoriniGame.py
- SantoriniLogic.py
- SantoriniPlayers.py
- - __init__.py
  - NNet.py
  - TaflNNet.py
- - __init__.py
  - NNet.py
  - TaflNNet.py
- __init__.py
- Digits.py
- GameVariants.py
- mainTafl.py
- pitTafl.py
- TaflGame.py
- TaflLogic.py
- TaflPlayers.py
- - __init__.py
  - NNet.py
  - TicTacToeNNet.py
- - best.pth.tar.data-00000-of-00001
- __init__.py
- README.md
- TicTacToeGame.py
- TicTacToeLogic.py
- TicTacToePlayers.py
- - __init__.py
  - NNet.py
  - TicTacToeNNet.py
- README.md
- TicTacToeGame.py
- TicTacToeLogic.py
- TicTacToePlayers.py
.gitignore
Arena.py
Coach.py
Game.py
LICENSE
main.py
MCTS.py
NeuralNet.py
pit.py
README.md
requirements.txt
setup_env.sh
test_all_games.py
utils.py

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Alpha Zero General (any game, any framework!)
A simplified, highly flexible, commented and (hopefully) easy to understand implementation of self-play based reinforcement learning based on the AlphaGo Zero paper (Silver et al). It is designed to be easy to adopt for any two-player turn-based adversarial game and any deep learning framework of your choice. A sample implementation has been provided for the game of Othello in PyTorch and Keras. An accompanying tutorial can be found [here](https://suragnair.github.io/posts/alphazero.html). We also have implementations for many other games like GoBang and TicTacToe.

To use a game of your choice, subclass the classes in ```Game.py``` and ```NeuralNet.py``` and implement their functions. Example implementations for Othello can be found in ```othello/OthelloGame.py``` and ```othello/{pytorch,keras}/NNet.py```.

```Coach.py``` contains the core training loop and ```MCTS.py``` performs the Monte Carlo Tree Search. The parameters for the self-play can be specified in ```main.py```. Additional neural network parameters are in ```othello/{pytorch,keras}/NNet.py``` (cuda flag, batch size, epochs, learning rate etc.).

To start training a model for Othello:
```bash
python main.py
```
Choose your framework and game in ```main.py```.

### Docker Installation
For easy environment setup, we can use [nvidia-docker](https://github.com/NVIDIA/nvidia-docker). Once you have nvidia-docker set up, we can then simply run:
```
./setup_env.sh
```
to set up a (default: pyTorch) Jupyter docker container. We can now open a new terminal and enter:
```
docker exec -ti pytorch_notebook python main.py
```

### Experiments
We trained a PyTorch model for 6x6 Othello (~80 iterations, 100 episodes per iteration and 25 MCTS simulations per turn). This took about 3 days on an NVIDIA Tesla K80. The pretrained model (PyTorch) can be found in ```pretrained_models/othello/pytorch/```. You can play a game against it using ```pit.py```. Below is the performance of the model against a random and a greedy baseline with the number of iterations.
![alt tag](https://github.com/suragnair/alpha-zero-general/raw/master/pretrained_models/6x6.png)

A concise description of our algorithm can be found [here](https://github.com/suragnair/alpha-zero-general/raw/master/pretrained_models/writeup.pdf).

### Citation

If you found this work useful, feel free to cite it as

```
@misc{thakoor2016learning,
  title={Learning to play othello without human knowledge},
  author={Thakoor, Shantanu and Nair, Surag and Jhunjhunwala, Megha},
  year={2016},
  publisher={Stanford University, Final Project Report}
}
```

### Contributing
While the current code is fairly functional, we could benefit from the following contributions:
* Game logic files for more games that follow the specifications in ```Game.py```, along with their neural networks
* Neural networks in other frameworks
* Pre-trained models for different game configurations
* An asynchronous version of the code- parallel processes for self-play, neural net training and model comparison. 
* Asynchronous MCTS as described in the paper

Some extensions have been implented [here](https://github.com/kevaday/alphazero-general).

### Contributors and Credits
* [Shantanu Thakoor](https://github.com/ShantanuThakoor) and [Megha Jhunjhunwala](https://github.com/jjw-megha) helped with core design and implementation.
* [Shantanu Kumar](https://github.com/SourKream) contributed TensorFlow and Keras models for Othello.
* [Evgeny Tyurin](https://github.com/evg-tyurin) contributed rules and a trained model for TicTacToe.
* [MBoss](https://github.com/1424667164) contributed rules and a model for GoBang.
* [Jernej Habjan](https://github.com/JernejHabjan) contributed RTS game.
* [Adam Lawson](https://github.com/goshawk22) contributed rules and a trained model for 3D TicTacToe.
* [Carlos Aguayo](https://github.com/carlos-aguayo) contributed rules and a trained model for Dots and Boxes along with a [JavaScript implementation](https://github.com/carlos-aguayo/carlos-aguayo.github.io/tree/master/alphazero).
* [Robert Ronan](https://github.com/rlronan) contributed rules for Santorini.
* [Plamen Totev](https://github.com/plamentotev) contributed Go Text Protocol player for Othello.

Note: Chainer and TensorFlow v1 versions have been removed but can be found prior to commit [2ad461c](https://github.com/suragnair/alpha-zero-general/tree/2ad461c393ecf446e76f6694b613e394b8eb652f).