DQN-tensorflow - GitLite

KO | EN

gitlite — search

#javascript #ai #llm #golang #go #android #rust #nodejs #windows #awesome-list #vue #ios #mcp #openai

DQN-tensorflow

(★ 2,583)

Tensorflow implementation of Human-Level Control through Deep Reinforcement Learning

File Explorer

Download Latest Version (.zip)

- 0518_hist_step_m2.png
- 0518_hist_wall_m2.png
- 0518_scalar_step_m2.png
- 0518_scalar_wall_m2.png
- 0519_hist_step_all.png
- 0519_hist_wall_all.png
- 0519_scalar_step_all.png
- 0519_scalar_wall_all.png
- 0520_hist_step_all.png
- 0520_hist_wall_all.png
- 0520_scalar_step_all.png
- 0520_scalar_wall_all.png
- 0620_scalar_step_m2.png
- 0620_scalar_step_m3.png
- A1_0.00025lr_distributed.png
- A1_A2_A4_0.00025lr.png
- A1_A2_A4_0.00025lr_0.0025lr.png
- A1_A2_A4_0.0025lr.png
- A4_0.00025lr_distributed.png
- A4_duel_double.png
- best.gif
- model.png
- tensorboard_160516.png
- tensorboard_160518_histogram1.png
- tensorboard_160518_histogram2.png
- tensorboard_160518_scalar1.png
- tensorboard_160518_scalar2.png
- __init__.py
- agent.py
- base.py
- environment.py
- history.py
- ops.py
- replay_memory.py
- utils.py
.gitignore
config.py
LICENSE
main.py
README.md

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Human-Level Control through Deep Reinforcement Learning

Tensorflow implementation of [Human-Level Control through Deep Reinforcement Learning](http://home.uchicago.edu/~arij/journalclub/papers/2015_Mnih_et_al.pdf).

![model](https://raw.githubusercontent.com/devsisters/DQN-tensorflow/master/assets/model.png)

This implementation contains:

1. Deep Q-network and Q-learning
2. Experience replay memory
    - to reduce the correlations between consecutive updates
3. Network for Q-learning targets are fixed for intervals
    - to reduce the correlations between target and predicted Q-values

## Requirements

- Python 2.7 or Python 3.3+
- [gym](https://github.com/openai/gym)
- [tqdm](https://github.com/tqdm/tqdm)
- [SciPy](http://www.scipy.org/install.html) or [OpenCV2](http://opencv.org/)
- [TensorFlow 0.12.0](https://github.com/tensorflow/tensorflow/tree/r0.12)

## Usage

First, install prerequisites with:

$ pip install tqdm gym[all]

To train a model for Breakout:

$ python main.py --env_name=Breakout-v0 --is_train=True
    $ python main.py --env_name=Breakout-v0 --is_train=True --display=True

To test and record the screen with gym:

$ python main.py --is_train=False
    $ python main.py --is_train=False --display=True

## Results

Result of training for 24 hours using GTX 980 ti.

![best](https://raw.githubusercontent.com/devsisters/DQN-tensorflow/master/assets/best.gif)

## Simple Results

Details of `Breakout` with model `m2`(red) for 30 hours using GTX 980 Ti.

![tensorboard](https://raw.githubusercontent.com/devsisters/DQN-tensorflow/master/assets/0620_scalar_step_m2.png)

Details of `Breakout` with model `m3`(red) for 30 hours using GTX 980 Ti.

![tensorboard](https://raw.githubusercontent.com/devsisters/DQN-tensorflow/master/assets/0620_scalar_step_m3.png)

## Detailed Results

**[1] Action-repeat (frame-skip) of 1, 2, and 4 without learning rate decay**

![A1_A2_A4_0.00025lr](https://raw.githubusercontent.com/devsisters/DQN-tensorflow/master/assets/A1_A2_A4_0.00025lr.png)

**[2] Action-repeat (frame-skip) of 1, 2, and 4 with learning rate decay**

![A1_A2_A4_0.0025lr](https://raw.githubusercontent.com/devsisters/DQN-tensorflow/master/assets/A1_A2_A4_0.0025lr.png)

**[1] & [2]**

![A1_A2_A4_0.00025lr_0.0025lr](https://raw.githubusercontent.com/devsisters/DQN-tensorflow/master/assets/A1_A2_A4_0.00025lr_0.0025lr.png)

**[3] Action-repeat of 4 for DQN (dark blue) Dueling DQN (dark green) DDQN (brown) Dueling DDQN (turquoise)**

The current hyper parameters and gradient clipping are not implemented as it is in the paper.

![A4_duel_double](https://raw.githubusercontent.com/devsisters/DQN-tensorflow/master/assets/A4_duel_double.png)

**[4] Distributed action-repeat (frame-skip) of 1 without learning rate decay**

![A1_0.00025lr_distributed](https://raw.githubusercontent.com/devsisters/DQN-tensorflow/master/assets/A4_0.00025lr_distributed.png)

**[5] Distributed action-repeat (frame-skip) of 4 without learning rate decay**

![A4_0.00025lr_distributed](https://raw.githubusercontent.com/devsisters/DQN-tensorflow/master/assets/A4_0.00025lr_distributed.png)

## References

- [simple_dqn](https://github.com/tambetm/simple_dqn.git)
- [Code for Human-level control through deep reinforcement learning](https://sites.google.com/a/deepmind.com/dqn/)

## License

MIT License.