TF2-RL - GitLite

KO | EN

gitlite — search

#python #ai #typescript #android #machine-learning #rust #linux #docker #nodejs #windows #vue #pytorch #security #open-source

TF2-RL

(★ 316)

Reinforcement learning algorithms implemented for Tensorflow 2.0+ [DQN, DDPG, AE-DDPG, SAC, PPO, Primal-Dual DDPG]

File Explorer

Download Latest Version (.zip)

- - aeddpg_actor_epoch2300.h5
  - aeddpg_critic_epoch2300.h5
  - q_values.png
  - rewards.png
- - test_render_basic_reward500.gif
- TF2_AE_DDPG.py
- - ddpg_actor_episode124.h5
  - ddpg_critic_episode124.h5
  - loss.png
  - q_values.png
  - rewards.png
- - test_render_basic_reward500.gif
  - test_render_lstm_time_step5_reward500.gif
- Prioritized_Replay.py
- TF2_DDPG_Basic.py
- TF2_DDPG_LSTM.py
- - q_values_time_step1.png
  - q_values_time_step2.png
  - q_values_time_step3.png
  - q_values_time_step4.png
  - rewards_time_step1.png
  - rewards_time_step2.png
  - rewards_time_step3.png
  - rewards_time_step4.png
- - test_render_basic_time_step4_reward500.gif
  - test_render_lstm_time_step4_reward500.gif
- - q_values_time_step1.png
  - q_values_time_step2.png
  - q_values_time_step3.png
  - q_values_time_step4.png
  - q_values_time_step5.png
  - rewards_time_step1.png
  - rewards_time_step2.png
  - rewards_time_step3.png
  - rewards_time_step4.png
  - rewards_time_step5.png
- TF2_DQN_Basic.py
- TF2_DQN_LSTM.py
- - loss.png
  - ppo_episode176.h5
  - reward.png
- - test_render_basic_reward500.gif
- TF2_PPO.py
- Prioritized_Replay.py
- TF2_PD_DDPG_Basic.py
- TF2_SAC.py
.gitignore
hyperparam_tune.py
LICENSE.md
mytf2env.txt
README.md

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Reinforcement Learning Agents 
Implemented for Tensorflow 2.0+

## New Updates!
- DDPG with prioritized replay
- Primal-Dual DDPG for CMDP

## Future Plans
- SAC Discrete

## Usage
- Install dependancies imported ([my tf2 conda env as reference](https://github.com/anita-hu/TF2-RL/blob/master/mytf2env.txt))
- Each file contains example code that runs training on CartPole env
- Training: `python3 TF2_DDPG_LSTM.py`
- Tensorboard: `tensorboard --logdir=DDPG/logs`

## Hyperparameter tuning
- Install hyperopt https://github.com/hyperopt/hyperopt
- Optional: switch agent used and configure param space in `hyperparam_tune.py` 
- Run: `python3 hyperparam_tune.py`

## Agents
Agents tested using CartPole env.

| Name | On/off policy | Model | Action space support |
| --- | --- | --- | --- | 
| [DQN](https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf) | off-policy | Dense, LSTM | discrete |
| [DDPG](https://arxiv.org/pdf/1509.02971.pdf) | off-policy | Dense, LSTM | discrete, continuous | 
| [AE-DDPG](https://arxiv.org/pdf/1903.00827.pdf) | off-policy | Dense | discrete, continuous | 
| [SAC:bug:](https://arxiv.org/pdf/1812.05905.pdf) | off-policy | Dense | continuous |
| [PPO](https://arxiv.org/pdf/1707.06347.pdf) | on-policy | Dense | discrete, continuous |

#### Contrained MDP
| Name | On/off policy | Model | Action space support |
| --- | --- | --- | --- | 
| [Primal-Dual DDPG](https://arxiv.org/pdf/1802.06480.pdf) | off-policy | Dense | discrete, continuous|

## Models
Models used to generate the demos are included in the repo, you can also find q value, reward and/or loss graphs

## Demos
| DQN Basic, time step = 4, 500 reward | DQN LSTM, time step = 4, 500 reward |
| --- | --- |
| <img src="DQN/gifs/test_render_basic_time_step4_reward500.gif" height="200"> | <img src="DQN/gifs/test_render_lstm_time_step4_reward500.gif" height="200"> |

| DDPG Basic, 500 reward | DDPG LSTM, time step = 5, 500 reward |
| --- | --- |
| <img src="DDPG/gifs/test_render_basic_reward500.gif" height="200"> | <img src="DDPG/gifs/test_render_lstm_time_step5_reward500.gif" height="200"> |

| AE-DDPG Basic, 500 reward | PPO Basic, 500 reward |
| --- | -- |
| <img src="AE-DDPG/gifs/test_render_basic_reward500.gif" height="200"> | <img src="PPO/gifs/test_render_basic_reward500.gif" height="200"> |