phasic-policy-gradient - GitLite

KO | EN

gitlite — search

#python #ai #golang #machine-learning #deep-learning #macos #nodejs #cli #vue #pytorch #mcp #ios #openai #open-source

phasic-policy-gradient

(★ 265)

Code for the paper "Phasic Policy Gradient"

파일 탐색기

최종 버전 다운로드 (.zip)

- - e_aux-mean.pdf
  - e_aux.pdf
  - e_pi-mean.pdf
  - e_pi.pdf
  - n_pi-mean.pdf
  - n_pi.pdf
  - ppg-mean.pdf
  - ppg.pdf
  - ppg_single_network-mean.pdf
  - ppg_single_network.pdf
  - ppgkl-mean.pdf
  - ppgkl.pdf
  - ppo-mean.pdf
  - ppo.pdf
- __init__.py
- constants.py
- distr_builder.py
- envs.py
- graph.py
- graph_util.py
- impala_cnn.py
- log_save_helper.py
- logger.py
- minibatch_optimize.py
- ppg.py
- ppo.py
- reward_normalizer.py
- roller.py
- torch_util.py
- train.py
- tree_util.py
- vec_monitor2.py
.gitignore
environment.yml
LICENSE
README.md
setup.py

# 설치 가이드

1. 코드 내려받기

git clone https://github.com/openai/phasic-policy-gradient

깃허브에서 프로젝트 코드 전체를 내 컴퓨터로 내려받습니다.

cd phasic-policy-gradient

방금 내려받은 프로젝트 폴더 안으로 이동합니다.

2. Python

쉬움 추천

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Python 3 설치 시 'Add Python to PATH' 옵션을 꼭 체크하세요 (Windows).

pip install -e phasic-policy-gradient

requirements.txt 등에 명시된 파이썬 라이브러리를 설치합니다.

mpiexec -np 4 python -m phasic_policy_gradient.train

파이썬 스크립트(또는 모듈)를 실행합니다.

python -m phasic_policy_gradient.graph --experiment_name ppg

파이썬 스크립트(또는 모듈)를 실행합니다.

mpiexec -np 4 python -m phasic_policy_gradient.train --n_epoch_pi 3 --n_epoch_vf 3 --n_aux_epochs 0 --arch shared

파이썬 스크립트(또는 모듈)를 실행합니다.

python -m phasic_policy_gradient.graph --experiment_name ppo

파이썬 스크립트(또는 모듈)를 실행합니다.

✅ 에러 메시지 없이 실행되고 터미널에 안내 문구가 출력되면 정상입니다.

이 레포의 README에 적힌 실제 명령어를 그대로 가져왔습니다.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

**Status:** Archive (code is provided as-is, no updates expected)

# Phasic Policy Gradient

#### [[Paper]](https://arxiv.org/abs/2009.04416)

This is code for training agents using [Phasic Policy Gradient](https://arxiv.org/abs/2009.04416) [(citation)](#citation).

Supported platforms:

- macOS 10.14 (Mojave)
- Ubuntu 16.04

Supported Pythons:

- 3.7 64-bit

## Install

You can get miniconda from https://docs.conda.io/en/latest/miniconda.html if you don't have it, or install the dependencies from [`environment.yml`](environment.yml) manually.

```
git clone https://github.com/openai/phasic-policy-gradient.git
conda env update --name phasic-policy-gradient --file phasic-policy-gradient/environment.yml
conda activate phasic-policy-gradient
pip install -e phasic-policy-gradient
```

## Reproduce and Visualize Results

PPG with default hyperparameters (results/ppg-runN):

```
mpiexec -np 4 python -m phasic_policy_gradient.train
python -m phasic_policy_gradient.graph --experiment_name ppg
```

PPO baseline (results/ppo-runN):

```
mpiexec -np 4 python -m phasic_policy_gradient.train --n_epoch_pi 3 --n_epoch_vf 3 --n_aux_epochs 0 --arch shared
python -m phasic_policy_gradient.graph --experiment_name ppo
```

PPG, varying E_pi (results/e-pi-N):

```
mpiexec -np 4 python -m phasic_policy_gradient.train --n_epoch_pi N
python -m phasic_policy_gradient.graph --experiment_name e_pi
```

PPG, varying E_aux (results/e-aux-N):

```
mpiexec -np 4 python -m phasic_policy_gradient.train --n_aux_epochs N
python -m phasic_policy_gradient.graph --experiment_name e_aux
```

PPG, varying N_pi (results/n-pi-N):

```
mpiexec -np 4 python -m phasic_policy_gradient.train --n_pi N
python -m phasic_policy_gradient.graph --experiment_name n_pi
```

PPG, using L_KL instead of L_clip (results/ppgkl-runN):

```
mpiexec -np 4 python -m phasic_policy_gradient.train --clip_param 0 --kl_penalty 1
python -m phasic_policy_gradient.graph --experiment_name ppgkl
```

PPG, single network variant (results/ppgsingle-runN):

```
mpiexec -np 4 python -m phasic_policy_gradient.train --arch detach
python -m phasic_policy_gradient.graph --experiment_name ppg_single_network
```

Pass `--normalize_and_reduce` to compute and visualize the mean normalized return with `phasic_policy_gradient.graph`.

# Citation

Please cite using the following bibtex entry:

```
@article{cobbe2020ppg,
  title={Phasic Policy Gradient},
  author={Cobbe, Karl and Hilton, Jacob and Klimov, Oleg and Schulman, John},
  journal={arXiv preprint arXiv:2009.04416},
  year={2020}
}
```