Hierarchical-Actor-Critic-HAC-PyTorch - GitLite

KO | EN

gitlite — search

#ai #llm #golang #machine-learning #linux #docker #java #cli #awesome-list #pytorch #ios #mcp #openai #open-source

Hierarchical-Actor-Critic-HAC-PyTorch

(★ 334)

PyTorch implementation of Hierarchical Actor Critic (HAC) for OpenAI gym environments

파일 탐색기

최종 버전 다운로드 (.zip)

- __init__.py
- continuous_mountain_car.py
- pendulum.py
- rendering.py
- MountainCarContinuous-v0-3level.gif
- MountainCarContinuous-v0.gif
- Pendulum-v0-2level.gif
.gitignore
DDPG.py
HAC.py
LICENSE
README.md
requirements.txt
test.py
train.py
utils.py

# 설치 가이드

1. 코드 내려받기

git clone https://github.com/nikhilbarhate99/Hierarchical-Actor-Critic-HAC-PyTorch

깃허브에서 프로젝트 코드 전체를 내 컴퓨터로 내려받습니다.

cd Hierarchical-Actor-Critic-HAC-PyTorch

방금 내려받은 프로젝트 폴더 안으로 이동합니다.

2. Python

쉬움 추천

사전 준비물

Git GitHub에서 프로젝트 코드를 내려받으려면 필요합니다.
Python 3 설치 시 'Add Python to PATH' 옵션을 꼭 체크하세요 (Windows).

pip install -r requirements.txt

requirements.txt 등에 명시된 파이썬 라이브러리를 설치합니다.

python <실행할 파일명>.py # README에서 정확한 실행 파일명을 확인하세요

파이썬 스크립트(또는 모듈)를 실행합니다.

✅ 에러 메시지 없이 실행되고 터미널에 안내 문구가 출력되면 정상입니다.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Hierarchical-Actor-Critic-HAC-PyTorch

This is an implementation of the Hierarchical Actor Critic (HAC) algorithm described in the paper, [Learning Multi-Level Hierarchies with Hindsight](https://arxiv.org/abs/1712.00948) (ICLR 2019), in PyTorch for OpenAI gym environments. The algorithm learns to reach a goal state by dividing the task into short horizon intermediate goals (subgoals).

## Usage
- All the hyperparameters are contained in the `train.py` file.
- To train a new network run `train.py`
- To test a preTrained network run `test.py`
- For a detailed explanation of offsets and bounds, refer to [issue #2](https://github.com/nikhilbarhate99/Hierarchical-Actor-Critic-HAC-PyTorch/issues/2)
- For hyperparameters used for preTraining the pendulum policy refer to [issue #3](https://github.com/nikhilbarhate99/Hierarchical-Actor-Critic-HAC-PyTorch/issues/3)

## Implementation Details

- The code is implemented as described in the appendix section of the paper and the Official repository, i.e. without target networks and with bounded Q-values.
- The Actor and Critic networks have 2 hidded layers of size 64.

## Citing

Please use this bibtex if you want to cite this repository in your publications :

@misc{pytorch_hac,
        author = {Barhate, Nikhil},
        title = {PyTorch Implementation of Hierarchical Actor-Critic},
        year = {2021},
        publisher = {GitHub},
        journal = {GitHub repository},
        howpublished = {\url{https://github.com/nikhilbarhate99/Hierarchical-Actor-Critic-HAC-PyTorch}},
    }

## Requirements

- Python 3.6
- [PyTorch](https://pytorch.org/)
- [OpenAI gym](https://gym.openai.com/)

## Results

### MountainCarContinuous-v0
 (2 levels, H = 20, 200 episodes)  |  (3 levels, H = 5, 200 episodes)  |
:-----------------------------------:|:-----------------------------------:|
![](https://github.com/nikhilbarhate99/Hierarchical-Actor-Critic-HAC-PyTorch/blob/master/gif/MountainCarContinuous-v0.gif)  | ![](https://github.com/nikhilbarhate99/Hierarchical-Actor-Critic-HAC-PyTorch/blob/master/gif/MountainCarContinuous-v0-3level.gif)  |

(2 levels, H = 20, 200 episodes)  |
:---------------------------------:|
![](https://github.com/nikhilbarhate99/Hierarchical-Actor-Critic-HAC-PyTorch/blob/master/gif/Pendulum-v0-2level.gif) |

## References

- Official [Paper](https://arxiv.org/abs/1712.00948) and [Code (TensorFlow)](https://github.com/andrew-j-levy/Hierarchical-Actor-Critc-HAC-)