hindsight-experience-replay

KO | EN

gitlite — search

#python #hacktoberfest #typescript #llm #go #machine-learning #linux #macos #java #nodejs #vue #pytorch #security #open-source

hindsight-experience-replay

(★ 451)

This is the pytorch implementation of Hindsight Experience Replay (HER) - Experiment on all fetch robotic environments.

File Explorer

Download Latest Version (.zip)

- pick.gif
- push.gif
- reach.gif
- results.png
- slide.gif
- __init__.py
- her.py
- __init__.py
- mpi_utils.py
- normalizer.py
- __init__.py
- ddpg_agent.py
- models.py
- replay_buffer.py
.gitignore
arguments.py
demo.py
LICENSE
README.md
train.py

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Hindsight Experience Replay (HER)
This is a pytorch implementation of [Hindsight Experience Replay](https://arxiv.org/abs/1707.01495).

## Acknowledgement:
- [Openai Baselines](https://github.com/openai/baselines)

## Requirements
- python=3.5.2
- openai-gym=0.12.5 (mujoco200 is supported, but you need to use gym >= 0.12.5, it has a bug in the previous version.)
- mujoco-py=1.50.1.56 (~~**Please use this version, if you use mujoco200, you may failed in the FetchSlide-v1**~~)
- pytorch=1.0.0 (**If you use pytorch-0.4.1, you may have data type errors. I will fix it later.**)
- mpi4py

## TODO List
- [x] support GPU acceleration - although I have added GPU support, but I still not recommend if you don't have a powerful machine.
- [x] add multi-env per MPI.
- [x] add the plot and demo of the **FetchSlide-v1**.

## Instruction to run the code
If you want to use GPU, just add the flag `--cuda` **(Not Recommended, Better Use CPU)**.
1. train the **FetchReach-v1**:
```bash
mpirun -np 1 python -u train.py --env-name='FetchReach-v1' --n-cycles=10 2>&1 | tee reach.log
```
2. train the **FetchPush-v1**:
```bash
mpirun -np 8 python -u train.py --env-name='FetchPush-v1' 2>&1 | tee push.log
```
3. train the **FetchPickAndPlace-v1**:
```bash
mpirun -np 16 python -u train.py --env-name='FetchPickAndPlace-v1' 2>&1 | tee pick.log
```
4. train the **FetchSlide-v1**:
```bash
mpirun -np 8 python -u train.py --env-name='FetchSlide-v1' --n-epochs=200 2>&1 | tee slide.log
```

### Play Demo
```bash
python demo.py --env-name=<environment name>
```
### Download the Pre-trained Model
Please download them from the [Google Driver](https://drive.google.com/open?id=1dNzIpIcL4x1im8dJcUyNO30m_lhzO9K4), then put the `saved_models` under the current folder.

## Results
### Training Performance
It was plotted by using 5 different seeds, the solid line is the median value. 
![Training_Curve](https://raw.githubusercontent.com/TianhongDai/hindsight-experience-replay/master/figures/results.png)
### Demo:
**Tips**: when you watch the demo, you can press **TAB** to switch the camera in the mujoco.

FetchPush-v1| FetchPickAndPlace-v1| FetchSlide-v1
-----------------------|-----------------------|-----------------------|
![](https://raw.githubusercontent.com/TianhongDai/hindsight-experience-replay/master/figures/push.gif)| ![](https://raw.githubusercontent.com/TianhongDai/hindsight-experience-replay/master/figures/pick.gif)| ![](https://raw.githubusercontent.com/TianhongDai/hindsight-experience-replay/master/figures/slide.gif)

hindsight-experience-replay

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

hindsight-experience-replay

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

Download README