robotic-arm-safeRL

KO | EN

gitlite — search

#python #react #nodejs #typescript #tailwindcss #html #python3 #reactjs #ai #data-science #rust #javascript

robotic-arm-safeRL

★ 40 Open GitHub ↗

A student project comparing reinforcement learning algorithms on a panda robotic arm. Implemented PPO and its safety version, PPO Lagrangian, taking into account costs such as avoiding obstacles. Explore the potential of safeRL in robotics.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Safe Reinforcement Learning on a Robotic Arm

![Panda-test](https://raw.githubusercontent.com/lukakovac99/robotic-arm-safeRL/main/panda-test.gif)

A student project comparing reinforcement learning (RL) algorithms on a panda robotic arm. Implemented PPO and its safety version, PPO Lagrangian, taking into account costs such as avoiding obstacles. Explore the potential of safeRL in robotics.

## Acknowledgement

For this project we are using reinforcement learning agents developed by OpenAI from [Safety Starter Agents](https://github.com/openai/safety-starter-agents).
For the environment we modified an existing OpenAI Gym environment for pybullet [gym-panda](https://github.com/mahyaret/gym-panda).

## Requirements

This environment was built on Ubuntu 22.04.

Requires **Python 3.6**.

## Installation

To install this project and run and modify the code in this repository:

```
gh repo clone lukakovac99/robotic-arm-safe-RL

cd robotic-arm-safeRL
```

For the environment setup use [Miniconda Environment Manager](https://docs.conda.io/projects/conda/en/latest/user-guide/install/linux.html):

```
conda create --name py36 python=3.6 mpi4py

conda activate py36

pip install -r requirements.txt
```

## Important Files

- **gym_panda/envs/panda_env.py**
In this file you can change the RL environment with which the agent is interacting.
That includes changing the method of the action execution, changing the observation information, obstacle generation, episode length etc.

- **run_training.py**
File with which you run the training.
In this file it's specified which environment, algorithm and training configuration the agent should use. 
Also you can set the name for the output file.

- **plot.py**
File to plot the training's data (reward, cost).

- **test_policy.py**
File to test the learned policy.

## Getting started

### Runing training:

1. Set the name of the output file of the experiment in *run_training.py*.
2. Set the configuration of the experiment (type of the algorithm, max episode length, number of epochs etc.).
Make sure that max episode length is of the same value as defined in *panda_env.py*.
3. To change method of action exectution or obstacle generation method, just (de)comment out the desired options in *panda_env.py*.
4. To change the type of the algorithm, change the code in *run_training.py*:
```
ppo(
	env_fn = lambda : gym.make('panda-v0'),
	ac_kwargs = dict(hidden_sizes=[64,64]),
	logger_kwargs = dict(output_dir='exp-results/'+current_time+file_name, exp_name=(file_name))
	)
```
If you want to run PPO Lagrangian change the code to:
```
ppo_lagrangian(
	env_fn = lambda : gym.make('panda-v0'),
	ac_kwargs = dict(hidden_sizes=[64,64]),
	logger_kwargs = dict(output_dir='exp-results/'+current_time+file_name, exp_name=(file_name))
	)
```
5. Run this command in the terminal:
```
python run_training.py
```
6. If error shows up, check if *MODE* in *panda_env.py* is set to *p.DIRECT*:
```
MODE = p.DIRECT
```

### Plotting training progress:

Run this command in the terminal:
```
python plot.py /path/to/experiment-results-folder
```
Make sure to change */path/to/experiment-results-folder* to the folder that was generated by training.

### Test learned policy in a simulation:

1. Make sure that *MODE* in *panda_env.py* is set to *p.GUI*:
```
MODE = p.GUI
```
2. Run this command in the terminal:
```
python test_policy.py /path/to/experiment-results-folder
```

*For the detailed documentation and description of the project look at the report file.*

robotic-arm-safeRL

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README