gbrl - GitLite

KO | EN

gitlite — search

#python #react #llm #go #golang #android #linux #nodejs #macos #java #awesome #kubernetes #security #ios

gbrl

(★ 148)

Gradient Boosting Reinforcement Learning (GBRL)

파일 탐색기

최종 버전 다운로드 (.zip)

- - ci.yml
  - deploy.yml
- - gbrl_diagram.png
  - relative_ppo_performance.png
- - base_learner.rst
  - gbt_learner.rst
  - index.rst
  - multi_gbt_learner.rst
  - separate_actor_critic_learner.rst
  - shared_actor_critic_learner.rst
- - actor_critic.rst
  - base_gbt.rst
  - continuous_critic.rst
  - discrete_critic.rst
  - gaussian_actor.rst
  - gbt.rst
  - index.rst
  - parametric_actor.rst
- conf.py
- examples.rst
- index.rst
- make.bat
- Makefile
- quickstart.rst
- - __init__.py
  - compression.py
  - config.py
  - utils.py
- - __init__.py
  - actor_critic_learner.py
  - base.py
  - gbt_learner.py
  - multi_gbt_learner.py
- - __init__.py
  - actor.py
  - actor_critic.py
  - base.py
  - critic.py
  - gbt.py
- __init__.py
- __init__.py
- test_device_transfer.py
- test_dimension_handling.py
- test_gbt_multi.py
- test_gbt_single.py
- test_monotonic_constraints.py
- test_multi_gbt_learner.py
- test_scheduler.py
.coderabbit.yaml
.gitignore
CLA.md
CMakeLists.txt
LICENSE
LICENSES.txt
MANIFEST.in
poetry.lock
pyproject.toml
README.md
requirements.txt
setup.py
test.py
tutorial.ipynb

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Gradient Boosting Reinforcement Learning (GBRL)
GBRL is a Python-based Gradient Boosting Trees (GBT) library, similar to popular packages such as [XGBoost](https://xgboost.readthedocs.io/en/stable/), [CatBoost](https://catboost.ai/), but specifically designed and optimized for reinforcement learning (RL). GBRL is implemented in C++/CUDA aimed to seamlessly integrate within popular RL libraries.

[![License](https://img.shields.io/badge/license-MIT-green.svg)](https://github.com/NVlabs/gbrl/blob/master/LICENSE)
[![PyPI version](https://badge.fury.io/py/gbrl.svg)](https://badge.fury.io/py/gbrl)

## Overview

GBRL adapts the power of Gradient Boosting Trees to the unique challenges of RL environments, including non-stationarity and the absence of predefined targets. The following diagram illustrates how GBRL uses gradient boosting trees in RL:

![GBRL Diagram](https://github.com/NVlabs/gbrl/raw/master/docs/images/gbrl_diagram.png)

GBRL features a shared tree-based structure for policy and value functions, significantly reducing memory and computational overhead, enabling it to tackle complex, high-dimensional RL problems.

## Key Features: 
- GBT Tailored for RL: GBRL adapts the power of Gradient Boosting Trees to the unique challenges of RL environments, including non-stationarity and the absence of predefined targets.
- Optimized Actor-Critic Architecture: GBRL features a shared tree-based structure for policy and value functions. This significantly reduces memory and computational overhead, enabling it to tackle complex, high-dimensional RL problems.
- Hardware Acceleration: GBRL leverages CUDA for hardware-accelerated computation, ensuring efficiency and speed.
- Seamless Integration: GBRL is designed for easy integration with popular RL libraries. We implemented GBT-based actor-critic algorithm implementations (A2C, PPO, and AWR) in stable_baselines3 [GBRL_SB3](https://github.com/NVlabs/gbrl_sb3).

## Performance

The following results, obtained using the `GBRL_SB3` repository, demonstrate the performance of PPO with GBRL compared to neural-networks across various scenarios and environments:

![PPO GBRL results in stable_baselines3](https://github.com/NVlabs/gbrl/raw/master/docs/images/relative_ppo_performance.png)

## Getting started
### Dependencies
- Python 3.9 or higher

### Installation
GBRL provides pre-compiled binaries for easy installation. Choose **one** of the following options:

**CPU-only installation** (default):  
```pip install gbrl```

**GPU-enabled installation** (requires CUDA 12 runtime libraries):  
```pip install gbrl-gpu```

For further installation details and dependencies see the documentation.

### Usage Example
For a detailed usage example, see `tutorial.ipynb`

## Current Supported Features
### Tree Fitting
- Greedy (Depth-wise) tree building - (CPU/GPU)  
- Oblivious (Symmetric) tree building - (CPU/GPU)  
- L2 split score - (CPU/GPU)  
- Cosine split score - (CPU/GPU) 
- Uniform based candidate generation - (CPU/GPU)
- Quantile based candidate generation - (CPU/GPU)
- Supervised learning fitting / Multi-iteration fitting - (CPU/GPU)
    - MultiRMSE loss (only)
- Categorical inputs
- Input feature weights - (CPU/GPU)
- Monotonic constraints - (CPU/GPU, oblivious trees; the whole model must use SGD, Adam is not allowed on any output)

### GBT Inference

- SGD optimizer - (CPU/GPU)
- ADAM optimizer - (CPU only)
- Control Variates (gradient variance reduction technique) - (CPU only)
- Shared Tree for policy and value function - (CPU/GPU)
- Linear and constant learning rate scheduler - (CPU/GPU, linear scheduler GPU only for Oblivious trees)
- Support for up to two different optimizers (e.g., policy/value) - (CPU/GPU if both are SGD)
- SHAP value calculation (optimizer-aware; `return_base=True` gives values that reconstruct the prediction)

# Documentation 
For comprehensive documentation, visit the [GBRL documentation](https://nvlabs.github.io/gbrl/).

# Contributing
To contribute to GBRL, please review and sign the Contributor License Agreement (CLA) available at: [https://github.com/NVlabs/gbrl/blob/master/CLA.md](https://github.com/NVlabs/gbrl/blob/master/CLA.md)

# Citation
``` 
@inproceedings{
fuhrer2025gradient,
title={Gradient Boosting Reinforcement Learning},
author={Benjamin Fuhrer and Chen Tessler and Gal Dalal},
booktitle={Forty-second International Conference on Machine Learning},
year={2025},
url={https://arxiv.org/abs/2407.08250}
}
```
# Licenses
Copyright © 2024-2026, NVIDIA Corporation. All rights reserved.

This work is made available under the NVIDIA MIT License. Click [here](https://github.com/NVlabs/gbrl/blob/master/LICENSE) to view a copy of this license.