pgbm - GitLite

KO | EN

gitlite — search

#javascript #python #ai #typescript #llm #go #machine-learning #linux #macos #java #vue #pytorch #mcp #open-source

pgbm

(★ 163)

Probabilistic Gradient Boosting Machines

파일 탐색기

최종 버전 다운로드 (.zip)

- - bug_report.md
  - feature_request.md
- - build_source.sh
  - build_wheels.sh
  - test_source.sh
  - test_wheels.sh
- - build_wheels.yml
- - quick_start_firstexample.png
  - quick_start_tunedexample.png
- conf.py
- distributed_learning.md
- examples.md
- features.md
- function_reference.rst
- index.rst
- installation.md
- make.bat
- Makefile
- parameters_sklearn.md
- parameters_torch.md
- quick_start.md
- requirements.txt
- support.md
- - datasets.py
  - example01_housing_cpu.py
  - example04_housing_validation.py
  - example05_housing_vsngboost.py
  - example06_trainingtimevslightgbm.py
  - example07_hyperparamoptimization.py
  - example09_housing_featimportance.py
  - example11_housing_saveandload.py
  - example12_housing_checkpointing.py
  - example13_monotone_constraints.py
  - example14_probregression.py
  - README.md
- - datasets.py
  - example01_housing_cpu.ipynb
  - example01_housing_cpu.py
  - example02_housing_gpu.ipynb
  - example02_housing_gpu.py
  - example04_housing_validation.py
  - example05_housing_vsngboost.py
  - example06_trainingtimevslightgbm.py
  - example07_optimizeddistribution.py
  - example08_housing_autodiff.py
  - example09_housing_featimportance.py
  - example09a_housing_featimportance_shaps.py
  - example10_covidhospitaladmissions.py
  - example11_housing_saveandload.py
  - example12_housing_checkpointing.py
  - example15_monotone_constraints.py
  - README.md
- - dist_template.py
  - example13_housing_dist.py
  - example14_higgs_dist.py
  - README.md
- README.md
- - __init__.py
  - datasets.py
- plots.py
- - __init__.py
  - _binning.pyx
  - _bitset.pyx
  - _gradient_boosting.pyx
  - _predictor.pyx
  - binning.py
  - common.pyx
  - distributions.py
  - gradient_boosting.py
  - grower.py
  - histogram.pyx
  - predictor.py
  - splitting.pyx
  - utils.pyx
- - __init__.py
  - pgbm.py
  - pgbm_dist.py
  - splitgain_cpu.cpp
  - splitgain_cuda.cpp
  - splitgain_kernel.cu
- __init__.py
.gitignore
.readthedocs.yaml
changelog.md
LICENSE
MANIFEST.in
NOTICE
pyproject.toml
README.md
setup.py

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# PGBM <img src="https://icai.ai/wp-content/uploads/2020/01/AIRLabAmsterdam-10-6-gecomprimeerd-transparant.png" width="300" alt="Airlab Amsterdam" align="right"> #
[![PyPi version](https://img.shields.io/pypi/v/pgbm)](https://pypi.org/project/pgbm/)
[![Python version](https://img.shields.io/pypi/pyversions/pgbm)](https://docs.conda.io/en/latest/miniconda.html)
[![GitHub license](https://img.shields.io/pypi/l/pgbm)](https://github.com/elephaint/pgbm/blob/main/LICENSE)

_Probabilistic Gradient Boosting Machines_ (PGBM) is a probabilistic gradient boosting framework in Python based on PyTorch/Numba, developed by Airlab in Amsterdam. It provides the following advantages over existing frameworks:
* Probabilistic regression estimates instead of only point estimates. ([example](https://github.com/elephaint/pgbm/blob/main/examples/torch/example01_housing_cpu.py))
* Auto-differentiation of custom loss functions. ([example](https://github.com/elephaint/pgbm/blob/main/examples/torch/example08_housing_autodiff.py), [example](https://github.com/elephaint/pgbm/blob/main/examples/torch/example10_covidhospitaladmissions.py))
* Native GPU-acceleration. ([example](https://github.com/elephaint/pgbm/blob/main/examples/torch/example02_housing_gpu.py))
* Distributed training for CPU and GPU, across multiple nodes. ([examples](https://github.com/elephaint/pgbm/blob/main/examples/torch_dist/))
* Ability to optimize probabilistic estimates after training for a set of common distributions, without retraining the model. ([example](https://github.com/elephaint/pgbm/blob/main/examples/torch/example07_optimizeddistribution.py))
* Full integration with scikit-learn through a fork of HistGradientBoostingRegressor ([examples](https://github.com/elephaint/pgbm/tree/main/examples/sklearn))

It is aimed at users interested in solving large-scale tabular probabilistic regression problems, such as probabilistic time series forecasting.

For more details, [read the docs](https://pgbm.readthedocs.io/en/latest/index.html) or [our paper](https://arxiv.org/abs/2106.01682) or check out the [examples](https://github.com/elephaint/pgbm/tree/main/examples).

Below a simple example to generate 1000 estimates for each of our test points:
```py
from pgbm.sklearn import HistGradientBoostingRegressor
from sklearn.model_selection import train_test_split
from sklearn.datasets import fetch_california_housing

X, y = fetch_california_housing(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1)
model = HistGradientBoostingRegressor().fit(X_train, y_train) 
yhat_test, yhat_test_std = model.predict(X_test, return_std=True)
yhat_dist = model.sample(yhat_test, yhat_test_std, n_estimates=1000)
```

See also [this example](https://github.com/elephaint/pgbm/blob/main/examples/sklearn/example14_probregression.py) where we compare PGBM to standard gradient boosting quantile regression methods, demonstrating that we can achieve comparable or better probabilistic performance whilst only training a single model.

### Installation ###

See [Installation](https://pgbm.readthedocs.io/en/latest/installation.html) section in our [docs](https://pgbm.readthedocs.io/en/latest/index.html).

### Support ###
In general, PGBM works similar to existing gradient boosting packages such as LightGBM or xgboost (and it should be possible to more or less use it as a drop-in replacement).

* Read the docs for an overview of [hyperparameters](https://pgbm.readthedocs.io/en/latest/parameters.html) and a [function reference](https://pgbm.readthedocs.io/en/latest/function_reference.html).
* See the [examples](https://github.com/elephaint/pgbm/tree/main/examples) folder for examples.

In case further support is required, [open an issue](https://github.com/elephaint/pgbm/issues).

### Reference ###
[Olivier Sprangers](mailto:o.r.sprangers@uva.nl), Sebastian Schelter, Maarten de Rijke. [Probabilistic Gradient Boosting Machines for Large-Scale Probabilistic Regression](https://arxiv.org/abs/2106.01682). Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining ([KDD 21](https://www.kdd.org/kdd2021/)), August 14–18, 2021, Virtual Event, Singapore.

The experiments from our paper can be replicated by running the scripts in the [experiments](https://github.com/elephaint/pgbm/tree/main/paper/experiments) folder. Datasets are downloaded when needed in the experiments except for higgs and m5, which should be pre-downloaded and saved to the [datasets](https://github.com/elephaint/pgbm/tree/main/paper/datasets) folder (Higgs) and to datasets/m5 (m5).

### License ###
This project is licensed under the terms of the [Apache 2.0 license](https://github.com/elephaint/pgbm/blob/main/LICENSE).

### Acknowledgements ###
This project was developed by [Airlab Amsterdam](https://icai.ai/airlab/).