PyXAB

KO | EN

gitlite — search

#javascript #llm #machine-learning #rust #deep-learning #linux #docker #nodejs #windows #awesome #cli #ios #openai #open-source

(★ 127)

PyXAB - A Python Library for X-Armed Bandit and Online Blackbox Optimization Algorithms

File Explorer

Download Latest Version (.zip)

- - codeql.yml
  - testing.yml
- - plot_new_algo.py
  - plot_new_obj.py
  - plot_new_partition.py
  - README.rst
- - plot_HCT_Himmelblau.py
  - plot_HOO_Garland.py
  - plot_VHCT_Landmine.py
  - README.rst
- - cumulative_leaderboard.ipynb
  - simple_leaderboard.ipynb
- - conf.py
  - index.rst
  - sg_execution_times.rst
- make.bat
- Makefile
- requirements.txt
- - Ackley.png
  - DifficultFunc.png
  - DoubleSine.png
  - Garland.png
  - Himmelblau.png
  - Rastrigin.png
- HCT_heatmap.gif
- HCT_trajectory.gif
- HCT_visual.gif
- partition.png
- - __init__.py
  - Algo.py
  - DOO.py
  - GPO.py
  - HCT.py
  - HOO.py
  - PCT.py
  - POO.py
  - SequOOL.py
  - SOO.py
  - StoSOO.py
  - StroquOOL.py
  - VHCT.py
  - VPCT.py
  - VROOM.py
  - Zooming.py
- - landmine_formated_data.pkl
  - landmine_obj.py
  - run_landmine.py
- - __init__.py
  - BinaryPartition.py
  - DimensionBinaryPartition.py
  - KaryPartition.py
  - Node.py
  - Partition.py
  - RandomBinaryPartition.py
  - RandomKaryPartition.py
- - __init__.py
  - Ackley.py
  - Cexample.py
  - DifficultFunc.py
  - DoubleSine.py
  - Garland.py
  - Himmelblau.py
  - Objective.py
  - Rastrigin.py
- - __init__.py
- - __init__.py
  - plot.py
  - run_synthetic.py
  - visualize_gif.py
- __init__.py
.coveragerc
.gitignore
.readthedocs.yaml
CITATION.cff
LICENSE
README.md
setup.cfg
setup.py

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# PyXAB - Python *X*-Armed Bandit

PyXAB is a Python open-source library for *X*-armed bandit algorithms, a prestigious set of optimizers for online black-box optimization and 
hyperparameter optimization.

PyXAB contains the implementations of **10+** optimization algorithms, including the classic ones such as [Zooming](https://arxiv.org/pdf/0809.4882.pdf), 
 [StoSOO](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/algos/StoSOO.py), and [HCT](https://proceedings.mlr.press/v32/azar14.html), and the most
recent works such as [GPO](https://proceedings.mlr.press/v98/xuedong19a.html), [StroquOOL](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/algos/StroquOOL.py) and [VHCT](https://arxiv.org/abs/2106.09215).
PyXAB also provides the most commonly-used synthetic objectives to evaluate the performance of different algorithms and the implementations for different hierarchical partitions

**PyXAB is featured for:**

- **User-friendly APIs, clear documentation, and detailed examples**
- **Comprehensive library** of optimization algorithms, partitions and synthetic objectives
- **High standard code quality and high testing coverage**
- **Low dependency** for flexible combination with other packages such as PyTorch, Scikit-Learn

**Reminder**: The algorithms are maximization algorithms!

## Quick Links

- [Quick Example](#Quick-Example)
- [Documentations](#Documentations)
- [Installation](#Installation)
- [Features](#Features)
  * [*X*-armed bandit algorithms](#Stochastic-X-armed-bandit-algorithms)
  * [Hierarchical partition ](#Hierarchical-partition)
  * [Synthetic objectives](#Synthetic-objectives)
- [Contributing](#Contributing)
- [Citations](#Citations)

## Quick Example
PyXAB follows a natural and straightforward API design completely aligned with the online blackbox
optimization paradigm. The following is a simple 6-line usage example.

First, we define the parameter domain and the algorithm to run. 
At every round  `t`, call `algo.pull(t)` to get a point and call 
`algo.receive_reward(t, reward)` to give the algorithm the objective evaluation (reward)

```python3
from PyXAB.algos.HOO import T_HOO

domain = [[0, 1]]               # Parameter is 1-D and between 0 and 1
algo = T_HOO(rounds=1000, domain=domain) 
for t in range(1000):
    point = algo.pull(t)
    reward = 1                  #TODO: User-defined objective returns the reward
    algo.receive_reward(t, reward)
```

More detailed examples can be found [here](https://pyxab.readthedocs.io/en/latest/getting_started/auto_examples/index.html)

## Documentations

* The most up-to-date [documentations](https://pyxab.readthedocs.io/)

* The [roadmap](https://github.com/users/WilliamLwj/projects/1) for our project    
  
  * Our [manuscript](https://arxiv.org/abs/2303.04030) for the library

## Installation

To install via pip, run the following lines of code
```bash
pip install PyXAB                 # normal install
pip install --upgrade PyXAB       # or update if needed
```

To install via git, run the following lines of code
```bash
git clone https://github.com/WilliamLwj/PyXAB.git
cd PyXAB
pip install .
```

## Features:

### *X*-armed bandit algorithms

* Algorithm starred are meta-algorithms (wrappers)

| Algorithm                                                                           | Research Paper                                                                                                                                                                           | Year |
|-------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------|
| [Zooming](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/algos/Zooming.py)     | [Multi-Armed Bandits in Metric Spaces](https://arxiv.org/pdf/0809.4882.pdf)                                                                                                              | 2008 |
| [T-HOO](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/algos/HOO.py)           | [*X*-Armed Bandit](https://jmlr.org/papers/v12/bubeck11a.html)                                                                                                                           | 2011 |
| [DOO](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/algos/DOO.py)             | [Optimistic Optimization of a Deterministic Function without the Knowledge of its Smoothness](https://proceedings.neurips.cc/paper/2011/file/7e889fb76e0e07c11733550f2a6c7a5a-Paper.pdf) | 2011 |
| [SOO](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/algos/SOO.py)             | [Optimistic Optimization of a Deterministic Function without the Knowledge of its Smoothness](https://proceedings.neurips.cc/paper/2011/file/7e889fb76e0e07c11733550f2a6c7a5a-Paper.pdf) | 2011 |
| [StoSOO](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/algos/StoSOO.py)       | [Stochastic Simultaneous Optimistic Optimization](http://proceedings.mlr.press/v28/valko13.pdf)                                                                                          | 2013 |
| [HCT](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/algos/HCT.py)             | [Online Stochastic Optimization Under Correlated Bandit Feedback](https://proceedings.mlr.press/v32/azar14.html)                                                                         | 2014 |
| [POO*](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/algos/POO.py)            | [Black-box optimization of noisy functions with unknown smoothness](https://papers.nips.cc/paper/2015/hash/ab817c9349cf9c4f6877e1894a1faa00-Abstract.html)                               | 2015 |
| [GPO*](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/algos/GPO.py)            | [General Parallel Optimization Without A Metric](https://proceedings.mlr.press/v98/xuedong19a.html)                                                                                      | 2019 |
| [PCT](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/algos/PCT.py)             | [General Parallel Optimization Without A Metric](https://proceedings.mlr.press/v98/xuedong19a.html)                                                                                      | 2019 |
| [SequOOL](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/algos/SequOOL.py)     | [A Simple Parameter-free And Adaptive Approach to Optimization Under A Minimal Local Smoothness Assumption](https://arxiv.org/pdf/1810.00997.pdf)                                        | 2019 |
| [StroquOOL](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/algos/StroquOOL.py) | [A Simple Parameter-free And Adaptive Approach to Optimization Under A Minimal Local Smoothness Assumption](https://arxiv.org/pdf/1810.00997.pdf)                                        | 2019 |
| [VROOM](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/algos/VROOM.py)         | [Derivative-Free & Order-Robust Optimisation](https://arxiv.org/pdf/1910.04034.pdf)                                                                                                      | 2020 |
| [VHCT](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/algos/VHCT.py)           | [Optimum-statistical Collaboration Towards General and Efficient Black-box Optimization](https://openreview.net/forum?id=ClIcmwdlxn)                                                     | 2023 |
| [VPCT](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/algos/VPCT.py)           | N.A. ([GPO](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/algos/GPO.py) + [VHCT](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/algos/VHCT.py))                               | N.A. |

### Hierarchical partition

| Partition                                                                                                             | Description                                                                                                        |
|-----------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------|
| [BinaryPartition](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/partition/BinaryPartition.py)                   | Equal-size binary partition of the parameter space, the split dimension is chosen uniform randomly                 |
| [RandomBinaryPartition](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/partition/RandomBinaryPartition.py)       | Random-size binary partition of the parameter space, the split dimension is chosen uniform randomly                |
| [DimensionBinaryPartition](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/partition/DimensionBinaryPartition.py) | Equal-size partition of the space with a binary split on each dimension, the number of children of one node is 2^d |
| [KaryPartition](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/partition/KaryPartition.py)       | Equal-size K-ary partition of the parameter space, the split dimension is chosen uniform randomly                  |
| [RandomKaryPartition](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/partition/RandomKaryPartition.py) | Random-size K-ary partition of the parameter space, the split dimension is chosen uniform randomly                 |

### Synthetic objectives

* Some of these objectives can be found [on Wikipedia](https://en.wikipedia.org/wiki/Test_functions_for_optimization)

| Objectives <img width=200/>| Image | 
| --- |--- |
| [Garland](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/synthetic_obj/Garland.py) | <img src="https://raw.githubusercontent.com/WilliamLwj/PyXAB/main/figs/synthetic/Garland.png" alt="Garland" width="100"/> |
| [DoubleSine](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/synthetic_obj/DoubleSine.py)  | <img src="https://raw.githubusercontent.com/WilliamLwj/PyXAB/main/figs/synthetic/DoubleSine.png" alt="DoubleSine" width="100"/>  |
| [DifficultFunc](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/synthetic_obj/DifficultFunc.py) | <img src="https://raw.githubusercontent.com/WilliamLwj/PyXAB/main/figs/synthetic/DifficultFunc.png" alt="DifficultFunc" width="100"/>  |
| [Ackley](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/synthetic_obj/Ackley.py) | <img src="https://raw.githubusercontent.com/WilliamLwj/PyXAB/main/figs/synthetic/Ackley.png" alt="Ackley" width="100"/>  |
| [Himmelblau](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/synthetic_obj/Himmelblau.py) | <img src="https://raw.githubusercontent.com/WilliamLwj/PyXAB/main/figs/synthetic/Himmelblau.png" alt="Himmelblau" width="100"/>  |
| [Rastrigin](https://github.com/WilliamLwj/PyXAB/blob/main/PyXAB/synthetic_obj/Rastrigin.py) |  <img src="https://raw.githubusercontent.com/WilliamLwj/PyXAB/main/figs/synthetic/Rastrigin.png" alt="Rastrigin" width="100"/>  |

## Contributing

We appreciate all forms of help and contributions, including but not limited to

* Star and watch our project
* Open an issue for any bugs you find or features you want to add to our library
* Fork our project and submit a pull request with your valuable codes

Please read the [contributing instructions](https://pyxab.readthedocs.io/en/latest/info/contributing.html) before submitting
a pull request.

## Citations
If you use our package in your research or projects, we kindly ask you to cite our work

```text
@article{Li2023PyXAB,
  doi = {10.21105/joss.06507},
  url = {https://joss.theoj.org/papers/10.21105/joss.06507},
  author = {Li, Wenjie and Li, Haoze and Song, Qifan and Honorio, Jean},
  title = {PyXAB -- A Python Library for $\mathcal{X}$-Armed Bandit and Online Blackbox Optimization Algorithms},
  journal={Journal of Open Source Software},
  year = {2024},
  issn={2475-9066},
}
```

We would also appreciate it if you could cite our related works.

```text
@article{li2023optimumstatistical,
    title={Optimum-statistical Collaboration Towards General and Efficient Black-box Optimization},
    author={Wenjie Li and Chi-Hua Wang and Guang Cheng and Qifan Song},
    journal={Transactions on Machine Learning Research},
    issn={2835-8856},
    year={2023},
    url={https://openreview.net/forum?id=ClIcmwdlxn},
    note={}
}
```

```text

@article{Li2022Federated,
  title={Federated $\chi$-armed Bandit}, 
  volume={38}, 
  url={https://ojs.aaai.org/index.php/AAAI/article/view/29267}, 
  DOI={10.1609/aaai.v38i12.29267}, 
  number={12},
  journal={Proceedings of the AAAI Conference on Artificial Intelligence}, 
  author={Li, Wenjie and Song, Qifan and Honorio, Jean and Lin, Guang}, 
  year={2024}, 
  month={Mar.}, 
  pages={13628-13636}

}
```

```text

@InProceedings{Li2024Personalized,
  title = 	 {Personalized Federated $\chi$-armed Bandit},
  author =       {Li, Wenjie and Song, Qifan and Honorio, Jean},
  booktitle = 	 {Proceedings of The 27th International Conference on Artificial Intelligence and Statistics},
  pages = 	 {37--45},
  year = 	 {2024},
  editor = 	 {Dasgupta, Sanjoy and Mandt, Stephan and Li, Yingzhen},
  volume = 	 {238},
  series = 	 {Proceedings of Machine Learning Research},
  month = 	 {02--04 May},
  publisher =    {PMLR},
  pdf = 	 {https://proceedings.mlr.press/v238/li24a/li24a.pdf},
  url = 	 {https://proceedings.mlr.press/v238/li24a.html},
}
```

PyXAB

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

PyXAB

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

Download README