KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #ai-agents #dsh-plugin #codex #developer-tools #claude-code #windows #desktop-app #typescript #rust #ai-agent #cli

Casmopolitan

★ 35 Open GitHub ↗

[ICML'21] Think Global and Act Local: Bayesian Optimisation for Categorical and Mixed Search Spaces

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Code for 'Think Global and Act Local: Bayesian Optimisation for Categorical and Mixed Search Spaces'

This repo contains the current implementation of the Casmopolitan algorithm.

[Link to paper](https://arxiv.org/abs/2102.07188)

If you find our paper or this repo to be useful for your research, please consider citing:
```
@article{wan2021think,
  title={Think Global and Act Local: Bayesian Optimisation over High-Dimensional Categorical and Mixed Search Spaces},
  author={Wan, Xingchen and Nguyen, Vu and Ha, Huong and Ru, Binxin and Lu, Cong and Osborne, Michael A},
  journal={International Conference on Machine Learning (ICML) 38},
  year={2021}
}
```

## Bayesian Optimization over Discrete and Mixed Spaces via Probabilistic Reparameterization 
If you are interested in Bayesian optimization in discrete and/or mixed search spaces, we have a new work tackling this problem using probabilistic reparameterisation of the acquisition functions, allowing continuous optimization to be used even when the input space is discrete. Please check out the paper accepted at NeurIPS 2022 [here](https://openreview.net/forum?id=WV1ZXTH0OIn) and code [here](https://github.com/facebookresearch/bo_pr)!

# Dependencies
```
Anaconda Python 3.6
gpytorch==0.3.5
pytorch==1.6.0
numpy==1.16.4
pandas==0.25.1
tqdm (for progress bar visualisation)==4.32.1
xg-boost==0.90 (to run XGBoost on MNIST task)
scikit-learn==0.21.2
```

# Examples to reproduce results in paper
```bash
    python3 main.py -p pest
    python3 main.py -p func2C --max_iters 200
    python3 main.py -p ackley53 --max_iters 400
```
where ```-p``` specifies the problem, ```-n``` specifies the number of trust region to initialise. Look at ```main.py``` to see other settings that can be tuned. ```-a``` specifies the acquisition function.

# To run a new custom problem
1. Go to ```./test_funcs/base.py``` for the implementation of the base class. A new problem should be a derived class
   that properly implements all the required methods of the base class. Note that by default the new problem has property
   ```problem_type == 'categorical'```. If your new problem is in the mixed space, you need to change this to ```mixed``` (See examples
   in ```./mixed_test_func```).
2. Edit the appropriate imports and other specifications and run from ```main.py``` as usual.

### Update on 11 April 2022

Added the adversarial attack example (see details in the paper. The code is adapted from https://github.com/rubinxin/BayesOpt_Attack).

1. To run the experiment, first download the ```cifar-10-batches-bin``` data and place them under ```mixed_test_func/AdvAttack/tf_models/cifar-10-batches-bin```
2. Use ```run_adversarial_attack.py``` as the main script to run attack using Casmpolitan.

# Acknowledgements

This code repository uses materials from the following public repositories. The authors thank the respective repository maintainers

1. TuRBO: Eriksson, D., Pearce, M., Gardner, J. R., Turner, R., & Poloczek, M. (2019). Scalable global optimization via local Bayesian optimization. Advances in Neural Information Processing Systems, 32 (NeurIPS).
   Code repo: https://github.com/uber-research/TuRBO/
2. CoCaBO: Ru, B., Alvi, A. S., Nguyen, V., Osborne, M. A., & Roberts, S. J. (2020). Bayesian Optimisation over Multiple Continuous and Categorical Inputs. International Conference on Machine Learning, 37 (ICML).
   Code repo: https://github.com/rubinxin/CoCaBO_code
3. COMBO: Oh, C., Tomczak, J. M., Gavves, E., & Welling, M. (2019). Combinatorial Bayesian Optimization using the Graph Cartesian Product. Advances in Neural Information Processing Systems, 32 (NeurIPS).
Code repo: https://github.com/QUVA-Lab/COMBO
4. Starting kit for NeurIPS 2020 Black-Box Optimisation Challenge. Code repo: https://github.com/rdturnermtl/bbo_challenge_starter_kit
5. Bliek, L., Guijt, A., Verwer, S., & de Weerdt, M. (2020). Black-box mixed-variable optimisation using a surrogate model that satisfies integer constraints. arXiv preprint arXiv:2006.04508. Code repo: https://github.com/lbliek/MVRSM
6. BayesOpt Adversarial Attack: https://github.com/rubinxin/BayesOpt_Attack

Casmopolitan

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README