BNN_NoBN

KO | EN

gitlite — search

#javascript #hacktoberfest #ai #typescript #golang #machine-learning #deep-learning #linux #nodejs #java #awesome #cli #openai #open-source

(★ 56)

[CVPRW 21] "BNN - BN = ? Training Binary Neural Networks without Batch Normalization", Tianlong Chen, Zhenyu Zhang, Xu Ouyang, Zechun Liu, Zhiqiang Shen, Zhangyang Wang

BNN_NoBN 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# BNN - BN = ? Training Binary Neural Networks without Batch Normalization

[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)

Codes for this paper [BNN - BN = ? Training Binary Neural Networks without Batch Normalization](https://arxiv.org/pdf/2104.08215.pdf). [CVPR BiVision Workshop 2021]

Tianlong Chen, Zhenyu Zhang, Xu Ouyang, Zechun Liu, Zhiqiang Shen, Zhangyang Wang.

## Overview

Batch normalization (BN) is a key facilitator and considered essential for state-of-the-art binary neural networks (BNN). However, the BN layer is costly to calculate and is typically implemented with non-binary parameters, leaving a hurdle for the efficient implementation of BNN training. It also introduces undesirable dependence between samples within each batch.

Inspired by the latest advance on Batch Normalization Free (BN-Free) training, we extend their framework to training BNNs, and for the first time demonstrate that BNs can be completely removed from BNN training and inference regimes. By plugging in and customizing techniques including adaptive gradient clipping, scale weight standardization, and specialized bottleneck block, a **BN-free BNN** is capable of maintaining competitive accuracy compared to its BN-based counterpart. Experimental results can be found in [our paper](https://arxiv.org/pdf/2104.08215.pdf).

## BN-Free Binary Neural Networks

## Reproduce

### Environment

```
pytorch == 1.5.0
torchvision == 0.6.0
timm == 0.4.5
```

### Training on ImageNet

```
./script/imagenet_reactnet_A_bf.sh (BN-Free ReActNet-A)
./script/imagenet_reactnet_A_bn.sh (with BN ReActNet-A)
./script/imagenet_reactnet_A_none.sh (without BN ReActNet-A)
```

## Citation

```
@misc{chen2021bnn,
      title={"BNN - BN = ?": Training Binary Neural Networks without Batch Normalization}, 
      author={Tianlong Chen and Zhenyu Zhang and Xu Ouyang and Zechun Liu and Zhiqiang Shen and Zhangyang Wang},
      year={2021},
      eprint={2104.08215},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}
```

## Acknowledgement

https://github.com/liuzechun/ReActNet

https://github.com/liuzechun/Bi-Real-net

https://github.com/vballoli/nfnets-pytorch

https://github.com/deepmind/deepmind-research/tree/master/nfnets

BNN_NoBN

BNN_NoBN 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

BNN_NoBN

BNN_NoBN 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README