acceltran

KO | EN

gitlite — search

#python #react #ai #typescript #golang #android #machine-learning #rust #deep-learning #docker #windows #kubernetes #ios #openai

(★ 62)

[TCAD'23] AccelTran: A Sparsity-Aware Accelerator for Transformers

acceltran Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# AccelTran: A Sparsity-Aware Monolithic 3D Accelerator for Transformer Architectures at Scale

![Python Version](https://img.shields.io/badge/python-v3.6%20%7C%20v3.7%20%7C%20v3.8-blue)
![Conda](https://img.shields.io/badge/conda%7Cconda--forge-v4.8.3-blue)
![PyTorch](https://img.shields.io/badge/pytorch-v1.8.1-e74a2b)
![Hits](https://hits.seeyoufarm.com/api/count/incr/badge.svg?url=https%3A%2F%2Fgithub.com%2FJHA-Lab%2Facceltran&count_bg=%2379C83D&title_bg=%23555555&icon=&icon_color=%23E7E7E7&title=hits&edge_flat=false)

AccelTran is a tool to simulate a design space of accelerators on diverse *flexible* and *heterogeneous* transformer architectures supported by the FlexiBERT 2.0 framework at [jha-lab/txf_design-space](https://github.com/JHA-Lab/txf_design-space).

The figure below shows the utilization of different modules in an AccelTran architecture for the BERT-Tiny transformer model.

![AccelTran GIF](https://github.com/JHA-Lab/acceltran/blob/main/simulator/figures/bert_tiny.gif)

## Table of Contents
- [Environment Setup](#environment-setup)
  - [Clone this repository and initialize sub-modules](#clone-this-repository-and-initialize-sub-modules)
  - [Setup python environment](#setup-python-environment)
- [Run synthesis](#run-synthesis)
- [Run pruning](#run-pruning)
- [Run simulator](#run-simulator)
- [Developer](#developer)
- [Cite this work](#cite-this-work)
- [License](#license)

## Environment setup

### Clone this repository and initialize sub-modules

```shell
git clone https://github.com/JHA-Lab/acceltran.git
cd ./acceltran/
git submodule init
git submodule update
```

### Setup python environment

The python environment setup is based on conda. The script below creates a new environment named `txf_design-space`:
```shell
source env_setup.sh
```
For `pip` installation, we are creating a `requirements.txt` file. Stay tuned!

## Run synthesis

Synthesis scripts use [Synopsys Design Compiler](https://www.synopsys.com/implementation-and-signoff/rtl-synthesis-test/dc-ultra.html). All hardware modules are implemented in SystemVerilog in the directory [synthesis/top](https://github.com/JHA-Lab/acceltran/tree/main/synthesis/top).

To get area and power consumption reports for each module, use the following command:
```shell
cd ./synthesis/
dc_shell -f 14nm_sg.tcl -x "set top_module <M>"
cd ..
```
Here, `<M>` is the module that is to be synthesized in: `mac_lane`, `ln_forward_<T>` (for layer normalization), `softmax_<T>`, etc. where `<T>` is the tile size among 8, 16, or 32.

All output resports are stored in [synthesis/reports](https://github.com/JHA-Lab/acceltran/tree/main/synthesis/reports).

To run the synthesis for the DMA module, run the following command instead:
```shell
cd ./synthesis/
dc_shell -f dma.tcl 
```

## Run pruning

To get the sparsity in activations and weights in an input transformer model and its corresponding performance on the GLUE benchmark, use the dynamic pruning model: DP-BERT.

To test the effect of different sparsity ratios on the model performance on the SST-2 benchmark, use the following script:
```shell
cd ./pruning/
python3 run_evaluation.py --task sst2 --max_pruning_threshold 0.1
cd ..
```
The script uses a weight-pruned [model](https://huggingface.co/echarlaix/bert-base-uncased-sst2-acc91.1-d37-hybrid), and so, the weights are not pruned futher. To prune the weights with a `pruning_threshold` as well, use the flag: `--prune_weights`.

## Run simulator

AccelTran supports a diverse range of accelerator hyperparameters. It also supports all \~10<sup>88</sup> models in the FlexiBERT 2.0 design space.

To specify the configuration of an accelerator's architecture, use a configuration file in [simulator/config](https://github.com/JHA-Lab/acceltran/tree/main/simulator/config) directory. Example configuration files are given accelerators optimized for BERT-Nano and BERT-Tiny. Accelerator hardware configuration files should conform with the design space specified in the [simulator/design_space/design_space.yaml](https://github.com/JHA-Lab/acceltran/blob/main/simulator/design_space/design_space.yaml) file.

To specify the transformer model parameters, use a model dictionary file in [simulator/model_dicts](https://github.com/JHA-Lab/acceltran/tree/main/simulator/model_dicts). Model dictionaries for BERT-Nano and BERT-Tiny have already been provided for convenience.

To run AccelTran on the BERT-Tiny model, while plotting utilization and metric curves every 1000 cycles, use the following command:
```shell
cd ./simulator/
python3 run_simulator.py --model_dict_path ./model_dicts/bert_tiny.json --config_path ./config/config_tiny.yaml --plot_steps 1000 --debug
cd ..
```

This will output the accelerator state for every cycle. For more information on the possible inputs to the simulation script, use:
```shell
cd ./simulator/
python3 run_simulator.py --help
cd ..
```

## Developer

[Shikhar Tuli](https://github.com/shikhartuli). For any questions, comments or suggestions, please reach me at [stuli@princeton.edu](mailto:stuli@princeton.edu).

## Cite this work

Cite our work using the following bitex entry:
```bibtex
@article{tuli2023acceltran,
  title={{AccelTran}: A Sparsity-Aware Accelerator for Dynamic Inference with Transformers},
  author={Tuli, Shikhar and Jha, Niraj K},
  journal={arXiv preprint arXiv:2302.14705},
  year={2023}
}
```
If you use the AccelTran design space to implement transformer-accelerator co-design, please also cite:
```bibtex
@article{tuli2023transcode,
  title={{TransCODE}: Co-design of Transformers and Accelerators for Efficient Training and Inference},
  author={Tuli, Shikhar and Jha, Niraj K},
  journal={arXiv preprint arXiv:2303.14882},
  year={2023}
}
```

## License

See License file for more details.

acceltran

acceltran Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

acceltran

acceltran Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README