NeuSight

KO | EN

gitlite — search

#typescript #ai-agents #deepseek-harness #dsh-plugin #open-source #ai #claude-code #cli #windows #codex #developer-tools #dsh

★ 83 Open GitHub ↗

No description available.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# NeuSight

NeuSight is a framework designed to predict the performance of deep learning training and inference on various GPUs. For more details, please refer to our paper, [Forecasting GPU Performance for Deep Learning Training and Inference](https://dl.acm.org/doi/10.1145/3669940.3707265).

## Installation

To install NeuSight as a Python package, run:

```bash
git clone https://github.com/sitar-lab/NeuSight.git
cd NeuSight
pip install -e .
```
- NeuSight was tested on Python 3.9 and PyTorch version 2.1.0.

## Quick Start

We provide two Python scripts for predicting the latency of deep learning execution and training the predictor:

- `scripts/pred.py` for making predictions
 - `scripts/train.py` for training the predictor

The scripts use GPU and deep learning description files in JSON format, and execution hyperparameters such as batch size. See the scripts in `scripts/example` for examples of how to use these scripts.

## Tool Inputs
NeuSight requires two input files to run: a device configuration file and a deep learning model configuration file.

### Device Configuration File

The device configuration file specifies the architectural parameters of the prediction target GPU. Example configuration files can be found in `data/device_configs`. The configuration file includes
  - `Device`      : User-specified name of the device
  - `Dev_Mem`     : Global memory in GB
  - `Mem_Bw`      : Memory bandwidth of global memory in GB/s
  - `Num_Sm`      : Number of SMs
  - `Core_Per_SM` : Number of CUDA cores per SM
  - `Freq`        : Compute frequency in GHz
  - `SingleFLOPs` : Peak FP32 performance in GFLOPS/s
  - `L2Cache`     : Size of L2 Cache in MB

### Deep Learning Model Configuration File

The deep learning model configuration file specifies the architectural parameters of the target deep learning model. Example configuration files can be found in  `data/DLmodel_configs`. Configuration file are specified in `Hugging Face` model description format.

## Code Structure
```bash
/ : NEUSIGHT_ROOT
|-- neusight                     : Source file directory for NeuSight
|   |-- Dataset                  : For collecting and processing dataset for training
|   |-- Model                    : For machine learning based predictor
|   |-- Opgraph                  : For manipulating operator graphs
|   |-- Prediction               : For NeuSight predictor
|   |-- Tracing                  : For tracing ML model graphs
|-- scripts                      : Main scripts for running NeuSight
|   |-- asplos                   : Training dataset and scripts used for ASPLOS 2025 paper
|   |   |-- data                 : Input files used for NeuSight
|   |   |   |-- dataset          : Datasets used for training and tile table (NVIDIA)
|   |   |   |-- dataset_amd      : Datasets used for training and tile table (AMD)
|   |   |   |-- device_configs   : GPU description files
|   |   |   |-- DLmodel_configs  : DL model description files
|   |   |   |-- predictor        : Model configuration and trained parameters for ML predictor
|   |   |-- label                : Measured latencies for ML models evaluated
|   |   |-- results              : Results of NeuSight prediction
|   |   |-- summary              : Summary of NeuSight prediction
|   |-- example                  : Example scripts for running NeuSight
```

## Citation
If you use NeuSight in your research, please cite our paper:

```
@inproceedings{neusight,
  author = {Lee, Seonho and Phanishayee, Amar and Mahajan, Divya},
  title = {Forecasting GPU Performance for Deep Learning Training and Inference},
  year = {2025},
  isbn = {9798400706981},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  url = {https://doi.org/10.1145/3669940.3707265},
  doi = {10.1145/3669940.3707265},
  booktitle = {Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1},
  pages = {493–508},
  numpages = {16},
  keywords = {deep learning, gpu performance forecasting, ml for systems, training and inference},
  location = {Rotterdam, Netherlands},
  series = {ASPLOS '25}
}

NeuSight

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README