Large-Wireless-Models

KO | EN

gitlite — search

#react #typescript #go #machine-learning #rust #deep-learning #docker #java #nodejs #cli #awesome-list #openai #ios #open-source

Large-Wireless-Models

(★ 19)

No description available.

Large-Wireless-Models Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Scaling Law for Large Wireless Models

This repository hosts the official reference implementation for the AAAI 2026 paper **"Scaling Law for Large Wireless Models"** by Ziheng Liu, Jiayi Zhang, Haoyu Wang, Bokai Xu, Chen Zhang, Yiyang Zhu, and Enyu Shi. The project releases the Large Wireless Model (LWM) family plus the full preprocessing, training, and downstream evaluation pipeline that powered the paper.

## Highlights
- **Scalable transformer encoder** (`scaling_lwm_model.py`) that covers 0.7M–13B parameter regimes for wireless channels.
- **DeepMIMO-based data interface** (`scaling_input_preprocess.py`) to sample scenarios, slice channels into patches, and create masked modeling targets.
- **End-to-end training/evaluation scripts** for masked channel modeling, inference, and supervised tasks (LoS/beam classification, etc.).
- **Pretrained checkpoints** (`model_weights.pth`) and a worked `tutorial.py` demonstrating how to reproduce the main results.

## Repository layout
```
.
├── scaling_lwm_model.py        # LWM backbone definitions and pretrained loader
├── scaling_input_preprocess.py # DeepMIMO ingestion, tokenization, and label helpers
├── scaling_pretraining.py      # Masked channel modeling training loop
├── scaling_inference.py        # Inference utilities (loss + embedding extraction)
├── tutorial.py                 # Example end-to-end downstream classification workflow
├── utils.py                    # Visualization, classifiers, and evaluation helpers
├── scenarios/                  # DeepMIMO scenario folders (see Data section)
└── model_weights.pth           # Example pretrained checkpoint
```

## Installation
1. Create an environment (Python ≥ 3.9 is recommended) and install Python requirements:
   ```bash
   pip install -r requirements.txt
   ```
2. Install **DeepMIMO v3**
   - Clone the official DeepMIMO repository and install it in editable mode:
     ```bash
     git clone https://github.com/DeepMIMO/DeepMIMO-codes.git
     pip install -e DeepMIMO-codes
     ```
   - Or install from PyPI if/when an official `DeepMIMOv3` package is released.
3. Ensure `scenarios/` contains the DeepMIMO scenario folders you intend to use (see next section).
4. (Optional) Download or generate `model_weights.pth` checkpoints and place them in the project root. The default scripts expect this filename.

## Data preparation
- All experiments rely on DeepMIMO scenarios. Download the required scenarios (e.g., `city_11_santaclara`) and place each folder inside `scenarios/`.
- `scaling_input_preprocess.tokenizer()` wraps DeepMIMO loading and patch extraction. Set `selected_scenario_names` when calling the tokenizer or change the defaults inside `scaling_pretraining.py`/`tutorial.py`.
- Noise augmentation can be added by passing `snr_db` to `tokenizer()` or `patch_maker()`.

## Training (masked channel modeling)
`scaling_pretraining.py` contains the training loop used in the paper.
```bash
python scaling_pretraining.py
```
Important knobs (edit the script or wrap it yourself):
- `model_names` / `selected_model_names`: choose the desired LWM scale (see `MODEL_CONFIG`).
- `scenario_names`: specify one or more DeepMIMO scenarios.
- `batch_size`, `total_batch_size`, `n_epochs`, and optimizer settings control efficiency.
- Set `load_model=True` to continue fine-tuning from `model_weights.pth`.

The script automatically:
1. Tokenizes DeepMIMO channels into masked sequences.
2. Builds train/validation loaders with random splits per epoch.
3. Trains with AMP + gradient accumulation.
4. Saves checkpoints (`model_weights.pth` by default).

## Inference & downstream evaluation
- `scaling_inference.py` exposes `lwm_inference` to compute masked modeling loss and extract CLS/channel embeddings from a checkpoint:
  ```python
  from scaling_inference import lwm_inference
  from scaling_lwm_model import lwm
  model = lwm.from_pretrained('model_weights.pth', device='cuda')
  loss, embeddings = lwm_inference(preprocessed_data, 'cls_emb', model, 'cuda')
  ```
- `tutorial.py` demonstrates end-to-end downstream classification:
  ```bash
  python tutorial.py
  ```
  It shows how to:
  1. Tokenize a scenario and run the pretrained LWM.
  2. Generate LoS/NLoS or beam labels via `create_labels`.
  3. Train feed-forward classifiers (`utils.FCN`) at different data scales.
  4. Plot F1 scores with `plot_metrics`.
- Raw channel tensors (no LWM) can be obtained with `create_raw_dataset` for ablations.

## Visualization & utilities
`utils.py` hosts helper functions for:
- Dimensionality reduction snapshots (`plot_dimensionality_reduction`).
- Coverage/pathloss maps (`plot_coverage`, `plot_pathloss`).
- Baseline classifiers (`FCN`, `classify_by_euclidean_distance`).
- Metric plotting (`plot_metrics`).

## Citation
If you use this codebase or any derived models, please cite the AAAI paper:
```
Ziheng Liu, Jiayi Zhang, Haoyu Wang, Bokai Xu, Chen Zhang, Yiyang Zhu, and Enyu Shi,
“Scaling Law for Large Wireless Models”, in AAAI 2026.
```
BibTeX:
```bibtex
@inproceedings{liu2026scaling,
  title     = {Scaling Law for Large Wireless Models},
  author    = {Liu, Ziheng and Zhang, Jiayi and Wang, Haoyu and Xu, Bokai and Zhang, Chen and Zhu, Yiyang and Shi, Enyu},
  booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year      = {2026}
}
```

## Questions & contributions
For issues or feature requests, please open a GitHub issue once the repository is public. Pull requests that improve documentation, add tasks, or extend coverage are welcome.

Large-Wireless-Models

Large-Wireless-Models Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Large-Wireless-Models

Large-Wireless-Models Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README