KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #dsh-plugin #codex #typescript #developer-tools #ai-agents #deepseek-harness #claude-code #ai-agent #cli #react #mcp

windowed-roformer

★ 46 Open GitHub ↗

Official Repository for "Efficient Vocal Source Separation Through Windowed RoFormer"

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Efficient Vocal Source Separation Through Windowed Sink Attention

State-of-the-art vocal separation models like Mel-Band-Roformer rely on full temporal self-attention mechanisms, where each temporal frame interacts with every other frames. This incurs heavy computational costs that scales quadratically with input audio length, motivating chunking and windowing approaches. Through analysis of a pre-trained vocal separation model, we discovered that temporal attention patterns are highly localized. Building on this insight, we replaced full attention with windowed sink attention (WSA) with small temporal attention window and attention sinks. We show empirically that fine-tuning from the original checkpoint recovers 92% of the original SDR performance while reducing FLOPs by 44.5x.

Paper: [![Paper](https://img.shields.io/badge/arXiv-2510.25745-blue.svg)](https://arxiv.org/abs/2510.25745)

HuggingFace Model: [![Hugging Face Model](https://img.shields.io/badge/Hugging%20Face-WindowedRoformer-yellow.svg)](https://huggingface.co/smulelabs/windowed-roformer)

A big part of this work's code is built on top of [ZFTurbo's MSS Training Repository](https://github.com/ZFTurbo/Music-Source-Separation-Training).

---

If you are only interested in using the implemented FlexAttention based windowed attention module, please directly copy the `flex_attention_utils.py` out. You can use it as a drop-in replacement for the full attention module in the original model.

---

## Getting Started
### Setting up environment
```bash
uv venv separate --python=3.10
source separate/bin/activate
uv pip install -r requirements.txt
```

Download the model checkpoint from [HuggingFace](https://huggingface.co/smulelabs/windowed-roformer) and place it in the root directory.
```bash
wget https://huggingface.co/smulelabs/windowed-roformer/resolve/main/mbr-win10-sink8.ckpt
```

### Running the model
```bash
python main.py {path-to-input} -o {path-to-output} -c {path-to-checkpoint}
```

More arguments are available as defined in `main.py`.

windowed-roformer

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README