multichannel-asr

KO | EN

gitlite — search

#typescript #llm #go #machine-learning #rust #deep-learning #linux #java #cli #kubernetes #vue #security #ios #openai

multichannel-asr

(★ 11)

A novel approach for multi-channel call transcription using mono-channel ASR models. Leverages VAD-based channel merging with silence insertion strategy to enable speaker diarization without complex separation models. Supports Whisper and other end-to-end ASR models.

multichannel-asr 최신버젼 다운로드

최종 버전 다운로드 (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Multichannel ASR

**[English](README.md) | [简体中文](README_zh-CN.md)**

---

## Overview

Multichannel ASR is a training-free solution for transcribing multi-channel audio recordings (e.g., call center conversations, remote meetings) using standard mono-channel ASR models like OpenAI Whisper. It achieves accurate speaker diarization through a novel VAD-based channel merging and silence insertion strategy.

## Key Features

- **Training-Free**: No model training or fine-tuning required
- **Model Agnostic**: Works with any mono-channel ASR model (Whisper, FunASR, SenseVoice, etc.)
- **Efficient**: Multi-processing support for batch processing

## How It Works

The pipeline follows six steps to transform multi-channel audio into diarized transcripts:

1. **Multi-Channel Input**: Load stereo or multi-channel audio
2. **VAD & Segmentation**: Detect voice activity in each channel separately
3. **Silence Insertion & Merging**: Merge channels with strategic silence gaps:
   - 1000ms silence between different speakers
   - 50ms silence within same speaker
4. **ASR Transcription**: Transcribe merged audio using Whisper
5. **Timestamp-based Diarization**: Match transcription timestamps to original speakers
6. **Structured Output**: Generate Excel files with speaker labels and timestamps

## Installation

### Prerequisites

- Python 3.10+
- CUDA 12.2+ (for GPU acceleration)
- FFmpeg: `sudo apt-get install ffmpeg` (Ubuntu) or `brew install ffmpeg` (macOS)

### Install Dependencies

```bash
# Clone the repository
git clone https://github.com/Deng-GuiFeng/multichannel-asr.git
cd multichannel-asr

# Install PyTorch with CUDA support (CUDA 12.6 is compatible with CUDA 12.2)
pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu126

# Install other dependencies
pip install -r requirements.txt
```

## Usage

```bash
python src/main.py \
    --model_id openai/whisper-large-v3-turbo \
    --device cuda:0 \
    --src_dir ./data/input \
    --tgt_dir ./data/output
```

**Parameters:**
- `--model_id`: Hugging Face model ID (e.g., `openai/whisper-large-v3-turbo`)
- `--device`: Device to use (`cuda:0` for GPU, `cpu` for CPU)
- `--src_dir`: Input directory containing audio files
- `--tgt_dir`: Output directory for transcripts

## Patent Notice

This project implements the method described in Chinese Patent **[CN120895028A](CN120895028A.PDF)** (Published). The code is released under the MIT License, but commercial use of the patented method may require separate authorization.

## License

[MIT License](LICENSE.txt)

## Acknowledgments

- [OpenAI Whisper](https://github.com/openai/whisper)
- [Hugging Face Transformers](https://github.com/huggingface/transformers)
- [WebRTC VAD](https://github.com/wiseman/py-webrtcvad)

## Citation

If you find this project useful, please cite the original paper:

```bibtex
@article{deng2025evaluating,
  title={Evaluating Large Language Models in Crisis Detection: A Real-World Benchmark from Psychological Support Hotlines},
  author={Deng, Guifeng and Rao, Shuyin and Lin, Tianyu and Dai, Anlu and Wang, Pan and Xie, Junyi and Song, Haidong and Zhao, Ke and Xu, Dongwu and Cheng, Zhengdong and others},
  journal={arXiv preprint arXiv:2506.01329},
  year={2025}
}
```

multichannel-asr

multichannel-asr 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

multichannel-asr

multichannel-asr 최신버젼 다운로드

오늘의 추천 프로젝트

Was this content helpful?

Recent Feedback

Download README