qwen3-tts-rknn-russian

KO | EN

gitlite — search

#javascript #python #hacktoberfest #react #ai #typescript #llm #go #golang #android #machine-learning #rust #deep-learning #linux

qwen3-tts-rknn-russian

★ 9 Open GitHub ↗

Qwen3-TTS inference on RK3588 with RKLLM + ONNX + RKNN NPU

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Qwen3-TTS on RK3588 (RKNN/RKLLM)

Pure Python pipeline for running [Qwen3-TTS](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-0.6B-Base) on Rockchip RK3588 SoC using RKLLM (NPU) for the talker LLM and RKNN for the vocoder.

> **For better performance**, see [qwen3-tts-axera-russian](https://github.com/nickovchinnikov/qwen3-tts-axera-russian) which achieves **RTF 2.0x** (vs 5.5x here) using llama.cpp + GGML code predictor.

## Architecture

```
                  ┌──────────────┐
Text ────────────>│  Tokenizer   │──> token_ids
                  └──────┬───────┘
                         │
                  ┌──────▼───────┐
                  │ TextEmbedder │──> embeddings [N, 1024]
                  │  (CPU numpy) │
                  └──────┬───────┘
                         │
                  ┌──────▼───────┐
                  │  TalkerRKLLM │──> hidden_state [1, 1024]
                  │  (RK3588 NPU)│    + code_0 token
                  └──────┬───────┘
                         │ (autoregressive loop)
                  ┌──────▼───────┐
                  │CodePredictor │──> codes 1-15
                  │ (ONNX / CPU) │
                  └──────┬───────┘
                         │
                  ┌──────▼───────┐
                  │   Vocoder    │──> audio PCM (24kHz)
                  │(RKNN NPU/CPU)│
                  └──────────────┘
```

**Key difference from CosyVoice3**: Qwen3-TTS uses a Code Predictor (15 groups x 5 transformer layers) instead of Flow DiT + HiFT vocoder.

## Performance

Tested on FriendlyElec CM3588 (RK3588, 32GB RAM):

| Configuration | Talker | Code Pred | Vocoder | RTF |
|---|---|---|---|---|
| **GGUF + ONNX-1t + RKNN-64** | 30ms/step | 374ms/step | 6.0s | **5.5x** |
| RKLLM + ONNX-1t + RKNN-64 | 60ms/step | 374ms/step | 6.0s | 6.2x |
| RKLLM + RKLLM-CP + RKNN-64 | 60ms/step | ~500ms/step | 6.2s | 7.8x |
| RKLLM + numpy + RKNN-64 | 60ms/step | ~500ms/step | 6.1s | 8.1x |

**Tip**: Using GGUF talker (auto-detected if `.gguf` model present in `--base_dir`) gives ~2x speedup over RKLLM.

## Quick Start

```bash
# On CM3588
python3 qwen3_tts_pipeline.py \
    --text "Hello, how are you today?" \
    --base_dir /root/tts-rknn \
    --language english \
    --output output.wav

# Russian
python3 qwen3_tts_pipeline.py \
    --text "Привет, как дела? Сегодня хорошая погода." \
    --base_dir /root/tts-rknn \
    --language russian \
    --output output.wav

# Voice cloning
python3 qwen3_tts_pipeline.py \
    --text "Text to speak in cloned voice" \
    --base_dir /root/tts-rknn \
    --prompt_dir examples/prompt_russian/ \
    --language russian
```

## Model Files

Required files in `--base_dir`:

```
base_dir/
├── qwen3_tts_talker_rk3588.rkllm     # Talker RKLLM (887 MB, w8a8)
├── qwen3_tts_talker_q4_k_m.gguf      # Talker GGUF (469 MB, recommended)
├── embeddings/
│   ├── text_embedding.npy              # [151936, 2048] (1.2 GB)
│   ├── text_projection_*.npy           # MLP weights
│   ├── codec_embedding.npy             # [3072, 1024] (12 MB)
│   └── codec_head.npy                  # [3072, 1024] (12 MB)
├── code_predictor/
│   ├── code_predictor_weights.npz      # All weights (240 MB)
│   └── code_predictor_decode_step.onnx # ONNX model (314 MB)
└── vocoder_traced_64_sim_q8.rknn       # Vocoder RKNN (134 MB)
```

## Model Conversion

See [docs/MODEL_CONVERSION.md](docs/MODEL_CONVERSION.md) for step-by-step conversion instructions.

## Supported Languages

chinese, english, german, russian, french, japanese, korean

## See Also

- [qwen3-tts-axera-russian](https://github.com/nickovchinnikov/qwen3-tts-axera-russian) — optimized multi-server pipeline, RTF 2.0x
- [cosyvoice3-axera-russian](https://github.com/nickovchinnikov/cosyvoice3-axera-russian) — CosyVoice3 on CM3588 with AX650N NPU, RTF 1.26x

## License

Apache-2.0. Model weights are subject to [Qwen3-TTS license](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-0.6B-Base).

qwen3-tts-rknn-russian

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README