CosyVoice2-GRPO

KO | EN

gitlite — search

#python #hacktoberfest #react #ai #golang #machine-learning #docker #nodejs #awesome #cli #awesome-list #security #pytorch #mcp

CosyVoice2-GRPO

(★ 39)

A simple implementation for improving CosyVoice2 by GRPO method

CosyVoice2-GRPO Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

## CosyVoice2-GRPO

[Demos](https://ryuclc.github.io/LLM-TTS-GRPO/); [Paper](https://arxiv.org/abs/2509.18798)

A simple implementation for improving CosyVoice2 by GRPO method.

This is the code of paper “Group Relative Policy Optimization for Text-to-Speech with Large Language Models”

We modify official CosyVoice2 code with trl to achieve GRPO fine-tune. Only need the CosyVoice2 environment, without need to install any other modules or frameworks.

Most codes are borrowed from official [CosyVoice2](https://github.com/FunAudioLLM/CosyVoice), https://github.com/channel-io/ch-tts-llasa-rl-grpo, https://github.com/SebastianBodza/blog_projects/tree/main/00_GRPO_LLaSa, https://github.com/huggingface/trl.

## Training

### Environment

The same as official

For the grpo reward service:
``` sh
pip install "fastapi[standard]" "uvicorn[standard]" openai-whisper jiwer zhon
```

### Step

examples/multi/cosyvoice2-rl contains the scripts for training CosyVoice2-GRPO inited from official CosyVoice2

1. first download the pretrained CosyVoice2 model and process data into parquet format

2. run the reward service 
``` sh
CUDA_VISIBLE_DEVICES=1 python services/main.py --port 8080 --model large-v3
```

3. Then Run the training scripts:

Use your own wandb key (cosyvoice/bin/train.py), change the reward server url in .py files (cosyvoice/llm/reward_tts.py)
``` sh
sh train_multi_llm_qwen_grpo_new_v1_beta0.04grad1.0gen8.sh 
```

## Inference example

Inference follow the official example

In our paper, the zero-shot objective set is seed_mycv_test/demo_zero_shot-obj.json

``` sh
cd seed_mycv_test
bash infer_cli_mp_wose_rl.sh 0 demo_zero_shot-obj.json results_rl_grpo_new_v1_beta0.04grad1.0gen8_step2000 /home/changliu/work2/CosyVoice-2.0/examples/multi/cosyvoice2-rl/exp/cosyvoice2/llm/torch_ddp/grpo_new_v1_beta0.04grad1.0gen8/epoch_0_step_2000.pt
```

## Note

Now the implementation is rough, plan to refine code or migrate to other framework in the future.

### To-Do

- [x] paper
- [x] code
- [x] training script
- [x] inference script

CosyVoice2-GRPO

CosyVoice2-GRPO Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

CosyVoice2-GRPO

CosyVoice2-GRPO Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README