tts-audiobook-tool

KO | EN

gitlite — search

#typescript #ai-agents #ai #dsh-plugin #deepseek-harness #open-source #claude-code #codex #cli #developer-tools #react #windows

tts-audiobook-tool

★ 187 Open GitHub ↗

Audiobook creation app supporting too many TTS models (Qwen3-TTS, OmniVoice, VibeVoice, etc), focused on high-quality output. Plus audio-synced reader web app and standalone server component.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Table of contents

- [Description](#description)
- [Installation](#installation)
- [Usage notes](#usage-notes)
- [Update highlights](#update-highlights)

# Description

`tts-audiobook-tool` is a generative-AI audiobook creation tool focused on high-quality output, supporting a growing list of text-to-speech models:

- [Chatterbox (Multilingual, Turbo)](https://github.com/resemble-ai/chatterbox)
- [Fish Speech S2-Pro)](https://github.com/fishaudio/fish-speech) (either locally or via SGL-Omni)
- [Fish Speech S1-mini](https://github.com/fishaudio/fish-speech)
- [GLM-TTS](https://github.com/zai-org/GLM-TTS)
- [Higgs Audio V2](https://github.com/boson-ai/higgs-audio)
- [Higgs Audio V3](https://github.com/boson-ai/higgs-audio) (via SGL-Omni backend)
- [IndexTTS2](https://github.com/index-tts/index-tts)
- [MiraTTS](https://github.com/ysharma3501/MiraTTS)
- [MOSS-TTS (v1.5 9B and Local-Transformer 1.7B)](https://github.com/OpenMOSS/MOSS-TTS) (either locally or via SGL-Omni)
- [OmniVoice](https://github.com/k2-fsa/OmniVoice)
- [Oute TTS](https://github.com/edwko/OuteTTS)
- [Pocket TTS](https://github.com/kyutai-labs/pocket-tts)
- [Qwen3-TTS](https://github.com/QwenLM/Qwen3-TTS) (either locally or via SGL-Omni)
- [VibeVoice](https://github.com/microsoft/VibeVoice)
- [ZONOS2](https://github.com/Zyphra/ZONOS2) (via SGL-Omni)

The app employs various techniques to make the nondeterministic output of generative text-to-speech models reliable enough for bulk long-form speech synthesis. For example:

- Error detection using speech-to-text verification, with retry logic that keeps the most accurate take
- Segmentation of long-form text at paragraph/sentence/phrase boundaries
- Silence trimming and reduction of excessive pauses within generated audio, plus semantically-aware pause modulation at segment boundaries to improve prosody
- EBU R128 loudness normalization plus optional 48 kHz [LavaSR v2](https://huggingface.co/YatharthS/LavaSR) generative upsampler

It also includes optional realtime modes for audiobook playback and live LLM chat, mainly for voice/model testing and interactive use.

The app uses a plain-text, hotkey-based console interface.

### How to create an audiobook (quick summary)

1. Assign a working project directory.
2. Select a short reference audio clip for the voice clone, and adjust related model parameters if desired.
3. Select the source EPUB or text file, and optionally define file split points.
4. Generate
5. Concatenate the generated sound segments to create the finished audiobook file/s.
6. Use the optional web browser player to play and read your audiobook.

### Browser player

The included browser player displays text highlighted in sync with the generated audio — similar to the Kindle Whispersync or Google Play Books read-along experience. This works because the app embeds the text and word-level timing data directly into the metadata of the FLAC and M4B files it produces.

The player is a static HTML page — no web server or install required. Open `browser_player/index.html` directly in any browser, or use the [hosted version](https://zeropointnine.github.io/tts-audiobook-tool/browser_player/).

**Sample outputs**

All examples use the same source text and the same 15-second voice clone sample using each model's default settings, unless noted:

- [Chatterbox](https://zeropointnine.github.io/tts-audiobook-tool/browser_player/?url=https://zeropointnine.github.io/tts-audiobook-tool-sample-output/waves-chatterbox.abr.m4a)
- [Fish S1-mini](https://zeropointnine.github.io/tts-audiobook-tool/browser_player/?url=https://zeropointnine.github.io/tts-audiobook-tool-sample-output/waves-s1-mini.abr.m4a)
- [Fish S2-Pro](https://zeropointnine.github.io/tts-audiobook-tool/browser_player/?url=https://zeropointnine.github.io/tts-audiobook-tool-sample-output/waves-s2-pro.abr.m4a)
- [GLM-TTS](https://zeropointnine.github.io/tts-audiobook-tool/browser_player/?url=https://zeropointnine.github.io/tts-audiobook-tool-sample-output/waves-glm.abr.m4a)
- [Higgs Audio V2](https://zeropointnine.github.io/tts-audiobook-tool/browser_player/?url=https://zeropointnine.github.io/tts-audiobook-tool-sample-output/waves-higgs.abr.m4a)
- [Higgs Audio V2](https://zeropointnine.github.io/tts-audiobook-tool/browser_player/?url=https://zeropointnine.github.io/tts-audiobook-tool-sample-output/waves-higgs-different-voice.abr.m4a) (using a different voice clone sample, at high temperature)
- [Higgs Audio V3](https://zeropointnine.github.io/tts-audiobook-tool/browser_player/?url=https://zeropointnine.github.io/tts-audiobook-tool-sample-output/waves-higgs-v3.abr.m4a)
- [IndexTTS2](https://zeropointnine.github.io/tts-audiobook-tool/browser_player/?url=https://zeropointnine.github.io/tts-audiobook-tool-sample-output/waves-indextts2.abr.m4a)
- [IndexTTS2](https://zeropointnine.github.io/tts-audiobook-tool/browser_player/?url=https://zeropointnine.github.io/tts-audiobook-tool-sample-output/waves-indextts2-plus-emo.abr.m4a) (with added emotional guidance voice sample)
- [MiraTTS](https://zeropointnine.github.io/tts-audiobook-tool/browser_player/?url=https://zeropointnine.github.io/tts-audiobook-tool-sample-output/waves-mira.abr.m4a)
- [MOSS-TTS v1.5](https://zeropointnine.github.io/tts-audiobook-tool/browser_player/?url=https://zeropointnine.github.io/tts-audiobook-tool-sample-output/waves-moss-v1.5.abr.m4a)
- [MOSS-TTS-Local-Transformer-v1.5](https://zeropointnine.github.io/tts-audiobook-tool/browser_player/?url=https://zeropointnine.github.io/tts-audiobook-tool-sample-output/waves-moss-local-transformer-v1.5.abr.m4a)
- [OmniVoice](https://zeropointnine.github.io/tts-audiobook-tool/browser_player/?url=https://zeropointnine.github.io/tts-audiobook-tool-sample-output/waves-omnivoice.abr.m4a)
- [Oute](https://zeropointnine.github.io/tts-audiobook-tool/browser_player/?url=https://zeropointnine.github.io/tts-audiobook-tool-sample-output/waves-oute.abr.m4a)
- [Pocket TTS](https://zeropointnine.github.io/tts-audiobook-tool/browser_player/?url=https://zeropointnine.github.io/tts-audiobook-tool-sample-output/waves-pocket.abr.m4a)
- [Pocket TTS](https://zeropointnine.github.io/tts-audiobook-tool/browser_player/?url=https://zeropointnine.github.io/tts-audiobook-tool-sample-output/waves-pocket-upscaled.abr.m4a) (upsampled to 48 kHz with LavaSR v2)
- [Pocket TTS](https://zeropointnine.github.io/tts-audiobook-tool/browser_player/?url=https://zeropointnine.github.io/tts-audiobook-tool-sample-output/waves-pocket-stuart-bell.abr.m4a) (predefined voice: stuart_bell)
- [Qwen3-TTS-1.7B-Base](https://zeropointnine.github.io/tts-audiobook-tool/browser_player/?url=https://zeropointnine.github.io/tts-audiobook-tool-sample-output/waves-qwen3-12hz-1.7b-base.abr.m4a)
- [VibeVoice 1.5B](https://zeropointnine.github.io/tts-audiobook-tool/browser_player/?url=https://zeropointnine.github.io/tts-audiobook-tool-sample-output/waves-vibevoice-1.5b.abr.m4a)
- [VibeVoice 1.5B](https://zeropointnine.github.io/tts-audiobook-tool/browser_player/?url=https://zeropointnine.github.io/tts-audiobook-tool-sample-output/waves-vibevoice-1.5b-lora-klett.abr.m4a) (LoRA example) ([LoRA link](https://huggingface.co/vibevoice-community/klett))
- [VibeVoice 7B](https://zeropointnine.github.io/tts-audiobook-tool/browser_player/?url=https://zeropointnine.github.io/tts-audiobook-tool-sample-output/waves-vibevoice-7b.abr.m4a)
- [ZONOS2](https://zeropointnine.github.io/tts-audiobook-tool/browser_player/?url=https://zeropointnine.github.io/tts-audiobook-tool-sample-output/waves-zonos2.abr.m4a)

### Enhance existing audiobooks

This feature extends the browser player to work with professionally produced audiobooks — not just ones generated with this tool. It uses speech-to-text to align the existing audio with the corresponding book text, then embeds the same timing metadata the app normally produces during generation.

Select `Tools` > `Enhance existing audiobook`, and choose your source audiobook file (M4A or M4B) and the corresponding book text. This feature is experimental.

### tts-server-tool

The project also includes an optional stand-alone REST service that runs the app's full configured TTS pipeline — your chosen model, voice clone, and project settings — and plays generated audio through the default sound device or serves it as an HTTP audio stream. This is aimed at developers who want to integrate the app's TTS capabilities into their own tools and workflows — for example, triggering spoken audio from a script, a home automation system, a chatbot, or any other application that can issue HTTP requests. See [server readme](tts_audiobook_tool/server/README-server.md) for setup and API details.

# Installation

### Step 1

First, [ffmpeg](https://ffmpeg.org/download.html) must be in your system path.

### Step 2

Clone the repository and cd into it:

git clone https://github.com/zeropointnine/tts-audiobook-tool
    cd tts-audiobook-tool

### Step 3

A separate virtual environment must be created for each model you want to use. Perform the operations as described in one or more of the sections below, and then return here.

### Step 4 (Windows only)

To enable torch CUDA acceleration on Windows, run the following commands (The project uses the same version of torch for each TTS model's virtual environments unless otherwise noted - v2.8.0/cu128). This extra step is not required when using Linux.

pip uninstall -y torch torchaudio
    pip install torch==2.8.0 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cu128

### Step 5

Run the app by entering:

python -m tts_audiobook_tool

Or, if you have multiple virtual environments installed, you can use `launch.py` to pick a venv and then start the app:

python launch.py # searches the project root for venv subdirectories by default
    python launch.py path/to/venvs-parent-dir # searches for venv subdirectories from the given path

Note that any settings and features that are specific to a given TTS model will be enabled automatically based on which virtual environment has been enabled.

## Virtual environment for Chatterbox:

Initialize a **Python v3.11** virtual environment named "venv-chatterbox":

python -m venv venv-chatterbox # linux/mac
    C:\path\to\python3.11\python.exe -m venv venv-chatterbox # windows

Activate the virtual environment:

source venv-chatterbox/bin/activate # linux/mac
    venv-chatterbox\Scripts\activate.bat # windows

Install dependencies:

pip install -r requirements-chatterbox.txt

Windows CUDA support (unlike the other models, we must use torch 2.6/cu124 here)

pip uninstall -y torch torchaudio
    pip install torch==2.6.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu124

## Virtual environment for Fish S1-mini:

Initialize a **Python v3.12** virtual environment named "venv-fish-s1":

python -m venv venv-fish-s1 # linux/mac
    C:\path\to\python3.12\python.exe -m venv venv-fish-s1 # windows

Activate the virtual environment:

source venv-fish-s1/bin/activate # linux/mac
    venv-fish-s1\Scripts\activate.bat # windows

Install dependencies:

pip install -r requirements-fish-s1.txt

Authenticate the model on HuggingFace:

1. Accept the license terms on the [HuggingFace Fish model page](https://huggingface.co/fishaudio/s1-mini).

2. Authenticate locally using your [access token](https://huggingface.co/settings/tokens) by running `hf auth login`

## Virtual environment for Fish S2-Pro:

> **ℹ️ Note!**
> Requires 24GB VRAM

Initialize a **Python v3.12** virtual environment named "venv-fish-s2":

python -m venv venv-fish-s2 # linux/mac
    C:\path\to\python3.12\python.exe -m venv venv-fish-s2 # windows

Activate the virtual environment:

source venv-fish-s2/bin/activate # linux/mac
    venv-fish-s2\Scripts\activate.bat # windows

Install dependencies:

pip install -r requirements-fish-s2.txt

Authenticate the model on HuggingFace:

1. Accept the license terms on the [HuggingFace Fish model page](https://huggingface.co/fishaudio/s2-pro).

2. Authenticate locally using your [access token](https://huggingface.co/settings/tokens) by running `hf auth login`

## Virtual environment for GLM-TTS

> **ℹ️ Requires CUDA**

### Linux

Initialize a **Python v3.11** virtual environment named `venv-glm`:

- Linux

python -m venv venv-glm

Activate the virtual environment:

- Linux/Mac

source venv-glm/bin/activate

Install dependencies:

pip install -r requirements-glm.txt

### Windows

Requires setting up a "hybrid conda environment" due to a transitive dependency which is difficult to install on Windows using only pip.

Install [Miniconda](https://www.anaconda.com/docs/getting-started/miniconda/install#quickstart-install-instructions) if not already.

Initialize the conda environment:

conda create --prefix "c:\path\to\conda-glm" python=3.11 -y

Activate the conda environment:

conda activate "c:\path\to\conda-glm"

Install this dependency using conda:

conda install pynini==2.1.6 -c conda-forge

Install the rest of the project dependencies using pip like normal:

pip install -r requirements-glm.txt

Note that we pull from [a fork of glm-tts](https://github.com/zeropointnine/glm-tts-packaged) that has been refactored for use as an installable package.

## Virtual environment for Higgs Audio V2:

> **ℹ️ Note!**
> 24GB VRAM recommended

On Linux and macOS, portaudio must be installed (eg, on Mac, `brew install portaudio`)

Initialize a **Python v3.12** virtual environment named `venv-higgs-v2`:

python -m venv venv-higgs-v2 # linux/mac
    C:\path\to\python3.12\python.exe -m venv venv-higgs-v2 # windows

Activate the virtual environment:

source venv-higgs/bin/activate # linux/mac
    venv-higgs\Scripts\activate.bat # windows

Install dependencies:

pip install -r requirements-higgs-v2.txt

Note that the above `requirements` file draws from a personal fork of the `higgs-audio` library due to the fact that the higgs repo is missing `__init__.py` files required for module use.

## Virtual environment for IndexTTS2

Initialize a **Python v3.11** virtual environment named `venv-indextts2`:

python -m venv venv-indextts2 # linux/mac
    C:\path\to\python3.11\python.exe -m venv venv-indextts2 # windows

Activate the virtual environment:

source venv-indextts2/bin/activate # linux/mac
    venv-indextts2\Scripts\activate.bat # windows

Install dependencies:

pip install -r requirements-indextts2.txt

## Virtual environment for MiraTTS

> **ℹ️ Requires CUDA**

Initialize a **Python v3.12** virtual environment named `venv-mira`:

python -m venv venv-mira # linux/mac
    C:\path\to\python3.12\python.exe -m venv venv-mira # windows

Activate the virtual environment:

source venv-mira/bin/activate # linux/mac
    venv-mira\Scripts\activate.bat # windows

Install dependencies:

pip install -r requirements-mira.txt

## Virtual environment for MOSS-TTS v1.5

> **ℹ️ Note!**
> 9B model requires 24GB+ VRAM

Initialize a **Python v3.12** virtual environment named `venv-moss`:

python -m venv venv-moss # linux/mac
    C:\path\to\python3.12\python.exe -m venv venv-moss # windows

Activate the virtual environment:

source venv-moss/bin/activate # linux/mac
    venv-moss\Scripts\activate.bat # windows

Update pip:

python -m pip install --upgrade pip setuptools wheel

Install dependencies:

pip install -r requirements-moss.txt

For CUDA on Windows, uninstall the vanilla version of torch, and install the CUDA version of torch 2.9.1:

pip uninstall -y torch torchaudio
    pip install torch==2.9.1 torchaudio==2.9.1 --index-url https://download.pytorch.org/whl/cu128

Finally, install [Flash Attention](#installing-flash-attention) (optional but recommended).

## Virtual environment for OmniVoice

Initialize a **Python v3.12** virtual environment named `venv-omnivoice`:

python -m venv venv-omnivoice # linux/mac
    C:\path\to\python3.12\python.exe -m venv venv-omnivoice # windows

Activate the virtual environment:

source venv-omnivoice/bin/activate # linux/mac
    venv-omnivoice\Scripts\activate.bat # windows

Install dependencies:

pip install -r requirements-omnivoice.txt

## Virtual environment for Pocket TTS:

Initialize a **Python v3.12** virtual environment named "venv-pocket":

python -m venv venv-pocket # linux/mac
    C:\path\to\python3.12\python.exe -m venv venv-pocket # windows

Activate the virtual environment:

source venv-pocket/bin/activate # linux/mac
    venv-pocket\Scripts\activate.bat # windows

Install dependencies:

pip install -r requirements-pocket.txt

Authenticate the model on HuggingFace:

Accept the license terms on the [HuggingFace Pocket TTS model page](https://huggingface.co/kyutai/pocket-tts).

Authenticate locally using your access token by running hf auth login

## Virtual environment for Qwen3-TTS

Initialize a **Python v3.12** virtual environment named `venv-qwen3tts`:

python -m venv venv-qwen3-tts # linux/mac
    C:\path\to\python3.12\python.exe -m venv venv-qwen3tts # windows

Activate the virtual environment:

source venv-qwent3tts/bin/activate # linux/mac
    venv-qwen3tts\Scripts\activate.bat # windows

Install dependencies:

pip install -r requirements-qwen3tts.txt

Install [Flash Attention](#installing-flash-attention) if using CUDA (optional but recommended).

## Virtual environment for VibeVoice

Initialize a **Python v3.11** virtual environment named `venv-vibevoice`:

python -m venv venv-vibevoice # linux/mac
    C:\path\to\python3.11\python.exe -m venv venv-vibevoice # windows

Activate the virtual environment:

source venv-vibevoice/bin/activate # linux/mac
    venv-vibevoice\Scripts\activate.bat # windows

Install dependencies:

pip install -r requirements-vibevoice.txt

Install [Flash Attention](#installing-flash-attention) if using CUDA (optional but recommended).

Note that because Microsoft famously removed the source code from their github repository, we pull from an archived, third-party fork, [vibevoice-community](https://github.com/vibevoice-community/VibeVoice).

## Virtual environment for Oute TTS:

Initialize a **Python v3.12** virtual environment named "venv-oute":

python -m venv venv-oute # linux/mac
    C:\path\to\python3.12\python.exe -m venv venv-oute # windows

Activate the virtual environment:

source venv-oute/bin/activate # linux/mac
    venv-oute\Scripts\activate.bat # windows

Install dependencies:

pip install -r requirements-oute.txt

### Oute TTS model configuration

Running the app optimally with Oute TTS requires extra steps due to the way the model supports multiple backends, model sizes and quantizations. You will need to review and hand-edit the source file **`config_oute.py`** accordingly.

The [OuteTTS Github project page](https://github.com/edwko/OuteTTS) documents these various options. But here are initial recommendations:

**CUDA:**

Prefer the *ExLllama2* backend if possible: `backend=outetts.Backend.EXL2` (See the example Oute config in `config_oute.py`). However, this requires installing:
- exllama2 library (`pip install exllamav2`)
- [Flash Attention](#installing-flash-attention)

Alternatively, `Backend.HF` is also hardware accelerated but slower. Flash Attention is optional in this case.

**Mac with Apple silicon:**

Use `Backend.LLAMACPP`.

## Virtual environment for SGL-Omni server

> **ℹ️ Note!**
> SGL-Omni is under active development. Should be treated as experimental for the time being.

The app supports server-based TTS inference using SGL-Omni. Install instructions can be found [here](https://sgl-project.github.io/sglang-omni/get_started/installation.html). Note that SGL-Omni is typically installed using Docker (especially for Windows). You may also need to perform additional per-model install steps as described in the docs.

The following models served through SGL-Omni are supported:
- [**Fish S2 Pro**](https://sgl-project.github.io/sglang-omni/cookbook/fishaudio_s2_pro.html) (24GB VRAM recommended)
- [**Higgs Audio V3**](https://sgl-project.github.io/sglang-omni/cookbook/higgs_tts.html) (24GB VRAM recommended)
- [**MOSS-TTS v1.5**](https://sgl-project.github.io/sglang-omni/cookbook/moss_tts.html) (>24GB VRAM required)
- [**Qwen3TTS-Base**](https://sgl-project.github.io/sglang-omni/cookbook/qwen3_tts.html) 
- [**ZONOS2**](https://sgl-project.github.io/sglang-omni/cookbook/zonos2.html) (16+GB VRAM recommended)

Launch SGL-Omni server, specifying one of the supported TTS model as described in their docs. Eg:
- `sgl-omni serve --model-path fishaudio/s2-pro --config examples/configs/s2pro_tts.yaml --port 8000`
- `sgl-omni serve --model-path bosonai/higgs-audio-v3-tts-4b --port 8000`
- `sgl-omni serve --model-path OpenMOSS-Team/MOSS-TTS-v1.5 --port 8000`
- `sgl-omni serve --model-path OpenMOSS-Team/MOSS-TTS-v1.5 --config examples/configs/moss_tts_24gb.yaml --port 8000` (for 24GB VRAM)
- `sgl-omni serve --model-path Qwen/Qwen3-TTS-12Hz-1.7B-Base --config examples/configs/qwen3_tts_1_7b.yaml --port 8000`
- `sgl-omni serve --model-path Qwen/Qwen3-TTS-12Hz-0.6B-Base --config examples/configs/qwen3_tts_0_6b.yaml --port 8000`
- `sgl-omni serve --model-path Zyphra/zonos2 --port 8000`

Once SGL-Omni is set up, continue to creating the app's virtual environment on your "client" computer:

Initialize a **Python v3.12** virtual environment named `venv-sgl-omni`:

python -m venv venv-sgl-omni # linux/mac
    C:\path\to\python3.12\python.exe -m venv venv-sgl-omni # windows

Activate the virtual environment:

source venv-sgl-omni/bin/activate # linux/mac
    venv-sgl-omni\Scripts\activate.bat # windows

Install the dependencies:

pip install -r requirements-sgl-omni.txt

Upon running the app for the first time, be sure to set the SGL-Omni server URL under `Options`.

### Installing Flash Attention

If instructed to do so, perform this step after successfully running `pip install requirements-[model].txt` without errors.

On Linux, enter:

pip install flash-attn==2.8.3 --no-build-isolation

On Windows, download and install a pre-compiled wheel from a trustworthy source (for example, [mjun0812](https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/tag/v0.4.10)). The filename should look something like this, depending on Python version and torch version:

- flash_attn-2.7.4+cu128torch2.8-cp311-cp311-win_amd64.whl
- flash_attn-2.7.4+cu128torch2.8-cp312-cp312-win_amd64.whl

# Usage notes

The app uses sane, conservative defaults, which should allow you to start generating your first audiobook in just a couple minutes. From there, you can explore and adjust the app's various knobs and settings to your preference.

The app saves its state between sessions, so you can interrupt the program at any time and resume later (important due to how long generating a full-length novel can take).

Additionally, setting "chapter cut points" can be useful to generate and export a long work in manageable chunks over time, allowing you to to use early chapter files before the full text is completed.

Note too that it's possible to utilize different voices and even different models over the course of generating the sound segments for a given project for whatever reason.

### Voice cloning

When prepping reference audio for voice cloning, it's worthwhile to prepare a few different sound samples from a given source (not just one), and then test each one out in turn on a short passage of the intended text, as the quality, characteristics, and word error rate resulting from each sample from the same source can vary quite a bit. Some TTS models are especially sensitive to this.

### VRAM considerations

The app ideally wants to use ~2-4 GB extra VRAM for the Whisper model, which needs to runs concurrently with the currently active TTS model to validate its output. If you are getting out of memory errors, try one of the following:

- Use large-v3-turbo instead of large-v3 (saves a gig or two of VRAM) (`Options` > `Whisper model` > `large-v3-turbo`)
- Force Whisper to use system memory instead of running on the GPU (runs much slower, ofc) (`Options` > `Whisper device` > `CPU`)
- Disable Whisper altogether (last resort) (`Options` > `Whisper model` > `Disabled`)

### Configurable model-specific features supported by the app

Zero-shot voice cloning is a first-class feature, supported for all models.

**Chatterbox**

- Multilingual and Turbo model variants
- Exaggeration
- CFG, temperature, top_p, top_k, repetition_penalty, seed

**IndexTTS2**

- Emotion voice sample
- Emotion vector 
- Emotion alpha
- FP16 / FP8
- Temperature, top_p, top_k, seed

**Fish S1-mini**

- Torch compile 
- Temperature, top_p, repetition penalty, seed

**Fish S2-Pro**

- Emotion tags
- Torch compile 
- Experimental rolling continuation mode
- Temperature, top_p, top_k, seed
- SGL-Omni backend inference support (including concurrent requests and streaming)

**GLM-TTS**

- Samplerate selection (24khz or 32 khz)
- Seed

**Higgs Audio V2**

- Temperature, top_p, top_k, seed

**Higgs Audio V3**
- Requires running SGL-Omni server
- Concurrent processing support
- Streaming support (for the stand-alone server and LLM chat mode)
- Temperature, top_p, top_k

**MiraTTS**

- Batch processing support
- Temperature, top_p, top_k, repetition_penalty, seed

**MOSS-TTS**

- Local inference support for both MOSS-TTS v1.5 9B and MOSS-TTS-Local-Transformer 1.7B models
- SGL-Omni backend inference support for MOSS-TTS v1.5
- Batch processing support
- Experimental rolling continuation mode
- Temperature, top_p, top_k, seed
- Music detection/rejection (for MOSS-TTS-Local-Transformer)

**OmniVoice**

- Voice design
- Inference steps
- Speed, CFG, seed

**Oute TTS**

- Temperature

**Pocket TTS**

- Selectable language models (english, french_241, german_241, italian, portuguese, spanish_241)
- Predefined voices (alba, anna, azelma, etc)
- Streaming support (for the stand-alone server and LLM chat mode)
- Supports CUDA acceleration (this is not mentioned in the project page)
- Temperature, seed

**Qwen3-TTS**

- CustomVoice and VoiceDesign model variants
- Support for Darwin-TTS-1.7B-Cross finetune
- Batch processing support
- Experimental rolling continuation mode
- Temperature, top_p, top_k, repetition_penalty, seed

**VibeVoice**

- Alternate VibeVoice models (eg, VibeVoice-7B or custom finetunes)
- LoRA support!
- Batch processing support
- Streaming support (for the stand-alone server and LLM chat mode)
- Music detection/rejection
- CFG, steps, seed

**ZONOS2** (via SGL-Omni)

- Concurrent requests
- Streaming support (for the stand-alone server and LLM chat mode)
- Temperature, top_k, repetition_penalty

### Inference speeds, expectations

Listed below are some anecdotal TTS inference speeds. The app adopts each respective model's reference inference implementation logic as much as possible. Note how CUDA inference speeds on Linux are usually significantly faster than on Windows.

| TTS Model               | Setup                | Speed           | Notes |
| ----------------------- | -------------------- | --------------- | ----- |
| Chatterbox Multilingual | GTX 4090, Windows    | ~190% realtime  | 
| Chatterbox Multilingual | GTX 3080 Ti, Windows | ~130% realtime  | 
| Chatterbox Multilingual | Macbook Pro M1 (MPS) | 20-35% realtime |
| Chatterbox Turbo        | GTX 3080 Ti, Linux   | 500%+ realtime  | 
| Chatterbox Turbo        | Macbook Pro M1       | ~70% realtime   |
| Fish S2-Pro             | GTX 4090, Windows    | 150% realtime   | 
| Fish S1-mini            | GTX 3080 Ti, Windows | 500%+ realtime  | 
| Higgs V2                | GTX 4090, Windows    | ~200% realtime  | 
| Higgs V3                | GTX 4090, Windows    | 300%+ realtime  | SGL-Omni; concurrent requests=1
| Higgs V3                | GTX 4090, Windows    | 2000%+ realtime | SGL-Omni; concurrent requests=10
| IndexTTS2               | GTX 4090, Windows    | ~150% realtime  | 
| IndexTTS2               | GTX 3080 Ti, Windows | ~90% realtime   |
| IndexTTS2               | Macbook Pro M1 (MPS) | ~20% realtime   |
| MOSS-TTS v1.5           | GTS 4090, Windows    | ~45% realtime (yes really) | batch size=1, flash attn
| MOSS-TTS v1.5           | GTS 4090, Windows    | ~80% realtime   | batch size=2, flash attn
| MOSS-TTS-Local-Transformer-v1.5           | GTS 4090, Windows    | 500% realtime   | batch size=20, flash attn
| OmniVoice TTS           | GTX 3080 Ti, Linux   | 300% realtime   | default steps
| OmniVoice TTS           | Macbook Pro M1 (MPS) | 20% realtime    | default steps
| Pocket TTS              | GTX 3080 Ti, Linux   | 1300% realtime  | 
| Pocket TTS              | Ryzen 7 7700, Linux  | ~200% realtime  | CPU (CUDA disabled)
| Pocket TTS              | Macbook Pro M1       | 350% realtime   | 
| Qwen3-TTS 1.6B          | GTX 3080 Ti, Linux   | 300% realtime   | batch size=5
| Qwen3-TTS 1.6B          | GTX 3080 Ti, Linux   | 100% realtime   | batch size=1
| GLM-TTS                 | GTX 3080 Ti, Linux   | 200%+ realtime  | 
| MiraTTS                 | GTX 3080 Ti, Linux   | 3000% realtime (yes really) | batch size=10
| MiraTTS                 | GTX 3080 Ti, Linux   | 800% realtime   | batch size=1
| Oute                    | GTX 3080 Ti, Windows | ~90% realtime   | using `outetts.Backend.EXL2`
| VibeVoice-Large 7B      | GTX 4090, Windows    | 600%+ realtime  | batch size=10, default steps
| VibeVoice 1.5B          | GTX 3080 Ti, Linux   | 1000% realtime  | batch size=10, default steps
| VibeVoice 1.5B          | GTX 3080 Ti, Linux   | 200%+ realtime  | batch size=1, default steps
| VibeVoice 1.5B          | GTX 3080 Ti, Windows | ~120% realtime  | batch size=1, default steps
| VibeVoice 1.5B          | Macbook Pro M1       | ~40% realtime   |
| ZONOS2                  | GTX 4090, Windows    | 200% realtime   | SGL-Omni- concurrent requests=1
| ZONOS2                  | GTX 4090, Windows    | 500% realtime   | SGL-Omni- concurrent requests=5

# Update highlights

**2026-08-11**

- Replaced Sidon with **[LavaSR v2](https://github.com/ysharma3501/LavaSR)** upscaler, which is better suited to TTS and produces more tonally faithful, consistent output.

**2026-08-11**

- `Generate > Select lines / review sound segments`: Full-screen interface for selecting lines to be queued for generation and for reviewing already-generated sound segments for deletion/quick-regeneration.

**2026-08-05**

- Added support for [**ZONOS2**](https://sgl-project.github.io/sglang-omni/cookbook/zonos2.html) using SGL-Omni.
- Added submenu `Create > Pauses between segments` with options for `Normal` and `Shorter`.

**2026-08-01**

- Added **voice selections editor**. When using multiple voice samples, voice samples can now be assigned on a line-by-line basis. `Voice > Edit voice selections`.

- Added **review-and-delete sound segments editor**. Useful for reviewing and selectively deleting unwanted sound generations (bad takes, etc). `Generate > Review/delete segments`.

- Added **text lines editor**. Allows for manually deleting unwanted lines from imported text (eg, table of contents, publisher info, etc) as well as splitting lines at inner phrase boundaries. The latter can be useful when manually assigning voice samples for direct quotes/dialog. `Text > View/edit text`

**2026-07-15**

- Added detection for potentially cut-off words at the end of generated audio
- Added detection for pathologically long audio generations

**2026-07-13**

- Added ability to add **multiple voice clone samples**, which get rotated after each generation.Applies to most of the supported TTS models.

**2026-06-07+**

- Added support for using [**SGL-Omni**](https://sgl-project.github.io/sglang-omni) as the backend inference engine. Currently supported TTS models using SGL-Omni are:

- [**Higgs V3**](https://huggingface.co/bosonai/higgs-audio-v3-tts-4b)
  - MOSS-TTS v1.5
  - Qwen3TTS
  - Fish S2

Also added LLM chat **custom system prompts** for Higgs V3 and Fish S2 Pro, which elicit "tagged output" from the LLM. Fun way to test the expressiveness of models' respective custom tag features.

**2026-06-03**

- Added experimental **rolling continuation mode** for Fish S2 Pro and Qwen3-TTS Base (this is already available for MOSS-TTS)

**2026-05-28**

- Added support for [**MOSS-TTS**](https://huggingface.co/OpenMOSS-Team/MOSS-TTS-v1.5) (MOSS-TTS v1.5 9B and MOSS-TTS-Local-Transformer 1.7B)

**2026-05-25**

- Added support for [**Darwin-TTS**](https://huggingface.co/FINAL-Bench/Darwin-TTS-1.7B-Cross) (use Qwen3-TTS virtual environment and select custom hf repo id: [`zeropointnine/Darwin-TTS-1.7B-Cross-Qwen3Tokenizer`](https://huggingface.co/zeropointnine/Darwin-TTS-1.7B-Cross-Qwen3Tokenizer))

**2026-05-15**

- Added support for **EPUB** text import

**2026-05-12**

- Added ability to import project settings from app-created m4b/flac files

**2026-05-11**

- Added **first-class language support for Spanish**. This results in more accurate dictionary-based audio word error validation (less false positives), alongside English, and makes using "Strictness level: High" more useful and more efficient.

**2026-05-07**

- Added support for [**OmniVoice TTS**](https://github.com/k2-fsa/OmniVoice) (thanks @GNBS711)

**2026-05-05/06**

- Added **streaming** option for LLM chat feature and stand-alone server, which minimizes "time to first audio". Currently supported for Vibevoice and Pocket TTS.

**2026-04-28**

- Added side feature: **Realtime LLM chat**
- Added **mlx-whisper** for Mac Apple Silicon (MLX acceleration for Whisper)

**2026-04-25**

- Added **multi-venv convenience launcher** (`python launch.py`)

**2026-04-24**

- Added **[Sidon](https://github.com/sarulab-speech/Sidon) 48khz upsampler** post-processing option. 
- Added `Concat` > **`Limit silence gaps`** (prevents long silences in sound generations, keeps narration flowing)

**2026-04-23**

- Added **high shelf filter** post-processing option
- Added strictness level "intolerant"
- Added AAC bitrate setting

**2026-04-22**

- Added support for [**Pocket TTS**](https://github.com/kyutai-labs/pocket-tts), lightweight model suitable for non-GPU systems.

**2026-04-13**

- Added side feature: **tts-server-tool**. View [server readme](tts_audiobook_tool/server/README-server.md).

**2026-04-11**

- Added configurable **top-p**, **top-k**, and **repetition penalty** hyperparameters to models where applicable given their architecture.

**2026-04-07**

- Added support for [**Fish S2-Pro**](https://huggingface.co/fishaudio/s2-pro)

- Updated torch, ctranslate2, and flash-attention dependencies for all virtual environments.

- Fish S1-mini and S2-Pro - Added torch compile toggle

**2026-01-26**

- Added support for [**Qwen3-TTS**](https://github.com/QwenLM/Qwen3-TTS) (Base, CustomVoice, and VoiceDesign models)

**2026-01-23**

- Added new text segmentation method **"multiple sentences"** (`Text` > `Text segmentation strategy`). Probably the ideal way to chunk text in combination with `max_words_per_segment = 80` when the TTS model can handle long prompts with high accuracy and without speeding up.

**2026-01-21**

- Added support for **VibeVoice LoRAs** [(recommended trainer)](https://github.com/voicepowered-ai/VibeVoice-finetuning)

**2026-01-18~**

- **Batching optimization** improves net TTS inference speed by up to 25% (VibeVoice and Mira).

- **VibeVoice batch support** (drastically speeds up inference).

- **VibeVoice seed support**

- **Music detection** - Validator will now reject audio with music hallucinations (VibeVoice)

- Added submenu: `Concat` > `Open audiobook file in the player app`

**2026-01-12**

- Improved per-phrase text segmentation (ie, `Concat` > `Subdivide into phrases`). For pre-existing projects, requires reimporting the source text.

- Prevent some TTS models from mangling the output of prompts starting with **all-caps phrases** (relevant for MiraTTS in particular, plus some others).

**2026-01-10**

- "Section markers" can now be used for creating **M4B chapters** and player bookmarks (`Concatenate` > `Section markers` > `Mode`)

- Note, app now saves AAC files using `.m4b` suffix instead of `.m4a`.

**2026-01-05**

- Added option to **add section markers using regular expressions** (thanks @AntitrustEnthusiast).

**2026-01-04**

- Added support for **Chatterbox-Turbo**.

- Improved logic for trimming audio generations with spurious words and noises at the beginning or end (important for Chatterbox).

**2025-12-31**

- Added support for **MiraTTS**, including batch mode. The model is exceptionally fast when running in batch mode.

- Added option: `Generate` > `Transcript validation strictness`

- Option to control seed value now available for: Chatterbox, Fish, GLM

**2025-12-23**

- Added option: **`Project`** > **`Word substitutions`**. This allows you to replace words in the prompt that the TTS model may mispronounce. Requires some experimentation, as you might imagine.

- Relatedly, added: `Project` > `Word substitutions` > `Inspect project text for uncommon words`.

- Also added option: `Generate` > `Transcript validation max retries`

**2025-12-21**

- Simplified and much-improved **speech-to-text validation logic**, which is now able to calculate the number of word errors for an audio generation (rather than simply adjudging "pass" vs "fail" using a patchwork of bespoke functions).

- When a prompt is re-generated due to excessive word errors, the program is able to cherrypick the generation with the least number of errors.

**2025-12-16**

- Added support for **GLM-TTS**.

- Added option: **`Realtime audio generation` > `Save output`**.

**2025-12-13**

- Added option **`Concatenate` > `Subdivide into phrases`**. This allows the player app to highlight text on a "per-phrase" basis by leveraging the STT word-level timing data (Note, on pre-existing projects, all audio would first need be re-generated).

**2025-12-09**

- Added option to **clone project** (load an existing project > make new project > "Do you want to carry over the current project's settings?" > "Yes")

- Added option to force use of **CPU torch device** even when CUDA or MPS is available.

**2025-12-07**

- Added **max text segment word length** option, allowing for up to 80 words per text segment prompt. Can be useful with VibeVoice and Fish (other models are best left at the default of 40 words).

- **Resolved Linux CUDA crashes** related to faster-whisper library by changing Linux torch requirements. If you were experiencing this problem, get latest and update your virtual environment (`pip install -r requirements-[modelname].txt`).

**2025-12-06**

- **Updated Chatterbox** requirement to v0.1.4, which is multilanguage capable, and **added language code** project setting. Thanks to @JuMGameN, co-author for this update.

- For existing users, the Chatterbox venv must be recreated using Python v3.11.

**2025-11-11**

- Added option to keep the **Whisper** model on the CPU.

**2025-11-05**

- Player/reader: Added **bookmarks** feature, ability to pin the audio player widget in desktop mode, plus.

**2025-10-24**

- Added option to **disable transcription validation** (`Options` > `Whisper model type` > `Disabled`). Doing so is only advisable while VRAM is at a premium or when using reliably accurate and stable TTS models (eg, IndexTTS2; definitely not VoiceVoice 1.5B or Chatterbox).

**2025-10-23**

- Added option to insert a "page turn" **sound effect at section breaks** (2+ blank lines in the text). I use this or text files converted from epub files using Calibre, which tends to insert two or more blank lines at chapter breaks, etc.

**2025-10-03**

- Added support for **IndexTTS2**, including secondary voice reference sample guiding emotion and "emotion vectors".

- Added option to **specify Whisper model** (large-v3 or large-v3-turbo)

**2025-09-15**

- Added support for **VibeVoice alternative models** (ie, large/7B model)

**2025-09-12**

- Added support for **VibeVoice**.

**2025-08-10**

- Migrated from openai-whisper to **faster-whisper** (faster, less memory, equivalent accuracy).

**2025-08-06**

- Added support for **Higgs Audio V2** (3B base model).

**2025-07-18**

- Added support for **Fish OpenAudio S1-mini**

- Logic to detect dropped phrases at end of generated sound segments (common occurrence with the Fish model)

- Added utility to transcode and concatenate directory of MP3 chapter files to M4A (meant for use with the "Enhance existing audiobook" tool)

**2025-07-02**

- New feature: **Real-time playback** (`Options > Real-time playback`)

This serves as a quicker and more "casual" alternative to the regular `Generate audio` UI flow, and allows for more-or-less immediate and uninterrupted audio playback (contingent on system performance, ofc). It employs the same quality control measures except for loudness normalization, and does not save its output.

**2025-06-28** (many)

- Generated sound segments now have silence trimmed off the ends, and in the concatenation step, stitched-together lines have pauses of varying lengths inserted at paragraph, sentence, and phrase boundaries, resulting in much improved prosody / flow.

- Loudness normalization is now being applied correctly (on the final audio file instead of per audio segment)

- Better detection of undesired repeating phrases (Oute especially)

- Better detection and fix for spooky Chatterbox noises at the end of prompts

- Short 1-2 word sentences now get grouped with adjacent sentences to mitigate Chatterbox and Oute's issues with short prompts

- Streamlined handling of audio data throughout

- Encoding audiobook files in AAC/M4A format no longer requires intermediate FLAC step

- Streamlined some UI

- Some improvements to the web player/reader

tts-audiobook-tool

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README