Zero-shot voice tuner for Kokoro-82M.
A 5-30 second reference clip in, a stock-shaped [510, 1, 256] voice pack out in about 0.3 seconds.
pip install inno-kokorofrom inno_kokoro.enroll import Tuner, enroll, read
from kokoro import KPipeline
tuner = Tuner() # downloads the weights to the huggingface cache on first use
pack, _ = enroll(*read("my_ref.wav"), tuner)
pipe = KPipeline(lang_code="a")
wav = next(pipe("Hello from a tuned voice.", voice=pack)).audioTuner("/models/model.safetensors") loads from a path of your own instead of the cache.
Currently only available for English; prefixes work like the stock packs e.g: af_, am_, bf_, bm_.
Command line
Write a voice pack plus a test render to disk:
inno-kokoro my_ref.wav am_me # -> voices/am_me.pt + voices/am_me_test.wav
inno-kokoro my_ref.wav am_me packs/ # write somewhere else
inno-kokoro my_ref.wav am_me --fmax 300 # override the pitch ceiling (set automatically otherwise)Weights download on first use. To pre-fetch them instead (Dockerfile layer, offline machine):
inno-kokoro --fetch /models # -> /models/model.safetensorsIntegrated into remsky/Kokoro-FastAPI (v0.9.0+) with ENABLE_INNO_TUNER=true
For best results, the reference audio should be:
- 3-second minimum, up to the first 30-seconds
- Single speaker (english).
- Reasonably clear of audio artifacts
Voice pack generation time:
- about 0.05 s per second of reference on CPU
- 0.1 to 0.3 s total on a GPU (after the model is loaded).
LibriSpeech test-clean, F5-TTS cross-sentence split: 1127 utterances, 39 held-out speakers.
Scored against the speaker's real recording. Normalized scores compare the render between sounding like a stranger (0) and a second sample of the same benchmark speaker (1).
RTF on an RTX 4060 Ti.
| system | SIM-o | normalized | UTMOS | RTF |
|---|---|---|---|---|
| ground truth (second recording) | 0.695 | 1.00 | 4.10 | |
| F5-TTS v1 base | 0.650 | 0.94 | 3.86 | 0.48 |
| StyleTTS2 (LibriTTS zero-shot) | 0.386 | 0.46 | 4.40 | 0.06 |
| Inno v0.2 | 0.288 | 0.32 | 4.45 | 0.07 |
| OpenVoice v2 | 0.227 | 0.23 | 3.80 | 0.12 |
| Kokoro, nearest stock pack | 0.167 | 0.15 | 4.25 | 0.06 |
UTMOS scoring stays high which reflects the priority placed on maintaining Kokoro's voice quality.
Inno can match about a third of the way to most identities, and avoids copying recording artifacts etc.
Apache-2.0. The speaker encoder is CC BY-SA 3.0.
Full model card available on HuggingFace: remsky/kokoro-inno-clone-tuner.
Only clone voices you have permission to clone, even the shallow cloning provided by this model and technique.