Voicebox

What is Voicebox?

Voicebox is a cutting-edge technology that excels in synthesizing speech across six languages, removing transient noise, editing content, transferring audio styles within and across languages, and producing diverse speech samples. It stands out by generating speech up to 20 times faster than state-of-the-art auto-regressive models. Specifically, Voicebox is a non-autoregressive flow-matching model designed to fill in speech based on audio context and text inputs. The model has been rigorously trained on a significant amount of data, including 60,000 hours for the English version and 50,000 hours for a multilingual version covering languages such as English, French, German, Spanish, Polish, and Portuguese. Voicebox's adaptability and efficiency make it a versatile tool for various speech synthesis and editing tasks .

Who created Voicebox?

Voicebox was created as a state-of-the-art speech generative model by Meta. The model is a non-autoregressive flow matching model that excels in text-guided speech infilling tasks. It outperforms single-purpose AI models across various speech tasks through in-context learning. Voicebox is capable of synthesizing speech across six languages, removing transient noise, editing content, transferring audio style within and across languages, and generating diverse speech samples. Additionally, it can generate speech up to 20 times faster than state-of-the-art auto-regressive models.

How to use Voicebox?

To use Voicebox, follow these steps:

1. Model Overview: Voicebox is a non-autoregressive flow-matching model designed to infill speech based on audio context and text.

2. Capabilities: Voicebox is a state-of-the-art speech generative model that can synthesize speech across six languages, remove transient noise, edit content, transfer audio style within and across languages, and generate diverse speech samples. It outperforms single-purpose AI models in speech tasks through in-context learning and can generate speech up to 20 times faster than traditional auto-regressive models.

3. Training: Voicebox is trained to solve a text-guided speech infilling task using a large-scale dataset, enabling it to excel in various speech-related tasks.

By following these steps, users can effectively utilize Voicebox for high-quality, multilingual speech generation and manipulation tasks.