Model Overview


The architecture is the same as that in the paper.

LibriSpeech test Samples

Speaker Prompt Text Generated Audio
Life was like a box of chocolates, you never know what you're gonna get.
In 2014, P&G recorded $83.1 billion in sales. On August 1, 2014, P&G announced it was streamlining the company, dropping and selling off around 100 brands from its product portfolio in order to focus on the remaining 65 brands, which produced 95% of the company's profits.
Mechanics is a branch of physics that deals with the behavior of physical bodies under the influence of various forces. The study of mechanics is important in understanding the behavior of machines, the motion of objects, and the principles of engineering. Mechanics has been an essential part of physics since ancient times and has continued to evolve with advancements in science and technology. This paper will discuss the principles of mechanics, the laws of motion, and the applications of mechanics in engineering and technology.
Life was like a box of chocolates, you never know what you're gonna get.
In 2014, P&G recorded $83.1 billion in sales. On August 1, 2014, P&G announced it was streamlining the company, dropping and selling off around 100 brands from its product portfolio in order to focus on the remaining 65 brands, which produced 95% of the company's profits.
Mechanics is a branch of physics that deals with the behavior of physical bodies under the influence of various forces. The study of mechanics is important in understanding the behavior of machines, the motion of objects, and the principles of engineering. Mechanics has been an essential part of physics since ancient times and has continued to evolve with advancements in science and technology. This paper will discuss the principles of mechanics, the laws of motion, and the applications of mechanics in engineering and technology.
Life was like a box of chocolates, you never know what you're gonna get.
In 2014, P&G recorded $83.1 billion in sales. On August 1, 2014, P&G announced it was streamlining the company, dropping and selling off around 100 brands from its product portfolio in order to focus on the remaining 65 brands, which produced 95% of the company's profits.
Mechanics is a branch of physics that deals with the behavior of physical bodies under the influence of various forces. The study of mechanics is important in understanding the behavior of machines, the motion of objects, and the principles of engineering. Mechanics has been an essential part of physics since ancient times and has continued to evolve with advancements in science and technology. This paper will discuss the principles of mechanics, the laws of motion, and the applications of mechanics in engineering and technology.
Life was like a box of chocolates, you never know what you're gonna get.
In 2014, P&G recorded $83.1 billion in sales. On August 1, 2014, P&G announced it was streamlining the company, dropping and selling off around 100 brands from its product portfolio in order to focus on the remaining 65 brands, which produced 95% of the company's profits.
Mechanics is a branch of physics that deals with the behavior of physical bodies under the influence of various forces. The study of mechanics is important in understanding the behavior of machines, the motion of objects, and the principles of engineering. Mechanics has been an essential part of physics since ancient times and has continued to evolve with advancements in science and technology. This paper will discuss the principles of mechanics, the laws of motion, and the applications of mechanics in engineering and technology.

Model Overview


Inspired by the architecture of Real-Time Voice Cloning, we add an emotion encoder to learn the emotion representation of the speaker prompt. The output of the emotion encoder is an emotion embedding. Then we concatenate the speaker embedding and the emotion embedding and reduce the dimension of the joint embedding. The final joint embedding is then plugged into the Tacotron2 model.
Since, we want to take advantage of the pretrained parameters of the real-time voice cloning checkpoints, we load the pretrained model, initialize the parameters of the fully connected layer added to the synthesizer, and continue training the Tacotron2 model with the joint speech embedding input rather than the speaker embedding input.

Emotion Speech Maintenance

Text Emotion Speaker Prompt Generated Audio
We have to reduce the number of plastic bags. Afraid
Angry
Disgusted
Happy
Surprise