#013 - Redub Film With AI: It's Karaoke Night at the Movies

A review of the AI audio dubbing app, Vozo

I promise I don't have an obsession with AI-generated mouths. I couldn't resist using it for the cover image. It was an unexpected prompt result I found fitting for today's featured app for audio dubbing.

THE PITCH

A screenshot of Vozo's website homepage.

I wonder if this app will introduce any copyright issues with dubbing movies.

With Vozo, you can rewrite, redub, and lip-sync your videos into new stories with prompts.

THE EXPERIENCE

I like AI apps that focus on one thing. That is because they aren't bloated with features, and navigating them is simple. Vozo is no different. When starting, you have three options: rewrite and redub, translate and redub, and repurpose. Each one takes you to a simple interface that transcribes the video's dialog to your left, showcasing a video preview on the right and an editable timeline toward the bottom of the screen.

Rewrite and redub let you rewrite the dialog and add it as a new dub to the video, which I tried first. In my test run, I used two movie clips: one from a favorite cheesy horror film, Tales From The Crypt: Demon Knight, and the other from the recent The Stand remake that ran on Paramount+. Starting with Demon Knight, the uploading and transcribing went fast with my 14-second clip. In it, I altered the intro dialog of the cryptkeeper character. Upon initial playback, the dub was on the high pitch and speed side of things, so I tweaked those settings to align it with the original audio track. Similar to other text-to-speech apps, you can edit each sentence individually.

Once I finished dubbing, I switched to "Lip-sync" to match the new dialog to the character's lips. This is where things did not work; due to the character's zombie-like appearance, the AI model could not detect a human, so it could not lip-sync. Where I had success was doing the same process but for my 32-second clip of The Stand. Since that has humans, the lip-syncing feature worked much better. I altered the dialog to include talking about dancing, which turned out to be just as silly as I thought. As before, after changing the dialog, making a few tweaks brought it close to the original audio to a solid effect. You can view my dubbed video results below.

@thegoodbadugly.ai

Tales From The Crypt: Demon Knight redubbed with Vozo.#AI #artificialintelligence #horrorfilm

@thegoodbadugly.ai

The Stand. Redubbed using Vozo.

Translating the video to Spanish worked smoothly. Each sentence is broken up and editable like before. Still, it also displays the translated version, letting you switch between the two as you listen to the audio playback. You can even adjust the timing of the translation. Finally, with repurpose, you're simply trimming your dubs and exporting them into formats for social media platforms. It has a handy feature of rating the "viral-ness" of the dialog in the video clip.

THE VERDICT

Vozo does one thing well. From an intuitive interface to solid text-to-speech conversion, I rank this one at the top of similar AI text-to-speech products like Play AI.

Besides the novelty of redubbing video clips for memes, translating videos can be more valuable. I honestly see this as the most long-tail use case for the product. That said, Vozo still has room to grow and could use additional features like different gender or ethnic voice options or include a sound effects library.

Good to Know 👍️ 

  • Free trial

  • Includes AI assistant that can rewrite a script

  • Adjust voice tone by adding text capitalizations, exclamation marks, etc.

  • Transcribing directly from YouTube links did not work for me

  • Lip-sync limited to a few minutes

Product Website 👇️ 

THE END

I'm Kevin, a former marketing technology director on a mission to provide brief yet informative reviews that respect your time. Please subscribe or share if you like. Thank you for reading.

Was this newsletter useful? Help me improve it! Click below to vote.