Project Intro

At the AZ conference we had a booth to collect My Stories. However, this room was a very busy and noisy room. It had the merch perch, book sales, and forums booths as well.

We did the best we could with a directional mic, but still a lot of background is picked up.

The request is to find and utilize tools and techniques that would result in a clean audio track.

One approach is to find tools that greatly reduce background noise and enhance the primary voice. I’m personally skeptical of how far we can take those tools.

The alternative idea is similar to what was used for the Javier Milei to translate is epic WEF speech to English while maintaining his voice, accent, mannerisms, etc.

https://twitter.com/aphysicist/status/1747868626948907325

We can use AI tools to extract a good transcript. Train another AI on their voice with what good clips we can find in the audio track. Then dub over based on the transcript with their trained voice. A completely clean audio track.

Replied in this thread, we will provide samples for experimentation.

Example of clip with okay sound:

“…in short, a successful entrepreneur is a hero…”

:heart_on_fire:

Incredible, okay… when you told me about this before I thought, “I’m sure it’s alright.”

…it’s virtually flawless

I am inspired but also terrified!!!

Hello and have to say I’m missing you guys, not a lot, but enough.

I’ve just listened to both recordings and the original audio is useable, but of course I was hoping for all my work to already have been done, so I could sit back and take all the credit.

May I ask the status of the deep fakes please. What can I do?

Thanks

Greg

We had a flood of post-conference stuff that ate up the week, but I reminded folk of this priority and are working on it today.

I’m about to upload m4v and aac/mp3 versions to make the work easier for others.

@Kyle @Ryan

Attached are the mp3 versions of the audio so folk can start playing without trying to convert/mux the AVI files.

Possible tools to use for the cloning. @coryreeve @SteveHicks if you have others, especially OSS, please reply here and recommend them.

Here’s an attempt at some cleanup on the original audio so folks can run it through any AI assistance or AI voiceover.

@coryreeve @SteveHicks @Gregton @Kyle Here are the subs, generated by WhisperAI. Definitely some mistakes, so we’ll need to fix the transcript up a bit.

You can download the transcripts here:

https://mega.nz/file/pAIVyTpI#-mVcr5jFqAEiujc5jOe9Cd-d_ekCGp8KsuyHML_4FTQ

You can download the video from CloudFlare, but here is the mp4 direct download as well. It has the original video, but the enhanced audio track.

https://mega.nz/file/UVgHgZQI#ZCvepBQUZliSx9ezFyV79EYI1UZu1M2gxmZt4xnRHWE

What I think needs to happen next:

  • Cut video to more appropriate length. Closer to a final cut overall.
  • Remove Greg audio entirely, maybe subtitle him if necessary
  • Train AI on her audio
  • Re-dub from trained audio clone

Signed up for Resemble AI as it said pricing was on demand. But you have to do the $100/mo to unlock custom voices. Going to try something else.

@Kyle @coryreeve @SteveHicks @Gregton This needs timing and other adjustments. But it’s pretty clean and sounds like her.

Hey @flccc-eric For some reason, I can’t play or download these here