AI Lip Sync Notes

2023-08-22 Research to see if there is an Open source Alternative to commercial Lip Sync / AI Avatar Services

Code Walkthrough:

• https://becominghuman.ai/deepfake-audio-with-wav2lip-263f0f0e84bc

Building your own:

• https://stackoverflow.com/questions/76284570/converting-real-time-audio-to-phonemes

The one everyone references that started it all: https://github.com/Rudrabha/Wav2Lip Research/Personal Use Only and low quality lip sync.

Spin Offs with Wav2lip codebase. So no commercial use allowed: https://github.com/indianajson/wav2lip-HD https://github.com/OpenTalker/SadTalker

• https://huggingface.co/spaces/vinthony/SadTalker

Most interesting one, but, published by ByteDance Team https://github.com/yerfor/GeneFace

Uses Wav2Lip as an option.. so maybe ok? https://github.com/numz/sd-wav2lip-uhq

Not sure if they use the Wav2Lip codebase. https://github.com/sahilg06/EmoGen

Other:

• https://colab.research.google.com/github/indianajson/wav2lip-HD/blob/main/colab.ipynb

Commerical Websites:

• https://www.animaze.us/allproducts
• https://gooey.ai/lipsync-maker/
• https://labs.everypixel.com/lipsync
• https://www.heygen.com/pricing