A workflow Make Video Character saying anything. By Cloning the voice, generate TTS, make video faces talk (lip sync)

flow1.1: Audio Generator (single language)

flow1.2: Audio Generator (cross language)

flow2: MuseTalk (input generated audio file path and the raw video)

MuseTalk's software env is a little bit tricky. recommand env here with cuda11.8:

torch==2.0.1
torchvision==0.15.2
torchaudio==2.0.2

pip install --no-cache-dir -U openmim
mim install mmengine
mim install "mmcv==2.0.1"
mim install "mmdet==3.2.0"
mim install "mmpose==1.1.0"