Zero-shot audio-driven digital human
Style-Talking
Style-Talking clones speaking style from a reference video, predicts LivePortrait expression motion from target audio, and renders a talking video with an optional RefGAN renderer while preserving the original non-face regions.
Speaking-style prompt
wav2vec audio features
RefGAN renderer
Video-driven identity
Trump / English style-clone result, driven by cloned English speech and rendered with RefGAN.
Pipeline
Audio to expression to video
source video
wav2vec
style-prompted expression
original or RefGAN renderer
face blend
Multilingual style clone
One reference, four target languages
English
Japanese
Korean
Cross-identity samples
Source video, original decoder, and RefGAN result
Michelle Yeoh / English source
Style-Talking
Style-Talking RefGAN
Zendaya / English source
Style-Talking
Style-Talking RefGAN
Liu Yifei / Chinese source
Style-Talking
Style-Talking RefGAN
Morgan Freeman / English source
Style-Talking
Style-Talking RefGAN
Jake Gyllenhaal / Japanese source
Style-Talking
Style-Talking RefGAN
Timothee Chalamet / Korean source
Style-Talking
Style-Talking RefGAN
Renderer comparison
Original LivePortrait decoder vs RefGAN renderer
The same Audio2Motion output and target audio are used in each pair; only the rendering path changes.
Trump / Chinese original decoder
Trump / Chinese RefGAN renderer
Trump / English original decoder
Trump / English RefGAN renderer
Trump / Japanese original decoder
Trump / Japanese RefGAN renderer
Trump / Korean original decoder
Trump / Korean RefGAN renderer
Michelle Yeoh / English original decoder
Michelle Yeoh / English RefGAN renderer
Zendaya / English original decoder
Zendaya / English RefGAN renderer
Liu Yifei / Chinese original decoder
Liu Yifei / Chinese RefGAN renderer
Morgan Freeman / English original decoder
Morgan Freeman / English RefGAN renderer
Jake Gyllenhaal / Japanese original decoder
Jake Gyllenhaal / Japanese RefGAN renderer
Timothee Chalamet / Korean original decoder
Timothee Chalamet / Korean RefGAN renderer
Method comparison