Lipsync
Lipsync is the match between visible mouth movement and heard audio. It is the most sensitive point of any production with speaking people, because a deviation of a few hundredths of a second registers without anyone being able to say why.
Three cases that get confused
- Generated speech: a produced figure speaks a text. Short lines work well, long passages in close-up stay demanding.
- Dubbing: a filmed recording gets a new audio track in the same language. The classic from cinema, routine today.
- Translation with mouth adaptation: a real person suddenly speaks English with matching lips. That alters the image of a real person and requires their consent, see deepfake.
Where it fails today
The longer a shot runs and the closer the camera sits to the face, the more visible deviations become. Strong emotion, fast speech and sounds with a distinct mouth shape are also difficult. One sentence almost always works, a monologue over thirty seconds rarely does.
Our recommendation in those cases: film the speaking passages and build the world around them generatively. That is hybrid production, handled with us by Dean Rocinante.
Frequently asked questions
What does lipsync mean?
Lipsync describes the match between mouth movement and sound. In AI production the term usually means that a generated or subsequently translated speech track matches the visible lip movements.
Does lipsync work in several languages?
Yes, technically well. With an AI avatar it is unproblematic, with a real person it requires consent and, depending on the case, a label. For training in several languages it is the fastest route.
How long can a speaking shot be?
Generatively we recommend up to about ten seconds at a time, then cut or move to other images. That is the better way to tell it anyway, because a talking head over thirty seconds rarely holds.
Related terms
Read on: All terms in the glossary • AI avatar
Someone has to speak in your film?
We tell you whether that holds generatively or whether half a shoot day is the better route.
Book a first call