Cyberpunk 2077 will be the first real-world application of a brand new machine-learning software that provides the complete automation of lifelike facial animations and high-end lip syncs.
Speaking with PC Gamer magazine for the latest issue, associate game director and animation head Sebastian Kalemba revealed that Cyberpunk 2077 uses JALI software to create realistic human expressions for every character model from in-game cutscenes to someone randomly walking on the street.
“Doing facial motion capture for every NPC in the game is impossible,” explained Kalemba. “You have to remember that we are talking about recording massive amounts of lines with numerous actors across different localizations, cooperating with studios all over the world for sessions, and doing VO pickups all the way until the very last moments sometimes.”
JALI allowed developer CD Projekt Red to overcome all of those difficulties and challenges, while being able to create something on a massive scale that would not be simply possible with regular motion capture.
The interesting thing to note about JALI is that the machine-learning software gives animators complete control to fine-tune the automated results, which further allows for even better animations. Cyberpunk 2077 hence has lifelike facial animations as well as reactions that go deeper than just the (automated) surface. Kalemba confirmed that every animation sequence in the game has the eyes, brows, neck movements, jaw, and lips working in tandem—all for the sake of having players “sense emotional stimuli even though we do not see it.”
JALI was first announced in 2016 and one of its co-founders Pif Edwards notes that “bad lip-sync gets noticed and great lip-sync gets ignored.” Cyberpunk 2077, while seeing CD Projekt Red put in an insane amount of work, promises such human-like responses and expressions from characters that players might probably end up completely ignoring them. That is exactly the level of immersion CD Projekt Red has been gunning for since the start.
Cyberpunk 2077 releases for PlayStation 4, Xbox One, and PC on November 19, 2020. The game will be playable on PlayStation 5 and Xbox Series X as well but through backwards compatibility. The real next-generation enhancements are slated to arrive later.