Cyberpunk 2077 is less than a month away from release and it’s been revealed that it’s utilizing AI to match lip-sync animations to 10 different languages. The game is set to support a total of 18 different languages at launch, with only the aforementioned 10 being fully voiced. For the rest, players will be able to activate the game’s subtitles, which can also be customized by players to their preference.
In other games, facial and speech animation is usually achieved with actors and motion capture rigs, and while that approach leads to more detailed and expressive animations, it usually limits the animations to a single language, which was the case with Ghost of Tsushima. While the game supported both English and Japanese dubbed dialogue, the animations were only available for the English language option, so dialogue in the Japanese dub looked unnatural with the lack of synced animations.
Click the button below to start this article in quick view.
For Cyberpunk 2077, CD Projekt Red sought the help of Jali Research’s technology to achieve the game’s AI-driven facial animations and multilingual dialogue. In a recent presentation by Jali Research, co-founder Chris Landreth explains how the company was able to achieve the AI-driven dialogue animation system for Cyberpunk 2077, which includes an acoustic model, a pronouncing dictionary, and a G2P (grapheme-to-phoneme) model for each of its 10 supported dialogue languages, which currently include English, French, German, Japanese, Brazilian Portuguese, Polish, Russian, Mandarin, Italian, and Spanish.
The presentation shows off how the system not only constructs the character’s lip-syncing animations but the rest of the facial animations as well depending on the specific language and how inflective the language is. Lastly, Jali also shows off how the technology uses transcript tagging to animate the dialogue for characters that are bilingual and switch between multiple languages in a single sentence. It’s not clear how much of the game’s dialogue will be voiced, but the amount of dialogue in Cyberpunk 2077 seems to be incredibly massive.
The presentation shows just how much work and effort CD Projekt RED is putting in to make Cyberpunk 2077 an incredibly impressive game, which unfortunately led the company to implement some last-minute crunch. Hopefully, all the time and work that the developers and its partners, like Jali Research, have invested in the game pays off and it becomes even more successful than the studio’s previous hit, The Witcher 3.
Cyberpunk 2077 is scheduled for release on November 19 for the PlayStation 5, Xbox Series X/S, PS4, Xbox One, PC, and Google Stadia.
Source: Jali Research/YouTube