Ubisoft made a prototype game with voice-controlled AI teammates

"Ubisoft has announced a new game prototype featuring voice-controlled AI teammates that understand visual context and natural language. This "Teammates" project builds on the Neo NPCs Ubisoft showed off with Nvidia in 2024 to demo in-game AI that can naturally respond to players. A key difference this time, besides the complexity of the interactions the prototype supports, is that Teammates is already being played in a closed playtest with "a few hundred players," Ubisoft says."

"Teammates, even if Ubisoft describes it as a playable "experimental research project," still uses the basic concepts of a first-person shooter. The prototype casts players as "a member of the resistance in a dystopian future, tasked with moving through an enemy base to locate five missing members of their team," where directing in-game AI characters is key to success. Ubisoft came up with three AI NPCs for the project,"

"Based on footage shared with Engadget, Ubisoft's AI characters not only understand voice commands, but also have a visual awareness of what the player is seeing. A direction to "stand behind a barrel" prompted Sofia to take into consideration where the player was looking and position itself appropriately. In the version of Teammates available in the closed playtest, Ubisoft also uses Jaspar to onboard and teach players about the basics of the game."

Ubisoft's Teammates prototype features voice-controlled AI teammates with visual context awareness and natural-language understanding, playable in a closed playtest with a few hundred players. The prototype uses three AI NPCs: Jaspar as an onboard assistant aware of lore and able to adjust settings, and Pablo and Sofia as in-world robotic companions responsive to commands. AI characters interpret both voice and visual cues, demonstrated when Sofia positioned itself "behind a barrel" based on player view. The project builds on Neo NPC work with Nvidia and uses AI middleware intended for incorporation into other engines and games. Personality presets and verbosity controls are under experimentation.

Read at Engadget