"We're in the middle of an industrial revolution for robotics," says Ge Yang, a postdoc at MIT's Computer Science and Artificial Intelligence Laboratory, who worked on the project. "This is our attempt at understanding the impact of these [generative AI] models outside of their original intended purposes, with the hope that it will lead us to the next generation of tools and models."
Robots trained using this method achieved a higher success rate in real-world tests than those trained using more traditional techniques. Researchers used the system, called LucidSim, to train a robot dog in parkour, getting it to scramble over a box and climb stairs even though it had never seen any real-world data.
Digital simulations are a rapid, scalable way to teach [robots] to do new things, but the robots often fail when they're pulled out of virtual worlds. This approach demonstrates how helpful generative AI could be when it comes to teaching robots to do challenging tasks.
LucidSim uses a combination of generative AI models to create the visual training data. First the researchers generated thousands of prompts for ChatGPT, getting it to create descriptions of a range of environments that represent the conditions the robot would encounter in the real world.
Collection
[
|
...
]