Google's Genie world model can now simulate real streets with Street View | TechCrunch

Street View is being integrated with DeepMind’s Project Genie to create immersive, interactive simulations of real streets and their surroundings. The capability supports changing conditions such as weather and viewing alternate scenarios, including future or extreme-weather possibilities. The integration is positioned for both robotics and human exploration. For robotics, simulated environments can model rare lighting events, such as sun glints in places with limited sunlight, helping prevent unexpected visual conditions from affecting new robots. For humans, users can simulate traveling to a city at a chosen time of year, such as seeing a specific block covered in snow. Google’s Street View data collection spans 20 years, producing hundreds of billions of images across many countries and continents, enabling realistic world simulation.

"Imagine being able to do that, but in a more immersive, interactive way that allows you to really simulate the street and its environs, and even do things like adjust the weather or see what it would look like in a "Day After Tomorrow" scenario. That's one of the goals of Google's latest integration. Starting today, Google DeepMind is connecting Street View to Project Genie, the company's general-purpose world model that can generate diverse, interactive environments."

""It's really powerful for both the agent [and robotics] use case and for humans to play with, and that's always been the thesis of Genie," Jack Parker-Holder, a research scientist on DeepMind's open-endedness team, told TechCrunch. He gave the example of a new robot being deployed in London, which rarely sees the sun. Genie could, Parker-Holder says, simulate those scarce occasions when the sun glints off the Victorian housing, so the rays don't shock the robot when it happens."

""Simultaneously, you might say, 'I'm going to New York City, but not this time of year,'" he continued. "It's going to be snowy. I want to see what that block looks like in the snow.""

""With Street View, we have imagery from a large quantity of the world," Jack said. "You can imagine how potentially powerful it is to combine this rich source of real-world information and data with an ability to simulate worlds.""

Read at TechCrunch