It’s 2016, and you’re wandering around the neighborhood at dusk. On the prowl, you’re swiping on a smartphone to catch that elusive Snorlax and grow your collection in Pokémon Go. You, alongside hundreds of millions of other players, are enjoying a mixed-reality game built around lovable characters. What you don’t realize is that these scans of your environment will one day help robots navigate obstacles and interact with objects.
Niantic, the creator of Pokémon Go, sold the game last year and pivoted hard into one of Silicon Valley’s buzziest AI topics right now: world models. “We’re very much about reconstructing the real world,” says Brian McClendon, Niantic Spatial’s chief technology officer. He hopes to leverage its trove of world data to build replicas. Unlike the text-based nature of large language models, world models use visual inputs and can generate digital environments that simulate physics.
This software is poised to seep into everyday life, without you necessarily ever interacting directly with world models. 2027 could be a major year for the expansion of home robotics and driverless cars, both of which are bolstered by advances in world models. It’s time to better understand how these bots think.
“So with a language model, you have a sentence, and it will predict the next word in the sentence,” says Oliver Cameron, the CEO of Odyssey, an AI lab focused on world models. “With a world model, you give it a visual state of the world, an image, and it will then predict the next visual state of the world.” These 3D generations can be used as digital training grounds. “They’re simulations, but they look and feel real,” he says.
McClendon sees these world models as helping robots practice without physical movement or being in intended environments. He says someone could take photos of where the robot is intended to function, generate a 3D model, and then let the software run “millions of simulations” to train the robot on the space’s parameters.
It’s not just humanoid robots that could use simulated practice; driverless cars also improve using world models. Without a human behind the wheel, split-second software reactions are paramount.
“Importantly, a world model must be able to react to the actions that we take in the world, and so it must understand physics, cause and effect, and how other objects and people change their actions in response to our own,” says Jamie Shotton, chief scientist at Wayve, which builds world models designed for cars.
Wayve’s generative world model is powered, in part, by dashcam footage. The startup uses it to recreate hazardous roads and other unsafe conditions that would not be practical to test in real life.
Beyond physical applications, developers are also analyzing how world models could impact video games. Since physics and 3D rendering are core to video games, major players like Tencent are investing in their own world models.
You can expect large AI companies, like Google, to keep boasting about their latest world-model improvements. Investors are also throwing money at insurgent big names in computer science that are turning their attention to the technology. Yann LeCun, Meta’s former chief AI scientist, raised over a billion dollars recently to fund his Advanced Machines Intelligence startup. Additionally, famed Stanford computer scientist Fei-Fei Li launched a startup in 2024 called World Labs, which AMD acquired this year for over $8 billion.
Even if you plan on avoiding autonomous cars and domestic bots altogether, world models are already warping the labor market behind the scenes. To simulate our physical world, AI labs need real people to record it. Earlier this year, I strapped an iPhone to my forehead and recorded hours of videos in my apartment. I recorded my hands folding laundry, washing dishes, and pouring Diet Coke, selling the video data through an app for a fairly measly payout. As more companies attempt to build better world models, even more gig work may demand this kind of data collection.
Between chasing virtual monsters in 2016 and selling videos of my chores in 2026, these systems are ingesting more and more of my physical life. Now I can’t stop picturing the digital ghost of some humanoid robot trapped inside a simulation of my bedroom, endlessly folding my boxers with its metallic claws—getting slightly better with every pass. I’m sorry I didn’t tidy up first.