This post is also available in:
עברית (Hebrew)
Artificial intelligence has long excelled at simple games and simulations, but most systems still depend on millions of trial-and-error interactions to learn. That approach, while effective in virtual settings, is impractical for real-world robots that can wear out or break during training. Researchers at Google DeepMind have now introduced Dreamer 4, an artificial agent that sidesteps this limitation by learning inside a world model; a virtual simulation that captures both visuals and physical dynamics.
Dreamer 4 is the first AI system to complete one of Minecraft’s most complex objectives – obtaining diamonds – without ever practicing in the real game. Instead, it trained entirely on offline gameplay videos. The agent learned to imagine how the Minecraft world behaves, from cutting trees to crafting tools, and used that internal model to plan actions in sequence, much like a human would reason before acting.
According to TechXplore, the model relies on a transformer-based architecture designed to predict future frames, actions, and rewards. Through a method called shortcut forcing, the system speeds up video generation and reinforcement learning by over 25 times compared to conventional video-based models. Once trained, Dreamer 4 could accurately simulate interactions such as mining, crafting, or using in-game objects, all in real time on a single GPU.
Unlike text-to-video generators such as Sora or Veo, Dreamer 4’s world model is interactive. Meaning, it allows agents to explore, make decisions, and learn within simulated environments. This capability could be particularly valuable for robotics, where real-world training is time-consuming and costly. By learning from limited action data and large collections of passive video, agents can generalize from observation rather than endless physical trials.
The research suggests a path toward scalable, imagination-based learning, where intelligent agents refine their skills in realistic virtual worlds before being deployed in physical ones. Future versions of Dreamer aim to incorporate long-term memory and language understanding, enabling systems that not only simulate reality but also collaborate with humans across a wide range of tasks.
The research was published here.
