Microsoft has unveiled Muse, an artificial intelligence model designed to generate both game visuals and gameplay actions. Developed in collaboration with Ninja Theory, a subsidiary of Xbox Game Studios, Muse aims to assist game development by predicting game dynamics based on human input.

According to an official blog by Katja Hofmann, Senior Principal Research Manager at Microsoft Research, Muse is built on the World and Human Action Model (WHAM) and has been trained on over one billion images and controller actions from the Xbox game Bleeding Edge, equivalent to seven years of recorded gameplay.

The model processes an initial one-second prompt to generate subsequent frames and actions, enabling AI-driven gameplay sequences.

Microsoft has already begun using Muse to develop real-time AI models for its first-party games. The research behind the model, published in Nature, details its training process and capabilities.

Microsoft has released Muse’s model weights, sample data, and a prototype interface through Azure AI Foundry to encourage further research and experimentation.

Microsoft and Ninja Theory have previously collaborated on AI research, with Bleeding Edge serving as a platform for AI-driven gameplay experiments. Muse extends these efforts by applying transformer-based models to human gameplay data, reinforcing Microsoft’s investment in AI-driven game development.