InteractiveWorld: A verifiable environment for human-centric MLLM assistants
In brief
- InteractiveWorld simulates an MLLM assistant guiding an autonomous human agent in real-time within a 3D environment.
- InteractiveBench offers 120 dynamic tasks to assess assistants under real-world conditions.
- The 12 tested models, including the best (Claude Fable 5.1), achieve less than 25% success on complex tasks.
Researchers introduce InteractiveWorld, a novel simulation environment to evaluate multimodal large language model (MLLM) assistants in closed-loop interaction with a human. Unlike existing robot-centric simulators, this platform positions the assistant as a guide while a human agent acts autonomously in a 3D environment. It comes with InteractiveBench, a dynamic benchmark of 120 tasks across six categories, where each assistant instruction shapes the final outcome.
Tests on 12 cutting-edge MLLM models show limited performance: the top model, Claude Fable 5.1, reaches at most 20% and 25% success rates on long-horizon Guidance and Planning tasks. The authors highlight the gap between current MLLM capabilities and the requirements for reliable assistance in the physical world.
This summary gives you the essentials. The full article, its figures and quotes are on Hugging Face Daily Papers.
Read the original articleAI-generated summary of the source, checked by rules. How we write · Report an error