MindTopo Highlights Geometric Failures in Vision Models
Vision models struggle with fundamental topological reasoning, preferring visually plausible patterns over strict geometric consistency.
The MindTopo benchmark evaluates how Multimodal Large Language Models and video generative models perform topological reasoning. This involves identifying spatial properties like continuity, separation, and order—relationships that remain constant even as shapes undergo deformation. The study examines 14 MLLMs and 3 video generative models across 11,030 instances of 13 procedurally generated task types.
The Disconnect Between Vision and Topology
The benchmark demonstrates that while models can recognize individual objects or textures, they do not preserve topological invariants. A model might successfully identify the components of an image, yet when asked to predict the state of a path or an enclosure, it fails to maintain the correct connectivity. For example, a model might correctly render a hole in a shape but fail to understand that a continuous line cannot pass through that hole without intersecting the boundary. This suggests the models rely on local visual cues rather than a global internal representation of space. The failure manifests even in simple tasks where the topological rule is fixed, such as determining whether two points are contained within the same enclosed region.
| Evaluation Mode | Task Complexity | Human Performance | Model Performance |
|---|---|---|---|
| Reasoning | Identification/Inference | High | Low/Moderate |
| Planning | Closed-loop Agent | High | Very Low |
Persistence and the Planning Gap
Planning requires maintaining state across sequential transitions, such as navigating an agent through an environment where obstacles must remain persistent. MindTopo shows that models struggle significantly with these temporal dependencies. When generating a series of frames, these systems often fail to keep spatial structures consistent from one step to the next. The system acts as if it is generating independent visual snapshots rather than updating a single, persistent world state. If a barrier is identified in the first frame of a sequence, the model frequently fails to honor that barrier as it predicts subsequent frames.
To understand this through a concrete example: imagine an agent asked to draw a closed loop around a point. The model might start correctly, but as it continues the path, it may overlap its own line or leave a gap, essentially 'forgetting' the spatial constraints it began with. The model does not treat the path as a persistent geometric object with a start and end point. Instead, it treats the next step as a probabilistic completion based on the current pixels, often resulting in a visually plausible but topologically incorrect endpoint.
Limitations in Model Improvement
Researchers tested whether fine-tuning could bridge this gap using the Qwen3-VL-2B-Instruct model, but improvements in planning were minimal. The core issue is that models reach plausible endpoints while failing to preserve topology across the intermediate transitions. For anyone building agentic workflows for robotics or simulation, the takeaway is that MLLMs cannot be relied upon to understand spatial constraints alone. Integrating these systems into physical environments requires offloading spatial constraints to a dedicated symbolic engine. These tools provide a hard state tracker—such as a vector map—that enforces geometric rules like non-collision. You should limit the model's role to high-level semantic labels, while the symbolic engine manages the geometry. The current gap between visual proficiency and spatial logic is not easily bridged by simple scaling, and we do not yet know if architectural changes or synthetic data alone can force these models to learn the underlying topological rules of a physical space.