WORLD UNDERSTANDING × EXECUTABLE REALIZATION

ExeWorldBench

Benchmarking Code-Based Executable 3D World Modeling

We do not benchmark whether models can reproduce what a world looks like,
but whether they can construct an executable world that works.

↗ Read the paper 🤗 Explore the dataset

ExeWorldBench overview: world understanding and executable code-based realization of 3D worlds

01 — THE IDEA

Executable World Modeling
- World Understanding
- Executable Realization

Visual-to-code reproduces what a world looks like.
Executable world modeling captures how a world actually works.

WORLD UNDERSTANDING

Recover the underlying mechanism

Reason about how components are organized, how they interact, and how effects propagate.

EXECUTABLE REALIZATION

Make behavior emerge from construction

Operationalize that understanding through the structure and interactions of a code-based world.

02 — THE BENCHMARK

Three tasks.
Increasing complexity and autonomy.

From recovering individual mechanisms to composing causal systems and designing worlds from intent.

T1 RECOVER

Mechanism Executability

Infer a mechanical mechanism from a reference image and realize its complete motion in Blender.

Recover component relationships and constraints
Infer motion from visible structural evidence
Evaluate visual structure and mechanical motion

Four chronological Blender frames of a mechanical system from T1 — Generated sequence · GPT-6 Astra (Codex CLI) · Manuscript example

T2 COMPOSE

Compositional Executability

Build multi-stage mechanical systems whose execution depends on coordinated spatial and causal relationships.

Connect mechanisms through causal handoffs
Resolve positions, orientations, and distances
Assess execution across the complete chain

Chronological frames of a multi-stage system from T2 — Qualitative sequence from the manuscript · Multi-stage mechanical system

T3 SYNTHESIZE

Intent-Driven Executable World Synthesis

Construct an executable world from high-level textual requirements, without a direct visual reference.

Component-ConstrainedGoal-Constrained

Organize specified components, or determine the mechanical realization from goals, rules, and state definitions.

Four chronological frames of component-constrained world synthesis — Generated sequence · Component-Constrained Synthesis · Manuscript example

03 — EVALUATION

Inspect the world.
Trace its execution.

Visual Structure

Geometry, spatial arrangement, mechanical assembly, and inter-component relationships.

T1 · T2 · T3 / 0–10

Mechanical Motion

Motion type, direction, range, and temporal evolution of individual components.

T1 / 0–10

Executable Causal Mechanism

Component behavior and the correctness of each stage-to-stage causal handoff.

T2 · T3 / 0–10

Goal Satisfaction

Strict judgments of initial state, intermediate behavior, and final state requirements.

T3-2 / Binary per requirement

04 — FINDINGS

Runnable code is a beginning.
Coherent worlds remain challenging.

Show results after constrained repair

Executability and goal satisfaction are percentages; Visual Structure, Motion, and ECM are scored out of 10. Harness-based settings are identified in model names. See the paper for protocol details.

Execution success ≠ physical coherence

Repair improves code executability substantially more than the corresponding world modeling metrics.

Single mechanisms do not guarantee causal chains

Strong T1 motion performance does not necessarily transfer to multi-stage causal execution.

An initial configuration is only the start

Models often satisfy initial-state requirements more reliably than intermediate behavior and final-state requirements.

05 — DATA CONSTRUCTION

Verified mechanisms.
Progressively open-ended worlds.

The paper's data construction pipeline for T1, T2, and T3 — Data construction pipeline from the manuscript. Select the figure to view it at full resolution.