An image becomes a world

WMMLLM

Turning MLLMs into World Models

The first interactive world model built on multimodal large language models: persistent, editable worlds from a single image.

Step 1 / 1

    Lagoon Pier Island · saved panorama states of a prepared demo · nothing is generated while you browse

    Drag to look around

    Baiang Li1,3 Enze Xie2,† Jincheng Yu2 Jin Wang2 Haozhe Liu2 Yitong Li2 Juanxi Tian3 Felix Heide1

    1Princeton University2NVIDIA3OpenEnvision†Project Lead

    Film

    WMMLLM in one minute

    One minute of the worlds on this page. Sound on.

    Worlds

    Explore generated worlds

    Step inside. Look around. Change what happens next.

    Requests

    One world, many requests

    Change the light. Bring in the weather. Walk somewhere new.

    Generated videos of prepared requests. Choose one to try it in the world.

    Watch the film

    Generated videos from prepared requests.

    Results

    Return to the same view

    WMMLLM completes 85.3% of the camera round trips on MBench-A; the six public world models complete 0.4% to 50.9%.

    A round trip is complete when the camera turns at least 15° in the requested direction during the first half and returns within 10° of the start. All methods are scored on the same 224 items with the same MBench-A scorer.

    Last frame, WM^MLLM
    WMMLLM
    Last frame, HY-WorldPlay
    HY-WorldPlay
    Last frame, Infinite-World
    Infinite-World
    Last frame, Matrix-Game 3.0
    Matrix-Game 3.0
    Last frame, Yume-1.5
    Yume-1.5
    Last frame, Matrix-Game 2.0
    Matrix-Game 2.0
    Last frame, LingBot-World
    LingBot-World

    Two MBench-A camera round trips. Every method starts from the same image, turns away and is asked to come back. Switch between the first frame, a frame during the turn away and the last frame of each video.

    Round trips completed

    85.3%

    Six public world models 0.4% to 50.9%

    of 224 MBench-A items

    Epipolar score

    91.5

    Six public world models 0.1 to 74.0

    MBench-A environment recovery, mean score

    Joint success

    77.5%

    Six public world models 0.0% to 32.5%

    camera out and back, starting content shown, usable: a blind judge counts 31 of 40 videos against 0 to 13

    Difference between revisits

    0.037

    Six public world models 0.32 to 0.66

    median LPIPS, lower is closer

    Camera round trips on all 224 MBench-A items. WMMLLM completes 191 round trips and the public models 1 to 114. It turns a median 43° and returns within 0.1° on 189 items; the public models turn a median 82° to 173°.
    The same pose before and after the camera leaves. The median LPIPS between the two views is 0.037 for WMMLLM and 0.32 to 0.66 for the six public world models. Bars mark quartiles and median.
    All numbers

    The table loads with the page data.

    How much of a generated world is usable. Over 36 directions, a blind model judge rates 87.6% of the views of the 219 generated worlds in the paper usable. Most defects lie 30° to 60° from the input view, around its border.

    How it works

    MLLMs act on an explicit world state

    Plan a request, propose a change, check it, and commit the new panorama. The video presents the transition.

    Where the world lives. A video model generates a revisit again from its context, a 3D-asset system renders an explicit scene (schematic), and WMMLLM renders a revisit from its panorama state.

    One request, four steps

    1. Plan

      GPT-6.1-sol reads the request, the current view and an optional box, and returns operations in order. Each operation takes one route: rotate, revisit, local, global or walk.

    2. Propose

      For an edit, GPT-Image proposes the endpoint image at the editing view. Rotations and revisits render the saved state and call no image generator.

    3. Select and check

      The MLLM compares the source with the proposal and marks the region to write: the old and new footprints of the object and effects such as shadows. The proposal is blended inside this region, and the operation is aborted if any panorama pixel outside it changes.

    4. Commit and present

      A verifier compares the candidate with the request. A candidate that fails is retried once; a second failure keeps the current state. After a commit, Sol-H3 generates the transition video from the source and endpoint views. Video frames never enter the state.

    Five routes change the state

    • rotate

      Renders the current revision at a new orientation. No generator.

    • revisit

      Reuses a saved view when its revision, camera and rendering profile match and the hashes agree. No generator.

    • local

      Writes a GPT-Image proposal into the panorama inside the selected region only.

    • global

      A change of lighting, weather or season replaces the panorama and retires earlier patches.

    • walk

      Generates the destination panorama for a new place.

    Full method figure and route table
    WMMLLM overview. Pretrained models read the explicit state, and only commits that pass the checks write it. (a) One MBench-A turn. (b) How each route changes the state.
    The five routes. The part of the visual state each route changes, the image generator it calls and its checks.

    Local edits

    Edits stay local

    A local edit leaves every pixel outside its region unchanged.

    Result with WM^MLLM: a fishing rod, a white float and dragged debris are added; the rest of the riverbed is unchanged.
    Source: a riverbed under clear shallow water with a rusty pipe.
    Source WMMLLM

    An MBench-A request adds a fishing rod that drags debris along a riverbed. Drag the divider to compare the source with the result.

    Direct editing installs the whole GPT-Image proposal, which also adds a rock and shifts the texture of the whole frame. WMMLLM writes the proposal inside the selected region only.

    Pixels changed outside the region

    • Direct editing54%
    • Threshold paste25%
    • WMMLLMnone

    Share of the pixels outside the region that change by more than 16 intensity levels, for this request. Threshold paste copies every pixel whose color change exceeds a threshold.

    In the audit of the benchmark run, all 97 local edits that pass the check leave every panorama pixel outside their region unchanged.

    Region selected after generation

    34.8% to 83.0%

    completion in blind evaluation on 112 local edits, with the region predicted before generation and with the region selected after it; a difference of 48.2 points

    CompBench image editing

    15 of 15 columns

    best value in every metric column against 15 baselines, on 2,709 GPT-Image proposals

    Background preservation, PSNR in dB

    GroupWMMLLMBest baseline
    Local (n = 2,465 edits)30.44027.692
    Multi-object (n = 144 edits)29.10426.849
    Implicit reasoning (n = 100 edits)32.86828.891
    Selecting the region after generation. Composited inside the region predicted before generation (blue), the same GPT-Image proposal loses the skateboarder who fell and the bowler; composited inside the region that the MLLM selects after generation (red), it keeps both.

    Questions

    About the demo

    Loading questions.
    What does not work yet

    The main limitation is completing the action of a request reliably.

    • Final endpoint rejections cause 276 of the 280 failure holds. A hold repeats the current view, so the motion in the request is missing.
    • The endpoint check and the evaluation judge are models without human calibration, and frame samples cannot verify the motion between them.
    • The world is a panorama around one center, so translation and regions that no view has shown require generation. The generation log flags a seam in 208 of the 227 environment items.
    • Failures also arise from weak camera motion and from placements that miss the relation in the request.
    Known issues

    Paper

    WMMLLM: Turning MLLMs into World Models

    Baiang Li, Enze Xie, Jincheng Yu, Jin Wang, Haozhe Liu, Yitong Li, Juanxi Tian, Felix Heide.

    BibTeX
    @article{li2026wmmllm,
      title   = {{WM}$^{\text{MLLM}}$: Turning {MLLMs} into World Models},
      author  = {Li, Baiang and Xie, Enze and Yu, Jincheng and Wang, Jin and Liu, Haozhe and Li, Yitong and Tian, Juanxi and Heide, Felix},
      journal = {arXiv preprint arXiv:2611.24193},
      year    = {2026}
    }