Abstract

Contributions

    Method

    PhysPlan operates in two stages: an agentic planner reasons about physical causality and emits a Chain-of-Visual-Thought, then a training-free guidance stage renders it into video while routing optimization to active objects.

    ▣ Pipeline figure — assets/pipeline.png Drop your final Stage 1 / Stage 2 overview (the clearer supplementary version is recommended)
    Figure 1. Overview of PhysPlan. Stage 1 (agentic planning) extracts a multimodal guidance signal; Stage 2 (graph-guided optimization) steers reverse diffusion.
    Stage 1

    Agentic Physical Planning

    A vision-language model acts as a cognitive simulator: it parses the scene into a graph, predicts a causal sequence of physical state transitions, renders intermediate keyframes, and extracts kinematic masks, depth, and a kinetic-intensity profile.

    Stage 2

    Training-Free Graph-Guided Optimization

    The extracted conditions steer the reverse diffusion process via masked spatial-trajectory and 3D structural-consistency losses, with Object-Centric Gradient Routing and a kinetics-aware schedule — no fine-tuning of the base model.

    Chain-of-Visual-Thought

    The planner decomposes a prompt into a sequence of grounded visual states. Select a step:

    Figure 2. A worked example: the planner advances physical state across keyframes, each grounded into a segmentation mask and depth map.

    Key mechanisms

    Object-Centric Gradient Routing

    The test-time gradient is routed exclusively through the active-object region defined by the downsampled mask, so the passive background receives no update and is preserved exactly.

    Figure 3. Global guidance perturbs the entire latent (the background painting drifts); element-wise masking with M zeroes the gradient outside the active object, confining the update to the object and locking the background.

    Kinetic Intensity Profiling

    The planner scores per-frame physical volatility. The guidance learning rate and step density scale with it, concentrating optimization on high-entropy events such as collisions and phase changes.

    Experiments

    Physical plausibility (PhyGenBench)

    Physical plausibility (Physics-IQ)

    Generalization across vision-language models

    Video quality (VBench)

    Ablation study

    User study

    Failure analysis & limitations

    Representative failure cases

    ▣ Failure cases — assets/failure.png Supplementary limitations figure: mask leakage · extreme fragmentation
    Failure modes. Mask leakage on transparent/refractive objects, and extreme fragmentation that exceeds the capacity of the tracking masks.

    BibTeX