SKIP — Slides

Sparse Keyframe Interpolation Paradigm

Event-Preserving Sparse-to-Dense Generation for Embodied World Models

Ziheng He, Yixiang Chen, Ning Yang, Zhanqian Wu, Qisen Ma, Yuan Xu, Jiabing Yang, Peiyan Li, Xiangnan Wu, Xiaofeng Wang, Zheng Zhu, Jing Liu, Nianfeng Liu, Yan Huang

UCAS · CASIA · NJU · GigaAI · THU · FiveAges

arXiv:2606.00664 · cs.RO

arXiv Full page ↗ 中文

The Problem

Dense video world models are expensive

Embodied world models predict how actions change the scene — but rollouts are generated frame by frame in pixel space.
Long-horizon manipulation videos make this redundant and slow; neighboring frames barely change.
You can't just drop frames: policies depend on sparse task-relevant events — approach, contact, grasp, release.

Our Idea

Generate only the keyframes, then interpolate

Stage 1

SKIP-Selector

Event-preserving keyframe selection via multimodal fusion of visual flow, semantic change (DINOv3) and proprioception — with explicit gripper-event protection.

Stage 2

SKIP-Generator

A sparse video diffusion model (Wan 2.2-TI2V-5B, fine-tuned) synthesizes only the selected keyframes from an initial frame + language prompt.

Stage 3

SKIP-Reconstructor

A learned gap predictor sets per-segment frame counts; an action-conditioned interpolator recovers the missing intervals from robot actions.

Method

SKIP architecture

Results at a glance

Faster, sharper, and policy-ready

4.16×

faster than dense generation

89.0%

aggregate FVD reduction

+56.9 pp

π₀.₅ LIBERO SR vs. dense gen.

+41.7 pp

real-robot SR vs. dense gen.

Visual fidelity

End-to-end video quality on LIBERO

Method	PSNR ↑	SSIM ↑	LPIPS ↓	FVD ↓
Wan 2.2 (chunked, dense baseline)	21.423	0.832	0.162	4177
Uniform + SKIP Gen./Recon.	21.345	0.847	0.130	716
RDP + SKIP Gen./Recon.	21.076	0.843	0.137	724
TriPSS + SKIP Gen./Recon.	21.033	0.827	0.157	640
SKIP (ours)	21.635	0.856	0.119	458

Averaged across four LIBERO suites. SKIP cuts aggregate FVD from 4177 → 458 (−89.0%) over the dense baseline.

Efficiency

4.16× faster than dense generation

Suite	SKIP total (s/traj) ↓	Wan 2.2 dense (s) ↓	Speedup ↑
LIBERO-10	39.79	243.86	6.13×
LIBERO-Goal	37.73	144.66	3.83×
LIBERO-Object	37.75	137.92	3.65×
LIBERO-Spatial	36.64	124.74	3.40×
Weighted average	37.86	157.37	4.16×

Per trajectory on one NVIDIA H20, 50 denoising steps, bf16.

The key result

SKIP videos can replace real demonstrations

π₀.₅ training data	LIBERO-10	Goal	Object	Spatial	Avg.
Real demos (all-real reference)	90.1	96.5	96.3	97.9	95.2
SKIP-Mix70 (30% real + 70% SKIP)	89.6	96.6	96.0	97.5	94.9
SKIP-Mix100 (100% SKIP)	88.8	96.0	94.2	96.4	93.9
Dense-Mix70 (30% real + 70% dense)	68.5	67.7	60.6	59.4	64.1
Dense-Mix100 (100% dense)	35.5	38.6	37.2	36.6	37.0

SKIP-Mix70 matches all-real training within 0.3 pp; dense generation collapses by 30–58 pp.

Qualitative · LIBERO-10 (soup + cream cheese, long-horizon)

SKIP vs. every baseline

Ground Truth

SKIP (ours)

Dense

Uniform

RDP

TriPSS

Ground truth (GT) is the reference: SKIP tracks GT, while Dense and Uniform / RDP / TriPSS all miss task-relevant events (matching the FVD gaps on slide 6).

Real-robot deployment

Deployed on a real Franka Panda

T1 · two mugs → basket

T2 · bowl → drawer

T3 · white mug → plate

T4 · stack bowls

Takeaways

Sparse keyframes are enough

Generate event-preserving keyframes, then reconstruct the dense rollout with action-conditioned interpolation.
4.16× faster than dense generation and −89.0% FVD on LIBERO.
SKIP videos match all-real policy training (−1.3 pp sim, −6.7 pp real); dense generation collapses by 48–58 pp.

arXiv:2606.00664 Full project page ↗