LeRobot v0.6.0: Imagine, Evaluate, Improve
The robotics community is currently engaged in a profound debate: do world models actually provide a tangible benefit to robot policies? With the launch of LeRobot v0.6.0, the team is providing the tools necessary to answer that question definitively. This release is fundamentally about closing the robot learning loop, introducing policies that can "imagine" future states before executing actions, reward models that provide objective success metrics, and a streamlined deployment CLI that transforms real-world failures into high-quality training data.
Beyond these core advancements, v0.6.0 delivers a suite of six new simulation benchmarks, native depth sensing, VLM-powered dataset annotation, custom video encoding, and seamless cloud training integration via Hugging Face Jobs.
---
World Models: Policies That Imagine
The quest to build robots that understand the consequences of their actions has led to the integration of three distinct world-model-based policies in LeRobot. Each approach offers a unique strategy for balancing the computational cost of "imagination" with performance.
VLA-JEPA
VLA-JEPA utilizes a compact Vision-Language-Action (VLA) model—built upon the Qwen3-VL-2B architecture—to predict future states in latent space. During the training phase, the JEPA (Joint-Embedding Predictive Architecture) world model anticipates upcoming frames based on the robot's own actions. Crucially, the world model is discarded during inference, allowing users to benefit from world-model supervision without incurring additional latency at runtime.
LingBot-VA
Taking a more direct approach, LingBot-VA functions as an autoregressive video-action model. It predicts future video frames and corresponding actions in chunks, continuously feeding real observations back into the model to keep its internal "imagination" grounded in reality. Users can even visualize these predictions by setting --policy.save_predicted_video=true to compare the robot’s mental model against actual outcomes.
FastWAM
FastWAM explores the necessity of test-time imagination. By pairing a ~5B parameter video-generation expert with a compact action expert within a single network, the model learns to "dream" its own rollouts during training. At inference, however, it bypasses the dreaming process entirely, directly denoising action chunks for high-speed performance.
---
The Expanding VLA Model Zoo
LeRobot v0.6.0 significantly expands its library of supported Vision-Language-Action models, ensuring developers have access to the latest state-of-the-art architectures.
- GR00T N1.7: The integration with NVIDIA’s cross-embodiment foundation model has been upgraded to N1.7. This version replaces the previous VLM with Cosmos-Reason2-2B, utilizing a flow-matching action head. It is fully parity-tested against NVIDIA’s Isaac-GR00T implementation.
- MolmoAct2: Developed by the Allen Institute for AI, this model is now fully supported for fine-tuning, evaluation, and real-world deployment. It is optimized to fit within ~12 GB of VRAM at bf16 precision.
- EO-1: A VLA pretrained on interleaved vision-text-action data, featuring a Qwen2.5-VL-3B backbone.
- Multitask DiT: Bringing the TRI Large Behavior Models recipe to the platform, this ~450M-parameter diffusion transformer is conditioned on CLIP embeddings, allowing a single model to master multiple tasks.
- EVO1: Proving that VLAs do not need to be massive, EVO1 packs its capabilities into just 0.77B parameters, making it ideal for real-time execution on modest hardware.
---
Reward Models: Quantifying Success
A major hurdle in robot learning has been the lack of automated success detection. LeRobot v0.6.0 introduces a unified lerobot.rewards API, providing a home for progress estimation and success detection.
"Success detection and progress estimation are the missing pieces in the robot learning loop. v0.6.0 gives them a home."
- Robometer: A pretrained, general-purpose reward model built on Qwen3-VL-4B. It scores task progress and success from raw video and language instructions without requiring task-specific fine-tuning.
- TOPReward: A zero-shot approach that leverages off-the-shelf VLMs. By calculating the log-probability of the token "True" given a video and instruction, it turns any capable VLM into a functional reward signal.
---
Datasets: Faster, Richer, and More Flexible
Data handling has received a massive overhaul. Users now have granular control over video encoding, with the ability to probe for hardware-accelerated encoders like NVENC, VideoToolbox, and VAAPI.
Furthermore, depth support is now a first-class citizen. By plugging in an Intel RealSense camera, users can record depth maps in millimeters, compressed as 12-bit streams alongside RGB data. Additionally, the new lerobot-annotate CLI automates the creation of rich language annotations, allowing VLMs to watch episodes and generate timestamped subtasks, plans, and VQA pairs automatically.
Performance-wise, data loading is now up to 2x faster. Multi-camera frames are decoded in parallel, and persistent workers ensure that decoder caches remain active across training epochs.
---
Benchmarks: One CLI to Evaluate Them All
LeRobot v0.6.0 solidifies its position as an evaluation hub by introducing six new simulation benchmarks, all accessible via the lerobot-eval CLI:
1. LIBERO-plus: Stress-tests policies with 10,000 perturbed variants. 2. RoboTwin 2.0: Focuses on 50 bimanual manipulation tasks with heavy domain randomization. 3. RoboCasa365: The largest task surface in the lineup, spanning 365 kitchen tasks. 4. RoboCerebra: Evaluates long-horizon behavior by chaining 3 to 6 sub-goals. 5. RoboMME: A memory-focused exam testing object tracking and procedure imitation. 6. VLABench: Tests reasoning and knowledge in complex manipulation scenarios.
---
Training & Inference: The Robot Learning Flywheel
The new lerobot-rollout CLI transforms deployment into a robust workflow. It includes a DAgger (Dataset Aggregation) strategy, allowing users to monitor policy execution, intervene instantly when a failure occurs, and record human corrections. These corrections are tagged with intervention flags, creating a perfect dataset for iterative fine-tuning.
For those training large models, FSDP (Fully Sharded Data Parallel) support is now available, allowing users to shard parameters, gradients, and optimizer states across multiple GPUs. For users lacking local compute, the lerobot-train command now supports HF Jobs, enabling one-click cloud training on hardware ranging from T4s to 8x H200s.
---
Codebase: Leaner and Cleaner
The installation process has been significantly streamlined, with roughly 40% fewer base dependencies. The codebase now favors a modular approach, where hardware-specific dependencies are only installed if needed.
- uv.lock: Now the authoritative dependency specification for CI and development.
- Foxglove Integration: Users can stream teleoperation and rollout data directly to Foxglove for professional-grade visualization.
- Wayland/macOS Support: Keyboard controls for recording have been improved to work across headless rigs, SSH sessions, and macOS without requiring complex permissions.
As the LeRobot ecosystem continues to grow, these improvements—from the new LeLab browser UI to the comprehensive compute hardware guide—ensure that both academic researchers and hobbyists have the infrastructure to push the boundaries of open-source robotics. The LeRobot team invites the community to explore these new features and continue contributing to the future of embodied AI.