Overview
This repository provides a comprehensive suite of Reinforcement Learning (RL) training environments for the Microduck bipedal robot. Built upon the mjlab (MuJoCo Warp) framework, it enables high-frequency (50 Hz) policy training using PPO. The trained policies are then exported to ONNX format and deployed on the physical Microduck robot, bridging the gap between simulation and real-world performance.
Key Features
- Sim2Real Recipe: Encodes a complete sim2real strategy, including advanced actuator modeling (BAM), sophisticated domain randomization, backlash simulation, and a proven reward design philosophy.
- Diverse Training Tasks: Offers a wide range of RL environments for Microduck, from basic walking and standing to complex maneuvers like roulades, sit-to-stand transitions, and even roller skating and ball kicking.
- High-Fidelity Actuator Modeling: Employs the BAM M6 actuator model, meticulously simulating voltage control, back-EMF, and various friction effects, crucial for the success of small bipedal robots.
- Backlash Simulation: Includes specific "Backlash" variants for most tasks, accurately modeling ±1° of gear play in each of the 14 servo joints, a significant factor in sim2real transfer.
- Flexible Robot Models: Provides multiple MJCF robot models tailored for different task families, optimizing simulation for specific scenarios like walking, general locomotion, and roller-based activities.
- Unified Observation Contract: Features a standardized 61-dimensional observation space across all policies, facilitating seamless hot-swapping of different trained behaviors on the real robot.
- Effortless Training & Deployment: Offers a streamlined quickstart guide with
uvcommands for training, playing back policies, and exporting them to ONNX for deployment on the physical Microduck. - GPU Acceleration & Cloud Training: Leverages CUDA GPUs for local training and supports execution on Hugging Face Jobs for users without local GPU resources.
Typical Use Cases
- Developing robust walking gaits: Train policies for stable and efficient locomotion on various terrains.
- Enabling complex dynamic maneuvers: Create policies for standing up from falls, sitting, rolling, and even performing tricks.
- Achieving precise environmental interaction: Train robots to interact with objects, such as kicking a ball.
- Bridging the sim2real gap: Utilize the repository's comprehensive sim2real techniques to deploy highly performant policies on the physical Microduck robot.
- Research and development in legged robotics: Serve as a foundational platform for exploring novel RL algorithms and robotic control strategies.
- Onboard runtime integration: Export trained ONNX policies for seamless integration with the Microduck's onboard runtime.
