Overview

This repository provides a comprehensive suite of Reinforcement Learning (RL) training environments for the Microduck bipedal robot. Built upon the mjlab (MuJoCo Warp) framework, it enables high-frequency (50 Hz) policy training using PPO. The trained policies are then exported to ONNX format and deployed on the physical Microduck robot, bridging the gap between simulation and real-world performance.

Key Features

• Sim2Real Recipe: Encodes a complete sim2real strategy, including advanced actuator modeling (BAM), sophisticated domain randomization, backlash simulation, and a proven reward design philosophy.
• Diverse Training Tasks: Offers a wide range of RL environments for Microduck, from basic walking and standing to complex maneuvers like roulades, sit-to-stand transitions, and even roller skating and ball kicking.
• High-Fidelity Actuator Modeling: Employs the BAM M6 actuator model, meticulously simulating voltage control, back-EMF, and various friction effects, crucial for the success of small bipedal robots.
• Backlash Simulation: Includes specific "Backlash" variants for most tasks, accurately modeling ±1° of gear play in each of the 14 servo joints, a significant factor in sim2real transfer.
• Flexible Robot Models: Provides multiple MJCF robot models tailored for different task families, optimizing simulation for specific scenarios like walking, general locomotion, and roller-based activities.
• Unified Observation Contract: Features a standardized 61-dimensional observation space across all policies, facilitating seamless hot-swapping of different trained behaviors on the real robot.
• Effortless Training & Deployment: Offers a streamlined quickstart guide with uv commands for training, playing back policies, and exporting them to ONNX for deployment on the physical Microduck.
• GPU Acceleration & Cloud Training: Leverages CUDA GPUs for local training and supports execution on Hugging Face Jobs for users without local GPU resources.

Typical Use Cases

• Developing robust walking gaits: Train policies for stable and efficient locomotion on various terrains.
• Enabling complex dynamic maneuvers: Create policies for standing up from falls, sitting, rolling, and even performing tricks.
• Achieving precise environmental interaction: Train robots to interact with objects, such as kicking a ball.
• Bridging the sim2real gap: Utilize the repository's comprehensive sim2real techniques to deploy highly performant policies on the physical Microduck robot.
• Research and development in legged robotics: Serve as a foundational platform for exploring novel RL algorithms and robotic control strategies.
• Onboard runtime integration: Export trained ONNX policies for seamless integration with the Microduck's onboard runtime.