How Sim-to-Real Works in Humanoid Robotics

Transfer a controller, not an animation

In sim-to-real locomotion, engineers design or learn a controller using a simulated body, then test that controller on a physical robot. At each control update it receives observations and produces commands. The physical motors and ground must respond closely enough for the next observation to remain useful.

Consider a left foot lifting while the right foot supports the body. In simulation, the policy has learned how ankle and hip commands affect the next landing. A slippery floor or delayed motor response changes that relationship. Replaying the same joint targets does not recreate the same forces.

For the mechanics of support and foot placement, Read how one walking step is controlled. This guide focuses on transferring the controller that coordinates them.

Build a model that can be checked

[1] [2]

Keep visual geometry separate from collision and inertial geometry. A detailed foot mesh can still use an approximate contact shape. For a controller that relies on toe or heel contact, that approximation belongs in the experiment record.

The MuJoCo tutorial makes these fields concrete. Inspect an actual G1 model and its recorded joint response

Train the observation-to-action mapping

An observation might contain body rotation, gyro readings, joint positions and velocities, a requested walking velocity and recent actions. A simulator also knows quantities that a physical robot may not measure. Giving the deployed actor perfect base velocity without an estimator would hide a transfer requirement.

The action space must match the controller architecture. Joint-position offsets followed by PD drives are different from torque commands. An offset vector needs a joint order, a scale and a reference pose. The observation normalization used during training must travel with the policy checkpoint.

Reinforcement learning repeatedly collects trajectories, scores them and updates policy parameters. Velocity tracking can reward useful movement; penalties can discourage extreme joint motion or foot slip. Resets expose different starting states. A fall termination limits an episode, while a timeout truncates a still-valid trajectory. Reward design does not itself prove a safe motion.

For the learning methods themselves, Compare reinforcement learning with imitation. The walking tutorial defines a specific policy and evaluation workflow.

Build a walking experiment with separate evaluation trials

Vary the conditions for a reason

Randomization asks one controller to work across a distribution of models. It does not measure the physical distribution. Excessively broad ranges can make the task impossible; narrow ones can miss the hardware. Keep held-out conditions that were not used for tuning, and report their outcomes separately.

A walking result with its denominator

Google DeepMind’s 2024 study trained 20-joint ROBOTIS OP3 miniature humanoids in MuJoCo, using dynamics randomization and perturbations. Policies issued joint-position targets at 40 Hz. Physical observations combined proprioception with external motion capture. [3]

[3]

These are measured task means, not product top speeds. The separate turning comparison reran failed attempts until ten successes; the learned policy fell in three of thirteen attempts. The study’s limits include miniature scale, motion capture, joint wear, recalibration and sensitivity to battery charge. It does not establish full-size workplace readiness. [3]

The authors provide Open the authors’ released data and notebooks. Those notebooks were not executed for this guide.

A larger robot raises different model questions

UC Berkeley researchers trained a history-based Transformer controller for Agility Robotics Digit in randomized Isaac Gym environments. Their model approximated closed-chain rods with stiff virtual springs. That is a concrete example of adapting training around an engine’s mechanical representation. [4]

The paper’s comparative ten-run success rates on slopes, steps and unstable ground are simulator evaluations. Physical unstable-plank trials were omitted because of damage risk. A hardware video and a simulated trial table should not be merged into a claimed physical success rate. [4]

For another full-size platform, Read the Atlas hardware analysis. Its specifications are separate from either research result here.

Check the interface before energizing a robot

• Match the policy to the exact robot variant, sensor layout, actuator mode and firmware.
• Check joint order, signs, units, quaternion convention, target offsets and limits with recorded data first.
• Measure the actual observation and command timing. Define handling for stale samples and missed updates.
• Verify available effort and temperature limits through the manufacturer’s documented interface.
• Use supervised, controlled trials with a defined test area, fault response and stop criteria appropriate to the machine.
• Record every attempt, including interventions, aborted runs and recovery, before widening the task.

These are engineering checks, not a complete commissioning or conformity procedure. Simulation can expose faults cheaply, but it cannot verify an untested stop circuit, an incorrect cable connection or a battery fault on a physical unit.

Read the machine safety and conformity guide

Keep the remaining gap visible

An ideal rigid link does not reproduce structural flex. A friction coefficient does not fully describe a worn sole. Motor heating can reduce available effort after the first few minutes, while sensor bias or calibration drift changes the observations. Network delay and camera processing add timing differences that may not appear in a basic mechanics model.

When the robot diverges, compare aligned logs before changing rewards at random. Determine whether the mismatch is in sensing, the actuator response, contact or the policy’s decision. A good fit on one motion should be tested on another.

Follow the return loop from physical logs to a revised simulation