Humanoid behavior generation and reinforcement learning policy development from live-action videosFrom live-action videos of human movements to humanoid movement packages that have been evaluated with physical simulation.
Retargeting conversion to joint trajectory data (NPZ/CSV) for humanoids such as Unitree G1 and Unitree R1 based on human movement videos taken with smartphones and cameras, MuJoCo / Isaac This is a development service that delivers a package for robot execution (ONNX model and set of settings) after learning a whole body tracking policy using reinforcement learning (PPO) using Lab (mjlab) and evaluating safety and completion using closed-loop physical simulation (Sim2Sim).
Consult about humanoid behavior generation and reinforcement learning policy development from live-action videos
Consultation content and deliverables
- Compatible robot
- Unitree G1 (23-29DoF, etc.), Unitree R1 (26DoF), etc. (Target configuration/controller version fixed in advance)
- Input conditions
- Video of a single person showing the whole body and toes (includes step-by-step actions excluding jumping, lying on the floor, vigorous rotation, etc.)
- Offer menu
- ① Reference trajectory data conversion (NPZ/CSV) ② Closed-loop physical evaluation with existing whole body controller ③ Motion-specific PPO reinforcement learning & ONNX delivery ④ Actual machine verification
- Physical evaluation items
- Quantitative inspection of completion rate, presence of falls, trajectory following error, foot slip, joint angle/speed limit, self-collision, and torque saturation
- Deliverables
- Normalized motion data (NPZ/CSV), evaluated ONNX/checkpoints, observation/behavior specifications/PD gain settings, evaluation report
- price
- Estimates based on length and number of videos, scope of physical simulation evaluation, individual reinforcement learning, and robot testing
How we work
We will ask you about the current situation, target robot, site conditions, and desired timing, and then we will begin work after agreeing on the scope of implementation and preconditions. Deliverables and prices will be specified in the individual quotation.
A 4-step motion generation/physical verification pipeline that prevents failures due to "just converted JSON"
01 Video quality inspection and high-precision retargeting to robot skeletons
Estimates the 3D human body posture, route position, and ground contact status from live-action videos, and generates a full-frame normalized trajectory (NPZ/CSV) that matches the link length, joint order, and range of motion constraints of the target robot (G1/R1).
02 Closed-loop physical simulation evaluation using whole body balance control (SONIC, etc.)
If only kinematic pose conversion is used, the actual robot will not be able to withstand the shift of the center of gravity and will fall. Input a reference trajectory into the whole body control policy on the physics simulator and verify stability against fluctuations in friction, delay, and initial posture.
03 Behavior-specific reinforcement learning (mjlab PPO) and ONNX policy output
PPO reinforcement learning is performed in Yushima Lab's GPU environment (RTX A6000 48GB, etc.) for gestures and repetitive movements that are difficult to reproduce with general-purpose tracking devices alone. Only checkpoints that pass the evaluation criteria are packaged in ONNX format.
04 Hardware verification option on owned actual machines (G1/R1)
Upon request, we will test the starting posture, motion replay, and return to stop on an robot in the EmplifAI laboratory, and deliver the product along with footage of the robot operating and safe operation parameters.
NVIDIA GEAR Sonic × Unitree G1: Real robot demonstration
We ran NVIDIA GEAR Sonic on a Unitree G1 robot and verified its operation. Watch the video to see the demonstration.
Gemini 4 Argon × Unitree G1: Turning Vision into Action
Unitree G1 just picked up a plastic bottle using Gemini 4 Argon. No hard-coded motions. Just AI turning vision into action. The future of robotics is getting real.
Even when the bottle is moved and placed again, the AI visually understands the scene and generates the grasping motion. We provide Physical AI and VLA implementation on the Unitree G1 as well as custom PoC and verification development tailored to your operations.
Frequently asked questions
What can I request for humanoid behavior generation/reinforcement learning policy development from live-action videos?
Retargeting conversion to joint trajectory data (NPZ/CSV) for humanoids such as Unitree G1 and Unitree R1 based on human movement videos taken with smartphones and cameras, MuJoCo / Isaac This is a development service that delivers a package for robot execution (ONNX model and set of settings) after learning a whole body tracking policy using reinforcement learning (PPO) using Lab (mjlab) and evaluating safety and completion using closed-loop physical simulation (Sim2Sim). The specific scope of work will be confirmed and agreed upon before starting work, including robot, site, and desired deliverables.
How are the costs and scope of delivery for humanoid behavior generation and reinforcement learning policy development from live-action videos determined?
We will confirm the target robot, number of machines, implementation location, and required deliverables, and provide an individual estimate of availability, scope of work, completion conditions, and cost.
What is the difference between commercially available video conversion tools and cheap motion APIs?
General video conversion APIs only output "time series data of joint angles", and if you send it to an robot as is, it will fall over due to gravity and the slippage of the soles of your feet. The biggest difference with EmplifAI is that it fixes the target robot's profile (joint order, PD gain, control cycle), goes through closed-loop physical simulation evaluation (overturning, torque saturation, self-collision inspection) and PPO reinforcement learning, and delivers it as a control package that can be run on an robot.
Can any video be converted into humanoid motion?
No. Jumps, intense spins on one leg, touching the floor, videos with multiple people overlapping, etc. cannot be reproduced directly due to physical constraints. Before starting, we check the sample video and agree on a possible range, including conditions for maintaining ground contact and speed smoothing.
Consultation on humanoid behavior generation and reinforcement learning policy development from live-action videos
Please let us know the target robot, location, issue you would like to solve, and desired timing. We will inform you about the implementation details and estimate.
Consult about humanoid behavior generation and reinforcement learning policy development from live-action videos