PICO VR / whole body teleoperation environment construction supportBy connecting a VR headset and whole-body reinforcement learning control, we realize intuitive whole-body remote control and data collection of a humanoid.
We support the construction of a real-time whole-body teleoperation environment for bipedal humanoids such as Unitree G1 by combining a VR headset/motion tracker (PICO, etc.) and a whole-body balance control policy (NVIDIA GEAR-SONIC, etc.). In addition to operating the arms and wrists of the upper body, it can remotely control whole body movements such as crouching, twisting, and moving the upper body while maintaining balance, and can be used as a basis for remote intervention in collecting demonstration data and events and demonstration experiments.
Consult about PICO VR/full body teleoperation environment construction support
Consultation content and deliverables
- Target robot
- Unitree G1 EDU (29DoF configuration, etc.) and other secondary development compatible humanoids
- operation device
- Configuration for use with PICO VR headset/tracker or leader arm/data glove
- control base
- Whole body reinforcement learning policy collaboration with NVIDIA GEAR-SONIC, DDS/ROS 2 communication, coordinate/joint mapping settings
- safety design
- Operational procedures including soft stop, initial attitude transition, lifting equipment and safety fence in case of communication delay or tracking loss
- Deliverables
- Teleoperation execution environment set, startup/calibration procedure manual, robot operation lecture
- price
- Individual estimates depending on whether or not you have equipment, hand configuration, and additional scope of data recording function.
How we work
We will ask you about the current situation, target robot, site conditions, and desired timing, and then we will begin work after agreeing on the scope of implementation and preconditions. Deliverables and prices will be specified in the individual quotation.
Three technical supports to expand from upper body teleoperation to "whole body cooperative teleoperation"
Real-time conversion from VR tracking to robot whole body coordinates
The movements of the operator's head, hands, and hips are captured using a VR device and retargeted to the humanoid's joint space with minimal delay. We will develop a calibration procedure to absorb differences in height and reach.
Maintaining dynamic balance through whole body reinforcement learning policy (GEAR-SONIC, etc.)
An integrated whole-body control system allows the lower body to automatically compensate for the center of gravity even when extending the arms forward or bending the knees to reach for low-level objects. We build dependent libraries and communication settings based on the G1 EDU robot verification knowledge in our own laboratory.
Simultaneous recording of multimodal performance data for imitation learning and VLA learning
Record robot-view camera images, whole-body joint angles and torques, and VR operation commands during full-body teleoperation in a time-synchronized manner. Connect to a conversion pipeline to LeRobot format, etc., and accumulate data that can be used as is for later autonomous learning.
NVIDIA GEAR Sonic × Unitree G1: Real robot demonstration
We ran NVIDIA GEAR Sonic on a Unitree G1 robot and verified its operation. Watch the video to see the demonstration.
Gemini 4 Argon × Unitree G1: Turning Vision into Action
Unitree G1 just picked up a plastic bottle using Gemini 4 Argon. No hard-coded motions. Just AI turning vision into action. The future of robotics is getting real.
Even when the bottle is moved and placed again, the AI visually understands the scene and generates the grasping motion. We provide Physical AI and VLA implementation on the Unitree G1 as well as custom PoC and verification development tailored to your operations.
Frequently asked questions
What can I request for PICO VR/full body teleoperation environment construction support?
We support the construction of a real-time whole-body teleoperation environment for bipedal humanoids such as Unitree G1 by combining a VR headset/motion tracker (PICO, etc.) and a whole-body balance control policy (NVIDIA GEAR-SONIC, etc.). In addition to operating the arms and wrists of the upper body, it can remotely control whole body movements such as crouching, twisting, and moving the upper body while maintaining balance, and can be used as a basis for remote intervention in collecting demonstration data and events and demonstration experiments. The specific scope of work will be confirmed and agreed upon before starting work, including robot, site, and desired deliverables.
How are the costs and scope of delivery for PICO VR/full body teleoperation environment construction support determined?
We will confirm the target robot, number of machines, implementation location, and required deliverables, and provide an individual estimate of availability, scope of work, completion conditions, and cost.
Is it possible to perform everything from walking to complex tasks with complete stability using full-body teleoperation alone?
With the current state of technology, there are cases in which balance can be lost due to changes in floor friction, sudden arm swings, or excessive forward leaning. For this reason, we begin with a safe step-by-step verification using a fixed stand and gantry (hanging device), clarify the allowable speed and range of movement for standing and walking on the floor, and then build an operational environment.
Consultation on PICO VR/full body teleoperation environment construction support
Please let us know the target robot, location, issue you would like to solve, and desired timing. We will inform you about the implementation details and estimate.
Consult about PICO VR/full body teleoperation environment construction support