VLA and imitation learning developmentTeach the motion you cannot write out, from a human demonstration.
When the shape or position of the object changes every time, writing the motion out as a program is hard. We work on imitation learning, which trains motion from human demonstration, and on VLA, which connects visual and language understanding to motion generation.
Discuss VLA and imitation learning
Some work cannot be written as a program
If it is grasping a fixed object at a fixed position, writing the motion out is surer and faster. Bringing learning into it only makes sense for work you cannot write out.
Objects that differ one from another, that are placed differently each time, that deform like a bag or cloth. You cannot write out every branch for work like that, and teaching the motion from a human demonstration becomes the realistic approach.
Learning is not a cure-all, though. How much data is needed changes with the difficulty of the task and the success rate you require, and part of it cannot be read until you start collecting. We run a trial collection first and move to the full one once we can judge.
What we can take on
Separating out the target task
We separate what should be solved by learning from what is faster to write out. Not everything needs to be learned.
Designing the demonstration data collection
What to record, from which viewpoint, and how much. We design the target task, the machine and the quality criteria.
Demonstration under remote operation
The operator's motion is transferred to the robot, and data is gathered while it works on the real machine. Operation and collection advance together.
Implementing imitation learning
We train motion from the collected data and evaluate the success rate. Implemented on ROS 2 and LeRobot.
Implementing VLA
Visual and language understanding is connected to motion generation. The configuration decides the next motion from an instruction in words and the situation in front of it.
Evaluation on the real robot
We measure success rate and takt on the actual site, not on a benchmark. Conditions are varied to identify where it holds.
Data is the foundation of all of it
Whether learning works is decided by the data before the model. Too few viewpoints, no failure cases, task conditions that drift — train on data like that and it will not run on site.
We design the target task, the machine, the volume and the quality criteria, then take the data from the real robot. There is also a setup that records vision, action, dialogue and non-verbal signals in sync while the robot works under remote operation.
We can also take on the data collection alone, or advise only on building the collection environment.
Frequently asked questions
How much training data do I need?
It changes with the difficulty of the task, how much the environment varies, and the success rate you require. We run a trial collection first, judge the volume and quality criteria needed, and then move to the full collection. We do not promise a total up front.
We have no data. Can we still start?
Yes. We handle the collection design too. A setup that records while the robot works under remote operation lets you run operations and gather data at the same time.
Which robots can this be implemented on?
Any machine that can be controlled through ROS 2 or a vendor SDK can be discussed. Models built for secondary development are easier to work with. Tell us the target machine.
What is the difference between VLA and imitation learning?
Imitation learning is the general name for methods that acquire motion from human demonstration. VLA refers to a model configuration that connects visual and language understanding to motion generation, and differs in that it can handle instructions in words. We use them according to the task.
Discussing VLA and imitation learning
Tell us the target task, the machine and the state of any data you have. We start together by separating out what should be solved by learning.
Discuss VLA and imitation learning