Why Demos Can Mislead People
A clip can be completely real and still tell you almost nothing about how the robot behaves on a Tuesday afternoon in someone’s warehouse.
A demo can be completely real and still tell you almost nothing.
Most robot videos are honest about what they show. They are dishonest about what they do not show — the number of attempts, the level of human help, the setup time, the edits, and what happens the moment something unexpected occurs.
How many tries, how much human help, and how often does this work tomorrow?
Teleoperation is not the enemy — concealment is
Teleoperation means a person is helping the robot from a distance. It is legitimate. It is how new behaviours get learned, how rare failures get recovered, and how early commercial work gets done safely.
The problem is not that teleoperation exists. The problem is when a video uses teleoperation and presents it as autonomy. The public read “fully autonomous” and the small print says “supervised remote operation.”
Disclose the autonomy level somewhere viewers can see it.
A real public example
Tesla's 2024 “We, Robot” event showed Optimus robots interacting with attendees. TechCrunch and The Verge later reported that those interactions involved human remote assistance — not full autonomy on the night.
The event was real. The robots were real. The autonomy framing was misleading because the human role was not made clear at the time. The lesson is not that demos are fake. The lesson is that the missing label is the problem.
What better evidence looks like
- Named customer, named site, named task.
- Continuous, unedited runtime — not just a cut sequence.
- Reported intervention rate per hour or per task.
- Disclosed autonomy level: teleoperated, supervised, autonomous within scope.
- Outputs that can be checked against a customer's own records.
Better evidence is boring on camera and decisive in a buyer meeting.
What people often misunderstand
- Mistake 01
“If it's edited, it's fake.”
Editing is normal. Hiding the autonomy level or the failure rate is the issue.
- Mistake 02
“If a human helped, it doesn't count.”
Almost every real deployment involves humans somewhere. Disclosure decides whether the claim is honest.
- Mistake 03
“Demos prove the robot is ready.”
Demos prove the robot can do the task once, in a known setting, by people who know it.
- Mistake 04
“If a video is impressive, the product is impressive.”
Capability is not availability. The good demo can still be a bad shift.
A viewer's checklist for any robot video
What is being shown — and what is being left out of frame?
- What is the task?
- How long is the unedited run?
- Who is in the loop — operator, supervisor, no-one?
- What happens if the object or route changes?
- Where is this robot today — research lab, pilot, or paid work?
- Demos can be real and still mislead by omission.
- Hidden variables include attempts, setup, edits, teleoperation, and failure mode.
- Teleoperation is legitimate; concealing it is not.
- Better evidence names site, task, runtime, intervention rate, and autonomy level.
- Capability shown in a clip is not availability over a shift.
- Teleoperation
- A person remotely controlling or assisting the robot.
- Autonomy level
- How much of the task the robot does on its own versus with human help.
- Intervention rate
- How often a person has to step in to help the robot.
- Operational design domain
- The conditions a system is designed to work within.
- Unedited runtime
- Continuous footage without cuts — shows real timing and failure points.
- Disclosure
- Telling the viewer what kind of help, setup, or editing was involved.
Sources and evidence notes
What this essay leans on
| Claim | Evidence | Strength | Note |
|---|---|---|---|
| The gap between scripted robot demos and real-world robot work remains significant. | IEEE Spectrum commentary on robot demos and real-world performance. | Medium | Industry commentary. |
| Tesla's 2024 'We, Robot' Optimus interactions involved human remote assistance. | TechCrunch and The Verge reporting on the 'We, Robot' event. | Strong | Independent journalism. |
| Teleoperation is a legitimate part of real robot deployments and data collection. | General robotics literature on teleoperation as training and recovery. | Medium | Standard practice, varies by vendor. |