Hand actions on a timeline
Perceptron created EgoHome to test how AI systems interpret first-person household video. The benchmark includes annotated movements as brief as 100 milliseconds, so a model must distinguish nearby actions on a timeline instead of describing an entire chore with one label. [1]
A model that identifies dishwashing may still confuse the hand that holds a plate with the hand that scrubs it. EgoHome scores those descriptions separately. Every result discussed below comes from Perceptron’s published evaluation; TechniaHQRobot has not run the benchmark. [1]
137 clips from household video
EgoHome consists of 137 clips cut from 46 videos, totaling 1.99 hours. Head-mounted cameras recorded routine household tasks including washing dishes, cleaning a microwave, wiping a stovetop and folding laundry on a bed or sofa. The public video material is 1920 × 1080 at 30 frames per second according to the accompanying evaluation repository. [1] [2]
Each clip contains overlapping descriptions of the overall activity and of the left and right hands. This arrangement permits comparison of a system that identifies the chore with a system that can also distinguish the movements used to perform it. [1]
Four types of annotations
| Annotation | Published count | What it records |
|---|---|---|
| Hand bounding boxes | 417,912 boxes | Left/right hand location across frames |
| Subtask timelines | 1,924 segments | Steps such as wiping a surface |
| Hand actions | 5,032 segments | Left/right verbs from the Ego4D vocabulary |
| Hand captions | 5,277 segments | Open-ended left/right descriptions |
Across the three segment-based categories, Perceptron counts 3,371 segments shorter than one second and 773 shorter than half a second. The shortest segment is 0.10 second. Hand boxes were corrected by human annotators at 10 fps and then carried to 30 fps using Perceptron’s smoothing and extrapolation process; the 30 fps boxes were not all drawn by hand. [1]
The action vocabulary contains 118 possible Ego4D verbs, of which 47 occur in this release. Hand captions are free text rather than selections from that vocabulary. This distinction matters when comparing the hand action and hand caption scores. [1]
What a 100 ms boundary changes
Consider lifting a cup from a drying rack. A hand approaches, closes around the handle, rotates the cup, moves it and releases it. A single label such as “putting away dishes” loses the times at which contact begins and ends. A robot-learning pipeline using such labels could pair an early reach frame with a later release description. [1]
The same timing problem appears when two hands work together: one hand steadies a bowl while the other changes a sponge’s orientation. A video model can identify the correct object but assign the action to the wrong hand or mix two consecutive actions. [1]
EgoHome versus WGO-Bench
| Measure | EgoHome | WGO-Bench |
|---|---|---|
| Total duration | 1.99 hours | 1.24 hours |
| Clips / episodes | 137 clips from 46 videos | 100 episodes |
| Domains | Head-mounted household chores | Robot manipulation and egocentric sequences |
| Annotation categories | Boxes, subtasks, hand actions, hand captions | Subtask timelines |
| Shortest segment | 0.10 second | 0.19 second |
| Evaluation | Centroid hit fraction; permissible subsegment Jaccard | Segment and semantic F1; one-to-one matching |
WGO-Bench includes DROID and RoboCOIN robot manipulation recordings alongside HomER egocentric video. EgoHome concentrates on household hand activity and has three additional annotation categories. Their scores answer different questions: the two benchmarks should not be treated as interchangeable robot-control rankings. [1]
Perceptron’s launch article also states that WGO-Bench evaluates segment matches with a temporal overlap threshold of 0.75 and a semantic label check for semantic F1. EgoHome instead allows semantically valid coarse and fine splits to correspond. [1] [3]
Benchmark results
| System | Subtask | Hand action | Hand caption |
|---|---|---|---|
| Perceptron 4 fps | 0.787 | 0.421 | 0.613 |
| Perceptron 8 fps | 0.798 | 0.421 | 0.626 |
| GPT-6 Astra 4 fps | 0.797 | 0.498 | 0.749 |
| Gemini 3.8 Flash 4 fps | 0.663 | 0.405 | 0.558 |
The three columns use permissible subsegment Jaccard, which measures agreement between predicted and human-reference timelines while allowing multiple acceptable temporal subdivisions. The fourth annotation task, hand-box location, uses a different metric called centroid hit fraction: Perceptron reports 0.942 at 4 fps, 0.973 at 8 fps and 0.979 for GPT-6 Astra at 4 fps. [1]
Perceptron’s own settings were averaged over three runs at 4 fps and two at 8 fps; the other evaluated systems were run once. These are Perceptron’s measurements using its benchmark and scoring rules, not measurements by TechniaHQRobot. [1]
What the results reveal
The Perceptron 4 fps setting scores 0.787 for subtasks and 0.421 for hand actions, a 0.366-point gap. GPT-6 Astra at 4 fps scores 0.797 and 0.498, a 0.299-point gap. The tasks have different label spaces and error patterns, so that subtraction describes the reported scores rather than a unified percentage of correctly executed movements. [1]
Increasing Perceptron sampling from 4 fps to 8 fps changes its subtask score by +0.011 and hand caption by +0.013, while its hand action score stays at 0.421. More sampled frames alone did not improve hand action classification in this published comparison. [1]
Looking only at subtask segments shorter than 0.5 seconds, Perceptron reports 0.559 at 4 fps and 0.555 at 8 fps, versus 0.503 for GPT-6 Astra at 4 fps. Longer-segment averages can hide short-action failures. In the separate hand-action breakdown for segments under 0.5 seconds, Perceptron 4 fps scores 0.298 and GPT-6 Astra 4 fps scores 0.422. [1]
A permissive temporal metric solves one practical scoring problem. A model may split “sweeping stairs” into upper and lower stairs, while an annotator uses one longer label. If both descriptions cover the same physical action, one-to-one segment matching can penalize the extra split. Perceptron’s permissible subsegment Jaccard lets related segments match many-to-many after semantic judging. [1]
How EgoHome is annotated
Perceptron reports starting with machine-generated draft labels, having people correct them and having one of its researchers review every clip. Annotators completed calibration and agreement checks before labeling the evaluation set. [1]
The reported total annotation effort is 115.9 active human minutes per minute of video across all four categories, approximately 116 minutes. The category-level figures are 41.7 for boxes, 17.1 for subtasks, 24.6 for actions and 32.6 for captions. These are Perceptron’s production figures rather than an independently timed replication. [1]
Open-source access and licensing
Perceptron publishes the benchmark resource links, metric code and judge prompts. The GitHub code carries an MIT license. The EgoHome dataset is released under CC BY-NC 4.0 with additional permission for anyone, including commercial organizations, to evaluate models and publish evaluation results. [1]
Perceptron states that other commercial uses, including selling the data or using it to train commercial models, are not allowed under that permission. The license does not grant this publication unrestricted commercial reuse or redistribution of the underlying videos. [1]
Technical limitations and research questions
A useful next experiment would compare two models after training on the same human-labeled household clips, with grasp or placement success measured on a physical robot. EgoHome itself provides no such intervention study. Another test would isolate failures caused by fast motion, occlusion and interactions between hands, rather than averaging all time spans. [1]
The benchmark’s judge can accept descriptions at different granularities, but that flexibility also makes judge behavior a part of the measurement. Testing multiple judge models or hand-verifying disputed matches would indicate how stable a ranking is when the semantic scorer changes. The published tables do not settle those questions. [1]
Original research and sources
Research and benchmark data: Perceptron. Independent editorial analysis: TechniaHQRobot. Results are reported from Perceptron’s published evaluation.
Perceptron: original EgoHome research and results
Perceptron: Egocentric 1.0 launch and WGO-Bench evaluation
Sources and verification
- Perceptron EgoHome: A new benchmark for dense, sub-second egocentric understanding ↗Perceptron · Read 10 November 2026
Primary source for dataset statistics, methods, license and model evaluation; all reported measurements are Perceptron’s.
- EgoHome evaluation code, prompts and metrics ↗Perceptron on GitHub · Read 10 November 2026
Public repository; documents metric implementations, dataset link and code MIT license.
- Introducing Perceptron Egocentric 1.0 ↗Perceptron · Read 10 November 2026
Primary launch report for the API and additional WGO-Bench comparison.
Evidence limits
- The evaluation is based on household videos. No real-robot success rate or industrial transfer study follows from the reported scores.
- The metric uses an LLM judge to assess semantic agreement, and scores can depend on its acceptance criteria.
- Perceptron averaged several runs for its own settings while reporting single runs of other models.
- The full benchmark article supplies the evaluation and dataset licensing terms; independent replication was not performed by TechniaHQRobot.
Article history
Prepared an original source-credited editorial reading of Perceptron’s EgoHome release and reported model comparisons. Editorial approval remains pending.
Section headings now name the subject of each section instead of a shared template.
Report a correction