Insert Flower 1× speed
Grasp a flower and insert its stem into a vase.
Recognizing task completion and failure enables robots to stop upon success and initiate recovery or replanning after failure. Existing methods augment Vision Language Action (VLA) policies with task status signal prediction to support these execution decisions. However, learning and integrating these signals can require fine-grained supervision, external monitoring models, or complex architectural designs. Moreover, the potential of task status signals to directly guide action generation remains underexplored. We present Echo, a framework that combines task status signal learning without frame-level progress annotations with a compact design for status prediction and action generation within a single VLA. Specifically, Echo represents task status with stop and failure signals, added as two dimensions of each action chunk and denoised together with robot actions. Their combination distinguishes ongoing execution, successful completion, and failure. To learn these signals without fine-grained progress annotations, Echo constructs successful, failed, and recovery trajectories with complementary supervision properties. To enable status signal-guided action generation, Echo controls interactions between the signals and actions and denoises the signals faster than the actions. These allow later action denoising steps to condition on cleaner status estimates while preserving accurate signal prediction. Extensive real-world manipulation experiments show that Echo accurately predicts task status and consistently improves policy performance.
Echo augments a flow-based VLA with two task-status dimensions and couples their denoising with robot actions. Four types of trajectories provide complementary supervision without additional frame-level progress annotations.
A stop signal (SS) and a failure signal (FS) are added to each action chunk. Low SS indicates ongoing execution; high SS with low FS indicates success; high SS with high FS indicates terminal failure. Low SS with elevated FS indicates a recoverable failure during ongoing execution.
Successful demonstrations and recovery demonstrations provide nominal and corrective actions. Terminal-failure rollouts and pre-intervention failure segments supervise task status, while their actions are excluded from imitation. Signal targets are constructed from segment types, boundaries, and positions within recovery segments.
Actions, SS, and FS use separate token groups and projections. Actions attend to both signal groups, while direct SS–FS attention is blocked to preserve their distinct semantics. Signals are denoised faster than actions and held fixed after reaching their clean endpoint, allowing later action updates to condition on cleaner status estimates.
We evaluate Echo on five real-world manipulation tasks using π0.5 as the policy backbone. Echo and the action-only baseline use the same successful and recovery demonstrations for action supervision. Echo additionally learns task-status signals from failure data.
Each Echo video includes the corresponding signal trace below the robot view. Baseline and Echo videos show independent recorded rollouts. Open Drawer is shown at 5× speed, Capsule Toy at 2×, and the remaining tasks at 1×.
Grasp a flower and insert its stem into a vase.
Operate the capsule-toy machine to dispense and retrieve a toy.
Pick up a pen and insert it into the holder.
Use a skewer to assemble a tanghulu fruit snack.
Grasp the handle and pull the drawer open.
Echo improves task completion scores over the action-only π0.5 baseline on all five tasks. Frame-level Macro-F1 evaluates status recognition throughout each trajectory; trajectory-level F1 evaluates outcome recognition within a fixed decision window.
| Task | π0.5 score | Echo score | Echo frame F1 | Echo traj. F1 |
|---|---|---|---|---|
| Insert Flower | 5.67 ± 0.92 | 8.33 ± 0.69 | 92.18 ± 0.71 | 96.67 ± 3.33 |
| Capsule Toy | 8.40 ± 0.18 | 8.57 ± 0.16 | 88.31 ± 0.57 | 100.00 ± 0.00 |
| Insert Pen | 6.87 ± 0.84 | 8.60 ± 0.46 | 89.33 ± 1.45 | 96.67 ± 3.33 |
| Make Tanghulu | 4.33 ± 0.92 | 5.33 ± 0.93 | 91.11 ± 1.13 | 93.33 ± 4.63 |
| Open Drawer | 5.40 ± 0.77 | 7.90 ± 0.26 | 88.01 ± 1.22 | 90.00 ± 5.57 |
Echo's gains are larger under out-of-distribution (OOD) conditions. In-distribution (ID) differences are modest and task-dependent, while status prediction remains broadly stable across conditions.
| Model | Insert Flower | Open Drawer | ||
|---|---|---|---|---|
| ID | OOD | ID | OOD | |
| π0.5 | 9.33 ± 0.67 | 2.00 ± 1.07 | 8.40 ± 0.13 | 2.40 ± 1.06 |
| Echo | 8.67 ± 0.91 | 8.00 ± 1.07 | 8.93 ± 0.18 | 6.87 ± 0.29 |
To examine how task status influences action generation, we prescribe signal targets during denoising on Insert Flower. With SS fixed to zero, FS = 0 produces task-directed motions, while sustained FS = 1 elicits repeated upward recovery motions.
