Echo: Coupled Action and Task Status Signal Denoising for Vision Language Action Models

1 Shanghai Jiao Tong University 2 Spirit AI 3 Tsinghua University 4 USTC 5 Tongji University 6 HIT

* Equal contribution† Corresponding authors‡ Project leaderWork done during internship at Spirit AI.

Abstract

Recognizing task completion and failure enables robots to stop upon success and initiate recovery or replanning after failure. Existing methods augment Vision Language Action (VLA) policies with task status signal prediction to support these execution decisions. However, learning and integrating these signals can require fine-grained supervision, external monitoring models, or complex architectural designs. Moreover, the potential of task status signals to directly guide action generation remains underexplored. We present Echo, a framework that combines task status signal learning without frame-level progress annotations with a compact design for status prediction and action generation within a single VLA. Specifically, Echo represents task status with stop and failure signals, added as two dimensions of each action chunk and denoised together with robot actions. Their combination distinguishes ongoing execution, successful completion, and failure. To learn these signals without fine-grained progress annotations, Echo constructs successful, failed, and recovery trajectories with complementary supervision properties. To enable status signal-guided action generation, Echo controls interactions between the signals and actions and denoises the signals faster than the actions. These allow later action denoising steps to condition on cleaner status estimates while preserving accurate signal prediction. Extensive real-world manipulation experiments show that Echo accurately predicts task status and consistently improves policy performance.

Example Insert Flower trajectories show stop and failure signals during recovery, adjustment, success, and failure.
Echo jointly generates actions and task-status signals. Stop (SS) and failure (FS) signals distinguish ongoing execution, successful completion, and terminal failure. During ongoing execution, elevated FS identifies recoverable failures and can guide corrective behavior.

Overview Video

Method

Echo augments a flow-based VLA with two task-status dimensions and couples their denoising with robot actions. Four types of trajectories provide complementary supervision without additional frame-level progress annotations.

Echo framework: four data sources, signal target construction, structured attention, and asynchronous denoising.
Overview of Echo. Successful, failed, pre-intervention, and recovery trajectories supervise different parts of the joint action–signal representation. Structured attention and asynchronous denoising allow cleaner signals to guide later action updates.

Task-status representation

A stop signal (SS) and a failure signal (FS) are added to each action chunk. Low SS indicates ongoing execution; high SS with low FS indicates success; high SS with high FS indicates terminal failure. Low SS with elevated FS indicates a recoverable failure during ongoing execution.

Learning from successful, failed, and recovery data

Successful demonstrations and recovery demonstrations provide nominal and corrective actions. Terminal-failure rollouts and pre-intervention failure segments supervise task status, while their actions are excluded from imitation. Signal targets are constructed from segment types, boundaries, and positions within recovery segments.

Status-guided action generation

Actions, SS, and FS use separate token groups and projections. Actions attend to both signal groups, while direct SS–FS attention is blocked to preserve their distinct semantics. Signals are denoised faster than actions and held fixed after reaching their clean endpoint, allowing later action updates to condition on cleaner status estimates.

Real-World Experiments

We evaluate Echo on five real-world manipulation tasks using π0.5 as the policy backbone. Echo and the action-only baseline use the same successful and recovery demonstrations for action supervision. Echo additionally learns task-status signals from failure data.

Each Echo video includes the corresponding signal trace below the robot view. Baseline and Echo videos show independent recorded rollouts. Open Drawer is shown at 5× speed, Capsule Toy at 2×, and the remaining tasks at 1×.

Task-status prediction. Stop (SS) and failure (FS) signals during an Insert Flower rollout. Shown at 1× speed.

Insert Flower 1× speed

Grasp a flower and insert its stem into a vase.

π0.5 baseline
Echo (ours)

Capsule Toy 2× speed

Operate the capsule-toy machine to dispense and retrieve a toy.

π0.5 baseline
Echo (ours)

Insert Pen 1× speed

Pick up a pen and insert it into the holder.

π0.5 baseline
Echo (ours)

Make Tanghulu 1× speed

Use a skewer to assemble a tanghulu fruit snack.

π0.5 baseline
Echo (ours)

Open Drawer 5× speed

Grasp the handle and pull the drawer open.

π0.5 baseline
Echo (ours)

Results

Task completion and status prediction

Echo improves task completion scores over the action-only π0.5 baseline on all five tasks. Frame-level Macro-F1 evaluates status recognition throughout each trajectory; trajectory-level F1 evaluates outcome recognition within a fixed decision window.

Task completion and task-status prediction. Mean ± SE over 30 trials per model and task; F1 scores are percentages.
Taskπ0.5 scoreEcho scoreEcho frame F1Echo traj. F1
Insert Flower5.67 ± 0.928.33 ± 0.6992.18 ± 0.7196.67 ± 3.33
Capsule Toy8.40 ± 0.188.57 ± 0.1688.31 ± 0.57100.00 ± 0.00
Insert Pen6.87 ± 0.848.60 ± 0.4689.33 ± 1.4596.67 ± 3.33
Make Tanghulu4.33 ± 0.925.33 ± 0.9391.11 ± 1.1393.33 ± 4.63
Open Drawer5.40 ± 0.777.90 ± 0.2688.01 ± 1.2290.00 ± 5.57

Generalization to out-of-distribution conditions

Echo's gains are larger under out-of-distribution (OOD) conditions. In-distribution (ID) differences are modest and task-dependent, while status prediction remains broadly stable across conditions.

Task completion score on Insert Flower and Open Drawer (0–10). Mean ± SE over 15 trials per model and condition.
ModelInsert FlowerOpen Drawer
IDOODIDOOD
π0.59.33 ± 0.672.00 ± 1.078.40 ± 0.132.40 ± 1.06
Echo8.67 ± 0.918.00 ± 1.078.93 ± 0.186.87 ± 0.29

Effect of Signal Intervention

To examine how task status influences action generation, we prescribe signal targets during denoising on Insert Flower. With SS fixed to zero, FS = 0 produces task-directed motions, while sustained FS = 1 elicits repeated upward recovery motions.

Prescribed failure signals produce different actions: FS equals one induces upward recovery motions, while FS equals zero produces task-directed movement.
This qualitative intervention shows that status inputs can steer action generation toward corrective behavior. See the paper for component ablations, failure-data scaling, and implementation details.