Diff-DAgger: Uncertainty Estimation with Diffusion Policy for Robotic Manipulation

1University of Virginia,

Abstract

Recently, diffusion policy has shown impressive results in handling multi-modal tasks in robotic manipulation. However, it has fundamental limitations in out-of-distribution failures that persist due to compounding errors and its limited capability to extrapolate. One way to address these limitations is robot-gated DAgger, an interactive imitation learning with a robot query system to actively seek expert help during policy rollout. While robot-gated DAgger has high potential for learning at scale, existing methods like Ensemble-DAgger struggle with highly expressive policies: They often misinterpret policy disagreements as uncertainty at multi-modal decision points. To address this problem, we introduce Diff-DAgger, an efficient robot-gated DAgger algorithm that leverages the training objective of diffusion policy. We evaluate Diff-DAgger across different robot tasks including stacking, pushing, and plugging, and show that Diff-DAgger improves the task failure prediction by 37%, the task completion rate by 14%, and reduces the wall-clock time by up to 540%. We hope that this work opens up a path for efficiently incorporating expressive yet data-hungry policies into interactive robot learning settings.

Can diffusion loss predict task failure?

Stacking

Pushing

Plugging

Real World Tasks

WIP

BibTeX

@misc{lee2024diffdaggeruncertaintyestimationdiffusion,
      title={Diff-DAgger: Uncertainty Estimation with Diffusion Policy for Robotic Manipulation}, 
      author={Sung-Wook Lee and Yen-Ling Kuo},
      year={2024},
      eprint={2410.14868},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2410.14868}, 
}