Imitation learning, especially with interventions, has driven so much recent robotics progress. However, improving a policy via targeted interventions until it reaches a useful and deployable success rate is a time and labor intensive process. Instead, wouldn’t it be great if policies could improve on their own?
That’s what
@VarunGiridhar3 and
@animesh_garg join us to talk about. In Q-Planning, they start with a large policy like pi-0.5, and add a Q-function estimator to predict value instead of just actions, then use both successful and failed rollouts to update this Q-function online, then use it to guide sampling and trajectory selection. With just a few rollouts they can dramatically improve policy performance online. This provides a way to do really difficult tasks like inserting a credit card into a wallet, increasing success rate from 25% to 80% in just a few iterations.
Learn more in Episode 105 of RoboPapers, hosted by
@micoolcho,
@chris_j_paxton, and
@ruijie_sg.