Mathematics

Economic growth

Mathematical optimization

Gradient method

Mathematical analysis

Economics

Balanced flow

Convergence (economics)

Computer science

Applied mathematics

Computer vision

Smoothing

IEEE Robotics and Automation Letters

The empirical success of derivative-free methods in reinforcement learning for planning through contact seems at odds with the perceived fragility of classical gradient-based optimization methods in these domains. What is causing this gap, and how might we use the answer to improve gradient-based methods? We believe a stochastic formulation of dynamics is one crucial ingredient. We use tools from randomized smoothing to analyze sampling-based approximations of the gradient, and formalize such approximations through the bundled gradient. We show that using the bundled gradient in lieu of the gradient mitigates fast-changing gradients of non-smooth contact dynamics modeled by the implicit time-stepping, or the penalty method. Finally, we apply the bundled gradient to optimal control using iterative MPC, introducing a novel algorithm which improves convergence over using exact gradients. Combining our algorithm with a convex implicit time-stepping formulation of contact, we show that we can tractably tackle planning-through-contact problems in manipulation.

Bundled Gradients Through Contact Via Randomized Smoothing