Abstract: While gradient-based methods can efficiently optimize trajectories and controllers by exploiting physical priors and differentiable simulators, contact-rich manipulation remains challenging due to discontinuous and vanishing gradients arising from hybrid contact dynamics. Recent methods smooth the dynamics to obtain continuous gradients, but the resulting model mismatch often causes controller failures when executed on real systems. We address this trade-off by developing a framework that plans efficiently with smoothed dynamics while explicitly quantifying and compensating for the induced modeling errors. Our method provides formal guarantees of constraint satisfaction and goal reachability on the true hybrid dynamics, enabling robust gradient-based synthesis for contact-rich manipulation. Specifically, we construct smooth approximations of both system dynamics and contact geometry to obtain a well-conditioned optimization landscape, and characterize the discrepancy from the true dynamics as a set-valued deviation. This deviation is incorporated into the optimization of time-varying affine feedback policies that admit analytical predictions of true system behavior under feedback, while relying solely on informative gradients from the smoothed dynamics. We evaluate our method on several contact-rich tasks, including planar pushing, object rotation, and in-hand dexterous manipulation, achieving guaranteed constraint satisfaction with lower goal error than baselines. By bridging differentiable physics with set-valued robust control, our method is the first certifiable gradient-based policy synthesis method for contact-rich manipulation.