Abstract: Neural network (NN) dynamics models and control policies achieve strong performance in robotics, but providing sound guarantees under uncertainty is difficult, especially when the NNs are components within the closed-loop system. Existing reachability tools offer formal over-approximations, yet are often non-differentiable, overly conservative, and too slow to integrate into modern learning and real-time planning pipelines. To address this, we present a parallelizable, differentiable reachability analysis tool in JAX that unifies continuous- and discrete-time systems and supports both analytical and NN-based dynamics and controllers. Our reachability tool uses Taylor-model flowpipe construction and CROWN-style linear bound relaxation and propagation, yielding a GPU-batched reachability primitive that can be differentiated and used in downstream objectives. Building on this primitive, we design (i) a certified training method that encourages the learning of reachability-friendly dynamics models and controllers, and (ii) a reachability-informed sampling-based MPC scheme that incorporates certified reachable sets during action selection and enables gradient-based refinement. Experiments on non-prehensile object manipulation and quadrotor control tasks show competitive performance to baseline planners while providing tight, certified reachability guarantees under uncertainty.