Abstract: Lifelong learning in robotics promises systems that can continually improve and adapt to changing environments, mirroring human adaptability. For robots, this capability is crucial in refining dynamics models, which are foundational for planning and control. However, enabling such adaptability faces significant challenges: safely incorporating new experience while avoiding catastrophic forgetting, managing outliers, reconciling exploration with exploitation and operating within the constraints of onboard resources. Towards this goal, we introduce a framework leveraging flow matching for online correction of misaligned robot models. Our method transforms the actions proposed by a model-based planner by mapping it to the action the robot would have ideally executed given a well-aligned dynamics model, based on observed deviations caused by model inaccuracies. This allows the robot to bridge the gap between planned actions and actual outcomes, resulting in faster adaptation to changes in dynamics such as altered surface friction or actuator degradation. We find that by transforming the actions themselves rather than exploring with a misaligned model, the robot collects informative data more efficiently, thereby accelerating robot adaptation. Moreover, we validate that the method can handle an evolving and possibly imperfect model while eliminating the dependency on replay buffers or legacy model snapshots. We validate our approach using an unmanned ground vehicle in both simulation and real-world settings. The results highlight the method’s adaptability and efficiency, demonstrating its potential towards enabling robot lifelong learning.