Abstract: Humanoid robots hold the potential for unparalleled versatility by performing human-like, whole-body skills. However, achieving agile and coordinated whole-body motions remains a significant challenge due to the dynamics mismatch between simulation and real-world physics. Existing approaches, such as system identification (SysID) and sim-to-real (Sim2Real) methods, often rely on labor-intensive parameter tuning or result in overly conservative policies that sacrifice agility. In this paper, we present ASAP (Aligning Simulation and Real Physics), a two-stage framework designed to tackle the dynamics mismatch and enable agile whole-body skills for humanoid robots. In the second stage, we deploy the policies in the real world and collect real-world data to train a residual action model that compensates for the dynamics mismatch, reducing tracking errors and improving agility. Then ASAP fine-tunes pre-trained policies with the residual action model integrated into the simulator to align effectively with real-world dynamics. We evaluate ASAP across three transfer scenarios—IsaacGym to IsaacSim, IsaacGym to Genesis, and IsaacGym to the real-world Unitree G1 humanoid robot. Our approach achieves significant improvements in agility and whole-body coordination across a variety of dynamic motions, reducing tracking error compared to SysID, Sim2Real baselines. ASAP enables highly agile motions previously unattainable, showcasing the effectiveness of residual action learning for bridging simulation and real-world dynamics. These results highlight a promising path toward more expressive and agile humanoid robots.