Removing CO2 from the atmosphere will reduce the amount of solar heat that does not get promptly radiated back to space from the sunlit surfaces.
But, the excess solar heat that has already gone into the ocean, especially the deep ocean, and into melting ice caps and glaciers, is another matter. It takes a long time for massive thermal reservoirs to readjust to the new (actually the old) thermal energy distributions.
If that wasn't the case, the Earth would already be much hotter than it is today. Those same reservoirs of thermal heat (or "cold") have kept Earth cooler than the equilibrium temperatures that would eventually result from the current CO2 levels.