Violent mergers might be robbing galaxies of their star-birthing material