There is more to it than going from cold to hot. Entropy would not be reversed unless and until there was a larger difference between the hots and the almost as hots at some point in time. Not clear how that would happen.
On the other hand, Bill sort of glossed over the assumed reason that there was ever any difference in temperature (or other energy forms) in the materials that comprise the universe. Cosmologists like to think of the universe as being uniform (except for "small" differences like galaxies and voids), and assume that the universe started as a tiny uniform spec that "inflated" to billions of light years. The tiny spec was so small that "quantum fluctuation" created even tinier non-uniformities, which then got "inflated" to enormous dimensions, so that was how the energy got separated into different regions and could be used to "do work" as it tries to become uniformly distributed, again.
But, it really is much more difficult to think about when you realize that the BBT starts with pure energy and somehow there is spontaneous development of order in the form of 6 types of quarks and various other forms of "sub-atomic matter" coming into existence, which then organized themselves into atomic matter, which then underwent fusion in stars to produce more complex atoms, which then combined chemically to produce, ultimately, life and us.
Another way of describing entropy is to relate it to order and disorder. But, when you look at the spontaneous development of all of that order in matter, starting from pure energy, it certainly looks like entropy must have decreased with that definition.
So, once we get out of the physical conditions of our experience with glasses of hot and cold water, and start trying to use the theory that entropy must always increase in anything the universe ever did or could ever do in the future, it seems to me that we should not be ruling out anything on the basis that it would require a decrease in entropy.
In particular, I do not believe that entropy should be used to rule out the possibility of a cyclic universe. Clearly, if the universe could gravitationally "crunch" back into a tiny spec like the BBT theorizes its beginning, then quantum fluctuations should be able to restore whatever entropy condition that existed when it was a tiny spec experiencing those fluctuations in the Big Bang, and then bang back to a new universe with nothing by hydrogen and energy (plus whatever dark matter and dark energy are, if they really exist).