Yes, time does vary wherever there is mass. But where it varies significantly is wherever you have significant density of mass. Neutron stars, for instance, have incredible strong surface gravity, so at their surface, time is noticeably different. Earth's gravitational field isn't significant, though GPS systems must include GR (Gen. Relativity) when calculating distances of, say, 1 cm. They also have to include SR (Special Relativity) as well since the satellites are moving rapidly in space. The two results work against one another, interestingly.
There is always a future so there must always be a past, at least until there was a beginning. This constant increase in time everywhere is known at the Arrow of Time, and it's a bit of a mystery on a purely physic's level.
It may help to think that if you had a trillion clocks -- estimates now suggest there are 2 trillion galaxies out there -- all beginning with the same time and had placed them evenly around the universe that they would all read about the same time, though variations in their local grav. field would alter them somewhat.
The observable universe, when we reverse time (with our imaginations and equations), take us back very close to t=0. But the wheels fly off the equation wagons when we get closer. It is very easy to infer a t=0 instant and beginning but we aren't able to get science involved since we usually do poorly with infinity results that shouldn't be there.
The fact that we can tweak this time frame to 13.8 Gyrs ago is astonishing given our humble existence in the universe, IMO.
Since light has a fixed speed, then the light we are seeing from all sources including your mirror, represents what happened in the past. Sunlight is about 8 minutes old. The oldest observable light (microwaves) comes from an incredible event known as the CMBR (Cosmic Microwave Background Radiation). This took place the instance the universe cooled to about 3000K due to its progressive expansion. At that temperature, electrons and protons came together to form atoms (9 of 10 were hydrogen). This eliminated all the light scattering and light suddenly was turned lose to travel freely. So we are seeing light in the CMBR that has taken about 13.8 Gyrs to get here. Those photons come from closer regions have already come and gone.