Harry, it depends on what is meant by the "Big Bang". Unfortunately, it appears that it seems frequently to include both the unknown metaphysical assertion at "time = zero" that there was a "singularity" and the scientific follow-up beginning one minute fraction of a second later.
There are some shades of meaning still open to discussion, but the above puts the question into first lines of consideration.
As things stand, it is totally unscientific to include both a metaphysical element and a scientific follow-up within the same term. Thus, "Big Bang" cannot be correctly used if it includes both elements. My suggestion is to use "Big Bang" for the scientifically adequate part of any discussion, and to use "t = 0" to cover the initial metaphysical element.
The crux of the question is that division by zero to produce an "infinite" quantity, is a mathematical operation, not a scientific procedure. For example, in mathematics, equations that are derived to explain natural processes may include 1/x. When x is equal to 0, there is a divide by zero error which may result in a singularity, e.g., a finite mass in an infinitesimally small volume, yielding an infinite density. In physics, however, other things usually obtain which make these equations break down as x gets close to zero, preventing these singularities from actually occurring."