The problem with the expectation that black holes must be a certain size has its foundation in the expectation of it being a positive density mass [ordinary matter] gravitational singularity, in accordance with the Schwartzchild radius calculations. However if we apply the understanding of a black hole as being a negative density mass [dark mater] gravitational well, the size is of no consequence because dark matter is expected to be more energy dense than ordinary matter. This consideration is better understood by thinking of gravity a bit differently. Where the universe's total energy is broken down to as 68% dark energy, 27% mass-energy via dark matter, and 5% mass-energy via ordinary matter, the percentage of energy distribution suggests a differing evolution. Wherein matter, as a whole, is composed of ordinary matter, or positive density matter, and dark matter, or negative density matter, a black hole is not infinitely dense but rather it is a degree of negative density mass.
If we assume that dark energy, being the largest distribution of total energy, represents the foundation for space-time and provides for a net zero inclusion of matter as a whole, then it starts as 100% of the total energy. Upon the advent of matter, as a whole, the distribution of total energy is shared among the universal components. It is a process that maintains the concept of retaining a zero sum net gain, by redistributing this total energy with the complementarily paired positive and negative density matter. This evolutionary distribution also takes on other aspects of universal purpose: dark energy being responsible for the increasing universal expansion of the newly created matter, dark matter insulating the ordinary matter from being torn apart within its dark energy medium, and ordinary matter building and evolving into whatever it can. In effect, dark matter is the 'force' created to insulate ordinary matter, or positive density mass, by warping the space-time fabric of dark energy away from its complementary partner, ordinary matter.
Subsequently when this complementary relationship is severed, ordinary matter is disintegrated as discarded out back into the cosmos, leaving dark matter to remain as a displacement in space-time. This is what happens when matter, as a whole, is separated upon the event horizon of a black hole. Upon this conjecture the black hole does not need to be infinitely dense ordinary matter, but rather it can form to become part of a gravitational well of dark matter. As a degree of negative density mass, the greater the negative mass density then the greater the space-time displacement (or warping). Currently there is no known calculation as to what degree of negative mass density displacement is considered to much or too little. The greater the space-time warp, the deeper the gravitational well and the greater the force of gravitational acceleration. So black holes can be very small or really chock-full of dark matter.
If you would like to explore this relationship better, read 'The Evolutioning of Creation: Volume 2'. Note ramifications of this theory in the science fiction novel, 'Shadow-Forge Revelations'.