Risk due to radiation, hard vacuum outside of any structure in space, reliability on machinery for all things necessary for life, high potential energy when orbiting the earth, risk due to collisions with space-faring objects...the list goes on. There are obviously many engineering issues to be addressed when it comes to building anything in space. The fuel is what makes launching costly, as it requires so much to do so little. When we are able to reliably store antimatter (probably positrons because it looks like we should be able to produce a significant amount of them), we will see an enormous boom in the number of humans with access to space.
Unfortunately, while antimatter is by far the most energy efficient fuel (that we know is physically possible), there is a very high price to pay if you don't treat the fuel with the utmost respect; those massive liquid fuel tanks we use now are dangerous, but the small antimatter-matter tanks which we would need to use are pretty much the same in terms of explosive power, only it is condensed into a small can-sized tank. Better hope there's no power failure on your spherical capacitor tank, because once the charge keeping the positrons in place dissipates, your life will be over before you know what happened...