Just to add to drwaynes explanation ...
In case you want to think of it according to Newton's laws, you can try to resolve the forces involved and then from F=MA you can figure out what happens. Of course in a rocket the M is changing as you throw mass overboard but still the concept remains valid. Let's start with an inflated balloon. You blow it up and pinch off the end (lets call this end the back end). If the back remains pinched off the forces internal to the balloon are the same everywhere. The gas molecules inside push just as hard to the right as the they do the left, up the same as down, front = back, etc, etc. Everything is in balance and so the forces cancel out. The net F = 0 so there's no A and so no motion.
Now open the pinched end. Gas rushes out through the hole. The force on the balloon at the opening is lower that the force on the balloon directly opposite it. While all the other forces remain in balance (right = left), there's now a net difference between the forces to the "front" and to the "back". With some nonzero net force there's now an acceleration and thus motion.
In the case of a shotgun when the charge goes off gas is created in the barrel. It pushes "sideways" on the barrel itself (equally outward trying to expand the barrel) and on the shot column and on the breach face. The push on the breach face pushes on the stock and eventually onto your shoulder where you feel the recoil. Since you are "big" relative to this force, you accelerate a little to the rear. The forces on the shot column are bigger (relatively speaking) and thus it accelerates pretty hard (up to almost supersonic velocities) until the column exits the barrel and the gases pushing it can "escape" out to the sides.
A rocket is like the balloon but with a burning propellant that makes a lot of (hot) gases (like the shotgun). While it's easier to compute what happens using momentums, the basic F=MA (where M and A are both variables) equation still holds and makes commonsense.
Hmmmm, not so sure this actually makes things any clearer. Oh well ....