momentum = mass * velocity
If you're hitting a framing chisel to hollow out a mortise in an oak beam, bigger is probably better. But the goal isn't always to hit the chisel as hard as possible. Given the equation above, you can see that you can achieve small amounts of momentum with a heavy mallet, but very small changes in velocity will produce relatively large changes in momentum because the mass of your mallet is so large.
If you need fine control over the energy you deliver to a chisel or gouge, a smaller mallet gives you more control within a given range by letting you use a wider range of velocities. The maximum momentum you can achieve with that smaller mallet will be less, of course, because there's a limit to how fast you can move the mallet. But the difference in velocities required to generate gentle taps compared to firm taps will be greater, which makes it easier to regulate.
It's the same reason that hammers come in sizes ranging from 6oz or less all the way up to a 20lb sledgehammer: different tools for different jobs. You could drive brads with a big sledgehammer, but you'd likely bend a number of them or hit your fingers because it's a lot harder to control than a small finishing hammer.