I'm always gonna have a handgun on me, period, holstered on a belt attached to my pants, worn on my body. I'll sleep with it, eat with it, swim with it, and take a dump with it. You will always have to set the rifle down for something, always. The one on your hip is always better than the one you leaned against a tree 2 yds away.
I know survival conversations somehow always wind up in a Mad Max, or Mogadishu type scenario. But a higher probability is that it will not be yourself against a well armed fire team. In which case a good handgun round delivered at appropriate ranges and well placed will definitely take the fight out of someone. Seriously, using a coked out heavyweight as the straw man to justify why you need a 45/70 to stop a person is just not realistic. More than likely the people you encounter in a survival scenario are also surviving ... tired, hungry, sick, scared, and not able to mend themselves readily. Any gunshot wound, by any caliber, will be fatal, and the knowledge of that will do more damage than the round itself. One shot stop is what a cop wants or a soldier wants it a modern context; me, I want them to stop fighting and leave me alone, crawl under a bush, and die a long painful death from septic shock. I don't plan on patching them up, or sticking around long enough to see what happens next. If I think it necessary, I'll walk over to the rifle I leaned against a tree (since chopping wood, or checking a trout line with it slung across my back is a PITA), and be ready for the next thing.
I'll also have a bunch of other things in my pockets, on the belt, in a chest pack ... because my backpack may be set down, my vehicle may be far away, and my house may be under water or a pile of ash. The knife in my pocket is better than the awesome $400 uber bushcrafter special strapped to the outside of the pack behind the seat in my truck.