I agree with you to a point. Now for a bit of background before I get flamed, I am 210# (not someones 85# wife) and I carry a Ruger Security Six .357 as my main duty weapon 90% of the time, and switch to either a 1911 .45 or a Glock 22 .40 for days when we have something exciting planned just for the sake of ammo commonality with the rest of my officers (I get to work with a partner once or twice a month, the rest of the time I'm alone with back up at least 20-30 minutes away) . As per my notes, I have fired 95K rounds through that Security Six since I have had it, and the vast majority of that total has been full power loads, so I am used to it and am comfortable with it. My back up is a Ruger SP101, loaded with the same loads, and it too is an old trusted friend that is comfortable. Now most of my guys give me all kinds of grief for carrying a revolver on duty still in 2017, but the thing just plain works. So I have to disagree with you about the .357 Magnum as a controllable and comfortable round, but I have spent a lot of years using it, and using it a lot, so I am a bit biased.
As to the .41 Mag, I can't say because I have never fired one, but I do have a couple .44 Mags. Now with the .44, I hunt with it, and target shoot with it, and have no trouble with it there, but I know for a fact that I can NOT control it in rapid fire like I can the .357, or any of my various semi-auto pistols. I have tried with various .44s, and I just can't do it. So have to concede you have a very good point at least when I am the one handling a .44 Mag. I have seen a lot of people try to show me how well they can control a .44, and only one has succeeded. I won't give a name, but the man is a close friend of mine, who bought his Ruger (Super?) Blackhawk .44Mag between deployments to Viet Nam, and has the paperwork signed by his old CO approving him shipping it into Viet Nam and back, and has been carrying ever since I have known him. He also occasionally carries a Redhawk .44 Mag, and when using a timer, his recovery times with that Redhawk are .04 seconds slower than mine with my Security Six. I don't see how he does it, but he does. To me, the .44 Mag is a hunting gun, and I treat it the same way as I do my .45 Colt that gets fed with "Ruger-only loads.
To me it is a matter of use what you can control. My limit, at least for now, is a .357 Mag, but a 9mm, .40, or .45 are easier, but also not quite as efficient and don't provide the range I may need. I say that because I work in a rural area, and have at least as much of a chance of having to put down a charging bull as a charging meth head. Also, with the open spaces involved, I may need a longer reach than most semi-auto rounds are capable of. Yes, I have a rifle too, but may be away from my patrol vehicle without it when it's needed, and the .357 increases my chances of getting back to it. Now if I could control a .44 Mag as well as I do a .357, I would carry that instead, but I can't, and am honest enough with myself to admit that.
As to over-penetration concerns, while valid, those can be minimized by ammo selection. For example, and I will use LE as an example again here. Most agencies that USED to issue the MP-5 in 9mm, have switched over to the AR-15 in .223, in large part due to that the .223 is much safer in an urban environment and less likely to over-penetrate than the 9mm. Just as a 9mm FMJ will penetrate deeper than a 9mm HP, the same is true of any caliber we can think of. I don't want to get into a brand X is better than brand Y argument here, but we can all agree that some types of ammo are safer in our guns than others. And each of us has to figure out where our personal perfect load is when we balance expansion and penetration. Nothing is perfect, and what is perfect for one of us is either under or over powered for another, but thankfully we can all choose. If I was in an urban environment, I would probably switch to either my 1911 or my Glock 22 permanently, as I would be less likely to need either the range to reach across a field or the extra power that I want in case of livestock.