Little example:
In 2, the encounter where the area is taken over by tons of soldiers including a couple chaingun "mutants." Pretty sure at some point blue guardians show up.
This encounter fills me with excitement before I get in there. How many guys can I stealth? Once I'm discovered, what are my tactics? Do pick off enough regular soldiers to get the crossbow guys to show up, so I have a powerful weapons against the chaingunners? Do I find RPG guy and use his stuff and grenades for the mutant instead? Do I let them fight each other? Maybe RPG a chaingunner, then chaingun the guardians? Technology > magic?
No matter what I generally choose, I know I'll be able to move around and react to events while overall flow of battle is one of the ones described above. Even if my stealth is detected after 1 guy, I can smoothly cause enough mayhem and switch tactics to make for a recordable battle.
Facing an encounter similar to that in 4/Hard means I have to steel myself. This is going to be repetitive. AND it's going to be stressful, in an annoying way. Plan out the stealth path, and then do it methodically. Toggle between 2 different approaches to keep the drudgery/trial-error factor down a bit. If I get caught right at the start, it's over. Everyone will shoot at me from EVERYWHERE with perfect accuracy, so my goose is cooked. I have to pick guys off in some grass very carefully and never gamble.
Finally, having stealthed in satisfactory fashion, I can take on the remainder. This isn't hard, because only a few guys are left. If I get killed, I might find myself restarting from a checkpoint in the MIDDLE of the encounter, where things are quiet for no reason, and half the soldiers are dead and maybe the guardians are now here (the whole too-many-checkpoints-in-too-difficult-mode paradox I mentioned in OP). This is annoying, but restarting the encounter just seems obnoxiously un-immersive, so just go with it.
Besides, if special enemies like guardians are involved, no way 4 would let me really improvise when they're added to the battle. They'd be so deadly (as are the "mutant" chaingunners) in the given level design that I can only start engaging them right after a checkpoint with guaranteed pre-scripted cover.
I'm tired and sleepy, so I may be rambling and being unclear; my apologies. I'm just trying to convey that this encounter in 2 (on Hard), which I just played a few minutes ago, filled me with care-free anticipation when the soldiers filed in. Then I imagine what I would feel in 4 (on Hard, not Moderate) and it's more like, "how do I get through this without too many deaths while also using some semblance of creative on the fly tactics"? Not bad... but anything but "care-free anticipation." And actually it takes me out of the world, my brain screaming VIDEO GAME at itself the whole time.