Many reasons. One, it's divisive. As much as artists want to preserve their desired look, they also want to cater to an audience that may be put off by it. For some people, stuff like CA and motion blur can actually induce nausea or headaches from prolonged exposure. So a toggle option helps address all that.
Two, it is often used as a crutch to mask other issues, like lack of texture detail or distant LoD. 007, for example, looks pretty raw while driving when you disable the radial blur. You actually start noticing LoD pop in. The other reason is to make it seem you are moving real fast when you actually aren't. IOI gave the toggle option, but another dev may not want their game to look like the cars are going too slow or the pop-in is bad. It's ultimately a developer decision on what to prioritize for each feature, the player preference or their product's presentation. And like most things, it's not black and white, so devs may fall anywhere on that spectrum for each effect.
Three, these are still post processing effects because actual lens simulation is extremely complicated and performance intensive. Real Depth of Field, CA simulation requires path tracing primary rays, something unlikely to happen till end of next generation, if not further out. Simulating motion blur during path tracing is an active research topic and several papers in the last couple of years that tackle it. So just having RT available does not mean actual lens simulation is within reach. But once primary rays are inevitably path traced for other benefits, adding lens simulation becomes a trivial task. And devs that want that will certainly include it.
Toggles should still be possible though. And that would be what I'd prefer too, so people can choose what suits them. I don't believe there is only one right answer on this topic. If a game calls back to the past, lens defects add a layer of immersion that you won't get otherwise. Even at 60 fps, some games look better with motion blur. Film grain always works well in horror. There are genre specific effects that always seem appropriate.
Should Halo have CA? I don't know. It's set in a sci fi future, so anachronistic lens defects don't make much sense, unless they are trying to invoke nostalgia. Given it is a remake of a classic, that may be their justification? I don't think it's a particularly strong one as they aren't going for a retro futuristic aesthetic, and 2001 isn't that long ago either. So the only explanation is a "filmic" look. Shallow as that may sound, it's the dev's prerogative to choose what presentation they want for their game.
Personally, I think CA, DoF, film grain and Motion Blur should all remain toggles/sliders in any game that has them. The devs can choose defaults, but gamers should be allowed to adjust it. Unlike a 2 hour film, this is significant time and interaction spent with the content and you really don't want to make it unpleasant for people who don't like it/can't physically handle it.