With consideration to AAA gaming trends how could Metroid be successful? What genre would it fit into at this point?
Not all games are destined to be "AAA multi-mullion sellers". Metroid is not different than something like Dark Souls and Bloodnorne. It's base game design has many niche elements that casual gamers or players with low attention spans can't comprehend. If you change those elements you simply end up with something that doesn't really play like Metroid.
This is what mostly happened with OtherM and even that bombed because they made it into a simple action shooter but there were much better action shooters on the market.
Nintendo has enough IPs that are mainstream successes. They have Mario and Zelda. These should be enough to sell multi-millions. And then they have the more niche IPs like Metroid, Pikmin, F-Zero, etc. They don't need to change those in order to make them more mainstream, they don't need that. They should just aim for the correct market and budget accordingly.
Heck, Metroid doesn't even need to be a high budget game in order to be a better Metroid game. And here's why.
-The game is at it's best when you are isolated. That means it doesn't need a bunch of characters to converse with so there is
no need for voice acting.
-The game is at it's best when the story is presented through the environments in a more subtle way. So
it doesn't need elaborate cutscenes.
-I don't remember any Metroid game having an orchestra for it's soundtrack and they were fine so far (except for OtherM which barely has any music). I believe that strange sounding alien-ish samples work better for this game instead of real instuments. So
it doesn't need orchestral music, just one talented guy composing with his computer.
Just with these you are cutting a huge amount of production costs. Metroid doesn't really need anything more than a good designed map, a great art direction and good game mechanics/ideas. Stuff that even indie developers can sometimes deliver. That's all Nintendo has to do really. I mean, how expensive was the first Metroid Prime to produce? And that even had a very advanced graphics engine to move it.