I haven't look closely at the released pictures yet but if they're smary it should be on the top of the console to make it accessible.
Anyway. The only downside I see is that carts seems to requre longer leadtime and possibly more upfront cost for devs, making restock of high selling games more difficult.
I think that if the carts are read-only (other than the space to save games), then patches would be my biggest question / potential downside, in addition to cost. It has been a while, I think, since a
home console had non-patchable games. In particular, will 3rd parties be OK with not being able to patch their games? It could lead to multi-month delays for the switch version of cross-platform games. Of course the 3DS has this scenario now, so I guess it will just scale to home platform games as well, but I'm having a hard time imagining certain game types being OK with no patches ever, if this is truly to be a home console that you can take on the go (as opposed to a new 3DS that plugs into a TV.)
Nintendo themselves take their time to test their games to the nth degree, and I think they will be OK, 3rd parties, less so.