It would make sense for it to be a separate app, with whatever underpinnings it needs built into the firmware. A large number of "stability" updates on the PS3, PSP, and Vita were for updating the emulators to fix bugs and allow more games to be added on the stores. With the PS4's firmware update schedule I think they'd rather update a single app not everyone is using. The downside is you'll need internet to get the app, but I think that's a trade-off most people are willing to make now.
Well, considering that we haven't heard anything about CD-based emulation, it may be entirely possible that you could only download the games anyway, thus the internet being a requirement would be for both the app and the games themselves.