The two biggest things for me is the Field of View and removing the gap on the bottom of the headset. I understand that some people may not want to be fully immersed in the headset and still want to be able to see their surroundings, but make this OPTIONAL, with the headset being able to be adjusted to your preferences. Don't make this a glaring difference from Oculus.
As for the 90 degree field of view, this has to be improved. Michael Abrash said that a 120 degree field of view should be possible on 2015. If this requires a higher resolution screen and increase in price, then so be it. But don't gimp the FOV from the beginning. I personally think $400.00 dollars isn't a bad price to start off (bundled with camera). The hardcore gamer and VR enthusiasts will support this from the beginning. 400 is pretty reasonable.
And since Sony is treating this as it's own medium, it needs to be marketed as such. Have a big launch for it, with big launch titles, and bundled with free to play stuff, such as VR demos, Virtual Tours, and other cool experiences to sell people on VR. Obviously have kiosks in stores such as Best Buy, Target, Walmart, etc, to blow peoples minds.
And one last hopeful thing would be to include the VR Cinema within the headset, untethered from the PS4. This will probably require some processing on board and may increase the price, but it doesn't even need to be a rendered movie theater. It can simply be a huge IMAX style screen that floats in front of you, and plays your downloaded movies on a memory stick or whatever. This will be great for the mass market. You can watch movies on an airplane, or laying in bed. The benefits of Sony's HMZ headset, but even better.
And obviously continue to improve the tracking and low persistence, but I'm sure that's the least of our worries. And keep the glowing LED Lights on the front, they look cool and are sure to attract attention.