12:50 into the video says that it's 2400x1080. I think Birdwell was talking about what resolution it appeared to be at because of proximity to the eyes.
Edit: Eek, just reread what you said. Will leave my post so people can see the res, anyway.
-
Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.
Valve engineer explains how Lighthouse (positional tracking) works (Tested interview)
- Thread starter Durante
- Start date
12:50 into the video says that it's 2400x1080. I think Birdwell was talking about what resolution it appeared to be at because of proximity to the eyes.
Edit: Eek, just reread what you said. Will leave my post so people can see the res, anyway.
ashecitism
Member
Well, Oculus and Morpheus gather tracking data by processing 60fps video stream by extrapolating 3D position from the 2D movement of IR dots. What Valve is doing is much more elegant, PC just receives stream of ID tags and timings in text form and rest is simple math.
So far everyone has very capable 3D tracking systems, but Valve has most streamlined solution for both sitting and walking in pretty large environment of up to 4.5x4.5 meters [they just need to make that headset less bulky]. Sony wins at ergonomics and complete ecosystem [plus some games will use 120fps rendering], Oculus has 1440p screen [~35% more pixels than Valve, but we don't know what pixel configuration both of them are using] but lacks motion controllers.
EventHorizon
Member
Lots of good info going up on twitter right now.
Alan Yates
Chief Pharologist at Valve Corporation. Tragically boring general purpose geek.
https://twitter.com/vk2zay
Alan Yates
Chief Pharologist at Valve Corporation. Tragically boring general purpose geek.
https://twitter.com/vk2zay
- One key missing component is actually the back-haul, most low-latency ISM band solutions are too narrowband or unreliable.
- The future of tracking systems for VR is not a single tracker, but combinations of the right tech for each part of the job.
- Lighthouse base station emissions are Class 1 (unconditionally eye safe), while it contains class 3 sources they are enclosed & interlocked.
- the plan has always been to ship hardware with lots of programmable logic so we can just keep shipping improvements.
- One base station is enough to track a rigid object with an IMU, five sensors need to be visible to acquire a pose, but it can hold with less
- Two or more base station visibility lets you track a single sensor, which may be on a non-rigid object.
- There are many modes Lighthouse can operate in, so far you've only seen one.
- Lighthouse was designed from day one to be scalable, you can in principle concatenate tracking volumes without limit like cell towers.
- While it can be used with single sensors and without an IMU, the IMU reduces its latency.
- Like all optical systems occlusion is a limiting factor. Spatial redundancy is very helpful, but finger tracking will be challenging.
- The most obvious approaches to finger tracking are mechanical, inertial or magnetic, fused with optical ground truth and a skeleton model.
- For many purposes finger tracking is actually less useful than a controller. Little is more haptically positive than a snappy switch
- Lighthouse is not the dual of a camera: The "dots" know their ID; the hardware gives you angles directly. Pose calc' can be decentralised.
- I'm glad to see @oculus talk about using a compositor model too, very handy when a VR app crashes. Makes apps easier to write too.
- I think we will eventually end up with an X Windowing System-like model, which runs asynchronously and VR apps connect to.
odds on Oculus moving to lighthouse as their tracking solution? Solves two problems for them - not having any controller solution as yet (they adopt steamVR controllers) - and having a limited tracking volume using their current camera solution.
Good news that a single lighthouse is enough to track larger objects - should be enough for seated VR with a couple of controllers.
Also pretty exciting to hear that two or more lighthouses could track single sensors - who'll be the first to start experimenting with kneepads or gloves?
Good news that a single lighthouse is enough to track larger objects - should be enough for seated VR with a couple of controllers.
Also pretty exciting to hear that two or more lighthouses could track single sensors - who'll be the first to start experimenting with kneepads or gloves?
I'm surprised they've been able to get this kind of performance out of a supposedly low cost solution. This is the kind of thinking that VR needs and will thrive from. Very glad to hear that they're being open with the tech, and excited to see where it all leads. Fingers crossed that they figure out movement across large landscapes. I'd think some kind of repositioning and trickery of the mind would be the obvious path to finding a solution.
odds on Oculus moving to lighthouse as their tracking solution? Solves two problems for them - not having any controller solution as yet (they adopt steamVR controllers) - and having a limited tracking volume using their current camera solution.
Good news that a single lighthouse is enough to track larger objects - should be enough for seated VR with a couple of controllers.
Also pretty exciting to hear that two or more lighthouses could track single sensors - who'll be the first to start experimenting with kneepads or gloves?
I don't know about how willing Oculus will be in using other's tech, regardless of it being open, but we'll know in time. Either way, this being out there can only be a good thing. I'm waiting impatiently to see what Oculus has been working on so far. With what Valve has shown, it seems like they will need to bring their A game.
EventHorizon
Member
I think it all comes down to the question of "Does Oculus think they can get a tracking solution as good as Valve's within a year?" If not then they just need to accept Valve's solution. I don't see how they can compete if they tried to push an inferior solution.
I mean, hopefully they have some tricks up their sleeves, but Valve's solution really does seem to be the best thing going. They've talked about VR being a collaboration and the necessity of standardization, and tracking is certainly an essential piece of the puzzle.
It might depend on price. If Valve's solution proves to be a more expensive option, then Oculus might prefer to not go that route in an effort to keep the headset as reasonably priced as possible. Producing expensive VR solutions is not going to be a great way to push adoption.
Yes, but it also depends on the cost of the lighthouse solution.
EventHorizon
Member
No doubt that price will play a part, but if price was the main driving factor then I think Project Morpheus would beat them both. Anyway, I initially thought that thr Vive was going to be a lot more expensive than the Rift, but now I'm not so sure. If there is a difference in price it will come down to the cost of the Vive's laser base station vs. the Rift's camera.
jediyoshi
Member
Whatever Oculus does from this point forward will be nuts. Either they: double down on being a seated experience with inferior tracking -or- do a complete 180 from THE OCULUS RIFT IS A SEATED EXPERIENCE line they've held to from day 1, in which case they spend more time developing their own solution hoping not to have to be pushed past this year for CV1 getting to market. I can't see Facebook happy with not having the reigns on whatever the defacto standard is for the Oculus and going with Lighthouse.
or they simply use this tech that Valve is offering to share. It makes no difference if you're sitting - Valve simply showed standing/walking demos to show how great the tracking was. 95% of people will be sitting at their desk in front of their PC - it'll still track that perfectly. A much better solution than a webcam on your monitor and you falling outside of its vision cone all the time.
jediyoshi
Member
But that's exactly what I was already alluding to? I'm just comparing their existing tech as it's been announced.
Morpheus is a console solution. Slightly different arena.
2 base stations plus two motion controllers plus the headset - I don't think we have any idea what its going to cost.
Betta Lines
Member
I wouldn't say it would be 'complete 180'. The 'seated experience' line is almost a meme at this point, often said by Oculus staff with a wry smile. Both Valve and Oculus are trying to show the impact of presence, and both have chosen to do it with standing demos, as that is the most effective way of convincing people. The practicalities of setting up either Crescent Bay or Vive demos in a typical living room or bedroom will remain problematic. It is likely that both Vive and Rift will run most of the same software, so you can be sure there will be plenty of seated experiences to be had on Vive, and plenty of standing ones on Rift.
efyu_lemonardo
May I have a cookie?
Sounds like they did the right thing. This is the most logical extension of the fundamental principles of the wiimote.
It's basically a multi wiimote radar system.
It's basically a multi wiimote radar system.
Pacote-san
Banned
Awesome explanation video, i so want ANY VR gear (not counting Gear VR) to come out this year
jediyoshi
Member
I just don't see their 'seated experience' shtick as being coy at all. It really feels like they're hyper cognizant of the potential liabilities and want to avoid them, even regardless of their tech's capabilities.
There'll definitely be overlap in most initial VR content, it's just not as smart a framework from Oculus' side from there on out. Unless their camera can also accommodate peripherals, you have the extra problem of a bunch of disparate techniques aimlessly trying to solve the same issue of tracking ontop of whatever they're trying to offer themselves. Whereas on the Lighthouse side, any traditional controller manufacturer can stick sensors on a peripheral and be immediately up to speed with the work something like STEM has done.
PoweredByHentai
Member
Betta Lines
Member
Are you suggesting that the Rift won't support any of the standing experiences offered by other headsets, or that they will, just with a liability disclaimer? Because you can be sure the other headsets will be plastered in disclaimers too...
Well he didn't go into the exact details of the way Valve have done it with photo cells and timing, but he explained the principle of inside-out tracking very well.
jediyoshi
Member
There's a difference between a hardware's capabilities and what they actually aim to accomplish and take advantage of. As it stands, they're taking the laissez faire approach of leaving that up to developers and add on peripherals to tackle as they wish. I don't recall any anecdotes of them approaching developers and indicating they aim to specifically work that aspect. Regardless, we already know that it won't be on par by virtue of the current camera's limitations.
Here's the other issue you run into: https://www.youtube.com/watch?v=U88VXJv_IRw&t=73 Dude naturally tries to squat but doesn't get any further feedback. If the standard had already been set, developers would be much more inclined to actually accommodate it rather than just what the headset itself will allow. By virtue of there not being standardized tracking and not ensuring accuracy when dealing with more obtuse poses, content creators are going to be more nebulous on how and when to incorporate it into experiences involving the entire body.
It really has very little to do with the Wiimote (though it's closer than the Move). The Wiimote uses a camera, there are no cameras here (that's what makes it so efficient, fast and scalable. And hopefully inexpensive).
DouglasteR
Member
no? You said if they double down on the 'seated' part with inferior tracking. I was saying that they could switch to this lighthouse for superior tracking - they can still have their 'seated' message. There are still clear benefits to this approach even when seated.
ScepticMatt
Member
only for the top of the screen
Norm clarrified this in his post on reddit.
http://www.reddit.com/r/oculus/comments/2y0vqy/testeds_steamvr_htc_vive_prototype_handson/cp56ipr
jediyoshi
Member
I didn't say anything contrary to that. I already accounted for what your original response was in the post you were responding to (though it should have been the preface for the first point), but I'll expand.
I can't see Facebook being happy with having purchased a $2 billion company, which was basically an investment in VR's future, and them not owning the patents and licenses on the tracking technology of their flagship product. It's really not as 'simple' as them dropping all the R&D they've worked on and then hopping on board another wagon.
mhayze
Member
My understanding of how that works - based on some other slides, it sounded like there's 2ms persistence per pixel, but it's a rolling shutter type display or simulated scanline (CRT style). Basically, not something you could easily simulate with IPS without a hardware scanning backlight (which should be appearing in real monitors soon, but is only available on TN monitors so far). On an HMD size display that would mean rows of LEDs serving as a backlight for the IPS display, probably approaching or exceeding the cost of a small OLED at this point.
But they were never going to persist with a camera solution forever. We knew that already. You think they aren't also in R&D stages with their own Lighthouse-like solutions? Palmer and Carmack have talked about and demonstrated this same concept before and they have several important ex-Valve VR guys onboard, so none of this is exactly going to be a surprise to them.
Betta Lines
Member
I see what you're saying, but I think it's too early to say what you're saying. It's still very early days and Valve are offering what appears to be a viable standard for positional tracking. But everyone is trying to solve this problem, including Oculus, and so we should wait for what they have to say. Whether they are 'targeting a seated experience' doesn't change the bigger picture of how VR is going to work over the next few years, so they will either have their own solution or embrace Valve's, otherwise they will be left behind, and there is no fear of that.
jediyoshi
Member
I'm sure they've covered every possible angle a dozen times over directly with Valve themselves, but then it doesn't align with any messaging they've indicated, or the fact that they've at least said no more developer kits. I would imagine if they truly aimed for tech that was much more capable, they would have been on the messaging ball sooner spreading the word, getting content creators to make more relevant content lined up to coincide with release.
I really believe there's a huge risk in going for room-scale experiences, and it might simply not be a risk Oculus is willing to take.
Some people (I'm not one of them) even doubt the viability of getting people to use headsets at all. But dedicating a large space or even an entire room to it is asking for a far larger commitment. I've been thinking about which room to use ever since the Vive announcement, but even I can't believe that that will be mainstream.
Some people (I'm not one of them) even doubt the viability of getting people to use headsets at all. But dedicating a large space or even an entire room to it is asking for a far larger commitment. I've been thinking about which room to use ever since the Vive announcement, but even I can't believe that that will be mainstream.
Betta Lines
Member
On the contrary, they've hinted that a devkit specifically about input could be on the way.
Cynn
Member
I agree. I think room movement will be a novelty for now.
TheHighFlyingBirds
Member
Will be interesting to see how accurate it is and how well it works.
My final year project was based on tracking, which used the response time to determine the location of a receiver in 3D space. This is far more refined though as I had to use 3 base transmitters; the solution to use multiple measure points on the receiver is genius. Would love to see the calculations and a bit more info.
My final year project was based on tracking, which used the response time to determine the location of a receiver in 3D space. This is far more refined though as I had to use 3 base transmitters; the solution to use multiple measure points on the receiver is genius. Would love to see the calculations and a bit more info.
Big Takeover
Member
The hardware risk is minimal because it scales down so easily. Full room software? That could very well turn into a niche thing, both to develop for and use. That's a risk they'll happily take. If people don't want to walk around to play video games, no problem. They're not actually tied to it. If it takes off, Gabe will move up a whole bunch of spots on the richest list.
No, he misunderstood. He thought the lasers were painting your room with a known pattern of IR dots, which were then picked up via cameras on the headset. This is inside-out tracking.
Lighthouse is outside in. But instead of trying to track dumb IR dots with a camera, the dots are *smart* and know exactly when they've been scanned.
This is an excellent analogy. For those who don't know, old-school lightguns worked like this:
The picture on a CRT is scanned out linearly by an electron beam. If you filmed one with a very high frame-rate camera, you'd see a streak of light racing from left to right as it moved down the screen, painting each scanline.
The lightgun itself just has a simple directional light sensor in it. When the trigger is pulled, the video output is momentarily switched to full brightness. The computer knows where the electron beam was pointing when it was on a valid target. If the lightgun senses the pulse of brightness at the correct time, then the computer knows that it was pointed at the target.
It may sound crazy to accurately time something so quick, but remember that to a computer, 60Hz is excruciatingly painfully slow. Even a NES can figure it out.
That's what makes this solution so robust - the computational requirements are NES-level, so you can have as many "smart dots" as you want, and the latency is incredibly low.
Compare that to inside-out image processing approaches. Even with IR dots an IR camera (like the wii), there's still an image processing stage where the computer has to interpret the arrangement of the dots and figure out what it's looking at.
Lighthouse just *knows* with absolute certainty where each dot is, using a couple of trigonometry sums.
If might not be for CV1, but they will go with this sort of system eventually. They've talked about it, they know its ultimately the better way to go, but it might not be that important for the first iteration to them.
And you're getting too caught up in the 'get up and move around' stuff. That's not going to be the main focus of home VR anytime soon. The seated/stationary experience will be.
IMACOMPUTA
Member
Wrong
The dots are dumb.
Fair point. Although, I thought oculus has almost no patents anyway? And a camera watching some IR dots wouldn't be patentable either?
One of the guys from tested already walked into a wall because the virtual barrier didn't pop up very quickly.
Right. Oculus aren't in this to get a monopoly or unbeatable lead in the headset hardware. They just want to help and be a part of the growth of VR tech. Business wise, they are more focused on the content/ecosystem side.
Yep, it's really awesome.
I still want to know more technical details though -- e.g. is there only one light frequency or distinct ones? How long and how often do the base stations pulse? What's the actual upper limit in tracked points? How many sensors do you need for accurate tracking with 2 base stations, and does this number change if you add more?
Well they're not little computers if that's what you mean. But they're sensors that know when they've been hit by the scanning laser. Smarter than an always-on LED.
Speculation:
Each base station uses a different frequency of light
Max. tracked sensors is essentially infinite, because no two sensors can occupy the same point in space, they will always produce time-distinct readings.