• Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.

Valve engineer explains how Lighthouse (positional tracking) works (Tested interview)

D

Durante

Member
Finally!

https://www.youtube.com/watch?v=leg2gS6ShZw

The suspense was killing me.

So, the sensors are simple photocells, the base stations sweep the room with lasers (thus "lighthouse") and from the timing of when they hit specific photocells (and due to the unique geometry of their placement) they can tell their absolute position very exactly and with absolutely minimal latency.

Amazing stuff.
 
A

ashecitism

Member
Really good vid. Just finished it.

Lighthouse and the controllers were the stars basically and not so much the headset.

Regarding the headset Will says it's OLED, but it's LCD, right?
 
M

Mrbob

Member
I'm pretty sure I'll buy this, yet still a little apprehensive. It sounds amazing, however I'm not sure how much I want to be walking around the room waggling my arms around. I'll keep an open mind though as the Tested guys gave it high praise.
 
N

Nzyme32

Member
It's hard not to be taken in by all their enthusiasm and excitement they have for the experience and possibilities. The only real issue seems to be having a decent enough space for it
 
Z

Zaph

Member
Absolutely incredible.

The Tested guys have tried every version of VR out (I believe including the latest in-house Rift), yet seeing them this blown away by Valve's solution is insane, especially considering all the talent at Oculus.

Can't wait.
 
G

Graphics Horse

Member
Sounds like each sensor works a bit like an old fashioned CRT lightgun then, so more like a bundle of lightguns where the screen is your room.
 
D

Durante

Member
Exile20 said:
Wow, does this put the other VR methods to shame?
Click to expand...
Of all positional tracking methods I know of, it certainly seems to have the best combination of tracking volume, accuracy, latency, number of tracked points, affordability and computational overhead.
 
B

Betta Lines

Member
ashecitism said:
Regarding the headset Will says it's OLED, but it's LCD, right?
Click to expand...
They (Valve/HTC) appear to be avoiding clarification on this officially. They (Tested) got the resolution the wrong way around, saying it is 2400x1080, so they might have got this detail wrong too.
 
D

Durante

Member
Betta Lines said:
They (Valve/HTC) appear to be avoiding clarification on this officially. They (Tested) got the resolution the wrong way around, saying it is 2400x1080, so they might have got this detail wrong too.
Click to expand...
I expect LCD, otherwise they'd say so.

One thing the Rift got going for it.
 
Z

Zaptruder

Banned
Will: "I might sound too effusive saying that this is the future of gaming... but it really is."

No Will. It's not just the future of gaming; it's the future of computing you're describing there.

With that said, big props to Valve for cracking that accuracy/money and computation cost puzzle.

But the controller design can stand to see significant improvement IMO. 2 buttons and a touchpad?

I like the touchpad... but it needs triggers, touchpad/thumbstick + 4 face buttons. Tactile and muscle memory is just so important for controls. It's not like touch screens either where you're at least looking at the screen where the buttons are (also they rely a lot of gesture, which activates proprioception and tactile feedback).

I don't know what the 'haptic touchpad' feels like... but it can succeed so long as it can let the user differentiate 'buttons' with light touch, without actually activating the buttons themselves... and it can of course provide a different feedback for when you actually push 1 button or two buttons or more buttons (similar to how a real controller will feel different between resting, pushing and pushing multiple buttons).

If it can't do that... then it'll either have to improve, or step aside and allow for real buttons.

Also, it needs velcro straps and a touch strip on the side. The idea is to allow you to open and close your hands without dropping the controller - allowing for natural grasping motion.

With that sort of setup... then you have a controller that can actually replicate the critical functionality of traditional controllers, full motion controllers and finger tracking controllers.

As it currently stands... it's a 'VR only' controller - which is limiting, and fails to take the VR paradigm as far as it could potentially go.

But it'll be fine as a first generation VR experience. I'm just saying there's room for improvement here.
 
F

Foggy

Member
Wow, I didn't even know you could set up more than 2 lighthouses and have multiple headsets in the same space. I can't imagine how feasible that is for even your most rabid consumer, but that's pretty amazing nonetheless.

I echoed this in the other thread, but I'm curious how applicable lighthouse is and how much it'll even be used outside of demos. The Valve rep briefly touched on playing with scale and teleporting, but it sounds like there's no real clear path to implementing this in a robust game outside of simple room experiences. I'd love for brilliant minds to show me the way though, since I don't see how this could replace seated experiences for the vast majority of VR applications(or at least in regards to gaming).
 
Nabs

Nabs

Member
Will buy Lighthouse enabled gloves so I can minority report the fuck out of everything.
 
N

Nafai1123

Banned
Foggy said:
Wow, I didn't even know you could set up more than 2 lighthouses and have multiple headsets in the same space. I can't imagine how feasible that is for even your most rabid consumer, but that's pretty amazing nonetheless.

I echoed this in the other thread, but I'm curious how applicable lighthouse is and how much it'll even be used outside of demos. The Valve rep briefly touched on playing with scale and teleporting, but it sounds like there's no real clear path to implementing this in a robust game outside of simple room experiences. I'd love for brilliant minds to show me the way though, since I don't see how this could replace seated experiences for the vast majority of VR applications(or at least in regards to gaming).
Click to expand...

This was touched on in another thread, but Valve is working on redirected walking, which supposedly can be used in surprisingly small spaces.
 
Z

Zaptruder

Banned
I'm looking forward to them releasing just the diode bits with velcro - strap it to your knees and elbows. Put on a vest with rumble and strap it to that. Full tracking. Should be good for full body emote and motion capture.

Nafai1123 said:
This was touched on in another thread, but Valve is working on redirected walking, which supposedly can be used in surprisingly small spaces.
Click to expand...

I'm interested to see how well it works.

I mean... it seems like there are a good number of situations in which you can break it.

Like getting near a physical wall, then turning directly towards the wall.

But they do have the 'holodeck' grid when you get too close to obstacles, so I guess that's how they're planning on solving for it.
 
A

AngryMoth

Member
I think lighthouse is neat and you could do a lot of interesting things but I don't see physical movement being the primary input for navigation in most games, just isn't practical. Still, can't wait to try it for myself eventually.
 
B

Big Takeover

Member
Foggy said:
Wow, I didn't even know you could set up more than 2 lighthouses and have multiple headsets in the same space. I can't imagine how feasible that is for even your most rabid consumer, but that's pretty amazing nonetheless.

I echoed this in the other thread, but I'm curious how applicable lighthouse is and how much it'll even be used outside of demos. The Valve rep briefly touched on playing with scale and teleporting, but it sounds like there's no real clear path to implementing this in a robust game outside of simple room experiences. I'd love for brilliant minds to show me the way though, since I don't see how this could replace seated experiences for the vast majority of VR applications(or at least in regards to gaming).
Click to expand...

That's not going to be their ambition. I don't think the seated gaming experience is ever going to be completely replaced. This just takes it in a whole new direction for new and unique gaming experiences.

The beauty of their system is that it can scale back down to seated VR with a traditional controller no problem. They have the highest ceiling as to what's possible, but that doesn't mean everything has to be funneled down that path.

The challenge will be building software that'll do it all. From PC w/monitor, to seated VR, to Lighthouse VR. Can it be done?
 
S

scitek

Member
Big Takeover said:
That's not going to be their ambition. I don't think the seated gaming experience is ever going to be completely replaced. This just takes it in a whole new direction for new and unique gaming experiences.

The beauty of their system is that it can scale back down to seated VR with a traditional controller no problem. They have the highest ceiling as to what's possible, but that doesn't mean everything has to be funneled down that path.

The challenge will be building software that'll do it all. From PC w/monitor, to seated VR, to Lighthouse VR. Can it be done?
Click to expand...

Put my character in a fucking wheelchair and I'll move around with a controller. I'll learn to get used to it for this shit.
 
cyberheater

cyberheater

PS4 PS4 PS4 PS4 PS4 PS4 PS4 PS4 PS4 PS4 PS4 PS4 PS4 PS4 PS4 PS4 PS4 Xbone PS4 PS4
Sounds amazing. Thanks for posting that.

I really can't wait for this to become a reality for me. I think I'm going to have to move a few things in my living room but it will be worth it.

This it super exciting stuff. The biggest paradigm shift in gaming in my lifetime. I'm fully on board.
 
R

RCSI

Member
I'm imagining a ride built out of Lighthouse, where instead of taking precious space to build animatronics, you have an "endless" virtual space along with the appropriate movement. It would be neat to try with this method of tracking.
 
D

Durante

Member
Big Takeover said:
That's not going to be their ambition. I don't think the seated gaming experience is ever going to be completely replaced. This just takes it in a whole new direction for new and unique gaming experiences.

The beauty of their system is that it can scale back down to seated VR with a traditional controller no problem. They have the highest ceiling as to what's possible, but that doesn't mean everything has to be funneled down that path.

The challenge will be building software that'll do it all. From PC w/monitor, to seated VR, to Lighthouse VR. Can it be done?
Click to expand...
I don't think it will necessarily be the same software across all of those. But that's fine.
 
J

jmga

Member
Durante said:
No, you can do low persistence on LCD.
Click to expand...
2ms of persistence?(I have just found this).

img_54f7aea5ed84b.jpg
 
D

Durante

Member
With 11ms frames and displaying for 2 ms you have 9 ms for pixel transitions. You can even do that with IPS these days!
 
B

Big Takeover

Member
Durante said:
I don't think it will necessarily be the same software across all of those. But that's fine.
Click to expand...

Certainly for 95% of the titles it won't be all three, but I'm very interested in seeing how Valve approaches the development of their IPs. What do you think you'll be expecting? Completely separate games? VR editions? Can seated tag along?

I guess it's too early to even speculate really.
 
A

ashecitism

Member
snip

this is better formatted

EventHorizon said:
Lots of good info going up on twitter right now.

Alan Yates @vk2zay
Chief Pharologist at Valve Corporation. Tragically boring general purpose geek.
https://twitter.com/vk2zay

  • One key missing component is actually the back-haul, most low-latency ISM band solutions are too narrowband or unreliable.
  • The future of tracking systems for VR is not a single tracker, but combinations of the right tech for each part of the job.
  • Lighthouse base station emissions are Class 1 (unconditionally eye safe), while it contains class 3 sources they are enclosed & interlocked.
  • the plan has always been to ship hardware with lots of programmable logic so we can just keep shipping improvements.
  • One base station is enough to track a rigid object with an IMU, five sensors need to be visible to acquire a pose, but it can hold with less
  • Two or more base station visibility lets you track a single sensor, which may be on a non-rigid object.
  • There are many modes Lighthouse can operate in, so far you've only seen one.
  • Lighthouse was designed from day one to be scalable, you can in principle concatenate tracking volumes without limit like cell towers.
  • While it can be used with single sensors and without an IMU, the IMU reduces its latency.
  • Like all optical systems occlusion is a limiting factor. Spatial redundancy is very helpful, but finger tracking will be challenging.
  • The most obvious approaches to finger tracking are mechanical, inertial or magnetic, fused with optical ground truth and a skeleton model.
  • For many purposes finger tracking is actually less useful than a controller. Little is more haptically positive than a snappy switch
  • Lighthouse is not the dual of a camera: The "dots" know their ID; the hardware gives you angles directly. Pose calc' can be decentralised.
Click to expand...
 
M

Mabase

Member
It's great to see how genuinely excited the engineer is. Not just PR people, but someone who actually understands what he's talking about.
 
M

mrklaw

MrArseFace
Mrbob said:
I'm pretty sure I'll buy this, yet still a little apprehensive. It sounds amazing, however I'm not sure how much I want to be walking around the room waggling my arms around. I'll keep an open mind though as the Tested guys gave it high praise.
Click to expand...


It'll still make for fantastic tracking while sitting down - no worries about going outside of a webcams cone of vision. They need to make the retail versions available in white thigh if they want them anywhere near my ceilings.
 
R

Raticus79

Seek victory, not fairness
Whew, this makes a lot more sense. The idea of marking the walls with laser dots was plausible, but it seemed like it would take too much processing power to crunch the views of a dozen or so cameras per device, especially to get a good sampling rate (100 times per second like they mention here).

Very impressive stuff. I'm looking forward to using the controllers. I've been keeping an eye out for six-axis input methods for space sims and these look like they'll beat the STEM controllers.
 
You must log in or register to reply here.

Similar threads

[DF Tech Interview] How God of War was brought from PlayStation to PC
Game Dev Hardware
23
3K
Justin9mm replied
Monster Hunter Rise director Yasunori Ichinose talks RE Engine on Switch
Game Dev Hardware
8
2K
Haggard replied
SteamVR Thread: Virtually a |Thread 2|
2 3 4 5
247
34K
Auto-Reply replied
GTA III: behind the scenes of the game that propelled Rockstar to the top of the world
Analysis Retro Game Dev
47
7K
Uhtred replied
Real Time Hardware Accelerated DirectX Ray Tracing - How Xbox Series X will show a generational leap in Video Game Graphics
Platform 2 3
108
17K
Bernkastel replied
Top Bottom