• Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.

SCEA files patent for context switching between architecturally distinct GPUs

androvsky

Member
Amd apu design are always one Gen behind the gpus. This is good news since it means thy are going with sea island gpu.

Amd apu must be pair with the match gpu. This means they are design around that and found a solution that fixes that problem.

Ah, in that case it makes perfect sense. :)
 
Can someone answer on this question.

If this is true and Sony ends up using this patent, what does it mean really, that MS can't have APU + GPU (low end gpu + powerful gpu)? Aren't both going for that? What is this patent making that only Sony can use?


and even more importantly, let's say low power GPU is 0.5 tflops and main is 2tflops, can those combined 2.5tflops be used for gaming or only 2tflops?

I think Sony is patenting it in case they get fucked like how they couldn't use audio chat.
 

Elios83

Member
Can someone answer on this question.

If this is true and Sony ends up using this patent, what does it mean really, that MS can't have APU + GPU (low end gpu + powerful gpu)? Aren't both going for that? What is this patent making that only Sony can use?


and even more importantly, let's say low power GPU is 0.5 tflops and main is 2tflops, can those combined 2.5tflops be used for gaming or only 2tflops?

We don't know what these companies are doing, it's all speculation.
Microsoft is supposed to have a CPU made of many low power cores to provide Windows 8 with great multitasking capabilities. No one so far has mentioned an APU option for them.

As for the second question the second GPU could be used to offload certain tasks off the main GPU and to add multitasking (even for 3D applications), also to help the CPU (more than the GPU) through GPGPU.
But honestly we don't know anything. What if the PS4 controller has a screen and the second GPU is used as a second video output?
 
Hard to determine wether this is something for a Vaio laptop, a Sony Tablet, Sony Phone, or even a Gaikai server park for cloud gaming rather than PS4 though ... but you never know. But the abstraction layer sounds like it would be more suitable for something Gaikai or mobile related than PS4 perhaps?

The patent was filed by SCEA, not Sony Corporation of America. It is for use in PlayStation products specifically.
 

Jhriad

Member
This means BC is in the PS4 right? RIGHT?!

No, I know it's not. *weeps silently in the corner*

I think Sony is patenting it in case they get fucked like how they couldn't use audio chat.

What's this referring to? Party chat or something else?
 
Hard to determine wether this is something for a Vaio laptop, a Sony Tablet, Sony Phone, or even a Gaikai server park for cloud gaming rather than PS4 though ... but you never know. But the abstraction layer sounds like it would be more suitable for something Gaikai or mobile related than PS4 perhaps?

Sony Computer Entertainment America Inc.
 

i-Lo

Member
I think Sony is patenting it in case they get fucked like how they couldn't use audio chat.

Which reminds me to ask, can they include in-game custom soundtrack for PS4 and if so then how are they circumventing the existing patent? Or is it patent valid for only one product, i.e. 360?
 
We don't know what these companies are doing, it's all speculation.
Microsoft is supposed to have a CPU made of many low power cores to provide Windows 8 with great multitasking capabilities. No one so far has mentioned an APU option for them.

As for the second question the second GPU could be used to offload certain tasks off the main GPU and to add multitasking (even for 3D applications).
But honestly we don't know anything. What is the PS4 controller has a screen and the second GPU is used as a second video output?

That's an interesting concept. Get a Vita, stick a Miracast chip in and turn it into a second screen controller for PS4 with the processing done on the APU given the lower power requirements for a 960x540 screen.

I still think multitasking is more likely though, in 2020 we will be watching 4k TV and Sony want a way to ensure they can cope with that on PS4. GPU based encoding for a DVR mode on PS4 is the most likely situation where the APU will find use.
 

Elios83

Member
This means BC is in the PS4 right? RIGHT?!

No, I know it's not. *weeps silently in the corner*



What's this referring to? Party chat or something else?

Hardware based BC won't be in the PS4 (unless they make people for it in a specific SKU or through a specific add-on) but all the other options (software emulation, Gaikai, etc) are still open.
 

USC-fan

Banned
You can see the performance gains with amd dual gfx in gaming.

http://www.pcper.com/reviews/Graphi...-Review-Can-AMD-compete-Sandy-Bridge/AMD-Dual

In a closed system the advantages should be even greater. Please also note that the first ps4 dev specs included apu and gpu that support dual gfx. As I stated back then when people were saying they were using weak hardware because they were designing weak system. I said it was for the dual gfx supports. Look like I was right again!

Dual gfx + hsa = bad ass system.
 
Something interesting to read. Inventor says "Rose; Robert W. (Bend, OR)"

Checking LinkedIn, he left SCEA in 2009.

Programmer
Sony Computer Entertainment America
August 2005 – January 2009 (3 years 6 months) Bend, Oregon

Engine Programmer (graphics, fx, resource management, UI, etc.) on three titles that won "Game of the Year" awards: Syphon Filter: Dark Mirror (PSP, PS2 2006), Syphon Filter: Logan's Shadow (PSP, PS2 2007), and Resistance: Retribution (PSP 2009).

Side projects: Developed the SPU/RSX decorator rendering system for MAG (released 2010), a geospatial data renderer for a PSP+GPS game concept (2006), and an audio synthesis engine for the Cell processor (2007).

What do you think "SPU/RSX decorator rendering system" is?
 
R

Rösti

Unconfirmed Member
Here are the two most relevant images, in my opinion, for better context:

scea_20120320068_fig346uyv.png
[0011] FIG. 3 is a block diagram of a computer graphics apparatus according to an embodiment of the present invention.

[0012] FIG. 4 is a block diagram of a computer readable medium containing computer readable instructions for implementing a computer graphics processing method in a computer graphics apparatus having a central processing unit (CPU) and architecturally dissimilar first and second graphics processing units (GPU) in accordance with an embodiment of the present invention.

Also, this should be noted as well:

[0018] The GPU 103, 104 are in general, architecturally dissimilar. As noted above, the term "architecturally dissimilar" means that graphical input formatted for one GPU 103 will not work with the other GPU 104 and vice versa. By way of example, and not by way of limitation, the two GPU may have different instruction sets, different display lists, or both. In addition, in some embodiments, the two GPU 103, 104 may have different processing performance and power consumption characteristics.

Nice find anyway. I'm liking the idea of having two GPUs. In what way they can work together will be interesting to see. Well, "together" may or may not be the correct term to use, but interoperability in general for Orbis could become quite spectacular with this present revelation in mind.
 

Jhriad

Member
I'm not joking, if the PS4 doesn't have backwards compatibility with my PS3 library, I'm going to pass on it.

Probably the same here. It's just the motivation I need to dump consoles entirely, hook a PC up to my television, and give Steam Big Picture a shot.

I haven't ever been a big mobile gamer either on phones or traditional handhelds and the last handheld I owned still had Gameboy in the name. Nor have I wanted to or bothered owning a "current" Nintendo home console in over a decade. I simply want to be able to consolidate my gaming experience to one platform, as much as possible, and have those games be playable on future iterations of the platform's hardware. If Playstation or Xbox won't provide that then PC gaming will. Whether they want my thousands of dollars every generation & me as a platform evangelist or taste maker for my group of friends is up to them at this point. That said I don't expect BC so I'm simply waiting for the inevitable truth of the matter to surface to begin the shift.
 
Hard to determine wether this is something for a Vaio laptop, a Sony Tablet, Sony Phone, or even a Gaikai server park for cloud gaming rather than PS4 though ... but you never know. But the abstraction layer sounds like it would be more suitable for something Gaikai or mobile related than PS4 perhaps?

Despite Gofreak saying this issue has already been discussed, the timing of time makes you link with PS4, due to the AMD announcements yesterday.

They started this in 2009, which would go in line with early PS4 development (closing in on specs and architechture).

It also goes with the theory that we would hear more from next gen before the end of the month.

I read the document and there´s one point where i think my limited computer knowledge makes me believe in extra power.

I thought the whole idea of the patent and the seamless switch from one to the other would mean both could be used in high demanding tasks... like the discrete one could be some sort of a turbo mode for the system... it looks possible, because having a GPU with 1/3 of the power of you main one (using proelite numbers) just to do XMB and other minor stuff would be kind of a waste.
 

Perkel

Banned
Rösti;45648624 said:
Here are the two most relevant images, in my opinion, for better context:

scea_20120320068_fig346uyv.png



.

preatty much confirms that PS4 will have split memory pool not shared like in WiiU.

2GB of GGDR5 and rest of slower ram.
 

Duffyside

Banned
If this is a power-saving measure, could it point to the PS4 being a tablet? Which would be good, because if the next gen consoles aren't tablets, they aren't thriving.
 

Plinko

Wildcard berths that can't beat teams without a winning record should have homefield advantage
If this is a power-saving measure, could it point to the PS4 being a tablet? Which would be good, because if the next gen consoles aren't tablets, they aren't thriving.

This is a horrible idea.
 

Nirolak

Mrgrgr
and even more importantly, let's say low power GPU is 0.5 tflops and main is 2tflops, can those combined 2.5tflops be used for gaming or only 2tflops?

Reading through this, the patent doesn't cover this usage, but mentions it as a potential.

For this patent, they shut off the GPU that isn't needed (either the low power one because you need more power, or the high power one because you don't need all the power).

However, the end notes that the concept could be extended to using architecturally similar processors (that were different power grades) to augment each other.

The guy who did the core research for this left in 2009, so it's unclear if they ever followed the proposed extension of this methodology to be able to unify the power of the GPUs. However, it was filed in July of this year, so it implies Sony is still interested in the technique at least on the level of power saving.
 

Perkel

Banned
I changed a little bit graph to point important parts from console perspective:

jbsRWNAGFv3fVu.jpg


Which would point that:

MAIN RAM will be slower and there will be a lot of it (cost effective)
VRAM1/VRAM2 will be ultra fast (GDDR5) but we don't know exactly which GPU A or B will be main gpu.

We can safely assume that one of them will have rather big fast memory (GDDR5) and other will have smaller memory and slower (GDDR3)

My bet:

MAIN RAM : 8GB DDR3/DDR4 ram
VRAM1 : 2GB GDDR5 ram
VRAM2 : 512MB GDDR3 ram

GPU2 will be mainly for system use and it will give also deliver multitasking without slowering games performance (voice chat in games, internet browser in games etc

That amount of RAM and type is cheap especially considering they will be buying tons of those.
We can safely assume Sony is not holding back and it's going bonkers.

With recent news of HD8000M we can even speculate that HD8000M chip like can be used for system and better GPU for games.
 

Perkel

Banned
A drawing in a patent isn't confirmation of anything.

Sure it isn't confirmation but if you consider drawing and how it is close to rumors and next gen systems you can deduce this drawing represent next gen PS4 console not TV or PC or Mobile phone. Patent alone also is more connected with consoles than PC or mobile gaming.
 

Perkel

Banned
Will not happen. If Sony is smart, they're going to integrate a much better remote play option with Vita. Also, Vita has that "magic port" that no one knows what it's used for. Anything to improve PS4/Vita connection would be a blessing.

As a gamer i love idea of remote play but i don't see it happening from Sony point of view. They don't want players to buy Vita and only stream PS3 games on it. They earn mainly on software not hardware.
 

jcm

Member
Sure it isn't confirmation but if you consider drawing and how it is close to rumors and next gen systems you can deduce this drawing represent next gen PS4 console not TV or PC or Mobile phone. Patent alone also is more connected with consoles than PC or mobile gaming.

Sure, but Sony patents all kinds of things, and doesn't bother to build lots of them. If a Sony patent app were confirmation of a product we'd all be chucking virtual tomatoes at our TVs now.
 

i-Lo

Member
I changed a little bit graph to point important parts from console perspective:

jbsRWNAGFv3fVu.jpg


Which would point that:

MAIN RAM will be slower and there will be a lot of it (cost effective)
VRAM1/VRAM2 will be ultra fast (GDDR5) but we don't know exactly which GPU A or B will be main gpu.

We can safely assume that one of them will have rather big fast memory (GDDR5) and other will have smaller memory and slower (GDDR3)

My bet:

MAIN RAM : 8GB DDR3/DDR4 ram
VRAM1 : 2GB GDDR5 ram
VRAM2 : 512MB GDDR3 ram

GPU2 will be mainly for system use and it will give also deliver multitasking without slowering games performance (voice chat in games, internet browser in games etc

That amount of RAM and type is cheap especially considering they will be buying tons of those.
We can safely assume Sony is not holding back and it's going bonkers.

With recent news of HD8000M we can even speculate that HD8000M chip like can be used for system and better GPU for games.

Fuck yea Crayons!

Btw, your estimates of RAM are a touch too optimistic for a console that is designed to be priced around $400. Also, I don't think the VRAM 2 will be using GDDR3.

It'll most likely be:

MAIN RAM : 2 - 4 GB DDR3/DDR4 ram
VRAM1+2 : 2GB GDDR5 ram
 

tokkun

Member
Here is a quick primer on context switching, since I don't think people are focusing enough on that aspect.

What is context switching?
It's when your operating system suspends the program that is currently running on a processor and stores its internal state into memory. At the same time, a different program that is stored in internal memory is swapped in so it can start running. This enables preemptive multi-tasking.

What about context switching between different processors?
This allows a program that started on one processor to stop running and resume on another. This, for example, allows programs to move between different cores on your quad-core processor for load balancing reasons. It is only slightly more complex than a single-processor context switch.

What about context switching between architecturally different processors?
Same idea as above, but a lot more complex since the two processors may not use the same instruction set and the internal state of the processor will be different.

What is a realistic gaming use for this on a console?
Picture-in-picture gaming. The windowed game is rendered on a weaker GPU while the full-screen game is rendered on the stronger one. If the gamers want to swap between windowed and full-screen, the game code would need to context swap between the GPUs (replace 'windowed' with 'streamed to a tablet' if that strikes your fancy).
 
Good, since it means you get the power of the main GPU + the mobile GPU.*

It also means they definitely have a higher powered GPU and a lower powered GPU, implying it's not just what is in the APU.

Assuming they're using the technology in the patent of course.

*Dependent on the main GPU being good, and this actually being used directly in game processing.

Exactly how I read it.
 

Jhriad

Member
Is your ps3 suddenly going to explode once you buy a ps4?

Are you going to sell your PS3?
I fail to get this no BC "problem"...

How many post-cartridge consoles have you had break or know to have broken on your friends?
How many devices do you want to have connected to your television at once?
How much controller & wire clutter is acceptable to you and/or your significant other?
What types of connections does your HDTV have? (admittedly more applicable to older generations of hardware)
Are you happy buying the same game digitally that you bought a physical copy of, perhaps multiple times (if current digital titles don't work on future consoles) should you want to play them?


Knowing that when/if my PSone died I could transition freely over to a PS2 was comforting and helps one retain the mental value you place on the software you buy for a console after the console has peaked. Backwards compatibility, particularly for digital titles, means that those titles will remain playable for the masses into the foreseeable future and not just the select few that were popular enough to be ported to the new generation of hardware. If backwards compatibility were available some kid in the future would be able to enjoy picking up Vanquish, Bayonetta, Yakuza 4, Mirror's Edge, Valkyria Chronicles & Asura's Wrath on the digital store and experiencing those games. Few if any of those games did well enough to be ported to future hardware. Instead they'll just receive a poor, curated list of "hits" and first party ip collections like 'The Call of Duty Collection' or 'Ratchet & Clank 1-27.' Just given the likely failure of hardware over time I would think it's easy to understand the desire for BC. You fail to get the problem because you're not even bothering to think about it.
 

tokkun

Member
Can someone answer on this question.

If this is true and Sony ends up using this patent, what does it mean really, that MS can't have APU + GPU (low end gpu + powerful gpu)?

No, it doesn't preclude Microsoft from having heterogenous multi-GPU.

Aren't both going for that? What is this patent making that only Sony can use?

The part about dynamically moving a program between the GPUs.

and even more importantly, let's say low power GPU is 0.5 tflops and main is 2tflops, can those combined 2.5tflops be used for gaming or only 2tflops?

This sort of model doesn't require context switching; it's a co-processor model. In other words, yes you can do this, but no, you don't need the patent to do this.

Where dynamic context switching would be interesting in such a model would be if, for example, you could dynamically predict when certain subsystems like the physics engine would need more power and switch them to a different GPU, thus achieving better utilization.
 

Perkel

Banned
Fuck yea Crayons!

Btw, your estimates of RAM are a touch too optimistic for a console that is designed to be priced around $400. Also, I don't think the VRAM 2 will be using GDDR3.

It'll most likely be:

MAIN RAM : 2 - 4 GB DDR3/DDR4 ram
VRAM1+2 : 2GB GDDR5 ram

DDR3 ram is cheap as few bottles of beer right now for end consumer for sony it will be even cheaper 8 GB is cheap.

If they will go with DDR4 which would be costly then we can assume 4-6GB.
 

i-Lo

Member
DDR3 ram is cheap as few bottles of beer right now for end consumer for sony it will be even cheaper 8 GB is cheap.

If they will go with DDR4 which would be costly then we can assume 4-6GB.

Gah, perkel, it's not about the cost per unit. It has to be integrated as chips on to the motherboard. And more chips=more complicated design. More complicated design=More cost, Greater heat issues and higher chance of HW failure. That said, I know they are doing something with stacking and DDR4 would be better choice to go with which not only has the same chip density as DDR3 (which is 2xGDDR5) but more importantly outperforms in raw numbers and power consumption. Still, at this point, I think it's unrealistic to expect 8GB system RAM when MS is rumoured to be using 8GB overall. That's why I state that 2 is min, 4 seems like the sweet spot and 6 would be the absolute max (making the total RAM the same as XB3).
 

Perkel

Banned
Gah, perkel, it's not about the cost per unit. It has to be integrated as chips on to the motherboard. And more chips=more complicated design. More complicated design=More cost, Greater heat issues and higher chance of HW failure. That said, I know they are doing something with stacking and DDR4 would be better choice to go with which not only has the same chip density as DDR3 (which is 2xGDDR5) but more importantly outperforms in raw numbers and power consumption. Still, at this point, I think it's unrealistic to expect 8GB system RAM when MS is rumoured to be using 8GB overall. That's why I state that 2 is min, 4 seems like the sweet spot and 6 would be the absolute max (making the total RAM the same as XB3).

Ok i can understand that. We know something about 3D stacking which may be or not be true.

I have other question then if you are knowledgeable about hardware design.

Why they can't use what we are using in PC for centuries. Ports. That would resolve heat issues and vertical space is not problem for homeconsoles.
 

ReaperXL7

Member
Hopefully that means that they've added multi-tasking background services. I would love to pause the game, go to a PSN Store or Browser and then come back. That or go to Netflix and return to the game. Multiple services running at the same time would be fantastic! Dedicate the bigger GPU for gaming and smaller GPU for everything else.

In simple words: Similar to how Vita or any smartphone works, but more Steam-like.

I could be remembering incorrectly, but I'm pretty sure I read something recently about sony having some kind of next gen tech demo presntation and one of the features they showed off was the ability to do stuff just like this.

And example was going to the playstation store seemlessly while playing a game being able to use the store as normal, and then exit right back into the game.

Sounds like such a small thing, but also very useful at the same time. I'm really looking forward to seeing what the next playstation and xbox are capable of not only from a games standpoint, but also possible features like this.
 

maverick40

Junior Member
How many post-cartridge consoles have you had break or know to have broken on your friends?
How many devices do you want to have connected to your television at once?
How much controller & wire clutter is acceptable to you and/or your significant other?
What types of connections does your HDTV have? (admittedly more applicable to older generations of hardware)
Are you happy buying the same game digitally that you bought a physical copy of, perhaps multiple times (if current digital titles don't work on future consoles) should you want to play them?


Knowing that when/if my PSone died I could transition freely over to a PS2 was comforting and helps one retain the mental value you place on the software you buy for a console after the console has peaked. Backwards compatibility, particularly for digital titles, means that those titles will remain playable for the masses into the foreseeable future and not just the select few that were popular enough to be ported to the new generation of hardware. If backwards compatibility were available some kid in the future would be able to enjoy picking up Vanquish, Bayonetta, Yakuza 4, Mirror's Edge, Valkyria Chronicles & Asura's Wrath on the digital store and experiencing those games. Few if any of those games did well enough to be ported to future hardware. Instead they'll just receive a poor, curated list of "hits" and first party ip collections like 'The Call of Duty Collection' or 'Ratchet & Clank 1-27.' Just given the likely failure of hardware over time I would think it's easy to understand the desire for BC. You fail to get the problem because you're not even bothering to think about it.

I will have two devices plugged in to my tv when the ps4 comes out. The ps3 and the ps4. When i want to play my back catalogue of ps3 games i will use my ps3. I fail to see a problem here. Ps4 will not have BC so get that out of your head. the percentage of people who use it is way too low for sony to warrant increasing the price of the console. Who buys a ps4 to play ps3 games...smh.
 

Elios83

Member
Fuck yea Crayons!

Btw, your estimates of RAM are a touch too optimistic for a console that is designed to be priced around $400. Also, I don't think the VRAM 2 will be using GDDR3.

It'll most likely be:

MAIN RAM : 2 - 4 GB DDR3/DDR4 ram
VRAM1+2 : 2GB GDDR5 ram

I guess that it's going to be that way, split memory architecture, 4GB of DDR3 main memory and 2GB high performance GDDR5 memory for the GPU.
Which would be pretty great imo.
 
Gah, perkel, it's not about the cost per unit. It has to be integrated as chips on to the motherboard. And more chips=more complicated design. More complicated design=More cost, Greater heat issues and higher chance of HW failure. That said, I know they are doing something with stacking and DDR4 would be better choice to go with which not only has the same chip density as DDR3 (which is 2xGDDR5) but more importantly outperforms in raw numbers and power consumption. Still, at this point, I think it's unrealistic to expect 8GB system RAM when MS is rumoured to be using 8GB overall. That's why I state that 2 is min, 4 seems like the sweet spot and 6 would be the absolute max (making the total RAM the same as XB3).

where has it been confirmed Sony is going to using less RAM than MS? Remember Sony is designing this thing to last past 2020 so they would definitely go for more DDR4 seeing how in the future it isn't going to be so expensive and they can stack it. And we all know how they love the redesign their hardware. I expect no less than 8 gigs of total ram in the PS4
 
I'm surprised AMD, ARM, Nvidia or Intel doesn't have a patent on this already. They've certainly used similar techniques.

I guess the important part here is the "architecturally distinct" aspect, which they likely wouldn't have worried about.
 

NateDrake

Member
I will have two devices plugged in to my tv when the ps4 comes out. The ps3 and the ps4. When i want to play my back catalogue of ps3 games i will use my ps3. I fail to see a problem here. Ps4 will not have BC so get that out of your head. the percentage of people who use it is way too low for sony to warrant increasing the price of the console. Who buys a ps4 to play ps3 games...smh.

Then why did so many people get mad when Sony pulled PS2 BC support from the PS3?
 
Top Bottom