Sunday, July 09, 2006

The ´Shade Tree´ and what lies underneath

Thank god, we have another patent to mull over. REVOLUTIONYOTTAX spotted it over on the Nintendo forums. And strap yourselves in, this one will have us baffled until the Wii hits the stores, no doubt.

The patent is entitled Recirculating shade tree blender for a graphics system and get ready for a number of mouthfuls, here are the details. This patent comprises:

* A generalized shade tree blender that can be used for multitexturing as well as a number of other flexible blending effects. (...)

* Recirculating shader hardware within a graphics pipeline can be controlled to provide a number of independently controllable blending stages. (...) Thus, relatively low cost and compact shader hardware can be used to implement arbitrarily complex shade trees. (...)

* The results of a first texture mapping operation is provided to a reconfigurable shader. The shader performs a blending operation in response to the first texture mapping operation. The shader is then reconfigured, and is connected to receive the results of a further texturing operation. The reconfigured shader combines its previous results with the results of the further texturing operation to provide a blended output. (...)

* A shader can be recirculated any desired number of times to implement an arbitrarily complex shading model (...) [in order to] provide great flexibility in implementing a variety of complex shading models. (...)

* A recirculating shade tree pixel blender is implemented in hardware to minimize processing time per stage. In more detail, (...) [this] provides a relatively low chip-footprint, versatile texture-environment processing subsystem including a hardware accelerated programmable texture shader/pixel blender that circulates computed color, opacity and other data over multiple cycles/stages. (...)

Right, I am sure that this is lost on the majority of readers. It´s certainly lost on the majority of writers (i.e. me), so let us move on to other parts.

What is a shade tree, then? The patent´s description reads:
Generally speaking, a shade tree is a tree of nodes each of which takes parameters from its children and produces parameters for its parent. For example, the parameters may be the terms of the illumination equation (e.g., specular coefficient or surface Normal). Other parameters might comprise atmospheric effects (e.g., haze) or projections.

The text goes on to note that this technique has been readily used in pre-rendered graphics. For further reference, a document is cited. It is a paper that was published in 1984. Its author, Robert L. Cook, has worked for the computer division of Lucasfilm, is co-creator of the RenderMan software and is now vice president of software engineering at Pixar Animation Studios.

Shade Trees seem to have been a very important discovery as far as pre-rendered graphics go, as Mark Schmelzenbach explains (spelling mistakes removed):
In his 1984 paper, Cook developed the idea of shade trees. This is a very powerful model that allows the user to develop complex shading models using a 'shading language' without having to delve into the actual framework of the rendering program. The language was extended in 1985 by Perlin to provide control and looping structures, as well as logical operators. This evolved to eventually form the basis for textures in RenderMan.

Shade trees provide the ability to contol every part of the shading calculation. This allows the user access to both 'classical' shading models and experimental. Shaders can even be developed for a specific scene, producing (for lack of a better term), elegant hacks.

So this patent may somehow enable this technique to be used in real-time rendering and, put very bluntly, will make things easier and graphics prettier for developers and consumers, respectively. By how much, however, we don´t know. So, instead of going into this deeper, I would like to focus on a few paragraphs that are not quite as prominent. (formatting is mine)
Certain of the above-described system components 50 could be implemented as other than the home video game console configuration described above. For example, one could run graphics application or other software written for system 50 on a platform with a different configuration that emulates system 50 or is otherwise compatible with it. If the other platform can successfully emulate, simulate and/or provide some or all of the hardware and software resources of system 50, then the other platform will be able to successfully execute the software.

Are you with me this far? The patent is related to a home console, so it must concern the Wii. And here it seems to suggest that you can use Wii software on other platforms. My first thought was Wii - DS connectivity, my second was Virtual Console. But read on:
As one example, an emulator may provide a hardware and/or software configuration (platform) that is different from the hardware and/or software configuration (platform) of system 50. The emulator system might include software and/or hardware components that emulate or simulate some or all of hardware and/or software components of the system for which the application software was written. For example, the emulator system could comprise a general purpose digital computer such as a personal computer, which executes a software emulator program that simulates the hardware and/or firmware of system 50.

Some general purpose digital computers (e.g., IBM or MacIntosh personal computers and compatibles) are now equipped with 3D graphics cards that provide 3D graphics pipelines compliant with DirectX or other standard 3D graphics command APIs. They may also be equipped with stereophonic sound cards that provide high quality stereophonic sound based on a standard set of sound commands. Such multimedia-hardware-equipped personal computers running emulator software may have sufficient performance to approximate the graphics and sound performance of system 50. Emulator software controls the hardware resources on the personal computer platform to simulate the processing, 3D graphics, sound, peripheral and other capabilities of the home video game console platform for which the game programmer wrote the game software. (...)

Host 1201 may be a general or special purpose digital computing device such as, for example, a personal computer, a video game console, or any other platform with sufficient computing power. Emulator 1303 may be software and/or hardware that runs on host platform 1201, and provides a real-time conversion of commands, data and other information from storage medium 62 into a form that can be processed by host 1201. For example, emulator 1303 fetches "source" binary-image program instructions intended for execution by system 50 from storage medium 62 and converts these program instructions to a target format that can be executed or otherwise processed by host 1201.

This seems to me most interesting. Because it seems to go beyond both the DS connectivity issue and the Virtual Console.

I cannot quite make sense of what really is being suggested here. But it would appear that at least some Wii software can be made compatible with other devices, including personal computers and other gaming consoles. How that can be achieved and to what end, I don´t know.

Obviously, making the bulk of Wii software available to consumers without them having to buy the console is just stupid for a console manufacturer. So that can´t be it. But, at least with regard to some software, this does seem to be suggested here, after all. I give up, please have a go at solving this riddle.

Sources: US Patent & Trademark Office, Wikipedia, Mark Schmelzenbach


Anonymous said...

They said earlier that Wii "will" be using both tv's "and" monitor's to play softwares...


The biggiest game related soft&hardware stuff - is via PC's.

= Wide userbase of "potential" customers who already know or atleast have knowleage how games works. So could this be one of the bigger market Nintendo wants to reach?


Short of thirdparty suppliers? Think WiiMote = PC Mouse/Keyboard. If WiiMote is implement in a exellent way, who will say that PC exclusive developers might try to develope games for Wii? If Ubisoft can pull it out with Reed Steel, it will then not be that far until they bring out their PC catalogs of IP's to Wii. :)

a litle word regarding the shader thing Fal.

They named it Hollywood right? Now we read about Renderman kinda like tech on/in Wii!? Wouldn't hten come to big surprice and exitement that some shoosen Wii titles will look graphicly somewhat the same as pre-rendered videos only now - you can control it? (Wow!)

Pokemon shows a similar visuals don't you think?


Anonymous said...

If this patent is indeed for Wii, this completely destroys the idea of the system not having any pixel-shading capabilities. However, the first section may relate to dev kits and their ability to prerender graphics. It definitely sounds like an efficient way to use pipelines. The second section just relates to emulation of hardware on different hardware, and the third appears to relate to specific 'wrappers' that convert one console's 'API' for rendering on another system.


Anonymous said...

Ok, what if it meant other systems games will play on the wii...WiiFTW!!!

Metaldave said...

Its kinda a dead end if you ask me. If this is one of the "secrets" Nintendo still has hidden from us, then we will be finding out in a few months.

I'm really interested in the "Shade Tree" though, the developers of the Wii hardware have said that a lot of new tech and research has gone into the making of this system to keep costs down while still having sufficent usage of power to maximize graphics and speed. I love efficient hardware!

D-Bone said...


I've read that in research they have found that there is a numerous amount females ( and others ) that play tiny point and click games on the PC. The connection I'm seeing is perhaps a virtual console application running on the pc that supports these small point and click games people are already playing, so, in essence, a bridge is created from the casual pc gamer to the Wii.

Games like Tetris and Bejeweled can be played on this pc 'Virtual Console' ( with a big Nintendo logo of course ) and that person who is normally afraid of video game consoles will be able to see how easy it is to download and play simple games on the Wii.

The Wii pointer is just like a mouse.

I don't think something like this, if implemented, is meant to make the more advanced Wii games playable on a pc. I think it would be intended to broaden the customer base of the simple Virtual Console games and used as a bridge to transition BeJeweled and solitaire gamers to the Wii.


The Afu said...

GC had Pixel Shaders capabilities so the anonymous who said Wii doesn't that would imply that nintendo downgraded the system in terms of abilities instead of upgrading, which clearly and definetively could be proven. People thinking that this system is a power saver in the wrong. Iwata has stated the system will consume about 50 watts during where as the GC typically takes about 25-35Watts.

Mark said...

It looks to me as if a system with less power could produce the graphics of a system of greater power. For example, with this technology you could take an X-box 360 game and emulate it onto the Wii, I don't believe that it's talking about emulating Wii games onto less powerful consoles. If you replace "System 50" with X-box 360 or PS3 the entire article makes a lot more sense.

wellsed said...

This is an update of a patent related to the GameCube. Look at all of the other known GameCube patents and they reference the previous version of this patent.

So, most everything in this should apply to GameCube or an enhanced GameCube-like system.

squeakypants said...

I think it's just a development tool, so you can emulate the wii on a PC.

Anonymous said...

Don't get your hopes up on the shader tree and whatnot, as it looks to me like a fancy way of describing simple blending. Think of a bunch of colored transparent sheets. Start with one sheet being placed over another. Based on the transparency of both layers, a relatively simple math equation is used to determine the change in the RGB value of what is represented on screen. The equation is repeated for that combination and the next sheet placed on top, etc.

In the context of a game, this would, for instance, calculate what the final color should be for a pixel that's part of a building obscured by fog that's being hit by light. Anything capable of 32 bit color can handle all of this, and it's a very basic thing (as far as image processing goes) that I don't see as being any more efficient on Wii as opposed to anything else.