Patent reveals 3D projection plans?

First of all, let me apologize for not posting anything about this patent earlier. The simple reason is that patents are notoriously hard to understand and this one may actually relate to stereoscopic 3D projection. It just needed a lot of time to dissect it properly and I was very busy in the last few days. To rush this post would not have done justice to this truly intriguing patent. I very much hope you feel that the delay in posting this story is made up for by its thoroughness.

To the topic at hand. The patent is entitled:

´Three-dimensional image generating apparatus, storage medium storing a three-dimensional image generating program, and three-dimensional image generating method´

The abstract reads:

´A video game apparatus as a three-dimensional image generating apparatus includes a CPU. The CPU works together with a GPU to generate and display a game image.´

Up to this point, the patent could relate to any videogame console I suppose. If you want to get acquainted with the type of language used in patents, I recommend you compare this one with a PlayStation2 patent I found.

It goes on:

´More specifically, some objects are arranged in a virtual three-dimensional space. The virtual three-dimensional space is converted into a viewpoint coordinate and then projected and converted.´

Now, this is unusual. If you check with the PlayStation2 patent, you will find no mention of the word ´projection´. Even an Eye Toy patent only ever mentions ´gaze projection´, meaning the direction in which the gamer looks.

´After the projection conversion, determination points are set corresponding to a specific object, and the degree of hiding of the specific object is calculated on basis of a Z value of the specific object and Z values of the determination points. According to the degree of hiding, the movement of a viewpoint is controlled or the brightness of the light source is changed.´

This is quite a mouthful. Let me try to translate this into plain English. First of all, the ´object´ mentioned is an in-game character (see FIG. 4 below). The term ´hiding´ really means a blocked view. The ´determination points´ must be some kind of anchor points around the character, a number of fixed points if you will. The ´Z value´ must be a depth coordinate, basically how far an in-game character stands away from the player. So this paragraph really talks about measuring the apprpriate distance between an in-game character projected into a room and the player; also to what extent the view of the character is blocked by objects in between him and the player. Let us turn to the images to see this more clearly:

---

FIG. 4

´FIG. 4 is an illustrative view showing one example of a virtual three-dimensional space developed on a video game apparatus presented in FIG. 1´

Note: FIG. 1 shows a Gamecube hooked up to a TV.
















Clearly, the player is on the right, behind some kind of screen that could be a stereoscopic sheet of glass (refer to my older post 3D after all? for more information on this technology and type of projection). The fixed points around the character suggest that his projected image should appear to be partly obstructed by an object. This information needs to be recognized when the character is projected, calculated and projected again. That must be the reason why the abstract mentions conversion, projection and more conversion:

´The virtual three-dimensional space is converted into a viewpoint coordinate and then projected and converted.´

---

FIG. 5

´FIG. 5 is illustrative views showing one example of a method for setting determination points in case where no hiding material exists between a viewpoint and a hidden object;´

This illustrates a case where the in-game character is projected in full, because no objects are blocking the view.

---

FIG. 6

´FIG. 6 is illustrative views describing one example of a degree of hiding in case where a hiding material exists between the viewpoint and the hidden object;´

This illustrates a case where the lower left half of the character is blocked by an object.

---

FIG. 7

´FIG. 7 is illustrative views describing another example of the degree of hiding in case where a hiding material exists between the viewpoint and the hidden object;´

This illustrates a case where the entire left half of the character is blocked by an object.

---

FIG. 8

´FIG. 8 is illustrative views describing other examples of the degree of hiding in case where no hiding material exists between the viewpoint and the hidden object and in case where a hiding material exists between the same;´

This illustrates a case where there is a light source behind the character. Imagine Mario standing in front of the sun. When the character is partly blocking the light source, the intensity of the light should be dimmed accordingly.

---

It seems very convincing so far that this patent deals with stereoscopic projection. However, there is a case against. In the description, it reads:

[0004] In the so-called computer graphics (CG) technology field well known to the general public, a virtual three-dimensional space is established, objects are rendered and a virtual screen is provided in this space, and a rendered (displayed) image is projected and converted, and displayed on the screen. According to the CG technology, it is necessary to control a viewpoint in order to determine an image seen from what position in the virtual space should be displayed. The control of a viewpoint is important especially in such a game where a three-dimensional image is rendered in real time according to input from a controller. More specifically, an object to be noted may be hidden by another object depending on the setting of a viewpoint, which makes it impossible to obtain an intended image. In such a case, it is a general practice to move the viewpoint to a position where the object to be noted is not hidden by another object.

Why is the screen sometimes referred to as a ´projection screen´ and sometimes as a ´virtual screen´?

[0005] For example, one example of such a conventional kind of three-dimensional image generating apparatus is disclosed in Japanese Patent Laying-open No. H9-167260 laid-open on Jun. 24, 1997 (document 1). According to document 1, if any hiding material such as another object comes in between an object to be operated and a camera in the game space, the camera is rotated and moved to a position where the hiding material does not hide the object to be operated. In this manner, the object to be operated is constantly displayed within the screen.

This sounds almost as if the patent may just relate to a method of generating an in-game camera according to the player´s controller input and based on objects that would block the view. At least, this is what some people have suspected it might only boil down to.

However, consider that showing a character on screen according to the objects that are in front of or behind him is neither new, nor is it a tricky operation when programming a game. There is simply no need to improve this technique, in my mind. Furthermore, the evidence in favour of this patent dealing with a technology that projects a 3D image into a room (much like ´Augmented Reality´) outweighs those few points that seem odd.

In my mind, the only real question is whether this patent relates to something that will be used with the Revolution console. And as far as that debate is concerned, we have had plenty mysterious comments that seem to suggest this. Again, I want to state that I am trying to remain sceptical about any kind of 3D projection. Especially the recent mock-ups by Seriousgamer007 (or other fakers) have made me more sceptical of this. And yet this patent seems to describe just that: 3D images projected right into your room.

On a final note, the apparent Japanese patent cited by Dead Rabbit in the comments to my anniversary post is unrelated to this. Once a patent is actually pending it must be given a different number. I did the same with one of the Sony patents mentioned above and I ended up with a Minolta patent to do with printers. I have also checked the German company that is mentioned in the apparently related patent (Heidenhain GmbH) and they are a company manufacturing precision measuring machinery and the like. The patent title may suggest that it could be related to the controller functionality (METHOD AND DEVICE FOR TRANSFERRING DATA BETWEEN ONE PROCESSING UNIT AND SEVERAL POSITION MEASURING DEVICES) but it is found only by searching for the publication number, rather than the application number (as mentioned in the US patent). And following the same routine with other patents leads to complete misses.

EDIT Since some readers doubt the significance of the patent, allow me to dissect the main claim given:

´A three-dimensional image generating apparatus for generating a three-dimensional image according to an operator's operation input, comprising:

1) ´an operation input detecting means for detecting the operator's operation input.´

This would have to be a receptacle of sorts, for example an infrared receptacle.

2) ´an object data storing means for storing object data for rendering objects forming a three-dimensional image.´

This would have to be the console´s memory.

3) ´a displaying means for displaying an image projected onto a projection screen within a virtual three-dimensional space.´

This would have to be a screen of sorts.

4) ´a virtual three-dimensional space generating means for generating said virtual three-dimensional space based on the operation input detected by said operation input detecting means and the object data stored in said object data storing means.´

This may be either the CPU or GPU or both, rendering 3D images.

5) ´a viewpoint controlling means for controlling a viewpoint within the virtual three-dimensional space generated by said virtual three-dimensional space generating means.´

This would have to be the controller.

6) ´a display image generating means for generating said image of said virtual three-dimensional space seen from the viewpoint controlled by said viewpoint controlling means, which is projected onto said projection screen.´

This I cannot identify. If this is different from 4) what is meant by it? And if it is, again, the chips generating a 3D image, why list this separately?

7) ´a depth value storing means for storing a depth value of each pixel of the image generated by said display image generating means.´

Again, this would have to be the console´s memory.

8) ´an object arrangement position storing means for storing at least an arrangement position of a specific object in said virtual three-dimensional space.´

Again, memory.

9) ´a determination point setting means for setting a predetermined number of determination points in a predetermined area containing the arrangement position of said specific object in said image.´

This may relate to a part of the program that calculates the reference points talked about above.

10) ´a determining means for determining whether or not a first depth value of the predetermined number of determination points set by said determination point setting means is smaller than a second depth value of said specific object.´

This may relate to a part of the program that calculates the depth and distance between two objects.

11) ´a degree-of-hiding calculating means for calculating a degree of hiding of said specific object according to the number of said determination points where it is determined by said determining means that said first depth value is smaller than said second depth value.´

This may relate to a part of the program that calculates whether an object is hidden by another object.

In conclusion, I don´t understand why there is a difference between 4) ´virtual three-dimensional space generating means´ and 6) ´display image generating means for generating said image of said virtual three-dimensional space´, unless the display mentioned in the latter is indeed a tangible display, i.e. hardware.

Source: United States Patent and Trademark Office, Patent Abstracts of Japan

Thanks to: Dead Rabbit