Thursday, December 15, 2005

Is parallax mapping the last secret?

Please welcome Nintendojo, Revo-Europe and 4 Color Rebellion to our little discussion. All of them have picked up on this debate (all without quoting me as the original source, I should mention), alongside uncountable forum posts. I would like to take this opportunity to thank every single person visiting my blog for helping to push the daily visits to a staggering count of 8.500.

Concerning the topic at hand, it has now been suggested by a number of people commenting on this story that Nintendo may in fact be planning to use yet another type of mapping called ´parallax mapping´. As ever, I have direct evidence in the form of new patents, but let´s look at the technique itself first.

Wikipedia is a good starting point again:
Parallax Mapping (also, Photonic Mapping, Offset Mapping or Virtual Displacement Mapping) is an alternative to the bump mapping and normal mapping techniques applied to textures in 3D rendering applications such as video games. To the end user, this means that textures (such as wooden floorboards) will have more apparent depth and realism with less of an influence on the speed of the game.

Parallax mapping is done by displacing the texture coordinates such that the texture occludes itself in accordance with a height map. Next-generation 3D applications may employ parallax mapping as new graphics algorithms are developed.

An easy way to understand this concept is to close one eye, take a pencil, point it at your eye, and move your head left and right. Parallax mapping takes that pixel on the far left of the pencil when it was facing you directly and stretches it accordingly to simulate your angle in comparison to the pencil.

Parallax mapping is also a way of faking displacement mapping.
To see exactly what parallax mapping does, let us look at some samples, courtesy of InfiScape.
The effects are obvious, particularly in the second example where it is combined with bump mapping. Basically, it works like this: Imagine looking at a raised point on a surface and then moving sideways. Due to perspective, the raised point will appear to move in the opposite direction. Parallax mapping can modify bump mapping to take account of such a move. This is referred to as offsetting. No extra polygons are needed. A height map is enough to calculate the offset value. It´s very simple to implement, is easy on the CPU and can thus be combined with other mapping techniques to achieve results, easily comparable to displacement mapping (though it does not actually alter the height of the pixels, I should point out).

Now, there are plenty of quotes in the patents mentioned in my earlier posts below that seem to relate to this technique. But I also came across this new one entitled ´ Method and apparatus for interleaved processing of direct and indirect texture coordinates in a graphics system´, which seemingly referrs (in some part, at least) to parallax mapping, as it often mentions an offset matrix that is applied to the textures:
In a graphics system having a memory containing texture data, a method of indirect texture referencing comprising the steps of: (a) using a set of indirect texture-coordinates to retrieve offset data from the memory; (b) multiplying the offset data by predetermined values forming elements of a texture offset matrix to produce a set of texture offset coordinates; and (c) using said set of offset texture coordinates for referencing texture data stored in the memory when mapping a predetermined texture to a rendered polygon.
Reading further, we learn about what effects this technique might have on a hardware arrangement (formatting is mine):
To provide an enhanced repertoire of texturing capabilities for a 3-D graphics system, the present invention provides a versatile texturing pipeline arrangement achieving a relatively low chip-footprint by utilizing a single texture address coordinate/data processing unit that interleaves the processing of logical direct and indirect texture coordinate data and provides a texture lookup data feedback path for "recirculating" retrieved indirect texture lookup data from a single texture retrieval unit back to the texture address coordinate/data processing unit. The interleaved coordinate processing and recirculated/feedback data arrangement of the present invention allow efficient processing of any number of logical direct and/or indirect texture mapping stages from a smaller number of hardware texture processing units while preserving a fine granularity in the overall data processing flow.
Concluding, it seems to me that this technique is the most likely to be used in the Revolution. It almost has the visual quality of displacement mapping while requiring next to no more calculations.

Sources: Wikipedia
Image source: InfiScape

39 comments:

Anonymous said...

First post! Woo!

I like the idea of displacement mapping better, but it's probable that they'll use all three techniques somehow next gen.

Anonymous said...

Please be patient, as there is a point after the long-winded stuff.

The "displacement mapping" patent was for using a hardware accelerated bump mapping. Contrary to what others wrote, there are other methods of displacement mapping that do not increase polygon count by several orders of magnitude.

For instance, the thesis paper linked from that article mentioned a ray-casting method in eye-space (i.e., what you see, not the model as it is defined), to build the image with about 12 extra polygons per viewable polygon. That could be tricky, but the Gamecube had a very fast frambuffer/z-buffer that would aid such a method very well.

However, I doubt that is the subject of the "emboss style bump mapping" patent application. I think it really is accelerated bump mapping, producing effects similar to the normal mapping that is very common on xbox360 and many PC titles (e.g., Half-life2, and the doom3-based games).

Now, *THIS* patent is a novel, and UNIQUE, means to reduce the amount of texture/pixel/texel processors necessary to produce the bump effects. It streamlines and optimizes the functionality of "pixel shaders" (the xbox360 has up to 48 and the PS3 24).

In fact, image number 5 shows a box labled "Texture Coordinate/Bump Processing". This patent is for the hardware used to accelerate the math/software/technique defined in the previously discussed "bump mapping" patent application.

Nothing more or less, but still a pretty nice look at what could be in Revolution: making bump mapping really cheap to use on all in-game objects.

Also, this hardware accelerates pretty much any effect/technique that uses texture lookups/offsets and such between multiple textures. The recent ATI work on "Parallax Offset Mapping" (not to be cofused with "parallax mapping" as they are different) provides a superior fake displacement mapping to bump/normal/parallax mapping for not-much-higher processing cost. This new hardware could make implimenting it in Nintendo games more of a reality, which would make for some great looking games.

--wellsed

zerorules44 said...

woo hoo!!!!!!!!
U-S-A
U-S-A

Anonymous said...

Hi Falafelkid,
What I don't quite get is(bei all dem Fach-Chinesisch)..Why should Nintendo patent all this stuff when in fact Ati is in charge of the contract(developing Hollywood) that Nintendo hand over to them ?
Strange..

-Infamous-anonymous-light shedder.

Anonymous said...

Looking at some of the other patents Nintendo filed in the same period shows that they seem to be heavily reliant on texture processing to make up for less geometry.

There's a lot of patent applications for Nintendo in this domain and this new hardware patent helps *ALL* of them.

Arsenis said...

nintendojo calls you german blogger so don be so mad, falafelkid....

lol.

Arsenis said...

now revo-europe uses nintendojo so you are still a german blogger, but 4color rebellion will not mention you.

i would be mad at them, german blogger...

lol

Arsenis said...

zerorules44


castlevania huh!!

Johnny B said...

@ light shedder: I also am a bit confused as to why Nintendo rather than ATI would be patenting these graphic optimizations. Does Nintendo "own" the rights to the R&D that they have paid ATI to do? I don't know the answer to this, but I'm guessing that Nintendo has to patent the aspects of this technology that relate to gaming.

Although I do see this being used in the Revolution, along with many other graphical tricks, I don't really think this is the final secret. It might relate to the final secret in some way, but its not the secret itself. That said, it is these kinds of optimizations that will make Revolution's graphics comparable to the competition's.

Anonymous said...

Hi Falafelkid,
What I don't quite get is(bei all dem Fach-Chinesisch)..Why should Nintendo patent all this stuff when in fact Ati is in charge of the contract(developing Hollywood) that Nintendo handed over to them ?
Strange..

-Infamous-anonymous-light shedder.

leander said...

the last secret bla bla why should the last secret be some complicated rendering technique no-one would be excited about that.
My theory it is something mire tangible like a micro... and online chat or phoning (for free of course)
now that would be cool huh?

Anonymous said...

Exactly. The secret isn't some rendering technique. I mean come on. That's the one thing I do know for sure.

I like the guy in the last article comments section who said the 'A' button is deceptively simple and overly large. I think his idea that it has multiple functionality is as good a guess as any.

It seems to fit right into what Nintendo wants to do. NOt scare off new gamers, but provide functionality for the hardcore. Kind of like Apple with their new Mighty Mouse.

Again it's probably the best guess I've heard yet.

Kevin said...

Great article Falafelkid. All you need now is one on the rumored Nurbs. ;)

Pekka said...

@wellsed,

I couldn't find anything on parallax offset mapping. Did you mean parallax occlusion mapping? Reverse parallax mapping where the bumps can overlap each other and cast dynamic shadows?

I suggest everyone read this PDF presentation for more on the technique. Check pages 31, 41 and 48 for samples.

For a sample of this applied on an in-game model, check the page 26 from this pdf.

There's a line in the first file around page 54 and I quote: -But you thought it wasn’t possible to do this at interactive rates -IT IS! Good hardware architecture helps it

Humm...?

Anonymous said...

Hey! I've been following your blog for quite a while, it's really the best rev site I've found.
Anyway, I noticed how your last three posts have been about mapping techniques which make games look better. But I think we've established that Nintendo's not interested in making games look shinier but basicly to suit up to the on going generation.

I think lookin at their tech specs is good but knowing Nintendo their secret is bound to have something that goes a little more beyond the screen of the TV. Maybe you should try getting somethin a lil more juicy. Even if it's just speculation, exercising the imagination is one of Nintendo's principles so maybe that's what they're tryin to go for?

Still, it's pretty interesting how you brought up those three mapping styles.. they might even use all three at the same time and use that other digital space patent thingy that makes graphics stick out of the screen. That'd definitely make somethin great

Pekka said...

I found a neat videoclip where some people applied relief texturing (an earlier form of parallax occlusion mapping) to Doom3 engine. Visit and download the video from the bottom of the page.

Now that is just beautiful.

You should also check the many papers on that site for more info.
Real-Time Relief Mapping on Arbitrary Polygonal Surfaces
Hardware-Assisted Relief Texture Mapping

In the last one, they used relief texturing to sort of build a hologram of a pre-rendered object inside a cube. I think some people have mentioned this technique before, but I just now realise what they meant by it.

Pekka said...

This paper is really interesting. It explains the 'hologram rendering' in good detail.

An interesting point from the paper:

7.4.1 Hardware implementation
One important area for future investigation is the design of efficient hardware implementations for relief texture mapping using the derived pre-warping equations. Adding this pre-warping capability to the texture memory of a graphics accelerator may allow relief texture mapping to become as commonly used as conventional texture mapping.


I'll need to read that patent on revolution's texture handling again.

ColdBlooder said...

no way. the last secret is never parallax mapping. the game "serious sam 2 already supports that technique and it really dosent look much better when you turn that on... in fact it just slows the game down a LOT on an x8000gto graphics card.

nz guy said...

the last big secret to the rev will be somthing that will make sence to the average person that nintendo is targeting. If nintendo anounced a new way of mapping people all around the world will unite in thinking...

so?

Anonymous said...

The point that a majority of you doesn't seem to get, is that it's not equivocal that Nintendo won't simply announce that the secret is supposed to be Parallax- Cube- or Displacement-mapping verbatim


- these techniques merely represent different components, that to an extent compose what some developers reffer to -qouting the recent GamesIndustry.biz article- as a "a black box" and "nobody's seen the hardware yet" in other words the Hollywood GPU of course.


I keenly presume that Nintendo will debunk all the rumors by flaunty displaying the last secret that is Hollywood, with all patented inventions we all wittnessed lately included.

I guess he will make it clear to everyone by presnting the whole package respectively the games,a quick summary or contemplation of the philosophy,concept and intentions behind the GPU,the interaction of it with Broadway and probably what kind of benefits this approach means for developers.

"While Broadway is well-understood by developers, the ATI part remains "a bit of a black box", according to one senior developer we spoke to. "We have theoretical throughput figures and stats from Nintendo, but nobody's seen the hardware yet - we're just treating it like it's a faster version of the GameCube GPU, at the moment."

- The-Infamous-Anonymous-Light Shedder

Anonymous said...

Very intresting ideed.

My only qualm is using wikopedia as a source. Wikopedia can't be relied upon what-so-ever.

Infact I believe they are finally getting sued do to the fact that they dony have enough staff to research every entry.

Anonymous said...

Prallax mapping is not the last secret. The technique is nothing new and nothing special. Unreal Engine 3 uses Parallax mapping. It's been done.

Pekka said...

Sorry for spamming your comments, but if Nintendo some way manages to pull relief mapping off or something akin to it, that would be just mind-boggling.

The Doom video I pointed to earlier is running on a current generation computer in real-time. If you look at the details, every surface was relief mapped and not just the door, floor and the pillar. Also, check those columns in the other video on that site. For what I can gather, that may be only a cylinder, and the profile of the pillar is calculated from the relief map.

Relief mapping has been around for about four years, give or take. nVidia is starting to utilize it in their next line of GPUs.

From the patents shown here, Nintendo's putting many eggs into the texture basket. Could this be their trump card?

Anonymous said...

Pekka:

Sorry I wasn't around to answer your question, but yes, "Parrallel Occlusion Mapping" was the correct term.

The Relief mapping seems very similar in effect from the videos, but I haven't the time to read the papers just yet. Maybe next week ;-)

However, both of those techniques subtract the bump and silouhette depth details from the geometry. Nintendo's patents specifically add the depth details to the geometry.

Now, with the 2003 patent entitled "Z Texturing" (these other things are patent applications, not real patents), Nintendo has a texture/depth buffer-based method to supply real per-pixel deformation of a model (including silouhette and occlusion by the deformations). I think the end results will be comparable to the relief texturing videos to which you've linked.

"From the patents shown here, Nintendo's putting many eggs into the texture basket. Could this be their trump card?"

YES!

I'm disregarding all notions of "final secret" and such, but YES!, this is a trump card Nintendo's waiting to reveal.

They have ~10 patents related to using the depth buffer (a.k.a., z-buffer) and textures to efficiently impliment bump mapping , per-pixel z texturing (i.e., a poor-man's displacemnt mapping, and self-shadowin provided by the bump-mapping), per light full-scene lighting, and some other effects.

The secret is much like the Gamecube: put low-latency, pretty-high bandwidth, cheap memory on the GPU die. Also, the patent linked by this article is for a Texture coordinate stage in the GPU that gets shared by all texturing and pixel processing that generates the appropriate coordinates in the color, bump, shadow, and "z-texturing" maps and hands them to the pixel engines.

So, Nintendo graphics WILL, beyond any doubt, look very similar to what we're seening of the xbox360 and PS3. They're just done using textures and it should be *very* easy for developers to generate the textures (which they already do with the other systems and PC today) and apply them (API calls insted of shader programs).

Anonymous said...

Crud, I keep forgetting to add my tag.

The last comment at 11:54 a.m. is mine.

--wellsed

Anonymous said...

light-shedder and jonny b:

"Why should Nintendo patent all this stuff when in fact Ati is in charge of the contract(developing Hollywood) that Nintendo hand over to them ?"

The patent applications are from 2000, when ArtX was finishing the development of the Gamecube Flipper GPU. I believe most of this technology is not in the Flipper, but the patent applications preceed ATI's purchase of ArtX and Nintendo's current contract with ATI.

The likelyhood of the deal is that ATI is providing Nintendo with the desired chips and gets some licensing rights to the patents.

There are around a dozen patents from this time period that all interlock. This patent, discussed by this article, is for a hardware feature that makes using texture-based effects more efficient on low-power, low-cost (read, Revolution) hardware.

--wellsed

Anonymous said...

?


Like great Post Production houses. Nintendo, desire to make and or bring most of its own console techology to the fullest. And like Industrial Light & Magic and the Soho based Glasswork who did not wait for Aliaswavefront's next Maya versions to knock on their door. No, they developed their own 3d sofwares. Hired great talented software programmers ect. This is what Nintendo is doing. Developing its own unique R&D, in this case, 3d graphic advancement.

Its just an evelution that some god company need to become great!

PS: guys, Falafelkid has explain over a couples of times that this texture mapping stuff was not "the" secret Nintendp has to offer, but that allong with it they might be a possible team of "other" tech to be reveald as "the" Secret-s

thx

/madox

Arsenis said...
This comment has been removed by a blog administrator.
Arsenis said...

@ wellsed.

when posting just choose other and type your name, webpage is optional as is you name.

just so if you need the info...

Pekka said...

Well, that cleared things a bit! Thanks, wellsed!

(Z-texturing patent)

wellsed said...

arsenis:

Thanks, I didn't realize that.

Anywho, it appears a lot of these patents *ARE* actually Gamecube related.

The document:

http://cubemedia.ign.com/media/
news/documents/
embeddedforumkeynote.pdf

shows the TEV and texture coordinate parts of Flipper on page 30. These were the important parts related by the patent linked from the article.

Arsenis said...

anyhow if they can do graphics like RE4 on gamecube, Rev will defenetly will have good graphics as being 2 - 2.5 more powerful.

lets find a new topic will ya!

or just continue...

Anonymous said...

Kojima Discusses Game Ideas, Revolution's Appeal
by Shawn White (12/15/05)


In a recent interview conducted by Japenese publication Famitsu, Metal Gear Creator Hideo Kojima discussed a few ideas concerning his in-development Revolution game as well as the console's appeal to a wider demographic.

During the interview, Kojima expressed his intent on producing games that can be considered stylish and artistic in nature; a goal he considers best accomplished on the Nintendo Revolution.

Kojima mentioned that he wants to make games outside of his multi-million dollar franchise Metal Gear and that with Revolution, he can create games "based on life, [or] something organic." He did not further elaborate on his comment.

On a final note, Kojima pointed out that some aspect of the Revolution will make it "popular with girls" without providing explicit details.

Revolution Report will have more enigmatic statements from Hideo Kojima as they become available.

Anonymous said...

Ha...people that revealing the revolution guy is back.
He has no new posts but his site isn't forzen anymore

nz guy said...

yea we no hes back woo... er no

Anonymous said...

Yeah that might be special, if not the fact that several Xbox 360 games that are already out such as Kameo, Project Gotham Racing 3, and PDZ already used it and several PS3, and Xbox 360 games use the mapping technique.

Seriously did you really think it was some sort of secret or are you just that bored dude?

jon said...

nintendo has been saying all along that it has new techniques to reduce development costs and production times. I dont think that that process will be the last secret. Remember the secret of the controller was a very big change. I think the secret is something else all together different. It probably has to do with the way we view the video game. Maybe 3d projection or somehting like that would be cool.

dfadf said...

Microsoft Office
Office 2010
Microsoft Office 2010
Office 2010 key
Office 2010 download
Office 2010 Professional
Microsoft outlook
Outlook 2010
Windows 7
Microsoft outlook 2010

pandorasell said...

Silver pandora jewelry store comprises replacement atomic number 79 equally the metal by choice in that pandora jewelry website flavors manners. .925 alright superlative Ag Jewelry checks 92.5% complete silver pandora jewelry online and comprises assorted with extra debases to beef up them and abbreviate maculating pandora style jewelry coming through a hard-nosed alloy as pandora bracelet beads . The rise in metal prices, especially gold, have begun to alter pandora leather bracelet dashes. Designers are applying sterling pandora style bracelet a lot of today because of its gold pandora bracelet appeal and affordability. Designers care Jacques Louis pandora charm necklace David Yurman and those from pandora’s cause changed their focus and are making A-list assembles by .925 Fine Sterling pandora necklace charms Although Silver is affordable its price has quietly risen 300% inch the last 5 years and is astir across 50% pandora silver beads these class entirely. Many investment advisers* recommend a portion of your wealth be inward Ag and amber. Trend setters who comprised the first to begin wearing pandora necklace leather metal to match tatto last word and additional electric current styles as well do good along accepting pandora bracelet charms worth more than they paid for it.