Fool me, this is real good shit ! [the VR thread]

[gilesgoat]

The stagnation thing is a technology one. The pace we can innovate at has slowed a bit as the process nodes and frequency jumps aren't cascading along at the rate they were. Moore's Law is long gone, I fear, although we're still managing to generate steady growth.

For this kind of stuff you should really get in touch with people like Carmack or such ..

We are well plugged in up and down the whole software stack.

On a similar topic I had a pint with Callum last week and he was very impressed with TxK and very surprised I did a part of the soundtrack .

Oh that's nice, we still hadn't hear from him back since we gave him "a thing to look at"

Mind you if your company would ever be interested in some "VR killer app" let us know ... and send GPUs this way

[Dio]

I have mentioned you.

Not that I've seen it yet or owt...

[gravy]

Merging the pixel shading is pretty hard if you think about it. As soon as you've got interpolation in the shader then you're dependent on the barycentrics, and those are going to be different for the two eyes. The final result may be very similar but it is slightly different (and I imagine if you tried to eliminate that difference you'd start seeing snapping or shimmering).

I know nothing about this! But that's never stopped me before

You render a triangle twice, once per eye, and they're different shapes on screen

Are the fetched texels in the smaller triangle a subset of the texels used by the bigger one?

Or is there an different triangle that would satisfy that for both triangles?

If you could get one set of main memory access for each triangle that'd decrease bandwidth to main memory which my understanding (probably hugely wrong) is one the main power consumers in portable gfx chips.

It'd be a horrible VR specific bodge that'd work for two minor view changes but if this ends up being a big platform it's a bodge that'll get made.

Assuming it's useful in the first case

[Dio]

You render a triangle twice, once per eye, and they're different shapes on screen

Are the fetched texels in the smaller triangle a subset of the texels used by the bigger one?

Or is there an different triangle that would satisfy that for both triangles?

Yes, for the fetched texels, given a bunch of likely assumptions (like mipmapping is in play). You'll hit the same texels, but the bilinear filtering coefficients will tend to be different, since they're dependent on the interpolated texture coords which in turn are generated from the triangle barycentric coords.

If you could get one set of main memory access for each triangle that'd decrease bandwidth to main memory which my understanding (probably hugely wrong) is one the main power consumers in portable gfx chips.

Sure. But what percentage of memory bandwith is shading, rather than depth/colour, especially with compression? Plus it's not just power, low-end chips have really crappy access to memory generally.

So yes, this sort of thing does help there. But so far colour me unconvinced that VR and that end of the market is going a long way, when other people are asking for 8kx8k per eye at guaranteed 100+fps and wanting much more shading horsepower to compensate for the eye's enhanced ability to spot techniques like bump mapping being fake in stereo environments. The thing I really want to think about is reduced compute requirements and most eye-sharing tricks don't seem to immediately offer that.

[gilesgoat]

Merging the pixel shading is pretty hard if you think about it. As soon as you've got interpolation in the shader then you're dependent on the barycentrics, and those are going to be different for the two eyes. The final result may be very similar but it is slightly different (and I imagine if you tried to eliminate that difference you'd start seeing snapping or shimmering).

I know nothing about this! But that's never stopped me before

You render a triangle twice, once per eye, and they're different shapes on screen

Are the fetched texels in the smaller triangle a subset of the texels used by the bigger one?

Or is there an different triangle that would satisfy that for both triangles?

If you could get one set of main memory access for each triangle that'd decrease bandwidth to main memory which my understanding (probably hugely wrong) is one the main power consumers in portable gfx chips.

It'd be a horrible VR specific bodge that'd work for two minor view changes but if this ends up being a big platform it's a bodge that'll get made.

Assuming it's useful in the first case

My very little understanding in that kind of stuff specifically "for mobile" but "by nose" ( i.e. seeing what happens in certain things and what is done ) is that "the most killer is bandwidth" ( and the other is that raster performance/pixel shader in mobile stuff is 'better to avoid it' because it's so poor ).

I got quite shocked to realize that ( well it's a supposition based on some 'hints' ) that even if you put an uncompressed texture in a GPU quite likely the driver/whatever will actually compress it because sounds to me it's "cheaper/faster" to have an HW decompressor that decompresses it "on the fly on the chip" rather than moving uncompressed data around, so you maybe better put/use textures compressed in its "native compressed format" you may save something.

I WONDER if anyone ever tought/done stuff like that for vertexes too.

I think the fact is for any rendering scene at the very least vertexes and textures would be the same, if that could be cached already would be something good I think stuff anyway "diverges" at the point you may need a second GPU the moment a matrix is changed ( as in per-eye ) because from that on "it's two different worlds already" all bets/optimisations/things are off, my guess is "in the name of simplicity" maybe two GPUs with some "synchronization" of such sharing just a common vertex/texture database could be the way to go ?

Mind you if we want to be "more picky" or "even more into detail" the two eyes are NOT "symmetrical", there can be various issues where effectively even if you start from a scene ( buffer of polys ) that are the same from there on camera/world and especially projection/distorsion per eye can be totally different so I don't know how much a single CPU can do to "share" stuff.

[venusian]

I'm firmly in the a-little-knowledge camp here, but if I was a 3D genius....

You only need really hi-def detail in the centre of each image (or rather the center as received by the eye). We need a render technique which can scale compute complexity with distance from the focus point. This might be extensible to temporal complexity. Ray tracing seems like an obvious good fit.

Dropping the frame rate is not acceptable in a VR environment. There needs to be a scene management technique that can scale the compute complexity of an scene, in terms of render time, before it's rendered so each frame arrives on time. Hard realtime rendering, if you like, rather the soft we typically have now.

For the former I remember seeing games in the 16-bit era which appeared to render objects to separate frame buffers asynchronously with the screen refresh then z-composite them together using the most recent version. E.g. for a racing game the track is drawn at 25Hz and opponent cars are positioned on it at 25Hz but only rerendered at (say) 5Hz. It's kind of like on-the-fly billboarding, I guess. I wonder how much this could be used for low-def peripheral objects in VR and whether it might even help with fast response to motion; you can increase the screen update rate without linearly scaling the render rate.

I want more spare time.

[Dio]

There needs to be a scene management technique that can scale the compute complexity of an scene, in terms of render time, before it's rendered so each frame arrives on time. Hard realtime rendering, if you like, rather the soft we typically have now.

Given arbitrary shading programs, doesn't that mean solving the halting problem?

[venusian]

I WONDER if anyone ever tought/done stuff like that for vertexes too.

I'm in full-on bullshit mode this afternoon.

I wonder if one could apply fractal image compression to vertex data. It's good for naturalistic image so might be good for naturalistic animated meshes too. I'm given to understand it's relatively cheap to decompress, at least in software, but maybe it's too recursive to do efficiently in hardware

[venusian]

There needs to be a scene management technique that can scale the compute complexity of an scene, in terms of render time, before it's rendered so each frame arrives on time. Hard realtime rendering, if you like, rather the soft we typically have now.

Given arbitrary shading programs, doesn't that mean solving the halting problem?

Yes. Sort of. You'd need to solve the halts-this-week problem. Scaling the difficulty from halts-ever to halts-by-wednesday means it's gonna be infinitely easy doesn't it?

[Dio]

My very little understanding in that kind of stuff specifically "for mobile" but "by nose" ( i.e. seeing what happens in certain things and what is done ) is that "the most killer is bandwidth" ( and the other is that raster performance/pixel shader in mobile stuff is 'better to avoid it' because it's so poor ).

Mmmm. Not really. It depends, a lot.

I got quite shocked to realize that ( well it's a supposition based on some 'hints' ) that even if you put an uncompressed texture in a GPU quite likely the driver/whatever will actually compress it because sounds to me it's "cheaper/faster" to have an HW decompressor that decompresses it "on the fly on the chip" rather than moving uncompressed data around, so you maybe better put/use textures compressed in its "native compressed format" you may save something.

It was common for a while. Indeed, I have a patent related to doing that smarter - I derived a method for predicting the amount visual distortion introduced by BC1.

Not so much nowadays as far as I'm aware, not on PC at least.

[Dio]

Yes. Sort of. You'd need to solve the halts-this-week problem. Scaling the difficulty from halts-ever to halts-by-wednesday means it's gonna be infinitely easy doesn't it?

I suspect it's impossible for any program that's not a single basic block. Even then, predicting execution time on a GPU can be a nightmare because of cases like non-mipmapped textures; combine a pathologically missing sampling pattern with something sat on the wrong side of the PCIE bus and you could take a thousand-plus-cycle miss on every single texel - so you need to predict the sampling pattern of your textures to get a reasonably accurate estimate of time taken. Even with mipmapping you can see some variance...

[gravy]

You render a triangle twice, once per eye, and they're different shapes on screen

Are the fetched texels in the smaller triangle a subset of the texels used by the bigger one?

Or is there an different triangle that would satisfy that for both triangles?

Yes, for the fetched texels, given a bunch of likely assumptions (like mipmapping is in play). You'll hit the same texels, but the bilinear filtering coefficients will tend to be different, since they're dependent on the interpolated texture coords which in turn are generated from the triangle barycentric coords.

If you could get one set of main memory access for each triangle that'd decrease bandwidth to main memory which my understanding (probably hugely wrong) is one the main power consumers in portable gfx chips.

Sure. But what percentage of memory bandwith is shading, rather than depth/colour, especially with compression? Plus it's not just power, low-end chips have really crappy access to memory generally.

So yes, this sort of thing does help there. But so far colour me unconvinced that VR and that end of the market is going a long way, when other people are asking for 8kx8k per eye at guaranteed 100+fps and wanting much more shading horsepower to compensate for the eye's enhanced ability to spot techniques like bump mapping being fake in stereo environments. The thing I really want to think about is reduced compute requirements and most eye-sharing tricks don't seem to immediately offer that.

It wouldn't surprise me if a bunch of current realtime rendering techniques changed or were dropped to get better VR fidelity.

I'm also pretty much convinced the killer app for VR is tetherless which in turn could mean mobile GPUs are going to be important. (Though it could mean just being a receiver for the grunty machine).

But it was a bit of a surprise how much nicer using a tetherless Gear VR was over the other systems currently in prototype.

It's interesting though and probably kind of scary for AMD / Nvidia. There's the potential for a massive requirements change for GPUs and the last time that happened Imagine and various others swooped in and eat their lunch. Same kind of thing with Intel and ARM.

Exciting times though

[Dio]

(Though it could mean just being a receiver for the grunty machine).

They need new wireless standards for this. Lot of data to shift a short distance with realtime requirements. Nothing that currently exists is anywhere near the latency figures needed.

It's interesting though and probably kind of scary for AMD / Nvidia. There's the potential for a massive requirements change for GPUs and the last time that happened Imagine and various others swooped in and eat their lunch. Same kind of thing with Intel and ARM.

Huge opportunity for us. Definitely not scared here. My biggest fear is that a percentage of the games get it wrong, make everyone ill and VR dies back off for another ten years. Getting across the geek threshold is the challenge.

[Yak]

It's surprising how much tetherless VR and a spinny chair reduces the nausea effect in first-person VR games. It sounds silly but I think it's significant.

[gilesgoat]

*SNIP*
Huge opportunity for us. Definitely not scared here. My biggest fear is that a percentage of the games get it wrong, make everyone ill and VR dies back off for another ten years. Getting across the geek threshold is the challenge.

This is where we hope to have an edge/advantage, in fact "it's SO EASY TO GET IT WRONG, lot harder to get it right" ( but NOT impossible ).

My hope that is "a number of really good things" will show that "YES VR CAN BE DONE RIGHT IF DONE RIGHT" and that the "not doing it right" will be a kind of "natural threshold" for "those that just thinks to jump on a wagon and those actually aiming at doing really good SW".

What scares me is if "another situation a la' iOS market" is going to happen, but it SHOULD not happen "that soon" because there are NOT the numbers in place for anything like that to happen also we are talking about "a different type of quality required", if the SW is not done "right" as you say it just turns out shit and/or makes people ill.

Thing is VR should not die because "it can definitely be proven that it can be done right" .. now I wonder what's going to happen if/when "big names" like EA or such would get into "for real" making some really "big title that shows how is done".

That could maybe convince and go "beyond the geek" but it could also spell "a bit of an end" for others, however I think that at least for some time "people doing VR right" won't be that many HOPEFULLY "those doing it wrong and trying to sell it" won't spoil the market and the whole experience .. fingers crossed ..

"In my ideal world", I suppose, I think "there should be a place for everyone doing VR right" .. and there should be NOT "for all those doing stuff that makes people sick" .. I think anyway VR is maybe "too big to die too soon" because "watch a really good demo and is SO impressive you simply can't forget it".

Real coffee did not vanish despite all the millions of "infamous and horrendous black beverages" sold around PRETEND to be coffee

Actually those kinda show the difference between what REAL coffee is and what not