In a deferred rendering setup (see Game Angst for a good discussion of deferred shading & lighting), lights are applied using data from screen-space buffers. Position, normal and other things are reconstructed from buffers and lighting is computed “in screen space”.

Because each light is applied to a portion of the screen, the pixels it computes can belong to different objects. If in any place of lighting computation you use textures with mipmaps, be careful. Most common use for mipmapped light textures is light “cookies” (aka Gobo).

Let’s say we have a very simple scene with a spot light:

Light’s angular attenuation comes from a texture like this:

If the texture has mipmaps and you sample it using the “obvious” way (e.g. tex2Dproj), you can get something like this:

Black stuff around the sphere is no good! It’s not the infamous half-texel offset in D3D9, not a driver bug, not a shader compiler bug and not the nature trying to prevent you from writing a deferred renderer.

It’s the mipmapping.

Mipmaps of your cookie texture look like this (128×128, 16×16, 8×8, 4×4 shown):

Now, take two adjacent pixels, where one belongs to the edge of the sphere, and the other belongs to the background object (technically you take a 2×2 block of pixels, but just two are enough to illustrate the point). When the light is applied, cookie texture coordinates for those pixels are computed. It can happen that the coordinates are very different, especially when pixels “belong” to entirely different surfaces that are quite far away from each other.

What the GPU does when texture coordinates of adjacent pixels are very different? Chooses a lower mipmap level so that texel to pixel density roughly matches 1:1. On the edges of this “wrong” screenshot, it happens that very small mipmap level is sampled, which is either black or white color (see 4×4 mip level).

What to do here? You could disable mip-mapping (which is not good for performance and not good for image quality). You could drop some smallest mip levels which might be enough and not that bad for performance. Another option is to manually supply LOD level or derivatives to sampling instructions, using something else than cookie texture coordinates. For example, derivative in view space position, or something like that. This might not be possible on lower shader models though.

So here are the D3D9 hacks known to me in one place.

Let me know if I missed something or got something wrong. I also want to figure out if Intel GPUs/drivers implement any of them.

Bits and pieces I found useful

• SSAO can be generated at a smaller resolution than screen, with depth+normals aware upsample/blur step.
• If random offset vector points away from surface normal, flip it. This makes random vectors be in the upper hemisphere, which reduces false occlusion on flat surfaces. Of course this requires having surface normals.
• When generating random vectors for your AO kernel:
  • Generate vectors inside unit sphere (not on unit sphere).
  • Use energy minimization to distribute your samples better, especially at low sample counts. See malmer.ru blog post.
• In your AO blurring/upsampling step: no need to sample each pixel for blur. Just skip some of them, i.e. make kernel offsets larger. See below.

Strided blur for AO

Normally you’d blur AO term using some sort of standard blur, for example separable Gaussian: horizontal blur, followed by vertical blur. How one can imagine horizontal blur kernel:

Here’s how Rune taught me how to blur better:

Rune:

The other thing is the blur. I tried to make the blur 4 times stronger, and it looks much better IMO without any artifacts I could see. I could even use 4x downsampling with that blur amount and still get acceptable results.

Aras:

how did you make it 4x stronger? (I was going to say that blur step is already quite expensive, and I don’t want to add more samples to make it even more expensive, yadda yadda)

Rune:

m_SSAOMaterial.SetVector (“_TexelOffsetScale”, m_IsOpenGL ?
  new Vector4 (4,0,1.0f/m_Downsampling,0) :
  new Vector4 (4.0f/source.width,0,0,0));
And similar for vertical.

Aras:

hmm. that’s strange :)

Rune:

I have no idea what I’m doing of course but it looks good.

Aras:

so this way it does not do Gaussian on 9×9 pixels, but instead only takes each 4th pixel. Wider area, but… it should not work! :)

Rune:

It creates a very fine pattern at pixel level but it’s way more subtle than the noise you get otherwise.

Aras:

ok (hides in the corner and weeps)

So yeah. The blur kernel can be “spread” to skip some pixels, effectively resulting in a larger blur radius for the same sample count:

Or even this:

Yes, it’s not correct blur. But that’s okay, we’re not building nuclear reactors that depend on SSAO blur being accurate. If you are, SSAO is probably a wrong approach anyway, I’ve heard it’s not that useful for nuclear stuff.

I’m not sure how this blur should be called. Strided blur? Interleaved blur? Interlaced blur? Or maybe everyone is doing that already and it has a well established name? Let me know.

Some images of blur in action. Raw AO term (very low – 8 – sample count and increased contrast on purpose):

Regular 9×9 blur (does not blur over depth+normals discontinuities):

Blur that goes in 2 pixel stride (effectively 17×17):

It does create a fine interleaved pattern because it skips pixels. But you get wider blur!

Blur that goes in 3 pixel stride (effectively 25×25):

At 3 pixel stride the artifacts are becoming apparent. But hey, this is very
low AO sample count, increased contrast and no textures in the scene.

For sake of completeness, the same raw AO term, but computed at 2×2 smaller resolution (still using low sample count etc.):

Now, 2×2 smaller AO, blurred with 3 pixels stride:

Happy blurring!

Here are my findings – with error visualization and shader performance numbers for some GPUs.

If you know any other method to encode/store normals in a compact way, please let me know!

Turns out some folks have figured out an approach that finally seems to work.

Here it is for my own reference:

• gamedev.net forum post by gjaegy
• Suggestion right there on my previous blog post comments
• Repost gamerendering blog
• Repost on gamedev.net forums again.

So here’s the proper way:

inline float4 EncodeFloatRGBA( float v ) {
  float4 enc = float4(1.0, 255.0, 65025.0, 160581375.0) * v;
  enc = frac(enc);
  enc -= enc.yzww * float4(1.0/255.0,1.0/255.0,1.0/255.0,0.0);
  return enc;
}
inline float DecodeFloatRGBA( float4 rgba ) {
  return dot( rgba, float4(1.0, 1/255.0, 1/65025.0, 1/160581375.0) );
}

That is, the difference from the previous approach is that the “magic” (read: hardware dependent) bias is replaced with subtracting next component’s encoded value from the previous component’s encoded value.

Well, finally I implemented it for upcoming Unity 2.6. I wrote some sort of a technical report here.

In short, I’m combining assembly fragments and doing simple temporary register allocation, which seems to work quite well. Performance is very similar to using fixed function (I know it’s implemented as vertex shaders internally by the runtime/driver) on several different cards I tried (Radeon HD 3xxx, GeForce 8xxx, Intel GMA 950).

What was unexpected: the most complex piece is not the vertex lighting! Most complexity is in how to route/generate texture coordinates and transform them. Huge combination explosion there.

Otherwise – I like! Here’s a link to the article again.

Got different lighting models (BRDFs) working. Without further ado, code snippets that produce real actual working shaders that work with lights & shadows and whatnot:

Simple Lambert (single color):

Properties
    Color _Color
EndProperties
Surface
    o.Albedo = _Color;
EndSurface
Lighting Lambert

Let’s add a texture:

Properties
    2D _MainTex
    Color _Color
EndProperties
Surface
    o.Albedo = SAMPLE(_MainTex) * _Color;
EndSurface
Lighting Lambert

Change light model to Half-Lambert (a.k.a. wrapped diffuse):

// ...everything the same
Lighting HalfLambert

Blinn-Phong, with constant exponent & constant specular color, modulated by gloss map in main texture’s alpha:

Properties
    2D _MainTex
    Color _Color
    Color _SpecColor
    Float _Exponent
EndProperties
Surface
    half4 col = SAMPLE(_MainTex);
    o.Albedo = col * _Color;
    o.Specular = _SpecColor.rgb * col.a;
    o.Exponent = _Exponent;
EndSurface
Lighting BlinnPhong

The same Blinn-Phong, with added normal map:

Properties
    2D _MainTex
    2D _BumpMap
    Color _Color
    Color _SpecColor
    Float _Exponent
EndProperties
Surface
    half4 col = SAMPLE(_MainTex);
    o.Albedo = col * _Color;
    o.Specular = _SpecColor.rgb * col.a;
    o.Exponent = _Exponent;
    o.Normal = SAMPLE_NORMAL(_BumpMap);
EndSurface
Lighting BlinnPhong

I also made an illustrative-style BRDF (see Illustrative Rendering in Team Fortress 2), but that only requires above sample to have “Lighting TF2″ at the end.

Another thing I tried is surface that has Albedo dependent on a viewing angle, similar to Layered Car Paint Shader. It works:

Properties
    2D _MainTex
    2D _BumpMap
    2D _SparkleTex
    Float _Sparkle
    Color _PrimaryColor
    Color _HighlightColor
EndProperties
Surface
    half4 main = SAMPLE(_MainTex);
    half3 normal  = SAMPLE_NORMAL(_BumpMap);
    half3 normalN = normalize(SAMPLE_NORMAL(_SparkleTex));
    half3 ns = normalize (normal + normalN * _Sparkle);
    half3 nss = normalize (normal + normalN);
    i.viewDir = normalize(i.viewDir);
    half nsv = max(0,dot(ns, i.viewDir));
    half3 c0 = _PrimaryColor.rgb;
    half3 c2 = _HighlightColor.rgb;
    half3 c1 = c2 * 0.5;
    half3 cs = c2 * 0.4;
    half3 tone =
        c0 * nsv +
        c1 * (nsv*nsv) +
        c2 * (nsv*nsv*nsv*nsv) +
        cs * pow(saturate(dot(nss,i.viewDir)), 32);
    main.rgb *= tone;
    o.Albedo = main;
    o.Normal = normal;
EndSurface
Lighting Lambert

Up next:

• How and where emissive terms should be placed. I cautiously omitted all emissive terms from the above examples (so my layered car shader is without reflections right now).
• Where should things like rim lighting go? I’m not sure if it’s a surface property (increasing albedo/emission with angle) or a lighting property (a back light).

My impressions so far:

• I like that I don’t have to write down vertex-to-fragment structures or the vertex shader. In most cases all the vertex shader does is transform stuff and pass it down to later stages, plus occasional computations that are linear over the triangle. No good reason to write it by hand.
• I like that the above shaders do not deal with how the rendering is actually done. For Unity’s case, I’m compiling them into single pass per light forward renderer, but they should just work with multiple lights per pass, deferred etc. Of course, that still has to be proven!

So far so good.

Series index: Shaders must die, Part 1, Part 2, Part 3.

Right now my experimental pipeline is:

1. Write a surface shader file
2. Perl script transforms it into Unity 2.x shader file
3. Which in turn is compiled by Unity into all lighting/shadows permutations, for D3D9 and OpenGL backends. Cg is used for actual shader compilation.

I have very simple cases working. For example:

Properties
    2D _MainTex
EndProperties
Surface
    o.Albedo = SAMPLE(_MainTex);
EndSurface

This is a “no bullshit” source code for a simple Diffuse (Lambertian) shader, 87 bytes of text.

The Perl script produces a Unity 2.x shader. This will be long, but bear with me – I’m trying to show how much stuff has to be written right now, when we’re operating on vertex/pixel shader level. See Attenuation and Shadows for Pixel Lights in Unity docs for how this system works.

Shader "ShaderNinja/Diffuse" {
Properties {
  _MainTex ("_MainTex", 2D) = "" {}
}
SubShader {
  Tags { "RenderType"="Opaque" }
  LOD 200
  Blend AppSrcAdd AppDstAdd
  Fog { Color [_AddFog] }
  Pass {
    Tags { "LightMode"="PixelOrNone" }
CGPROGRAM
#pragma fragment frag
#pragma fragmentoption ARB_fog_exp2
#pragma fragmentoption ARB_precision_hint_fastest
#include "UnityCG.cginc"
uniform sampler2D _MainTex;
struct v2f {
    float2 uv_MainTex : TEXCOORD0;
};
struct f2l {
    half4 Albedo;
};
half4 frag (v2f i) : COLOR0 {
    f2l o;
    o.Albedo = tex2D(_MainTex,i.uv_MainTex);
    return o.Albedo * _PPLAmbient * 2.0;
}
ENDCG
  }
  Pass {
    Tags { "LightMode"="Pixel" }
CGPROGRAM
#pragma vertex vert
#pragma fragment frag
#pragma multi_compile_builtin
#pragma fragmentoption ARB_fog_exp2
#pragma fragmentoption ARB_precision_hint_fastest
#include "UnityCG.cginc"
#include "AutoLight.cginc"
struct v2f {
    V2F_POS_FOG;
    LIGHTING_COORDS
    float2 uv_MainTex;
    float3 normal;
    float3 lightDir;
};
uniform float4 _MainTex_ST;
v2f vert (appdata_tan v) {
    v2f o;
    PositionFog( v.vertex, o.pos, o.fog );
    o.uv_MainTex = TRANSFORM_TEX(v.texcoord, _MainTex);
    o.normal = v.normal;
    o.lightDir = ObjSpaceLightDir(v.vertex);
    TRANSFER_VERTEX_TO_FRAGMENT(o);
    return o;
}
uniform sampler2D _MainTex;
struct f2l {
    half4 Albedo;
    half3 Normal;
};
half4 frag (v2f i) : COLOR0 {
    f2l o;
    o.Normal = i.normal;
    o.Albedo = tex2D(_MainTex,i.uv_MainTex);
    return DiffuseLight (i.lightDir, o.Normal, o.Albedo, LIGHT_ATTENUATION(i));
}
ENDCG
  }
}
Fallback "VertexLit"
}

Phew, that is quite some typing to get simple diffuse shader (1607 bytes)! Well, at least all the lighting/shadow combinations are handled by Unity macros here. When Unity takes this shader and compiles into all permutations, it results in 58 kilobytes of shader assembly (D3D9 + OpenGL, 17 light/shadow combinations).

Let’s try something slightly different: bumpmapped, with a detail texture:

Properties
    2D _MainTex
    2D _Detail
    2D _BumpMap
EndProperties
Surface
    o.Albedo = SAMPLE(_MainTex) * SAMPLE(_Detail) * 2.0;
    o.Normal = SAMPLE_NORMAL(_BumpMap);
EndSurface

This is 173 bytes of text. Generated Unity shader is 2098 bytes, which compiles into 74 kilobytes of shader assembly.

In this case, the processing script detects that surface shader modifies normal per pixel, and does the necessary tangent space light transformations. It all just works!

So this is where I am now. Next up: detect which lighting model to use based on surface parameters (right now it always uses Lambertian). Fun!

Ok, now that the controversial bits are done, let’s continue.

Most of this can be (and probably is) wrong, and I haven’t given it enough thought yet. But here’s my thinking about shaders of “regular scene objects”. All of below is about things that need to interact with lighting; I’m not talking about shaders for postprocessing, one-off uses, special effects, GPGPU or kitchen sinks.

Operating on vertex/pixel shader level is a wrong abstraction level

Instead, it should be separated out into “surface shader” (albedo, normal, specularity, …), “lighting model” (Lambertian, Blinn Phong, …) and “light shader” (attenuation, cookies, shadows).

• Probably 90% of the cases would only touch the surface shader (mostly mix textures/colors in various ways), and choose from some precooked lighting models.
• 9% of the cases would tweak the lighting model. Most of the things would settle for “standard” (Blinn-Phong or similar), with some stuff using skin or anisotropic or …
• The “light shader” only needs to be touched once in a blue moon by ninjas. Once the shadowing and attenuation systems are implemented, there’s almost no reason for shader authors to see all the dirty bits.

Yes, current hardware operates on vertex/geometry/pixel shaders, which is a logical thing to do for hardware. After all, these are the primitives it works on when rendering. But those primitives are not the things you work on when authoring how a surface should look or how it should react to a light.

Simple code; no redundant info; sensible defaults

In the ideal world, here’s a simple surface shader (the syntax is deliberately stupid):

Haz Texture;
Albedo = sample Texture;

Or with bump mapping added:

Haz Texture;
Haz NormalMap;
Albedo = sample Texture;
Normal = sample_normal NormalMap;

And this should be all the info you have to provide. This would choose the lighting model based on used things (in this case, Lambertian). It would somehow just work with all kinds of lights, shadows, ambient occlusion and whatnot.

Compare to how much has to be written to implement a simple surface in your current shader technology, so that it would work “with everything”.

From the above shader, proper hardware shaders can be generated for DX9, DX11, DX1337, OpenGL, next-gen and next-next-gen consoles, mobile platforms with capable hardware, etc.

It can be used in accumulative forward rendering, forward rendering with multiple lights per pass, hybrid (light pre-pass / prelight) rendering, deferred rendering etc. Heck, even for a raytracer if you have one at hand.

I want!

Now of course, it won’t be as nice as more complex materials have to be expressed. Some might not even be possible. But shader text complexity should grow with material complexity; and all information that is redundant, implied, inferred or useless should be eliminated. There’s no good reason to stick to conventions and limits of current hardware just because it operates like that.

Shaders must die!

I don’t really know why that happens, because it seems that most modern cards don’t have fixed function units, so internally they are running shaders anyway. DX9 runtime on Vista’s WDDM also seems to be only handling shaders to the driver internally. Still, for some reason somewhere the precision does not match…

How such a task should be approached?

My requirements are:

• Should handle any possible state combination in D3D fixed function T&L.
• D3D 9.0c, using vertex shader 2.0 is ok. For now I don’t care about OpenGL.
• No HLSL at runtime. I don’t want to add a megabyte or more to Unity web player just for HLSL. DX9 shader assembly is ok, because we already have the assembler code.
• Should work as fast (or close to) as the regular fixed function pipeline.

I looked at ATI’s FixedFuncShader sample. It’s an ubershader approach; one large (230 instructions or so) shader with static VS2.0 branching. It had some obvious places to optimize, I could get it down to 190 or so instructions, kill some rcp’s and reduce the amount of constant storage by 2x.

Still, it did not handle some things in the D3D T&L or had some issues:

• It assumes one input UV, one output UV and no texture matrices. This place in T&L gets quite convoluted – any input UVs or a texgen mode can be transformed by matrices of various sizes, and routed into any output UVs.
• It was not using full T&L lighting model. No biggie here.
• I haven’t checked with NVShaderPerf or AMD ShaderAnalyzer yet, but last time I checked the static branch instruction was taking two clocks on some NV architecture. So ubershader approach does not come for free.

Another thing I’m considering, is to combine final shader(s) from assembly fragments, with some simple register allocation.

In T&L shader code, there’s only limited set of could-be-redundant computations, mostly computing world space position, camera space normal, view vector and so on (those could be used lighting, texgen or fog). Those computations can be explicitly put into separate fragments, and later fragments could just use their result.

What is left then is some register allocation. A shader assembly fragment could want some temporary registers for internal use (this is simple, just give it a bunch of unused registers), also want some registers as input (from previous fragments), and save some output in registers.

Again, I haven’t checked with shader performance tools, but I think, guess and hope that the drivers do additional register allocation, liveness analysis etc. when converting D3D shader bytecode into hardware format. This would mean that I can be quite sloppy with it, i.e. don’t have to implement some super smart allocation scheme.

I wrote some experimental code for the shader assembly combiner and so far it looks like a reasonable approach (and not too hard either).

Does that make sense? Or did everyone solve those problems eons ago already?

Edit: half a year later, I wrote a technical report on how I implemented all this: http://aras-p.info/texts/VertexShaderTnL.html

Everyone knows the Valve’s hardware survey. But what if your target game players are not the traditional “big budget AAA game” type? For example, at the moment most Unity Web Player games are oriented to much more casual market, so hardware there might be very different. And indeed, turns out it is quite different.

Without further ado, here’s the data we have: Unity Web Player hardware statistics.

It’s about two million data points since we started gathering it earlier this year.

Some subjective points of interest (I’ll be using current data for 2008 Q3 here):

• Operating systems: Mac OS X is 2.5%, the rest is Windows. 64 bit Windows haven’t really picked up yet (0.7%). Windows 2000 is dying fast (0.7%). OS X Leopard already took over OS X Tiger.
• CPUs: poor Transmeta :) Dual core CPUs are becoming the norm (46%).
• Graphics cards: quite sad, in fact… top 15 cards are slow or horribly slow. Capability wise, they are quite good, with about 70% having shader model 2.0 or higher. Shader model 1.x cards are dead. “Can has DX10” is 2.7%.
• Casual machines don’t have lots of RAM. Nor lots of VRAM.
• Most popular nvidia driver? 56.73. Looks like this is the driver that comes integrated in XP SP2… Now, who says regular people ever update their drivers? Likewise, vga.dll (i.e. standard VGA) is 1.6% of machines; additional 1.5% don’t report any driver (not sure how that happens…).

So yeah. Casual machines: capabilities quite okay, performance low, low, low. That’s life.

From AnandTech’s Radeon HD 4850 & 4870 review: I can has vertex data?

Edit: gee, nowadays the reviews have funny performance measures. Like, FPS per square centimeter (of GPU die size)! It does actually make (some) sense, but it’s still funny. Frames per second per square centimeter… mmm… delicious.

Here’s the “best available” code that I have now:

inline float4 EncodeFloatRGBA( float v ) {
  return frac( float4(1.0, 255.0, 65025.0, 160581375.0) * v ) + bias;
}
inline float DecodeFloatRGBA( float4 rgba ) {
  return dot( rgba, float4(1.0, 1/255.0, 1/65025.0, 1/160581375.0) );
}

Before I thought that bias should be +0.5/255.0 normally, except it had to be around -0.55/255.0 on Radeon cards (older than Radeon HD series). Well, turns out I was wrong, the bias mostly has to be around -0.5/255.0.

Here’s the list (same bias on Windows/D3D9 and OS X/OpenGL, so it seems to be hardware dependent, and not something in API/drivers):

• Radeon 9500 to X850: -0.61/255
• Radeon X1300 to X1900: -0.66/255
• Radeon HD 2xxx/3xxx: -0.49/255
• GeForce FX, 6, 7, 8: -0.48/255
• Intel 915, 945, 965: -0.5/255

Those are the best bias values I could find. Still, every once in a while (rarely) encoding the value to RGBA texture and reading it back would produce something where one channel is half a bit off. Not a problem if you were encoding numbers were originally 0..1 range, but for example if you were encoding something that spans over whole range of the camera, then 0..1 range gets expanded into 0..FarPlane…

And all of a sudden there are huge precision errors, up to the point of being unusable. I just tried doing a quick’n'dirty depth of field and soft particles implementation using depth encoded this way… not good.

Oh well. Has anyone successfully used encoding of high precision number into RGBA channels before?

My prediction? OpenCL should be something along lines of CUDA or BrookGPU. Will work on various DX10-level graphics cards, and on the CPU. I think trying to target older graphics cards does not make sense – using real actual integer types is useful in general purpose computing (DX10 tech), and Apple will probably only be shipping DX10 level graphics cards in a year (at the moment only Intel cards in Macs are DX9 level; the rest is GeForce 8s and Radeon HDs). With a multithreaded CPU fallback any older machines will be taken care of anyway (and leaves the future open for Larrabees). So yeah, quite similar to BrookGPU actually.

It has “open” in the title, so maybe they will make it for other platforms as well. I doubt that they will ship implementation though; perhaps just make it royalty/patent/whatever free and publish the spec. Which is about the same level of “openness” as other technologies with “open” in their name (OpenGL, OpenAL, OpenMP, OpenCV, …) – not exactly open, but not the worst kind either.

Oh, and suddenly there are new uses for other technologies recently developed at Apple, like LLVM or clang.

We’ll see how it goes.

In short, I’m impressed.

It runs rendering tests almost correctly; the only minor bugs seem to be somewhere in attenuation of fixed function vertex lights. Everything else, including shaders, shadows, render to texture works without any problems.

Performance wise, of course it’s dozens to hundreds times slower than a real graphics card, but hey. I also tested with Intel 965 (aka GMA X3000) integrated graphics for comparison. All this on Intel Core2 Quad (Q6600), 3 GB RAM, Windows XP SP2.

• Avert Fate demo: Radeon HD 3850 about 300 FPS, SwiftShader about 5 FPS (about 15 FPS if per-pixel lighting is turned off), Intel 965 about 22 FPS (about 50 FPS if per-pixel lighting is turned off).
• Scene with lots of objects and lots of shadow-casting lights: Radeon HD 3850 about 76 FPS, SwiftShader 2.5 FPS, Intel – shadows not supported, duh.
• High detail terrain with lots of vegetation and four cameras rendering it simultaneously: Radeon HD 3850 about 68 FPS, SwiftShader about 3 FPS, Intel 965 about 12 FPS.

Ok, so SwiftShader loses on performance to Intel 965, but the difference is only “a couple of times”, and not in order of magnitude or so. Pretty good I’d say.

Not so long ago, some well known graphics guys went to work for Intel. A while ago Intel acquired Neoptica…

Signs of Larrabee coming? Intel starting to take GPUs seriously? Something else?

Meet kindernoiser – 4 kilobytes, quaternion Julia fractal on the GPU, screen space ambient occlusion and so on. iq has a nice article on the tech behind SSAO.

Keep ‘em coming!

Here is what I am doing right now:

inline float4 EncodeFloatRGBA( float v ) {
  return frac( float4(1.0, 255.0, 65025.0, 160581375.0) * v ) + 0.5/255.0;
}
inline float DecodeFloatRGBA( float4 rgba ) {
  return dot( rgba, float4(1.0, 1/255.0, 1/65025.0, 1/160581375.0) );
}

And this seems to work fine almost everywhere (see below). Why am I doing this – good question, I don’t have a hard theory on which bits go where and so on. I think I saw someone on gamedev.net forums saying that in hardware 0 == 0.0 and 255 == 1.0, and that truncation is actually done on the values (not rounding). So that would mean you multiply by 255 and add a half of a bit.

Now, the trick: the above does not quite work on Radeons (at least the X1600 that I’m mostly developing on while I’m on a Mac). Instead of adding 0.5/255.0, you have to subtract 0.55/255.0 – and that value is still not perfect, but that’s the best I could come up with by plowing through various combinations. I have no idea why this must be performed (24 bit internal precision? or does it round up? something else?). On GeForces and even Intel’s shader-capable hardware, the expected +0.5/255.0 value works.

…anyone up to figuring out the mathematical proof on why encoding/decoding this way actually works? :) And yes, the last component (the one that uses 160581375) is pretty much meaningless.

inline float4 EncodeFloatRGBA( float v ) {
  return frac( float4(1.0, 256.0, 65536.0, 16777216.0) * v );
}
inline float DecodeFloatRGBA( float4 rgba ) {
  return dot( rgba, float4(1.0, 1.0/256.0, 1.0/65536.0, 1.0/16777216.0) );
}

and everything is fine until sometimes, somewhere there’s “something wrong”. Must be rounding or quantizations errors; or maybe I should use 255 instead of 256; plus optionally add or subtract 0.5/256.0 (or would that be 0.5/255.0?). Or maybe the error is entirely somewhere else, and I’m just chasing ghosts here!

What would you do then? Why, of course, build an Encoding Floats Into Textures Studio 2007! (don’t tell me it’s not a great idea for a commercial software package! game studios would pay insane amounts of money for a tool like this!) The images here are exactly that – render into a texture, encoding UV coordinate as RGBA, then read from that texture, displaying RGBA and error from the expected value in some weird way. Turns out image postprocessing filters in Unity are a pretty good tool to do all this. Yay!

Sometimes in situations like this I figure out that graphics hardware still leaves a lot to be desired. This last image shows some calculations that depend only on the horizontal UV coordinate, so they should produce some purely vertical pattern (sans the part at the bottom, that is expected to be different). Heh, you wish!

The graphics hardware has been quite successful so far at hiding memory latencies (i.e. when sampling textures). It does so (according to my understanding) by having a looong pixel pipeline, where hundreds (or thousands) pixels might be at one or another processing stage. ATI talks about this in big letters (R520 dispatch processor) and speculations suggest that GeForceFX had something like that (article). I have no idea about the older cards, but presumably they did something similar as well.

I am not sure how the vertex texture fetches are pipelined – pretty slow performance on GeForce6/7 suggest that they aren’t :) Probably vertex shaders in current cards operate in a simpler way – just fetch the vertices and run whole shaders on them (in contrast to pixel shaders, which seem to run just several instructions, then go to another pixels, return back, etc.).

With DX10, we have arbitrary memory fetches in any stage of the pipeline. Even the boundary between different fetch types is somewhat blurry (constant buffers vs. arbitrary buffers vs. textures) – perhaps they will differ only in bandwidth/latency (e.g. constant buffers live near the GPU while textures live in video memory).

So, with arbitrary memory fetches anywhere (and some of them being high latency), everything needs to have long pipelines (again, just my guess). This is all great, but the longer the pipeline, the worse it performs in non-friendly scenarios: pipeline flush is more expensive, drawing just a couple of “things” (primitives, vertices, pixels) is inefficient, etc.

I guess we’ll just learn a new set of performance rules for tomorrow’s hardware!

Back to GS pipelining: I imagine that the “slow” scenarios would be like this: vertices have shaders with dynamic branches or memory fetches differing vastly in execution lengths – so GS has to wait for all vertex shaders of the current primitive (optional: plus topology) to finish; and then each GS has dynamic branches or memory fetches, and outputs different number of primitives to the rasterizer. If I’d were hardware, I’d be scared :)

The reason seems to be that on older Radeons MSAA does not resolve alpha channel, which obsiously messes things up in my case. I’m using RGBE8 encoding for the main rendertarget, and it RGB gets MSAA’d and exponent not – then oh well, no good anti aliasing most of the time.

Of course I could always manually supersample everything, but this would defeat the whole point of the demo. Or I could render everything in two passes, one for RGB and one for exponent – but this also is not very nice…

Probably I’ll just release the demo as it is now and wait for possible feedback. Or dig up an old Radeon somewhere and debug more – but replacing the video card in my Shuttle XPC is not an easy task :)

I’m trying to support both HDR and FSAA at the same time on ordinary DX9 hardware (no Radeons 1k) by using RGBE8 rendertarget for the main scene. It’s all okay so far.

The most difficult task right now is making it look good. Once I have that I’ll post the results.

We have a high-poly model for the main character (~200k tris), for the demo we use low-poly (~6500 tris) and a normalmap. Now, I’ve put 128 lights scattered on the hemisphere above him, each using shadow buffer. I have 4 shadow buffers, render to these from four lights, then render the character, fetching shadows from four shadowmaps at once. The result is that it’s almost realtime ambient occlusion for the animating character, and it runs at ~40FPS on my geforce 6800gt!

This is of course pretty useless, we don’t need realtime AO in the demo. But it has been nice :)