The saga continues! In short, I need to pack a floating point number in [0..1) range into several channels of 8 bit/channel render texture. My previous approach is not ideal.

Turns out some folks have figured out an approach that finally seems to work.

Here it is for my own reference:

• gamedev.net forum post by gjaegy

• Suggestion right there on my previous blog post comments

• Repost gamerendering blog

• Repost on gamedev.net forums again.

So here’s the proper way:

inline float4 EncodeFloatRGBA( float v ) {
  float4 enc = float4(1.0, 255.0, 65025.0, 16581375.0) * v;
  enc = frac(enc);
  enc -= enc.yzww * float4(1.0/255.0,1.0/255.0,1.0/255.0,0.0);
  return enc;
}
inline float DecodeFloatRGBA( float4 rgba ) {
  return dot( rgba, float4(1.0, 1/255.0, 1/65025.0, 1/16581375.0) );
}

That is, the difference from the previous approach is that the “magic” (read: hardware dependent) bias is replaced with subtracting next component’s encoded value from the previous component’s encoded value.

Almost half a year ago I was wondering how to implement T&L; in vertex shaders.

Well, finally I implemented it for upcoming Unity 2.6. I wrote some sort of a technical report here.

In short, I’m combining assembly fragments and doing simple temporary register allocation, which seems to work quite well. Performance is very similar to using fixed function (I know it’s implemented as vertex shaders internally by the runtime/driver) on several different cards I tried (Radeon HD 3xxx, GeForce 8xxx, Intel GMA 950).

What was unexpected: the most complex piece is not the vertex lighting! Most complexity is in how to route/generate texture coordinates and transform them. Huge combination explosion there.

Otherwise - I like! Here’s a link to the article again.

Continuing the series (see Part 1, Part 2)…

Got different lighting models (BRDFs) working. Without further ado, code snippets that produce real actual working shaders that work with lights & shadows and whatnot:

Simple Lambert (single color):

 Properties
     Color _Color
 EndProperties
 Surface
     o.Albedo = _Color;
 EndSurface
 Lighting Lambert

Let’s add a texture:

 Properties
     2D _MainTex
     Color _Color
 EndProperties
 Surface
     o.Albedo = SAMPLE(_MainTex) * _Color;
 EndSurface
 Lighting Lambert

Change light model to Half-Lambert (a.k.a. wrapped diffuse):

 // ...everything the same
 Lighting HalfLambert

Blinn-Phong, with constant exponent & constant specular color, modulated by gloss map in main texture’s alpha:

 Properties
     2D _MainTex
     Color _Color
     Color _SpecColor
     Float _Exponent
 EndProperties
 Surface
     half4 col = SAMPLE(_MainTex);
     o.Albedo = col * _Color;
     o.Specular = _SpecColor.rgb * col.a;
     o.Exponent = _Exponent;
 EndSurface
 Lighting BlinnPhong

The same Blinn-Phong, with added normal map:

 Properties
     2D _MainTex
     2D _BumpMap
     Color _Color
     Color _SpecColor
     Float _Exponent
 EndProperties
 Surface
     half4 col = SAMPLE(_MainTex);
     o.Albedo = col * _Color;
     o.Specular = _SpecColor.rgb * col.a;
     o.Exponent = _Exponent;
     o.Normal = SAMPLE_NORMAL(_BumpMap);
 EndSurface
 Lighting BlinnPhong

I also made an illustrative-style BRDF (see Illustrative Rendering in Team Fortress 2), but that only requires above sample to have “Lighting TF2” at the end.

Another thing I tried is surface that has Albedo dependent on a viewing angle, similar to Layered Car Paint Shader. It works:

 Properties
     2D _MainTex
     2D _BumpMap
     2D _SparkleTex
     Float _Sparkle
     Color _PrimaryColor
     Color _HighlightColor
 EndProperties
 Surface
     half4 main = SAMPLE(_MainTex);
     half3 normal  = SAMPLE_NORMAL(_BumpMap);
     half3 normalN = normalize(SAMPLE_NORMAL(_SparkleTex));
     half3 ns = normalize (normal + normalN * _Sparkle);
     half3 nss = normalize (normal + normalN);
     i.viewDir = normalize(i.viewDir);
     half nsv = max(0,dot(ns, i.viewDir));
     half3 c0 = _PrimaryColor.rgb;
     half3 c2 = _HighlightColor.rgb;
     half3 c1 = c2 * 0.5;
     half3 cs = c2 * 0.4;    
     half3 tone =
         c0 * nsv +
         c1 * (nsv*nsv) +
         c2 * (nsv*nsv*nsv*nsv) +
         cs * pow(saturate(dot(nss,i.viewDir)), 32);
     main.rgb *= tone;
     o.Albedo = main;
     o.Normal = normal;
 EndSurface
 Lighting Lambert

Up next:

• How and where emissive terms should be placed. I cautiously omitted all emissive terms from the above examples (so my layered car shader is without reflections right now).

• Where should things like rim lighting go? I’m not sure if it’s a surface property (increasing albedo/emission with angle) or a lighting property (a back light).

My impressions so far:

• I like that I don’t have to write down vertex-to-fragment structures or the vertex shader. In most cases all the vertex shader does is transform stuff and pass it down to later stages, plus occasional computations that are linear over the triangle. No good reason to write it by hand.

• I like that the above shaders do not deal with how the rendering is actually done. For Unity’s case, I’m compiling them into single pass per light forward renderer, but they should just work with multiple lights per pass, deferred etc. Of course, that still has to be proven!

So far so good.

Series index: Shaders must die, Part 1, Part 2, Part 3.

I started playing around with the idea of “shaders must die”. I’m experimenting with extracting “surface shaders” for now.

Right now my experimental pipeline is:

1. Write a surface shader file
2. Perl script transforms it into Unity 2.x shader file
3. Which in turn is compiled by Unity into all lighting/shadows permutations, for D3D9 and OpenGL backends. Cg is used for actual shader compilation.

I have very simple cases working. For example:

 Properties
     2D _MainTex
 EndProperties
 Surface
     o.Albedo = SAMPLE(_MainTex);
 EndSurface

This is a “no bullshit” source code for a simple Diffuse (Lambertian) shader, 87 bytes of text.

The Perl script produces a Unity 2.x shader. This will be long, but bear with me - I’m trying to show how much stuff has to be written right now, when we’re operating on vertex/pixel shader level. See Attenuation and Shadows for Pixel Lights in Unity docs for how this system works.

 Shader "ShaderNinja/Diffuse" {
 Properties {
   _MainTex ("_MainTex", 2D) = "" {}
 }
 SubShader {
   Tags { "RenderType"="Opaque" }
   LOD 200
   Blend AppSrcAdd AppDstAdd
   Fog { Color [_AddFog] }
   Pass {
     Tags { "LightMode"="PixelOrNone" }
 CGPROGRAM
 #pragma fragment frag
 #pragma fragmentoption ARB_fog_exp2
 #pragma fragmentoption ARB_precision_hint_fastest
 #include "UnityCG.cginc"
 uniform sampler2D _MainTex;
 struct v2f {
     float2 uv_MainTex : TEXCOORD0;
 };
 struct f2l {
     half4 Albedo;
 };
 half4 frag (v2f i) : COLOR0 {
     f2l o;
     o.Albedo = tex2D(_MainTex,i.uv_MainTex);
     return o.Albedo * _PPLAmbient * 2.0;
 }
 ENDCG
   }
   Pass {
     Tags { "LightMode"="Pixel" }
 CGPROGRAM
 #pragma vertex vert
 #pragma fragment frag
 #pragma multi_compile_builtin
 #pragma fragmentoption ARB_fog_exp2
 #pragma fragmentoption ARB_precision_hint_fastest
 #include "UnityCG.cginc"
 #include "AutoLight.cginc"
 struct v2f {
     V2F_POS_FOG;
     LIGHTING_COORDS
     float2 uv_MainTex;
     float3 normal;
     float3 lightDir;
 };
 uniform float4 _MainTex_ST;
 v2f vert (appdata_tan v) {
     v2f o;
     PositionFog( v.vertex, o.pos, o.fog );
     o.uv_MainTex = TRANSFORM_TEX(v.texcoord, _MainTex);
     o.normal = v.normal;
     o.lightDir = ObjSpaceLightDir(v.vertex);
     TRANSFER_VERTEX_TO_FRAGMENT(o);
     return o;
 }
 uniform sampler2D _MainTex;
 struct f2l {
     half4 Albedo;
     half3 Normal;
 };
 half4 frag (v2f i) : COLOR0 {
     f2l o;
     o.Normal = i.normal;
     o.Albedo = tex2D(_MainTex,i.uv_MainTex);
     return DiffuseLight (i.lightDir, o.Normal, o.Albedo, LIGHT_ATTENUATION(i));
 }
 ENDCG
   }
 }
 Fallback "VertexLit"
 }

Phew, that is quite some typing to get simple diffuse shader (1607 bytes)! Well, at least all the lighting/shadow combinations are handled by Unity macros here. When Unity takes this shader and compiles into all permutations, it results in 58 kilobytes of shader assembly (D3D9 + OpenGL, 17 light/shadow combinations).

Let’s try something slightly different: bumpmapped, with a detail texture:

 Properties
     2D _MainTex
     2D _Detail
     2D _BumpMap
 EndProperties
 Surface
     o.Albedo = SAMPLE(_MainTex) * SAMPLE(_Detail) * 2.0;
     o.Normal = SAMPLE_NORMAL(_BumpMap);
 EndSurface

This is 173 bytes of text. Generated Unity shader is 2098 bytes, which compiles into 74 kilobytes of shader assembly.

In this case, the processing script detects that surface shader modifies normal per pixel, and does the necessary tangent space light transformations. It all just works!

So this is where I am now. Next up: detect which lighting model to use based on surface parameters (right now it always uses Lambertian). Fun!

It came in as a simple thought, and now I can’t shake it off. So I say:

Ok, now that the controversial bits are done, let’s continue.

Most of this can be (and probably is) wrong, and I haven’t given it enough thought yet. But here’s my thinking about shaders of “regular scene objects”. All of below is about things that need to interact with lighting; I’m not talking about shaders for postprocessing, one-off uses, special effects, GPGPU or kitchen sinks.

Operating on vertex/pixel shader level is a wrong abstraction level

Instead, it should be separated out into “surface shader” (albedo, normal, specularity, …), “lighting model” (Lambertian, Blinn Phong, …) and “light shader” (attenuation, cookies, shadows).

• Probably 90% of the cases would only touch the surface shader (mostly mix textures/colors in various ways), and choose from some precooked lighting models.

• 9% of the cases would tweak the lighting model. Most of the things would settle for “standard” (Blinn-Phong or similar), with some stuff using skin or anisotropic or …

• The “light shader” only needs to be touched once in a blue moon by ninjas. Once the shadowing and attenuation systems are implemented, there’s almost no reason for shader authors to see all the dirty bits.

Yes, current hardware operates on vertex/geometry/pixel shaders, which is a logical thing to do for hardware. After all, these are the primitives it works on when rendering. But those primitives are not the things you work on when authoring how a surface should look or how it should react to a light.

Simple code; no redundant info; sensible defaults

In the ideal world, here’s a simple surface shader (the syntax is deliberately stupid):

Haz Texture;
Albedo = sample Texture;

Or with bump mapping added:

Haz Texture;
Haz NormalMap;
Albedo = sample Texture;
Normal = sample_normal NormalMap;

And this should be all the info you have to provide. This would choose the lighting model based on used things (in this case, Lambertian). It would somehow just work with all kinds of lights, shadows, ambient occlusion and whatnot.

Compare to how much has to be written to implement a simple surface in your current shader technology, so that it would work “with everything”.

From the above shader, proper hardware shaders can be generated for DX9, DX11, DX1337, OpenGL, next-gen and next-next-gen consoles, mobile platforms with capable hardware, etc.

It can be used in accumulative forward rendering, forward rendering with multiple lights per pass, hybrid (light pre-pass / prelight) rendering, deferred rendering etc. Heck, even for a raytracer if you have one at hand.

I want!

Now of course, it won’t be as nice as more complex materials have to be expressed. Some might not even be possible. But shader text complexity should grow with material complexity; and all information that is redundant, implied, inferred or useless should be eliminated. There’s no good reason to stick to conventions and limits of current hardware just because it operates like that.

Shaders must die!