And what do you know – turns out we’re trying to pull this off in upcoming Unity 3. We call this Surface Shaders cause I’ve a suspicion “shaders must die” as a feature name wouldn’t have flied very far.

Idea

The main idea is that 90% of the time I just want to declare surface properties. This is what I want to say:

Hey, albedo comes from this texture mixed with this texture, and normal comes from this normal map. Use Blinn-Phong lighting model please, and don’t bother me again!

With the above, I don’t have to care whether this will be used in a forward or deferred rendering, or how various light types will be handled, or how many lights per pass will be done in a forward renderer, or how some indirect illumination SH probes will come in, etc. I’m not interested in all that! These dirty bits are job of rendering programmers, just make it work dammit!

This is not a new idea. Most graphical shader editors that make sense do not have “pixel color” as the final output node; instead they have some node that basically describes surface parameters (diffuse, specularity, normal, …), and all the lighting code is usually not expressed in the shader graph itself. OpenShadingLanguage is a similar idea as well (but because it’s targeted at offline rendering for movies, it’s much richer & more complex).

Example

Here’s a simple – but full & complete – Unity 3.0 shader that does diffuse lighting with a texture & a normal map.

  Shader "Example/Diffuse Bump" {
    Properties {
      _MainTex ("Texture", 2D) = "white" {}
      _BumpMap ("Bumpmap", 2D) = "bump" {}
    }
    SubShader {
      Tags { "RenderType" = "Opaque" }
      CGPROGRAM
      #pragma surface surf Lambert
      struct Input {
        float2 uv_MainTex;
        float2 uv_BumpMap;
      };
      sampler2D _MainTex;
      sampler2D _BumpMap;
      void surf (Input IN, inout SurfaceOutput o) {
        o.Albedo = tex2D (_MainTex, IN.uv_MainTex).rgb;
        o.Normal = UnpackNormal (tex2D (_BumpMap, IN.uv_BumpMap));
      }
      ENDCG
    }
    Fallback "Diffuse"
  }

Given pretty model & textures, it can produce pretty pictures! How cool is that?

I grayed out bits that are not really interesting (declaration of serialized shader properties & their UI names, shader fallback for older machines etc.). What’s left is Cg/HLSL code, which is then augmented by tons of auto-generated code that deals with lighting & whatnot.

This surface shader dissected into pieces:

• #pragma surface surf Lambert: this is a surface shader with main function “surf”, and a Lambert lighting model. Lambert is one of predefined lighting models, but you can write your own.
• struct Input: input data for the surface shader. This can have various predefined inputs that will be computed per-vertex & passed into your surface function per-pixel. In this case, it’s two texture coordinates.
• surf function: actual surface shader code. It takes Input, and writes into SurfaceOutput (a predefined structure). It is possible to write into custom structures, provided you use lighting models that operate on those structures. The actual code just writes Albedo and Normal to the output.

What is generated

Unity’s “surface shader code generator” would take this, generate actual vertex & pixel shaders, and compile them to various target platforms. With default settings in Unity 3.0, it would make this shader support:

• Forward renderer and Deferred Lighting (Light Pre-Pass) renderer.
• Objects with precomputed lightmaps and without.
• Directional, Point and Spot lights; with projected light cookies or without; with shadowmaps or without. Well ok, this is only for forward renderer because in Light Pre-Pass lighting happens elsewhere.
• For Forward renderer, it would compile in support for lights computed per-vertex and spherical harmonics lights computed per-object. It would also generate extra additive blended pass if needed for the case when additional per-pixel lights have to be rendered in separate passes.
• For Light Pre-Pass renderer, it would generate base pass that outputs normals & specular power; and a final pass that combines albedo with lighting, adds in any lightmaps or emissive lighting etc.
• It can optionally generate a shadow caster rendering pass (needed if custom vertex position modifiers are used for vertex shader based animation; or some complex alpha-test effects are done).

For example, here’s code that would be compiled for a forward-rendered base pass with one directional light, 4 per-vertex point lights, 3rd order SH lights; optional lightmaps (I suggest just scrolling down):

#pragma vertex vert_surf
#pragma fragment frag_surf
#pragma fragmentoption ARB_fog_exp2
#pragma fragmentoption ARB_precision_hint_fastest
#pragma multi_compile_fwdbase
#include "HLSLSupport.cginc"
#include "UnityCG.cginc"
#include "Lighting.cginc"
#include "AutoLight.cginc"
struct Input {
	float2 uv_MainTex : TEXCOORD0;
};
sampler2D _MainTex;
sampler2D _BumpMap;
void surf (Input IN, inout SurfaceOutput o)
{
	o.Albedo = tex2D (_MainTex, IN.uv_MainTex).rgb;
	o.Normal = UnpackNormal (tex2D (_BumpMap, IN.uv_MainTex));
}
struct v2f_surf {
  V2F_POS_FOG;
  float2 hip_pack0 : TEXCOORD0;
  #ifndef LIGHTMAP_OFF
  float2 hip_lmap : TEXCOORD1;
  #else
  float3 lightDir : TEXCOORD1;
  float3 vlight : TEXCOORD2;
  #endif
  LIGHTING_COORDS(3,4)
};
#ifndef LIGHTMAP_OFF
float4 unity_LightmapST;
#endif
float4 _MainTex_ST;
v2f_surf vert_surf (appdata_full v) {
  v2f_surf o;
  PositionFog( v.vertex, o.pos, o.fog );
  o.hip_pack0.xy = TRANSFORM_TEX(v.texcoord, _MainTex);
  #ifndef LIGHTMAP_OFF
  o.hip_lmap.xy = v.texcoord1.xy * unity_LightmapST.xy + unity_LightmapST.zw;
  #endif
  float3 worldN = mul((float3x3)_Object2World, SCALED_NORMAL);
  TANGENT_SPACE_ROTATION;
  #ifdef LIGHTMAP_OFF
  o.lightDir = mul (rotation, ObjSpaceLightDir(v.vertex));
  #endif
  #ifdef LIGHTMAP_OFF
  float3 shlight = ShadeSH9 (float4(worldN,1.0));
  o.vlight = shlight;
  #ifdef VERTEXLIGHT_ON
  float3 worldPos = mul(_Object2World, v.vertex).xyz;
  o.vlight += Shade4PointLights (
    unity_4LightPosX0, unity_4LightPosY0, unity_4LightPosZ0,
    unity_LightColor0, unity_LightColor1, unity_LightColor2, unity_LightColor3,
    unity_4LightAtten0, worldPos, worldN );
  #endif // VERTEXLIGHT_ON
  #endif // LIGHTMAP_OFF
  TRANSFER_VERTEX_TO_FRAGMENT(o);
  return o;
}
#ifndef LIGHTMAP_OFF
sampler2D unity_Lightmap;
#endif
half4 frag_surf (v2f_surf IN) : COLOR {
  Input surfIN;
  surfIN.uv_MainTex = IN.hip_pack0.xy;
  SurfaceOutput o;
  o.Albedo = 0.0;
  o.Emission = 0.0;
  o.Specular = 0.0;
  o.Alpha = 0.0;
  o.Gloss = 0.0;
  surf (surfIN, o);
  half atten = LIGHT_ATTENUATION(IN);
  half4 c;
  #ifdef LIGHTMAP_OFF
  c = LightingLambert (o, IN.lightDir, atten);
  c.rgb += o.Albedo * IN.vlight;
  #else // LIGHTMAP_OFF
  half3 lmFull = DecodeLightmap (tex2D(unity_Lightmap, IN.hip_lmap.xy));
  #ifdef SHADOWS_SCREEN
  c.rgb = o.Albedo * min(lmFull, atten*2);
  #else
  c.rgb = o.Albedo * lmFull;
  #endif
  c.a = o.Alpha;
  #endif // LIGHTMAP_OFF
  return c;
}

Of those 90 lines of code, 10 are your original surface shader code; the remaining 80 would have to be pretty much written by hand in Unity 2.x days (well ok, less code would have to be written because 2.x had less rendering features). But wait, that was only base pass of the forward renderer! It also generates code for additive pass, for deferred base pass, deferred final pass, optionally for shadow caster pass and so on.

So this should be an easier to write lit shaders (it is for me at least). I hope this will also increase the number of Unity users who can write shaders at least 3 times (i.e. to 30 up from 10!). It should be more future proof to accomodate changes to the lighting pipeline we’ll do in Unity next.

Predefined Input values

The Input structure can contain texture coordinates and some predefined values, for example view direction, world space position, world space reflection vector and so on. Code to compute them is only generated if they are actually used. For example, if you use world space reflection to do some cubemap reflections (as emissive term) in your surface shader, then in Light Pre-Pass base pass the reflection vector will not be computed (since it does not output emission, so by extension does not need reflection vector).

As a small example, the shader above extended to do simple rim lighting:

  #pragma surface surf Lambert
  struct Input {
      float2 uv_MainTex;
      float2 uv_BumpMap;
      float3 viewDir;
  };
  sampler2D _MainTex;
  sampler2D _BumpMap;
  float4 _RimColor;
  float _RimPower;
  void surf (Input IN, inout SurfaceOutput o) {
      o.Albedo = tex2D (_MainTex, IN.uv_MainTex).rgb;
      o.Normal = UnpackNormal (tex2D (_BumpMap, IN.uv_BumpMap));
      half rim =
          1.0 - saturate(dot (normalize(IN.viewDir), o.Normal));
      o.Emission = _RimColor.rgb * pow (rim, _RimPower);
  }

Vertex shader modifiers

It is possible to specify custom “vertex modifier” function that will be called at start of the generated vertex shader, to modify (or generate) per-vertex data. You know, vertex shader based tree wind animation, grass billboard extrusion and so on. It can also fill in any non-predefined values in the Input structure.

My favorite vertex modifier? Moving vertices along their normals.

Custom Lighting Models

There are a couple simple lighting models built-in, but it’s possible to specify your own. A lighting model is nothing more than a function that will be called with the filled SurfaceOutput structure and per-light parameters (direction, attenuation and so on). Different functions would have to be called in forward & light pre-pass rendering cases; and naturally the light pre-pass one has much less flexibility. So for any fancy effects, it is possible to say “do not compile this shader for light pre-pass”, in which case it will be rendered via forward rendering.

Example of wrapped-Lambert lighting model:

  #pragma surface surf WrapLambert
  half4 LightingWrapLambert (SurfaceOutput s, half3 dir, half atten) {
      dir = normalize(dir);
      half NdotL = dot (s.Normal, dir);
      half diff = NdotL * 0.5 + 0.5;
      half4 c;
      c.rgb = s.Albedo * _LightColor0.rgb * (diff * atten * 2);
      c.a = s.Alpha;
      return c;
  }
  struct Input {
      float2 uv_MainTex;
  };
  sampler2D _MainTex;
  void surf (Input IN, inout SurfaceOutput o) {
      o.Albedo = tex2D (_MainTex, IN.uv_MainTex).rgb;
  }

Behind the scenes

I’m using HLSL parser from Ryan Gordon’s mojoshader to parse the original surface shader code and infer some things from the AST mojoshader produces. This way I can figure out what members are in what structures, go over function prototypes and so on. At this stage some error checking is done to tell the user his surface function is of wrong prototype, or his structures are missing required members – which is much better than failing with dozens of compile errors in the generated code later.

To figure out which surface shader inputs are actually used in the various lighting passes, I’m generating small dummy pixel shaders, compile them with Cg and use Cg’s API to query used inputs & outputs. This way I can figure out, for example, that a normal map nor it’s texture coordinate is not actually used in Light Pre-Pass’ final pass, and save some vertex shader instructions & a texcoord interpolator.

The code that is ultimately generated is compiled with various shader compilers depending on the target platform (Cg for PC/Mac, XDK HLSL for Xbox 360, PS3 Cg for PS3, and my own fork of HLSL2GLSL for iPhone, Android and upcoming NativeClient port of Unity).

So yeah, that’s it. We’ll see where this goes next, or what happens when Unity 3 will be released.

Since shaders are more or less “assets”, having two different languages to deal with is not very nice. What, I’m supposed to write my shader twice just to support both (for example) D3D and iPad? You would think in 2010, almost a decade since high level realtime shader languages have appeared, this problem would be solved… but it isn’t!

In upcoming Unity 3.0, we’re going to have OpenGL ES 2.0 for mobile platforms, where GLSL ES is the only option to write shaders in. However, almost all other platforms (Windows, 360, PS3) need HLSL/Cg.

I tried a bit making Cg spit out GLSL code. In theory it can, and I read somewhere that id uses it for OpenGL backend for Rage… But I just couldn’t make it work. What’s possible for John apparently is not possible for mere mortals.

Then I looked at ATI’s HLSL2GLSL. That did produce GLSL shaders that were not absolutely horrible. So I started using it, and (surprise!) quickly ran into small issues here and there. Too bad development of the library stopped around 2006… on the plus side, it’s open source!

So I just forked it. Here it is: http://code.google.com/p/hlsl2glslfork/ (commit log here). There are no prebuilt binaries or source drops right now, just a Mercurial repository. BSD license. Patches welcome.

Note on the codebase: I don’t particularly like the codebase. It seems somewhat over-engineered code, that was probably taken from reference GLSL parser that 3DLabs once did, and adapted to parse HLSL and spit out GLSL. There are pieces of code that are unused, unfinished or duplicated. Judging from comments, some pieces of code have been in the hands of 3DLabs, ATI and NVIDIA (what good can come out of that?!). However, it works, and that’s the most important trait any code can have.

Note on the preprocessor: I bumped into some preprocessor issues that couldn’t be easily fixed without first understanding someone else’s ancient code and then changing it significantly. Fortunately, Ryan Gordon’s project, MojoShader, happens to have preprocessor that very closely emulates HLSL’s one (including various quirks). So I’m using that to preprocess any source before passing it down to HLSL2GLSL. Kudos to Ryan!

Side note on MojoShader: Ryan is also working on HLSL->GLSL cross compiler in MojoShader. I like that codebase much more; will certainly try it out once it’s somewhat ready.

You can never have enough notes: Google’s ANGLE project (running OpenGL ES 2.0 on top of Direct3D runtime+drivers) seems to be working on the opposite tool. For obvious reasons, they need to take GLSL ES shaders and produce D3D compatible shaders (HLSL or shader assembly/bytecode). The project seems to be moving fast; and if one day we’ll decide to default to GLSL as shader language in Unity, I’ll know where to look for a translator into HLSL :)

Unity

It seems that everyone is talking about Unity this year. At GDC 2009 some people have heard about us, some others were “where the f*** this came from?!”, and some had no idea what Unity is. This year it’s hard to find anyone who hasn’t heard about Unity. I was surprised by number of AAA developers who are playing around with Unity internally (for prototyping, mobile & whatnot) and/or are big fans of Unity. I like!

We had a cool booth that was very busy at all times. As a bonus, the Unity chairs could be used as weapons!

Awesome quote: CEO of censored (competing middleware company) said: “yeah, Unity is going up, we are going down”. This is taken completely out of context of course.

We were busy demoing upcoming Unity 3 which I think will be quite awesome. Three days before the conference were spent crunching on the demos for GDC :)

Cool Stuff

Only managed to go to two sessions :(

Stephen Hill‘s “Rendering Tools and Techniques of Splinter Cell: Conviction” had interesting bits & pieces of stuff. Nice work on hierarchical Z occlusion and ambient occlusion fields! (probably first time I see AO fields used in actual game production)

Mike Acton‘s “Three Big Lies: Typical Design Failures in Game Programming” was entertaining. Content wise I pretty much knew what to expect. If you aren’t following Mike – do it now! Talk slides are at Insomniac’s site.

RAD‘s Telemetry profiler looks totally sweet. I think they acquired this one and improved it. Some very good UI ideas in there. On a related note, Scaleform’s new profiler looks… kinda inspired by Unity’s (comparison: Scaleform on the left, Unity on the right).

Fun Stuff

Managed to sneak in some fun (dare I say “social”?) stuff.

Rendering folks dinner (thanks Johan!) was awesome, even if it made me feel kinda small & stupid among those super smart guys & gals. Shadow algorithms on receipts FTW! Middleware Meetup (thanks Dan!) was full of friendly competitors :) #gdcdrink tweetup (thanks Mike!) had lots of war stories, PS3 talk and how to do fluid simulation on 360′s pixel shaders.

Well ok, it all started a bit before:

I exchanged some emails with David and Joachim and they invited me for a gamejam in their office. Then one thing led to another, I was young and needed money (oops! wrong topic) and on January 2006 I started working on this thing called “Unity”.

Unity was at version 1.2.1 then. Since then we’ve released about a dozen new versions, added hundreds (or thousands?) of new features, a handful of new platforms and have grown a lot.

Also, we stopped saying “Sales are INSANE!!!!11″ whenever they exceeded a whopping ten thousand euros per week. Seriously, that much money in 2006 was a big thing. Our Windows build machine was a single core Celeron with 512MB RAM because that’s what we could afford! Well ok, we’re still saying “sales are insane!” from time to time, just the threshold has gone way up.

Occasionally we’d get excited about strangest things. I think this email is about some car model from ATI that was on front page of our website in 2006. It’s beyond me why we’d put a car on Unity website, but somehow it seemed to make sense at the time.

It would take too much space to list all the awesome things that happened in those four years. I got to work on some things too, like Windows Web Player, Direct3D renderer, shadows, editor for Windows and whatnot. But I mostly concentrate on creating trouble, which does not seem to hinder Unity that much. I need to get more efficient!

Seriously though, it has been an amazing ride so far, and I hope it will only become better. Thanks to everyone at Unity Technologies and the community!

Rock on!

In Unity we use Mono to do game code (well, Unity users are doing that, not us). Overall it’s great; it has tons of advantages, loads of awesome and a flying ninja here and there. But no technology is perfect, right?

Edit: Miguel rightly points out in the comments that Mono team is solving or has already solved some of these issues already. In some areas they are moving so fast that we at Unity can’t keep up!

#1: Garbage Collector

Most game developers do not like Garbage Collection (GC) very much. Typically, the more limited/hardcore their target platform is, the more they dislike GC. The reason? Most GC implementations cause rather unpredictable spikes.

Here’s a run of something recorded in the (awesome) Unity 2.6 profiler. Horizontal axis is time, vertical is CPU time spent in that frame:

At the bottom you see dark red thingies appearing once in a while. This is garbage collector kicking in, because some script code is allocating some memory at runtime.

Now of course, it is possible to write your script code so that it does no allocations (or almost no allocations). Preallocate your objects into pools, manually invoke GC when there’s a game situation when a small hickup won’t affect gameplay, etc. In fact, a lot of iPhone games made with Unity do that.

But that kind of side steps the whole advantage of “garbage collector almost frees you from doing memory management”. If you’re not allocating anything anyway, GC could just as well not be there!

A little side story. Me and Unity’s iPhone tech lead ReJ tried to explain what GC is to a non-programmer. Here’s what we came up with:

Garbage Collection is this cleaning service for lazy people. They can just leave any garbage on the floor in their house, and once in a while a garbage guy comes, collects all the garbage and takes it outside. Now, there are some intricacies in the service.

First, you never know when the garbage guy will come. You might be taking a shower, doing a meditation or having some “sexy time” – and it’s in the service agreement that when a garbage guy comes, you have to let him in to do his work.

Second thing is, the garbage guy is usually some homeless drunkard. He smells so bad that when he comes, you have to stop whatever you were doing, go outside and wait until he’s done with the garbage collection. Even your neighbors, who might be doing something entirely else in parallel, actually have to stop and idle while garbage is being collected in your house!

There are variations of this GC service. One variation is called “moving GC”, where the garbage guy also rearranges your furniture while collecting the garbage – he moves it all into one side of your house. This is so that you can buy a bigger piece of furniture, or throw a huge piece of garbage – and there will be enough unused space for you to do that! Of course this way GC process takes somewhat longer, but hey, you get all your stuff nicely packed into one corner.

Can’t you see that this service is the greatest idea of all time?

This is quite a harsh attitude towards GC, and of course it’s exaggerated. But there is some truth to it. So how could GC be fixed?

GC fix #1: more control

More explicit control on when & how long GC runs. I want to say to the garbage guy, “come everyday at 4PM and do your work for 20 minutes”. In the game, I’d want to call GC with an upper time limit, say 1 millisecond for each call, and I would be calling that 30 times per second.

GC fix #2: sometimes I want to clean garbage myself

Inefficiencies and unpredictability of GC cause people to do even more work than a normal, oldskool memory allocation. Why not provide an option to deal with deallocations manually? I.e. a keyword reallynew could allocate an object that is not part of garbage collected world. It would function as a regular .NET object, just it would be user’s responsibility to reallydelete it.

Mono is already extending .NET (see SIMD and continuations). Maybe it makes sense to add some way to bypass garbage collector?

#2: Distribution Size

Using C#/.NET in a game requires having .NET runtime. None of the interesting platforms are guaranteed to have it, and even on Windows you can’t count on it being present. Mono is great here in a sense that it can be used on many more platforms than Microsoft’s own .NET. It’s also great on distribution size, but only if you compare it to Microsoft’s .NET.

In Unity Web Player, we package Mono DLL + mscorlib assembly into something like 1.5 megabytes (after LZMA compression). Which is great compared to 20+ megabytes of .NET runtime, but not that great it you compare it so, say, Lua runtime (which is less than 100 kilobytes).

On some platforms (iPhone, Xbox 360, PS3, …) it’s not possible to generate code at runtime, so Mono’s JIT does not work. All code that’s written in C# has to be precompiled to machine code ahead of time (AOT compilation). This is not a problem per se, but because .NET framework was never designed with small size and few dependencies in mind, doing anything will ultimately pull in a lot of code.

We joke that doing anything in C# will result in an XML parser being included somewhere. This is not that far from the truth; e.g. calling float.ToString() will pull in whole internationalization system, which probably somewhere needs to read some global XML configuration file to figure out whether daylight savings time is active when Eastern European Brazilian Chinese calendar is used.

Size fix #1: custom core .NET libraries?

For game uses, most of “fat” stuff in .NET runtime is not really needed. float.ToString() could just always use period as a decimal separator. Core libraries could consist just of essential collections (list, array, hash table) and maybe a String class, with just essential methods. Maybe it’s worth sacrificing some of the generality of .NET if that could shave off a couple of megabytes from your iPhone game size?

Of course this is very much doable; “all that is needed” ™ is writing custom mscorlib+friends, and telling C# compiler to not ever reference any of the “real” libraries.

Size fix #2: make Mono runtime smaller

Uncompressed Mono DLL in our Windows build is 1.5 megabytes. We have turned off all the easy stuff (profiler, debugger, logging, COM, AOT etc.). But probably some more could be stripped away. Do our games really need multiple AppDomains? Some fancy marshalling? I don’t know, it just feels that 1.5MB is a lot.

#3: Porting to New Platforms

You know this classic: “There’s no portable code. There’s only code that’s been ported.”

Most existing gaming platforms are quite weird. Most upcoming smartphone platforms also are quite weird, each in their own interesting way. Porting a large project like Mono is not easy, especially since parts of it (JIT or AOT engine) highly depend on the platform.

For Unity iPhone, unexpected discovery that it’s not possible to JIT on iPhone made the initial release be delayed by something like 4 months. It did not help that in early iPhone SDK builds JIT was actually possible, and Apple decided to disable runtime generated code later. Making Mono actually work there required significant work both from Mono team and from Unity. We still have one guy working almost exclusively on Mono+iPhone issues!

Of course, maybe all the Mono iPhone work made porting to new platforms easier as a byproduct. But so far we don’t have Mono ported to any other platform, up to production quality. So judging from experience, we now always assume Mono port will be a pain, just because “some nasty surprises will come up” (and they always do).

#4: Small Stuff

There is a ton of small bits where extending .NET would benefit gaming scenarios. For example:

Suppose there is some array on the native engine side; for example vertex positions in a mesh (3xFloat for each vertex). Is it possible to make that piece of memory be represented as a native struct array for .NET side? So that it would not involve any extra memory copies, but N vertices somewhere in memory would look just like Vector3[N] for C#?

On a similar note, having “strided arrays” would be useful. For example, mesh data is often interleaved, so for each vertex there is a position, normal, UVs and so on. It would be cool if in C# position array would still look like Vector3[N], but internally the distance between each element would be larger than 12 bytes required for Vector3.

Where do we go from here?

The above are just random ideas, and I’m not complaining about Mono. It is great! It’s just not perfect. Mono being open source is a very good thing, which means pretty much any interested party can improve it as needed. So rock on.

The process is roughly this (all of this is DX9 level tech on PCs; later tech or consoles could and should use more optimizations):

1. Render shadow map cascades. All of them packed into one shadow map via viewports.
2. Collect shadows into screen sized render target. This is the shadow term.
3. Blur the shadow term.
4. In regular forward rendering, use shadow term in screen space.

More detail:

Render Shadow Cascades

Nothing fancy here. All cascades packed into a single shadow map. For example two 512×512 cascades would be packed into 1024×512 shadow map side by side.

Screen-space Shadow Term

Render all shadow receivers with a shader that “collects” shadow map term. In effect, shadows from all cascades are collected into a screen-sized texture. After this step, original cascaded shadowmaps are not needed anymore.

Unity supports up to 4 shadow map cascades, which neatly fit into a float4 register in the pixel shader. Correct cascade is sampled just once, without using static or dynamic branching. Pixel shader pseudocode:

float4 near = float4 (z >= _LightSplitsNear);
float4 far = float4 (z < _LightSplitsFar);
float4 weights = near * far;
float2 coord =
    i._ShadowCoord[0] * weights.x +
    i._ShadowCoord[1] * weights.y +
    i._ShadowCoord[2] * weights.z +
    i._ShadowCoord[3] * weights.w;
float sm = tex2D (_ShadowMapTexture, coord.xy).r;

Additionally, shadow fadeout is applied here (shadows in Unity can be cast up to specified distance from the camera, and they fade out when approaching that distance).

After this I end up having shadow term in screen space. Note that here I do not do any shadow map filtering; that is done in screen space later.

On PCs in DX9 there is (or there was?) no easy/sane way to read depth buffer in the pixel shader, so while collecting shadows the shader also outputs depth packed into two channels of the render target.

Screen-space Shadow Blur

Previous step results in screen space shadow term and depth. Shadow term is blurred into another render target, using a spatially varying Poisson disc-like filter.

Filter size depends on depth (shadow boundaries closer to the camera are blurred more). Filter also discards samples if difference in depth is larger than something, to avoid blurring over object boundaries. It's not totally robust, but seems to work quite well.

Using shadow term in forward rendering

In forward rendering, this blurred shadow term texture is used. Here shadow term already has filtering & fadeout applied, and the shaders do not need to know anything about shadow cascades. Just read pixel from the texture and use it in lighting computation. Done!

Fin

Back then I didn't know this would be called "deferred" (that would probably have scared me away!). I don't know if this approach is any good, but so far it works quite well for Unity needs. Also, reduces shader permutation count a lot, which I like.

In Unity editor pretty much any window can be “detached” from the main window. An obvious use case is putting it onto a separate monitor. But of course you can just end up having a ton of detached windows overlapping each other.

Here I have four windows in total on OS X:

Here I have four windows on Windows:

However, users of OS X and Windows are used to applications behaving differently.

On OS X, it is very common that a single application has many overlapping windows. Usually users don’t have problems finding their windows either, thanks to Exposé. Press a key, voilà, here they are:

On Windows, there is no Exposé. So there’s a problem: when a detached window is obscured by another window, how do you get to it? One would ask “well, what’s wrong in having windows partially overlapped, like in above screenshot?”, to which I’d say “you’re a Mac user”.

Windows users do not have a ton of windows on screen. They tend to maximize the application they are currently working with. I was doing this myself all the time, and it took 3 years of Mac laptop usage before I stopped maximizing everything on my Windows box!

So what a typical Windows user might see when using Unity is this. Now, where are the other three detached windows?

On Windows, it is very uncommon for a single application to have many overlapped windows. When an application does that, the “detached” windows are always positioned on top of the main window. There are some applications that do not do this (yes I’m looking at you GIMP), and almost everyone is not happy with their usability.

So we decided to take this context into account. Windows users do not have Exposé, and they expect “detached” windows to be always on top of the main window. Unity 2.6 will do this soon.

Of course, you still can dock all the windows together and this whole “windows are obscured by other windows” issue goes away:

Hmm… I think the screenshots above show two new big features in upcoming Unity 2.6. Preemptive note: UI of the stuff above is not final. Anything might change, don’t become attached to any particular pixel!

Well, finally I implemented it for upcoming Unity 2.6. I wrote some sort of a technical report here.

In short, I’m combining assembly fragments and doing simple temporary register allocation, which seems to work quite well. Performance is very similar to using fixed function (I know it’s implemented as vertex shaders internally by the runtime/driver) on several different cards I tried (Radeon HD 3xxx, GeForce 8xxx, Intel GMA 950).

What was unexpected: the most complex piece is not the vertex lighting! Most complexity is in how to route/generate texture coordinates and transform them. Huge combination explosion there.

Otherwise – I like! Here’s a link to the article again.

Got different lighting models (BRDFs) working. Without further ado, code snippets that produce real actual working shaders that work with lights & shadows and whatnot:

Simple Lambert (single color):

Properties
    Color _Color
EndProperties
Surface
    o.Albedo = _Color;
EndSurface
Lighting Lambert

Let’s add a texture:

Properties
    2D _MainTex
    Color _Color
EndProperties
Surface
    o.Albedo = SAMPLE(_MainTex) * _Color;
EndSurface
Lighting Lambert

Change light model to Half-Lambert (a.k.a. wrapped diffuse):

// ...everything the same
Lighting HalfLambert

Blinn-Phong, with constant exponent & constant specular color, modulated by gloss map in main texture’s alpha:

Properties
    2D _MainTex
    Color _Color
    Color _SpecColor
    Float _Exponent
EndProperties
Surface
    half4 col = SAMPLE(_MainTex);
    o.Albedo = col * _Color;
    o.Specular = _SpecColor.rgb * col.a;
    o.Exponent = _Exponent;
EndSurface
Lighting BlinnPhong

The same Blinn-Phong, with added normal map:

Properties
    2D _MainTex
    2D _BumpMap
    Color _Color
    Color _SpecColor
    Float _Exponent
EndProperties
Surface
    half4 col = SAMPLE(_MainTex);
    o.Albedo = col * _Color;
    o.Specular = _SpecColor.rgb * col.a;
    o.Exponent = _Exponent;
    o.Normal = SAMPLE_NORMAL(_BumpMap);
EndSurface
Lighting BlinnPhong

I also made an illustrative-style BRDF (see Illustrative Rendering in Team Fortress 2), but that only requires above sample to have “Lighting TF2″ at the end.

Another thing I tried is surface that has Albedo dependent on a viewing angle, similar to Layered Car Paint Shader. It works:

Properties
    2D _MainTex
    2D _BumpMap
    2D _SparkleTex
    Float _Sparkle
    Color _PrimaryColor
    Color _HighlightColor
EndProperties
Surface
    half4 main = SAMPLE(_MainTex);
    half3 normal  = SAMPLE_NORMAL(_BumpMap);
    half3 normalN = normalize(SAMPLE_NORMAL(_SparkleTex));
    half3 ns = normalize (normal + normalN * _Sparkle);
    half3 nss = normalize (normal + normalN);
    i.viewDir = normalize(i.viewDir);
    half nsv = max(0,dot(ns, i.viewDir));
    half3 c0 = _PrimaryColor.rgb;
    half3 c2 = _HighlightColor.rgb;
    half3 c1 = c2 * 0.5;
    half3 cs = c2 * 0.4;
    half3 tone =
        c0 * nsv +
        c1 * (nsv*nsv) +
        c2 * (nsv*nsv*nsv*nsv) +
        cs * pow(saturate(dot(nss,i.viewDir)), 32);
    main.rgb *= tone;
    o.Albedo = main;
    o.Normal = normal;
EndSurface
Lighting Lambert

Up next:

• How and where emissive terms should be placed. I cautiously omitted all emissive terms from the above examples (so my layered car shader is without reflections right now).
• Where should things like rim lighting go? I’m not sure if it’s a surface property (increasing albedo/emission with angle) or a lighting property (a back light).

My impressions so far:

• I like that I don’t have to write down vertex-to-fragment structures or the vertex shader. In most cases all the vertex shader does is transform stuff and pass it down to later stages, plus occasional computations that are linear over the triangle. No good reason to write it by hand.
• I like that the above shaders do not deal with how the rendering is actually done. For Unity’s case, I’m compiling them into single pass per light forward renderer, but they should just work with multiple lights per pass, deferred etc. Of course, that still has to be proven!

So far so good.

Series index: Shaders must die, Part 1, Part 2, Part 3.

Right now my experimental pipeline is:

1. Write a surface shader file
2. Perl script transforms it into Unity 2.x shader file
3. Which in turn is compiled by Unity into all lighting/shadows permutations, for D3D9 and OpenGL backends. Cg is used for actual shader compilation.

I have very simple cases working. For example:

Properties
    2D _MainTex
EndProperties
Surface
    o.Albedo = SAMPLE(_MainTex);
EndSurface

This is a “no bullshit” source code for a simple Diffuse (Lambertian) shader, 87 bytes of text.

The Perl script produces a Unity 2.x shader. This will be long, but bear with me – I’m trying to show how much stuff has to be written right now, when we’re operating on vertex/pixel shader level. See Attenuation and Shadows for Pixel Lights in Unity docs for how this system works.

Shader "ShaderNinja/Diffuse" {
Properties {
  _MainTex ("_MainTex", 2D) = "" {}
}
SubShader {
  Tags { "RenderType"="Opaque" }
  LOD 200
  Blend AppSrcAdd AppDstAdd
  Fog { Color [_AddFog] }
  Pass {
    Tags { "LightMode"="PixelOrNone" }
CGPROGRAM
#pragma fragment frag
#pragma fragmentoption ARB_fog_exp2
#pragma fragmentoption ARB_precision_hint_fastest
#include "UnityCG.cginc"
uniform sampler2D _MainTex;
struct v2f {
    float2 uv_MainTex : TEXCOORD0;
};
struct f2l {
    half4 Albedo;
};
half4 frag (v2f i) : COLOR0 {
    f2l o;
    o.Albedo = tex2D(_MainTex,i.uv_MainTex);
    return o.Albedo * _PPLAmbient * 2.0;
}
ENDCG
  }
  Pass {
    Tags { "LightMode"="Pixel" }
CGPROGRAM
#pragma vertex vert
#pragma fragment frag
#pragma multi_compile_builtin
#pragma fragmentoption ARB_fog_exp2
#pragma fragmentoption ARB_precision_hint_fastest
#include "UnityCG.cginc"
#include "AutoLight.cginc"
struct v2f {
    V2F_POS_FOG;
    LIGHTING_COORDS
    float2 uv_MainTex;
    float3 normal;
    float3 lightDir;
};
uniform float4 _MainTex_ST;
v2f vert (appdata_tan v) {
    v2f o;
    PositionFog( v.vertex, o.pos, o.fog );
    o.uv_MainTex = TRANSFORM_TEX(v.texcoord, _MainTex);
    o.normal = v.normal;
    o.lightDir = ObjSpaceLightDir(v.vertex);
    TRANSFER_VERTEX_TO_FRAGMENT(o);
    return o;
}
uniform sampler2D _MainTex;
struct f2l {
    half4 Albedo;
    half3 Normal;
};
half4 frag (v2f i) : COLOR0 {
    f2l o;
    o.Normal = i.normal;
    o.Albedo = tex2D(_MainTex,i.uv_MainTex);
    return DiffuseLight (i.lightDir, o.Normal, o.Albedo, LIGHT_ATTENUATION(i));
}
ENDCG
  }
}
Fallback "VertexLit"
}

Phew, that is quite some typing to get simple diffuse shader (1607 bytes)! Well, at least all the lighting/shadow combinations are handled by Unity macros here. When Unity takes this shader and compiles into all permutations, it results in 58 kilobytes of shader assembly (D3D9 + OpenGL, 17 light/shadow combinations).

Let’s try something slightly different: bumpmapped, with a detail texture:

Properties
    2D _MainTex
    2D _Detail
    2D _BumpMap
EndProperties
Surface
    o.Albedo = SAMPLE(_MainTex) * SAMPLE(_Detail) * 2.0;
    o.Normal = SAMPLE_NORMAL(_BumpMap);
EndSurface

This is 173 bytes of text. Generated Unity shader is 2098 bytes, which compiles into 74 kilobytes of shader assembly.

In this case, the processing script detects that surface shader modifies normal per pixel, and does the necessary tangent space light transformations. It all just works!

So this is where I am now. Next up: detect which lighting model to use based on surface parameters (right now it always uses Lambertian). Fun!

Of course this created some buzz (hey, it’s Google after all). And it is in some way a competing technology with Unity. I think it’s going to be interesting, so I say “welcome competition!”

Preemptive blah blah: this website is my personal opinion and does not represent the views of my employer, former employers or anyone else other than myself.

Unity is one of the players in “3D on the web” space. 3D graphics in the browser are in fact nothing new. Unity’s browser plugin has existed since 2005 and is now in eight digits installations count. There is VRML, X3D, Adobe Shockwave, 3DVIA/Virtools, software rendering approaches on top of Flash and so on.

In my view, major advantages that Unity has compared to O3D:

• It’s not only about the graphics. Unity has physics, audio, input, scripting, streaming, networking, asset pipeline and whatnot. O3D is only about the graphics, and at a lower level.
• Unity runs on wider range of hardware. O3D requires Shader Mode 2.0 or later hardware, so about 30% of the “machines on the internet” can’t run O3D (based on our 2009Q1 data). Couple that with lots of compatibility workarounds that we have and it’s probably safe to say that Unity is more stable and mature at this point.
• Unity is not only about the web. There’s support for iPhone, Nintendo Wii, standalone games, and with time more console and mobile platforms will come.
• Creating and improving Unity is our primary and only focus as a company. In Google’s case, O3D is just another technology in their vast portfolio.

Of course, O3D also has advantages:

• It’s done by Google! When Google does something anything, people notice immediately :)
• O3D is free and open source. Hard to beat the free price, and open source does have it’s benefits. O3D is not a “standard” of any sort right now, but it looks like Google would want it to become one.
• Only focusing on low level graphics has it’s benefits: it’s lightweight, it appeals to hackers and graphics programmers who want to be in control. Unity’s higher level is much easier and faster to use, but low level hacking can be fun.

Of course there are tons of other differences (I might have missed something important as well).

For me as a rendering guy, it’s interesting to see O3D taking similar decisions here and there (e.g. they don’t use GLSL on OpenGL either because it does not really work in the real world).

So… we’ll see where things will go. It’s going to be interesting!

Here’s what’s new. Here’s the download page.

My 11th Unity release since I joined 3+ years ago. This is quite a crazy release that involved almost complete editor tools rewrite and lots of other juggling. Was not exactly a walk in the park, but it’s done now. Meet me at GDC in San Francisco next week and I’ll tell you the war stories (Unity booth is 5110 NH).

Here’s the obligatory source code commits graph:

18 people involved in source code, 5315 commits, 18501 file changes. Of course, svn commits do not mean anything… I’m just fascinated by graphs and numbers.

I’m the master of svn deception. I do tons of useless commits just so that the stats look good. Yeah!

…ok, back to work.

Take a quick look at zillions of new posts.

Answer about five questions with “what’s the value of your camera’s near plane?”.

There should be some way to automate all of this. For every 20th question, reply with “increase your near plane!”, or something.

We showed a glimpse of Unity editor for Windows at the keynote, so it is public now – yes, we are working on Windows toolchain. About the time! This is the major area I’m spending time these days – Windows, Windows, Windows. Learning WinAPI as I cruise along :) Before Unity 2.1 I spent months fixing tons of small issues, now I’m spending months doing tons of small Windows related things. Someday I’ll get back to doing tons of small things on the rendering side.

Here’s a couple of random photos that I stoleborrowed from Mantas:

Keynote in front of a Sentinel from The Matrix.

Presenters talking.

People listening!

I don’t know that guy in the center. Probably some stupid outsider. Really!

I vote that we complain enough that they don’t share plans until everything is 200% done – I personally don’t want to know about a new version of Unity until after I have already shipped at least one game built with that version.

Alternately, UT could issue press releases that lack any information that might set expectations and/or require approval from third parties:

Unity Technologies ApS, announced today that they may be releasing an update to their popular game development software at an undisclosed future time.

From their offices in Copenhagen, Unity Technologies CEO and co-founder, David Helgason, said “We’re pretty excited not to announce anything specific about this potential upcoming release as we think that many of our users may be pleased by the addition of some number of features that may not have been present in previous versions of the software.”

Tom Higgins, Product Evangelist, added “The features in this software are making it possible for a number of undisclosed companies to work on various projects that may be released in the future. I think all of our users might be inspired by the things that may currently be under development with this future version of Unity.”

edit: Uhm, based on the replies that follow, apparently it wasn’t obvious that I am not seriously requesting this.

Way to go! :)

Everyone knows the Valve’s hardware survey. But what if your target game players are not the traditional “big budget AAA game” type? For example, at the moment most Unity Web Player games are oriented to much more casual market, so hardware there might be very different. And indeed, turns out it is quite different.

Without further ado, here’s the data we have: Unity Web Player hardware statistics.

It’s about two million data points since we started gathering it earlier this year.

Some subjective points of interest (I’ll be using current data for 2008 Q3 here):

• Operating systems: Mac OS X is 2.5%, the rest is Windows. 64 bit Windows haven’t really picked up yet (0.7%). Windows 2000 is dying fast (0.7%). OS X Leopard already took over OS X Tiger.
• CPUs: poor Transmeta :) Dual core CPUs are becoming the norm (46%).
• Graphics cards: quite sad, in fact… top 15 cards are slow or horribly slow. Capability wise, they are quite good, with about 70% having shader model 2.0 or higher. Shader model 1.x cards are dead. “Can has DX10” is 2.7%.
• Casual machines don’t have lots of RAM. Nor lots of VRAM.
• Most popular nvidia driver? 56.73. Looks like this is the driver that comes integrated in XP SP2… Now, who says regular people ever update their drivers? Likewise, vga.dll (i.e. standard VGA) is 1.6% of machines; additional 1.5% don’t report any driver (not sure how that happens…).

So yeah. Casual machines: capabilities quite okay, performance low, low, low. That’s life.

And herein lies the problem: most often precision of vertex transformations is not the same in fixed function versus programmable vertex pipelines. If you’d just draw some objects in multiple passes, mixing fixed function and programmable paths, this is roughly what you will get (excuse my programmer’s art):

Not pretty at all! This should have looked like this:

So what do we do to make it look like this? We “pull” (bias) some rendering passes slighly towards the camera, so there is no depth fighting.

Now, at the moment Unity editor runs only on the Macs, which use OpenGL. In there, most of hardware configurations do not need this depth bias at all – they are able to generate same results in fixed function and programmable pipelines. Only Intel cards do need the depth bias on Mac OS X (on Windows, AMD and Intel cards need depth bias). So people author their games using OpenGL, where it does not need depth bias in most cases.

How do you apply depth bias in OpenGL? Enable GL_POLYGON_OFFSET_FILL and set glPolygonOffset to something like -1, -1. This works.

How do you apply depth bias in Direct3D 9? Conceptually, you do the same. There are DEPTHBIAS and SLOPESCALEDEPTHBIAS render states that do just that. And so we did use them.

And people complained about funky results on Windows.

And I’d look at their projects, see that they are using something like 0.01 for camera’s near plane and 1000.0 for the far plane, and tell them something along the lines of “increase your near plane, stupid!” (well ok, without the “stupid” part). And I’d explain all the above about mixing fixed function and vertex shaders, and how we do depth bias in that case, and how on OpenGL it’s often not needed but on Direct3D it’s pretty much always needed. And yes, how sometimes that can produce “double lighting” artifacts on close or intersecting geometry, and how the only solution is to increase the near plane and/or avoid close or intersecting geometry.

Sometimes this helped! I was so convinced that their too-low-near-plane was always the culprit.

And then one day I decided to check. This is what I’ve got on Direct3D:

Ok, this scene is intentionally using a low near plane, but let me stress this again. This is what I’ve got:

Not good at all.

What happened? It happened in roughly this way:

1. First, depth bias documentation on Direct3D is wrong. Depth bias is not in 0..16 range, it is in 0..1 range which corresponds to entire range of depth buffer.
2. Back then, our code was always using 16 bit depth buffers, so the equivalent of -1,-1 depth bias in OpenGL was multiplied with something like 1.0/65535.0, and that was fed into Direct3D. Hey, it seemed to work!
3. Later on, the device setup code was modified to do proper format selection, so most often it ended up using 24 bit depth buffer. Of course no one I never modified the depth bias code to account for this change…
4. And it stayed there. And I kept deceiving myself that the content of the users is to blame, and not some stupid code of mine.

It’s good to check your assumptions once in a while.

So yeah, the proper multiplier for depth bias on Direct3D with 24 bit depth buffer should be not 1.0/65535.0, but something like 1.0/(2^24-1). Except that this value is really small, so something like 4.8e-7 should be used instead (see Lengyel’s GDC2007 talk). Oh, but for some reason it’s not really enough in practice, so something like 2.0*4.8e-7 should be used instead (tested so far on GeForce 8600, Radeon HD 3850, Radeon 9600, Intel 945, reference rasterizer). Oh, and the same value should be used even when a 16 bit depth buffer is used; using 1.0/65535.0 multiplier with 16 bit depth buffer produces way too large bias.

With proper bias values the image is good on Direct3D again. Yay for that (fix is coming in Unity 2.1 soon).

…and yes, I know that real men fudge projection matrix instead of using depth bias… someday maybe.

So it’s a small very small 2D game without any next-gen bells and whistles. It can probably be done casually on the side, by allocating an hour here and there. We’ll see how it goes. Hey, I never actually done any game in Unity, I only make or break some underlying parts…

Of course, first I start with no game, just imported graphics. Hey look, I can do sprites!

Then cook up some base things: define the game grid, throw in some basic user interface on the right hand side, and make it actually do something. This wasn’t so hard; that already gets me an almost working level building functionality. It does not have fancy block building delay or block deletion yet; that will come later.

Next come basic physics. Danc’s design calls for simple arcade-like physics (things moving at constant speeds, bouncing off at equal angles, and so on), but in Unity I have a fully fledged physics engine just waiting to be used. Let’s use that.

The design has sloped ramp pieces, which are hard to approximate using any primitive colliders, so instead I’ll use convex mesh colliders for them. Now, on this machine I only have Blender, which I totally don’t know how to use; and I was too lazy to go to PC and use 3ds Max there. What a coder does? Of course, just type in the mesh file in ASCII FBX format. Excerpt:

; scaled 2x in Z, by 0.85 in Y
Vertices: -0.5,-0.425,-1.0, 0.5,-0.425,-1.0, -0.5,-0.425,1.0, 0.5,-0.425,1.0, -0.5,0.425,-1.0, -0.5,0.425,1.0
PolygonVertexIndex: 0,1,-3,2,1,-4,1,0,-5,2,3,-6,0,2,-5,2,5,-5,3,1,-5,5,3,-5

It’s a left ramp mesh! So much for fancy asset auto-importing functionality, when you don’t know how to use those 3D apps :)

After a while I’ve got peas being controlled by physics, colliding with level and so on. Physics is very bad for productivity, as I ended up just playing around with pea-stacks!

So far there’s no game yet… Next up: implement some AI for the peas, so they can wander around, climb the walls, fall down and bounce around. I guess that will be more work and less playing around… We’ll see.

You drive a jeep with a spiky ball, and your goal is to chase down raptors and send them to the future, presumably to end world hunger. Or you can do stunts. And you the driver are a raptor, only you wear a hat and a monocle.

Just go and play it already: raptorsafari.com. It’s free.

Or watch a trailer if you want to miss all the real fun, or read a press release. Of course it’s made in Unity, in two months from start to finish.

To me, this is a perfect example of focus. Basically there are three things – 1) vehicle, 2) raptors, 3) physics mayhem – and that almost describes a game. Yes, there are crates and stunts and achievements and online leaderboards, but that’s just additional stuff on top of the core game.

Sounds like a good plan for making game prototypes:

1. Think up a game idea and describe it in one concise sentence. The idea may be totally crazy, like in this case. I guess an idea like Velociraptor Safari would not fly in a pitch at any publisher, but that does not matter at this point.
2. Get a small team of smart people. In Flashbang’s case, it seems they were 4 to 7 person team.
3. Choose a game engine/toolset that will allow you to make your game fast. *cough cough*
4. Do it!

All the above requires is a small smart team and groceries/rent for a couple of months.

Your original idea may be totally crazy, but with the actual working prototype at hand it might just work. Looks like Velociraptor Safari really clicked something on the internets (see Kotaku, JayIsGames, Destructoid, TIGSource, AtomicGamer, …).

One of our customers found an interesting bug the other day: embedding Unity Web Player into a web page makes some javascript animation libraries not work correctly. For example, script.aculo.us or Dojo Toolkit would stop doing some of their tasks. But only on Windows, and only on some browsers (Firefox and Safari).

Wait a moment… Unity plugin makes nice wobbling web page elements not wobble anymore!? Sounds like an interesting issue…

So I prepared for a debug session and tried the usual “divide by two until you locate the problem” approach.

• Unity Web Player is composed of two parts: a small browser plugin, and the actual “engine” (let’s call it “runtime”). First I change the plugin so that it only loads the data, but never loads or starts the runtime. Everything works. So the problem is not in the plugin. Good.
• Load the runtime and do basic initialization (create child window, load Mono, …), but never actually start playing the content – everything works.
• Load the runtime and fully initialize everything, but never actually start playing the content – the bug appears! By now I know that the problem is somewhere in the initialization.

Initialization reads some settings from the data file, creates some “manager objects” for the runtime, initializes graphics device, loads first game “level” and then the game can play.

What of the above could cause something inside browser’s JavaScript engine stop working? And do that only on Windows, and only on some browsers? My first guess was the most platform-specific part: intialization of the graphics device, which on Windows usually happens to be Direct3D.

So I continued:

• Try using OpenGL instead of Direct3D – everything works. By now it’s confirmed that initializing Direct3D causes something else in the browser not work.
• “A-ha!” moment: tell Direct3D to not change floating point precision (via a create flag). Voilà, everything works!

I don’t know how I actually came up with the idea of testing floating point precision flag. Maybe I remembered some related problems we had a while ago, where Direct3D would cause timing calculations be “off”, if the user’s machine was not rebooted for a couple of weeks or more. That time around we properly changed our timing code to use 64 bit integers, but left Direct3D precision setting intact.

Side note: Intel x86 floating point unit (FPU) can operate in various precision modes, usually 32, 64 or 80 bit. By default Direct3D 9 sets FPU precision to 32 bit (i.e. single precision). Telling D3D to not change FPU settings could lower performance somewhat, but in my tests it did not have any noticeable impact.

So there it was. A debugging session, one line of change in the code, and fancy javascript webpage animations work on Windows in Firefox and Safari. This is coming out in Unity 2.0.2 update soon.

The moral? Something in one place can affect seemingly completely unrelated things in another place!

It’s been an amazing ride so far. I’m working with amazing people, we have (mostly :)) amazing customers, we’ve done three major releases and five minor ones, I’ve seen sales grow from ridiculously low amount to feels good levels, I’ve seen the team expansion, and I took svn revision number 10000 for myself (that happened half a year ago):

> Wohoo! Congratulations on the 10,000th commit Aras! You have won a warm cup of freshly brewed coffee when you come to work! :)

You don’t know the whole exciting story behind this… :)

I had two tiny “cleanups” to two files – removing unused variable from each. Now, I’d so something on my Mac, start compiling, and while it’s compiling I’d check svn log on the windows machine.

As soon as Valdemar checked in revision number 9998, I knew what I had to do. Two trivial svn commits!

Some serious work, as you can see. Oh, some of my work also ended up being actually released, I’m not just sitting there claiming svn revisions for myself. Honestly!

I’ve moving back to Lithuania now, continuing to work on Unity from home. Gonna miss some of the office fun, but oh well. Tradeoffs have to be made.

Let’s see what the coming years will bring. Rock on.

It’s a cute little gem, or alternatively, “Puzzle Bobble meets Ageia’s PhysX engine”. Lots of levels, user level editor, some AJAX magic and so on, and everything made in about four weeks. Matthew has some technical details for the curious.

Here are some shots and a youtube trailer for the lazy ones. The youtube trailer uses music from 8bitpeoples which can’t be bad :)

But really, you should just go and play it: splume.flashbangstudios.com

* I stand corrected: When Orcs Attack also was made with Unity 2.0, and it was released earlier!

It’s been in active development for about a year (though some of 2.0 features were in development for much longer), with source control commits graph looking roughly like this. We pretty much spent the summer doing 800 svn commits/month with about four major code monkeys :) Now that the release is done I fully expect the graph to drop off to low values again (is it called “burnout”?).

Last week there was the first Unity conference, and it was a blast. It was like, oh my, it’s full of people and all about unity! In other words, really really cool.

The night before the conference was spent in the hotel, doing last tweaks to the website and launch demos. If the presenters during keynote talk looked confused or exhausted, that’s a combination of trans-Atlantic flight and this last night of work. In fact, the very last fixes to the website were done during the keynote… oh well, Murphy’s law for the win.

Now what? Time to start working on Unity 2.x release :)

This is actually me debugging point light shadows (that happen to use depth encoded into RGBA8 cubemaps).

This is what happens when you use a too wide Poisson disc blurring in screen space and no prevention of “shadow leakage” over different depths.

LOL! Internet!

Here is how we do it at work. This took quite some time to set up, but I think it’s very worth it.

First you need hardware to test things on. For a start just a couple of graphics cards that you can swap in and out might do the trick. A larger problem is integrated graphics cards – it’s quite hard to swap them in and out, so we bit the bullet and bought a machine for each integrated card that we care about. The same machines are then used to test discrete cards (we have several shelves of those by now, going all the way back to… does ATI Rage, Matrox G45 or S3 ProSavage say anything to you?).

Then you make the unit tests (or perhaps these should be called the functional tests). Build a small scene for every possible thing that you can imagine. Some examples:

• Do all blend modes work?
• Do light cookies work?
• Does automatic texture coordinate generation and texture transforms work?
• Does rendering of particles work?
• Does glow image postprocessing effect work?
• Does mesh skinning work?
• Do shadows from point lights work?

This will result in a lot of tests, with each test hopefully testing a small, isolated feature. Make some setup that can load all defined tests in succession and take screenshots of the results. Make sure time always progresses at fixed rate (for the case where a test does not produce a constant image… like particle or animation tests), and take a screenshot of, for example, frame 5 for each test (so that some tests have some data to warm up… for example motion blur test).

By this time you have something that you can run and it spits out lots of screenshots. This is already very useful. Get a new graphics card, upgrade to new OS or install a new shiny driver? Run the tests, and obvious errors (if any) can be found just by quickly flipping through the shots. Same with the changes that are made in rendering related code – run the tests, see if anything became broken.

The testing process can be further automated. Here we have a small set of Perl scripts that can either produce a suite of test images for the current hardware, or run all the tests and compare the results with “known to be correct” suite of images. As graphics cards are different from each other, the “correct” results will be somewhat different (because of different capabilities, internal precision etc.). So we keep a set of test results for each graphics card.

Then these scripts can be run for various driver versions on every graphics card. They compare results for each test case, and for failed tests copy out the resulting screenshot, the correct screenshot, log the failures into a wiki-compatible format (to be posted on some internal wiki), etc.

I’ve heard that some folks even go a step further – fully automate the testing of all driver versions. Install one driver in silent mode, reboot the machine, after reboot runs another script that launches the tests and proceeds with the next driver version. I don’t know if that is only an urban legend or if someone actually does this^*, but that would be an interesting thing to try. The testing per card then would be: 1) install a card, 2) run the test script, 3) coffee break, happiness and profit!

^{* My impression is that at least with the big games it works the other way around – you don’t test with the hardware; instead the hardware guys test with your game. That’s how it looks for a clueless observer like me at least.}

So far this unit test suite was really helpful in a couple of ways: making of the just-announced Direct3D renderer and discovering new & exciting graphics card/driver workarounds that we have to do. Making of the suite did take a lot of time, but I’m happy with it!

…OTEE crew playing dance minigame in WarioWare: Smooth Moves. The game is awesome – you get lots of fun and look stupid at the same time!

I like it! That says a lot, because last time I played FPS was ages ago. Finally, someone made a game that I can like again. How much I’m biased because it’s made with Unity… decide for yourselves :)

Now the killer part: the game is designed and developed solely by a 15 year old – Forest “Yoggy” Johnson. How cool is that?

Enough talking, just go and play it. And remember that you didn’t see cool until you get to the robot level!

Demos: In the other news, fellow ReJ with TBL just won Assembly 2006 demo competition with an Amiga demo, putting all PC demos faces’ to dust. Check it out. Art direction over hardware capabilities, one more time.

Drivers: why oh why the graphics drivers must be so bad? The other day I was thinking why can’t they auto-update themselves (with an option to turn it off for corporate users etc.). Now you’ve got a (not so recent!) driver that is able to parse vertex programs wrong, and a user who does not have a clue that he should update it. It’s bad enough to have a bug in the first place, but auto-update at least would fix…

Or you have a driver that says “I’m OpenGL 1.2!” but the 3D texture functions are null. And it’s the most recent driver for a particular graphics card that you can buy today! And it’s not even a hard problem! What the developers are thinking – they go over the required GL 1.2 functionality, see that some is actually missing and decide “ah, screw it, we’ll say it’s 1.2 anyways”?!

I just don’t get it. I could use some enlightenment on this.