I’m happy to say it did! That’s it, move on to read the rest of the internets.

The earlier post was more focused on hardware compatibility area (differences between platforms, GPUs, driver versions, driver bugs and their workarounds etc.). In addition to that, we do regression tests on a bunch of actual Unity made games. All that is good and works, let’s talk about what tests the rendering team at Unity is using in the daily lives instead.

Graphics Feature & Regression Testing

In daily life of a graphics programmer, you care about two things related to testing:

1. Whether a new feature you are adding, more or less, works.
2. Whether something new you added or something you refactored broke or changed any existing features.

Now, “works” is a vague term. Definitions can range from equally vague

Works For Me!

to something like

It has been battle tested on thousands of use cases, hundreds of shipped games, dozens of platforms, thousands of platform configurations and within each and every one of them there’s not a single wrong pixel, not a single wasted memory byte and not a single wasted nanosecond! No kittehs were harmed either!

In ideal world we’d only consider the latter as “works”, however that’s quite hard to achieve.

So instead we settle for small “functional tests”, where each feature has a small scene setup that exercises said feature (very much like talked about in previous post). It’s graphics programmer’s responsibility to add tests like that for his stuff.

For example, Fog handling might be tested by a couple scenes like this:

Another example, tests for various corner cases of Deferred Lighting:

So that’s basic testing for “it works” that the graphics programmers themselves do. Beyond that, features are tested by QA and a large beta testing group, tried, profiled and optimized on real actual game projects and so on.

The good thing is, doing these basic tests also provides you with point 2 (did I break or change something?) automatically. If after your changes, all the graphics tests still pass, there’s a pretty good chance you did not break anything. Of course this testing is not exhaustive, but any time a regression is spotted by QA, beta testers or reported by users, you can add a new graphics test to check for that situation.

How do we actually do it?

We use TeamCity for the build/test farm. It has several build machines set up as graphics test agents (unlike most other build machines, they need an actual GPU, or a iOS device connected to them, or a console devkit etc.) that run graphics test configurations for all branches automatically. Each branch has it’s graphics tests run daily, and branches with “high graphics code activity” (i.e. branches that the rendering team is actually working on) have them run more often. You can always initiate the tests manually by clicking a button of course. What you want to see at any time is this:

The basic approach is the same as 4 years ago: a “game level” (“scene” in Unity speak) for each test, runs for defined number of frames, run everything at fixed timestep, take a screenshot at end of each frame. Compare each screenshot with “known good” image for that platform; any differences equals “FAIL”. On many platforms you have to allow a couple of wrong pixels because many consumer GPUs are not fully deterministic it seems.

So you have this bunch of “this is the golden truth” images for all the tests:

And each platform automatically tested on TeamCity has it’s own set:

Since the “test controller” can run on a different device than actual tests (the case for iOS, Xbox 360 etc.), the test executable opens a socket connection to transfer the screenshots. The test controller is a relatively simple C# application that listens on a socket, fetches the screenshots and compares them with the template ones. The result of it is output that TeamCity can understand; along with “build artifacts” that consist of failed tests (for each failed test: expected image, failed image, difference image with increased contrast).

That’s pretty much it! And of course, automated tests are nice and all, but that should not get too much into the way of actual programming manifesto.

Preemptive warning: I’ve only ever used CVS, SourceSafe, Subversion, git and Mercurial as source contro systems (never used Perforce). I never really used a code review tool before Kiln. Everything below might be non-issues in other tools/systems, or not suitable for different setups/workflows!

The Story

At Unity we used Subversion for source code versioning as long as I remember. svn revision 1 — an import from CVS — happened in 2005. We don’t talk about CVS. Nor about SourceSafe. Subversion was fine while the number of developers was small; we had a saying that CVS scales up to 5 people, and experimentally found out that svn scales up to about 50.

Since merging branches in subversion does not really work well, everyone was mostly working on one trunk, carefully. We would do an occasional branch for “this will surely break everything” features; and would branch off trunk sometime before each Unity release, but that’s about it. Having something like 50 people and 10 platforms on a single branch in version control does get a bit uneasy.

So we looked at various options, like git, Mercurial, Perforce and so on. I don’t know why exactly we ended up with Mercurial (someone made a decision I guess…). It felt like distributed versioning systems are teh future and unlike most game developers we don’t need to version hundreds of gigabytes of binary assets (hence no big need for Perforce).

So while some people were at GDC, we did a big switch to several things at once: 1) replace Subversion with Mercurial, 2) replace “everyone works on the same trunk” workflow with “teams work on their own topic branches”, 3) introduce a bit more formal code reviews via Kiln.

In hindsight, maybe switching three things at once wasn’t the brightest idea; there’s only so much change a person can absorb per unit of time. On the other hand, everyone experienced a large initial shock but now that the debris is setting down they just continue working with no big shocks predicted in the near future.

Our Setup

We use Fogcreek’s Kiln and host it on our own servers. This is mostly for legal reasons I think (in our source code we have 3rd party bits which are under strict NDAs). Advantage of hosting ourselves is that we’re under complete control. Disadvantage is that we have to do some work; and we only get Kiln updates each couple of months (so for example everyone who lets Fogcreek host Kiln is on Kiln 2.4.x right now, while we’re still on 2.3.x).

Our source tree is about 12000 files amounting to about 600MB. Mercurial’s history (60000 revisions imported from svn) adds another 200MB. Additionally, we pull almost 1GB of binary files (see below for binary file versioning) into the source tree.

Each “team” (core, editor, graphics, ios, android, …) has it’s own “branch” (actually, a separate repository clone) of the codebase, and merge back and forth between “trunk” repository. The trunk is supposed to be stable and shippable at almost any time (in theory… :)); unfinished, unreviewed code or code that has any failing tests can’t be pushed into trunk. Additionally, long-lasting features get their own “feature branches” (again, actually full clones of the repository). So right now we have more than 40 of those team+feature branches.

We have almost 50 developers committing to the source tree. Additionally, there is a build farm of 30 machines building most of those branches and running automated test suites. All this does put some pressure on the Kiln server ;) Everything below describes usage of Kiln 2.3.x with Mercurial 1.7.x; with more recent versions anything might have changed.

Mercurial, or: I Have Two Heads!

Probably the hardest thing to grok is the whole centralized-to-distributed versioning transition. Not everyone has github as their start page yet, and DVCS is actually more complex than a simple centralized model that Subversion has.

Things like this:

OMG it says I have two heads now, what do I do?!

just do not happen in centralized systems. It’s not easy for a developer to accept he has two heads now, either. Or where this extra head came from…

And the benefits of distributed source control system are not immediately obvious to someone who’s never used one. The initial reaction is that suddenly everything got more complex for no good reason. Compare operations that you would use daily:

• Subversion: update, commit.
  • Since merges don’t really work: branch, switch & merge are rarely used by mere mortals.
• Mercurial: pull, update or merge, commit, push.
  • And you might find you have two heads now!
  • You should also see their faces when you go “well, let me tell you about rebase…”. You might just as well explain everything with easy to understand spatial analogies ;)

Thankfully, there’s this thing called the intertubes, which often has helpful tutorials.

Myself, I think maybe switching to git would have been a smaller overall shock. Mercurial is easier to get into, but it kind of pretends to work like ye olde versioning system, while underneath it is very different. Git, on the other hand, does not even try to look similar; it says “I’ll fuck with your brain” immediately after initial “hi how are you”. So it’s a larger initial shock, but maybe that forces people to get into this different mindset faster.

Versioning large binary files

Even if we mostly version only the code, there are occasional binaries. In our case it’s mostly 3rd party SDKs that are linked into Unity. For example, PhysX, Mono, FMOD, D3DX, Cg etc. We do have the source code for most of them, but we don’t need each developer to have 30000 files of Mono’s source code for example. So we build them separately, and version the prebuilt headers/libraries/DLLs in the regular source tree. Some of those prebuilt things can get quite large though (think couple hundred megabytes).

Most distributed version control systems (including git and mercurial) have trouble with this. Every version of every file is stored in your own local checkoutclone. Try having 50 versions of whole Mono build in there and you’ll wonder where the precious SSD space on your laptop did go!

Luckily, Kiln has a solution for this: kbfiles extension. For each file marked as “large binary file”, only it’s “stand in” SHA1 hash is versioned, and the file itself is fetched from a central server into your local machine on demand. Think of it as a centralized versioning model for those special binary files. kbfiles itself is based on bfiles extension, with a tighter integration into Mercurial.

So the good news, with Kiln large binary files are handled easy and with no pain. You can globally set “large size” threshold, filename patterns etc. that are turned into “big files” automatically; or manually indicate “big file” when adding new files. And then continue using Mercurial as usual.

The bad news, however, is that kbfiles still has occasional bugs. Of course they will be fixed eventually, but for example right now rebasing with an incoming bigfiles commit will result in the wrong bigfile version in the end. Or, presence of kbfiles extension makes various Mercurial operations (like hg status) be much slower than usual.

Kiln as Web Interface

Kiln itself is the server hosting Mercurial repositories, a web interface to view/admin them, and a code review tool. It’s fairly nice and does all the standard stuff, like show overview of all activity happening in a group of repositories:

And shows the overview of any particular repository:

And of course diff view of any particular commit:

My largest complaints about Kiln’s web interface are: 1) speed and 2) merge spiderwebs.

Speed: like oh so many modern fancy-web systems, Kiln sometimes feels sluggish. Sometimes, in a time taken for Kiln to display a diff, Crysis 2 would have rendered New York fifty times. We did various things to boost up our server’s oomph, but it still does not feel fast enough. Maybe we don’t know how to setup our servers right; or maybe Kiln is actually quite slow; or maybe our repository size + branch count + number of people hitting it are exceeding whatever limits Kiln was designed for. That said, this is not unique of Kiln, lots of web systems are slow for sometimes no good reasons. If you are a web developer, however, keep this in mind: latency of any user operation is super important.

Merge spiderwebs: distributed version control makes merges reliable and easy. However, merges happen all the time and can make it hard to see what was actually going on in the code. You can’t see the actual changes through the merge spiderwebs.

The change history is littered with “merge”, “merge remote repo”, “merge again” commits. The branch graph goes crazy and starts taking half of the page width. Not good! Now of course, this is where rebasing would help, however right now we’re not very keen on using it because of Kiln’s bigfiles bug mentioned above.

Kiln as Code Review Tool

Reviewing code is fairly easy: there’s a Review button that shows up when hovering over any commit. Each commit also shows how many reviews it has pending or accepted. So you just click on something, and voilà, you can request a code review:

Within each review you see the diffs, send comments back and forth between people, and highlight code snippets to be attached with each comment:

In Kiln 2.3.x (which is what we use at the moment) the reviews still have a sort of “unfinished” feeling. For example, if you want multiple people to review a change, Kiln actually creates multiple reviews that are only very loosely coupled. The good news is that in Kiln 2.4 they have improved this, and I’m quite sure more improvements will come in the future.

Another option that I’m missing right now: in the repository views, filter out all approved commits. As an occasional “merge master”, I need to see if my big merge had any unreviewed or pending-review commits — something that’s quite hard to see with a merge-heavy history.

Summary

I’m quite happy with how switch to Mercurial + Kiln turned out to be so far. With each team working on their own repository, it does feel like we’re much less stepping on each other’s toes. That said, we haven’t shipped any Unity release from Mercurial yet; doing that will be a future exercise.

Kiln is promising. It has some very good ideas (integrated code reviews & versioning of big files in Mercurial), but it still has quite a lot of rough edges. I’m not totally happy with the web side performance of it either. That said, Fogcreek’s support for us has been fantastic; we got some bugfixes in the matter of days and they’ve been really helpful with setup/workflow/optimization issues. So it seems like it has a good future. Fogcreek guys, if you’re reading this: keep up wrk!

You need to do ten thousand things for the gold master / release / ShipIt(tm) moment. And you have 40 people who do the actual work… this means each of them only has to do 10000/40=250 things, which is not that bad. Right?

Meanwhile in the real world… it does not actually work like that. And that’s something that has been on my mind for a long time. I don’t know how much of this is truth vs. perception, or what to do about it. But here’s my feeling, simplified:

20 percent of the people are responsible for getting 80 percent of the work done

I am somewhat exaggerating just to keep it consistent with the Pareto principle. But my feeling is that “work done” distribution is highly non uniform everywhere I worked where the team was more than a handful of people.

Here are some stupid statistics to illustrate my point (with graphs, and everyone loves graphs!):

Graph of bugs fixed per developer, over one week during the bug fixing phase. Red/yellow/green corresponds to priority 1,2,3 issues:

The distribution of bugs fixes is, shall we say, somewhat non uniform.

Is it a valid measure of “productivity”? Absolutely not. Some people probably haven’t been fixing bugs at all that week. Some bugs are way harder to fix than others. Some people could have made major part of the fix, but the finishing touches & the act of actually resolving the bug was made by someone else. So yes, this statistics is absolutely flawed, but do we have anything else?

We could be checking version control commits.

Or putting the same into “commits by developer”:

Of course this is even easier to game than resolving bugs. “Moving buttons to the left”, “Whoops, that was wrong, moving them to the right again” anyone? And people will be trolling statistics just because they can.

However, there is still this highly subjective “feeling” that some folks are way, way faster than others. And not in just “can do some mess real fast” way, but in the “gets actual work done, and done well” way.

Or is it just my experience? How is it in your company? What can be done about it? Should something be done about it? I don’t know the answers…

Well ok, it all started a bit before:

I exchanged some emails with David and Joachim and they invited me for a gamejam in their office. Then one thing led to another, I was young and needed money (oops! wrong topic) and on January 2006 I started working on this thing called “Unity”.

Unity was at version 1.2.1 then. Since then we’ve released about a dozen new versions, added hundreds (or thousands?) of new features, a handful of new platforms and have grown a lot.

Also, we stopped saying “Sales are INSANE!!!!11″ whenever they exceeded a whopping ten thousand euros per week. Seriously, that much money in 2006 was a big thing. Our Windows build machine was a single core Celeron with 512MB RAM because that’s what we could afford! Well ok, we’re still saying “sales are insane!” from time to time, just the threshold has gone way up.

Occasionally we’d get excited about strangest things. I think this email is about some car model from ATI that was on front page of our website in 2006. It’s beyond me why we’d put a car on Unity website, but somehow it seemed to make sense at the time.

It would take too much space to list all the awesome things that happened in those four years. I got to work on some things too, like Windows Web Player, Direct3D renderer, shadows, editor for Windows and whatnot. But I mostly concentrate on creating trouble, which does not seem to hinder Unity that much. I need to get more efficient!

Seriously though, it has been an amazing ride so far, and I hope it will only become better. Thanks to everyone at Unity Technologies and the community!

Rock on!

The process is roughly this (all of this is DX9 level tech on PCs; later tech or consoles could and should use more optimizations):

1. Render shadow map cascades. All of them packed into one shadow map via viewports.
2. Collect shadows into screen sized render target. This is the shadow term.
3. Blur the shadow term.
4. In regular forward rendering, use shadow term in screen space.

More detail:

Render Shadow Cascades

Nothing fancy here. All cascades packed into a single shadow map. For example two 512×512 cascades would be packed into 1024×512 shadow map side by side.

Screen-space Shadow Term

Render all shadow receivers with a shader that “collects” shadow map term. In effect, shadows from all cascades are collected into a screen-sized texture. After this step, original cascaded shadowmaps are not needed anymore.

Unity supports up to 4 shadow map cascades, which neatly fit into a float4 register in the pixel shader. Correct cascade is sampled just once, without using static or dynamic branching. Pixel shader pseudocode:

float4 near = float4 (z >= _LightSplitsNear);
float4 far = float4 (z < _LightSplitsFar);
float4 weights = near * far;
float2 coord =
    i._ShadowCoord[0] * weights.x +
    i._ShadowCoord[1] * weights.y +
    i._ShadowCoord[2] * weights.z +
    i._ShadowCoord[3] * weights.w;
float sm = tex2D (_ShadowMapTexture, coord.xy).r;

Additionally, shadow fadeout is applied here (shadows in Unity can be cast up to specified distance from the camera, and they fade out when approaching that distance).

After this I end up having shadow term in screen space. Note that here I do not do any shadow map filtering; that is done in screen space later.

On PCs in DX9 there is (or there was?) no easy/sane way to read depth buffer in the pixel shader, so while collecting shadows the shader also outputs depth packed into two channels of the render target.

Screen-space Shadow Blur

Previous step results in screen space shadow term and depth. Shadow term is blurred into another render target, using a spatially varying Poisson disc-like filter.

Filter size depends on depth (shadow boundaries closer to the camera are blurred more). Filter also discards samples if difference in depth is larger than something, to avoid blurring over object boundaries. It's not totally robust, but seems to work quite well.

Using shadow term in forward rendering

In forward rendering, this blurred shadow term texture is used. Here shadow term already has filtering & fadeout applied, and the shaders do not need to know anything about shadow cascades. Just read pixel from the texture and use it in lighting computation. Done!

Fin

Back then I didn't know this would be called "deferred" (that would probably have scared me away!). I don't know if this approach is any good, but so far it works quite well for Unity needs. Also, reduces shader permutation count a lot, which I like.

And that’s where ol’ George found himself out there at the FogBugz
Fixin’ the devil’s bugs
Now, a man figures it’s his bugs and he’ll assign whom he wants
But it don’t always work out that way
You see, some bugs are special for a certain target
A certain platform, or a certain person
And no matter whom you’re assignin’, that’s where the bug ‘ll end up
And in the moment of assigning your mouse turns into a dowser’s wand
And clicks where the bug wants to go.

Uhm. Yeah.

Bits and pieces I found useful

• SSAO can be generated at a smaller resolution than screen, with depth+normals aware upsample/blur step.
• If random offset vector points away from surface normal, flip it. This makes random vectors be in the upper hemisphere, which reduces false occlusion on flat surfaces. Of course this requires having surface normals.
• When generating random vectors for your AO kernel:
  • Generate vectors inside unit sphere (not on unit sphere).
  • Use energy minimization to distribute your samples better, especially at low sample counts. See malmer.ru blog post.
• In your AO blurring/upsampling step: no need to sample each pixel for blur. Just skip some of them, i.e. make kernel offsets larger. See below.

Strided blur for AO

Normally you’d blur AO term using some sort of standard blur, for example separable Gaussian: horizontal blur, followed by vertical blur. How one can imagine horizontal blur kernel:

Here’s how Rune taught me how to blur better:

Rune:

The other thing is the blur. I tried to make the blur 4 times stronger, and it looks much better IMO without any artifacts I could see. I could even use 4x downsampling with that blur amount and still get acceptable results.

Aras:

how did you make it 4x stronger? (I was going to say that blur step is already quite expensive, and I don’t want to add more samples to make it even more expensive, yadda yadda)

Rune:

m_SSAOMaterial.SetVector (“_TexelOffsetScale”, m_IsOpenGL ?
  new Vector4 (4,0,1.0f/m_Downsampling,0) :
  new Vector4 (4.0f/source.width,0,0,0));
And similar for vertical.

Aras:

hmm. that’s strange :)

Rune:

I have no idea what I’m doing of course but it looks good.

Aras:

so this way it does not do Gaussian on 9×9 pixels, but instead only takes each 4th pixel. Wider area, but… it should not work! :)

Rune:

It creates a very fine pattern at pixel level but it’s way more subtle than the noise you get otherwise.

Aras:

ok (hides in the corner and weeps)

So yeah. The blur kernel can be “spread” to skip some pixels, effectively resulting in a larger blur radius for the same sample count:

Or even this:

Yes, it’s not correct blur. But that’s okay, we’re not building nuclear reactors that depend on SSAO blur being accurate. If you are, SSAO is probably a wrong approach anyway, I’ve heard it’s not that useful for nuclear stuff.

I’m not sure how this blur should be called. Strided blur? Interleaved blur? Interlaced blur? Or maybe everyone is doing that already and it has a well established name? Let me know.

Some images of blur in action. Raw AO term (very low – 8 – sample count and increased contrast on purpose):

Regular 9×9 blur (does not blur over depth+normals discontinuities):

Blur that goes in 2 pixel stride (effectively 17×17):

It does create a fine interleaved pattern because it skips pixels. But you get wider blur!

Blur that goes in 3 pixel stride (effectively 25×25):

At 3 pixel stride the artifacts are becoming apparent. But hey, this is very
low AO sample count, increased contrast and no textures in the scene.

For sake of completeness, the same raw AO term, but computed at 2×2 smaller resolution (still using low sample count etc.):

Now, 2×2 smaller AO, blurred with 3 pixels stride:

Happy blurring!

In Unity editor pretty much any window can be “detached” from the main window. An obvious use case is putting it onto a separate monitor. But of course you can just end up having a ton of detached windows overlapping each other.

Here I have four windows in total on OS X:

Here I have four windows on Windows:

However, users of OS X and Windows are used to applications behaving differently.

On OS X, it is very common that a single application has many overlapping windows. Usually users don’t have problems finding their windows either, thanks to Exposé. Press a key, voilà, here they are:

On Windows, there is no Exposé. So there’s a problem: when a detached window is obscured by another window, how do you get to it? One would ask “well, what’s wrong in having windows partially overlapped, like in above screenshot?”, to which I’d say “you’re a Mac user”.

Windows users do not have a ton of windows on screen. They tend to maximize the application they are currently working with. I was doing this myself all the time, and it took 3 years of Mac laptop usage before I stopped maximizing everything on my Windows box!

So what a typical Windows user might see when using Unity is this. Now, where are the other three detached windows?

On Windows, it is very uncommon for a single application to have many overlapped windows. When an application does that, the “detached” windows are always positioned on top of the main window. There are some applications that do not do this (yes I’m looking at you GIMP), and almost everyone is not happy with their usability.

So we decided to take this context into account. Windows users do not have Exposé, and they expect “detached” windows to be always on top of the main window. Unity 2.6 will do this soon.

Of course, you still can dock all the windows together and this whole “windows are obscured by other windows” issue goes away:

Hmm… I think the screenshots above show two new big features in upcoming Unity 2.6. Preemptive note: UI of the stuff above is not final. Anything might change, don’t become attached to any particular pixel!

Here are my findings – with error visualization and shader performance numbers for some GPUs.

If you know any other method to encode/store normals in a compact way, please let me know!

Well, finally I implemented it for upcoming Unity 2.6. I wrote some sort of a technical report here.

In short, I’m combining assembly fragments and doing simple temporary register allocation, which seems to work quite well. Performance is very similar to using fixed function (I know it’s implemented as vertex shaders internally by the runtime/driver) on several different cards I tried (Radeon HD 3xxx, GeForce 8xxx, Intel GMA 950).

What was unexpected: the most complex piece is not the vertex lighting! Most complexity is in how to route/generate texture coordinates and transform them. Huge combination explosion there.

Otherwise – I like! Here’s a link to the article again.

Here’s what’s new. Here’s the download page.

My 11th Unity release since I joined 3+ years ago. This is quite a crazy release that involved almost complete editor tools rewrite and lots of other juggling. Was not exactly a walk in the park, but it’s done now. Meet me at GDC in San Francisco next week and I’ll tell you the war stories (Unity booth is 5110 NH).

Here’s the obligatory source code commits graph:

18 people involved in source code, 5315 commits, 18501 file changes. Of course, svn commits do not mean anything… I’m just fascinated by graphs and numbers.

In summary, Vista is not that bad.

Once you get used to changes in Explorer, different skin and so on – it’s actually usable. I think they have made some real improvements in the underlying technology, too bad they managed to “compensate” for all of that by inconsistencies and lack of polish in user interface.

At this point it’s minor quirks in UI that annoy me, but apart from that, Vista is okay. Look:

Who implemented blending of icon overlays and do they still have a job? No sir, that shield icon is not properly blended here!

Who thought it’s a good idea to make the Burn icon bright red? In 6 months, I never used it. Why is it the brightest thing in the whole Explorer window?

Try going one folder up without resorting to this drop down menu. Utilities is the current folder here. And no, there’s no keyboard shortcut for “go up” either (there was in XP, which was perfect).

And of course, the awesome shutdown menu. The two buttons – never used them. What I always use is “Shut Down” from the menu. And let’s not even talk about all the choices in the menu (no, more choices is not always better).

So yeah. It’s not stellar, it has tons of small annoyances (and some large ones – try developing web plugins with UAC on…), but it’s usable. I might have gotten used to it by now, actually.

I don’t really know why that happens, because it seems that most modern cards don’t have fixed function units, so internally they are running shaders anyway. DX9 runtime on Vista’s WDDM also seems to be only handling shaders to the driver internally. Still, for some reason somewhere the precision does not match…

How such a task should be approached?

My requirements are:

• Should handle any possible state combination in D3D fixed function T&L.
• D3D 9.0c, using vertex shader 2.0 is ok. For now I don’t care about OpenGL.
• No HLSL at runtime. I don’t want to add a megabyte or more to Unity web player just for HLSL. DX9 shader assembly is ok, because we already have the assembler code.
• Should work as fast (or close to) as the regular fixed function pipeline.

I looked at ATI’s FixedFuncShader sample. It’s an ubershader approach; one large (230 instructions or so) shader with static VS2.0 branching. It had some obvious places to optimize, I could get it down to 190 or so instructions, kill some rcp‘s and reduce the amount of constant storage by 2x.

Still, it did not handle some things in the D3D T&L or had some issues:

• It assumes one input UV, one output UV and no texture matrices. This place in T&L gets quite convoluted – any input UVs or a texgen mode can be transformed by matrices of various sizes, and routed into any output UVs.
• It was not using full T&L lighting model. No biggie here.
• I haven’t checked with NVShaderPerf or AMD ShaderAnalyzer yet, but last time I checked the static branch instruction was taking two clocks on some NV architecture. So ubershader approach does not come for free.

Another thing I’m considering, is to combine final shader(s) from assembly fragments, with some simple register allocation.

In T&L shader code, there’s only limited set of could-be-redundant computations, mostly computing world space position, camera space normal, view vector and so on (those could be used lighting, texgen or fog). Those computations can be explicitly put into separate fragments, and later fragments could just use their result.

What is left then is some register allocation. A shader assembly fragment could want some temporary registers for internal use (this is simple, just give it a bunch of unused registers), also want some registers as input (from previous fragments), and save some output in registers.

Again, I haven’t checked with shader performance tools, but I think, guess and hope that the drivers do additional register allocation, liveness analysis etc. when converting D3D shader bytecode into hardware format. This would mean that I can be quite sloppy with it, i.e. don’t have to implement some super smart allocation scheme.

I wrote some experimental code for the shader assembly combiner and so far it looks like a reasonable approach (and not too hard either).

Does that make sense? Or did everyone solve those problems eons ago already?

Edit: half a year later, I wrote a technical report on how I implemented all this: http://aras-p.info/texts/VertexShaderTnL.html

void GUIView::MakeVistaDWMHappyDance()
{
    // Looks like Vista has some bug in DWM. Whenever we maximize or dock
    // a view, we must do something magic, otherwise
    // white stuff appears in place of the view.
    // See http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=4208117&SiteID=1

    bool earlierThanVista = systeminfo::GetOperatingSystemNumeric() < 600;
    if( earlierThanVista )
        return;

    // What seems to work is drawing one pixel via GDI.
    // We draw it at (1,1) with usual background color.
    int grayColor = 0.61f * 255.0f;
    PAINTSTRUCT ps;
    BeginPaint(m_View, &ps);
    SetPixel(ps.hdc, 1, 1, RGB(grayColor,grayColor,grayColor));
    EndPaint(m_View, &ps);
}

I know. Reading from screen when Aero is on is slow, bad and wrong. But then, what do you do? It’s better than users staring an all-white window just because Vista decided to draw it white, no matter what you think you’re drawing into it.

…still, MakeVistaDWMHappyDance is not nearly as cool as

internal interface ICanHazCustomMenu { … }

that Nicholas added a while ago.

• Fixed import progress bar for movies with no audio
• Fixed first context menu click not working on Windows
• Eye dropper backend on Windows
• Export Package actually works on Windows
• Compare Binary works on Windows
• Add checkbox to project wizard to always open it on startup
• F1 in bundled text editor goes to scripting docs for current word
• Fixed q/w/e/r keys in password fields and text areas toggling active Tool on Windows
• Fixed panes not repainting on Windows after some change is done via context menu on them
• …and so on.

Boring tiny little details.

This probably best summarizes where lion’s share of time goes when developing anything. I’m not working on some cool spherical harmonics lightmap compression. Or on cunning ways to encode shadow map information for better filtering. Or on using CUDA to compute something interesting.

In other words, I’m not working on cool technology. Instead I’m adding missing menu items. Fixing obscure corner cases. Fighting inconsistencies in operating system APIs. Spotting misplaced pixels. Adding missing keyboard shortcuts.

Nothing interesting to blog about!

But still, methinks the difference between software that is merely “good” and software that is “great” is in the details. And only in the details.

I’ll just take care of tons of more details. Maybe it will result in something good.

I’m the master of svn deception. I do tons of useless commits just so that the stats look good. Yeah!

…ok, back to work.

Take a quick look at zillions of new posts.

Answer about five questions with “what’s the value of your camera’s near plane?”.

There should be some way to automate all of this. For every 20th question, reply with “increase your near plane!”, or something.

We showed a glimpse of Unity editor for Windows at the keynote, so it is public now – yes, we are working on Windows toolchain. About the time! This is the major area I’m spending time these days – Windows, Windows, Windows. Learning WinAPI as I cruise along :) Before Unity 2.1 I spent months fixing tons of small issues, now I’m spending months doing tons of small Windows related things. Someday I’ll get back to doing tons of small things on the rendering side.

Here’s a couple of random photos that I stoleborrowed from Mantas:

Keynote in front of a Sentinel from The Matrix.

Presenters talking.

People listening!

I don’t know that guy in the center. Probably some stupid outsider. Really!

For the sake of the nation,
this operator must die!

Seriously. Suppose there is some class, let’s say ColorRGBAf. That has four floats inside. Now, someone at some point decided to add this operator to it:

operator float* () { /**/ }
operator const float* () const { /**/ }

Probably because it’s easier to pass color to OpenGL this way, or something like that.

This is evil. Like, really evil. Especially if that class did not have comparison operators defined, and some totally unrelated code four years later does:

if (color != oldColor) { /* … */ }

Ouch! Sounds like someone will spend four hours debugging something that looks like an event routing issue that only happens on Windows and only with optimizations on (yes, I just did that…).

What happens here? The compiler takes pointers to two colors and compares the pointers. If for some reason both colors are temporary objects, then it can even happen that both get folded into the same variable/register/whatnot. The pointers are the same. Ouch!

Implicit “nice” operators are just disguised evil. Remove that operator, add something like GetPointer() to class if someone really wants to use that, and better even make the comparison operators private and without implementations. Yes. Much better.

static void WatchdogFunc()
{
    while( true )
    {
        time_t now = time(NULL);
        Mutex::AutoLock lock(g_WatchdogMutex);
        if( now - g_StartTime > kWatchdogTimeout )
            ComplainLoudlyAndDoSomething();
        Thread::Sleep( 0.1f );
    }
}

Mutex is taken because g_StartTime can be occasionally updated by the same tool. Yes, possibly a mutex is an overkill here, and aligned variable + some memory fences should be enough (or just nothing), but hey, this is some random offline tool code.

What is horribly wrong with it?

Mutex is held locked for the whole duration of Sleep! That is, almost all the time; and other thread(s) barely have a chance to ever update g_StartTime.

And this is the code I’ve written. Oh stupid me.

And herein lies the problem: most often precision of vertex transformations is not the same in fixed function versus programmable vertex pipelines. If you’d just draw some objects in multiple passes, mixing fixed function and programmable paths, this is roughly what you will get (excuse my programmer’s art):

Not pretty at all! This should have looked like this:

So what do we do to make it look like this? We “pull” (bias) some rendering passes slighly towards the camera, so there is no depth fighting.

Now, at the moment Unity editor runs only on the Macs, which use OpenGL. In there, most of hardware configurations do not need this depth bias at all – they are able to generate same results in fixed function and programmable pipelines. Only Intel cards do need the depth bias on Mac OS X (on Windows, AMD and Intel cards need depth bias). So people author their games using OpenGL, where it does not need depth bias in most cases.

How do you apply depth bias in OpenGL? Enable GL_POLYGON_OFFSET_FILL and set glPolygonOffset to something like -1, -1. This works.

How do you apply depth bias in Direct3D 9? Conceptually, you do the same. There are DEPTHBIAS and SLOPESCALEDEPTHBIAS render states that do just that. And so we did use them.

And people complained about funky results on Windows.

And I’d look at their projects, see that they are using something like 0.01 for camera’s near plane and 1000.0 for the far plane, and tell them something along the lines of “increase your near plane, stupid!” (well ok, without the “stupid” part). And I’d explain all the above about mixing fixed function and vertex shaders, and how we do depth bias in that case, and how on OpenGL it’s often not needed but on Direct3D it’s pretty much always needed. And yes, how sometimes that can produce “double lighting” artifacts on close or intersecting geometry, and how the only solution is to increase the near plane and/or avoid close or intersecting geometry.

Sometimes this helped! I was so convinced that their too-low-near-plane was always the culprit.

And then one day I decided to check. This is what I’ve got on Direct3D:

Ok, this scene is intentionally using a low near plane, but let me stress this again. This is what I’ve got:

Not good at all.

What happened? It happened in roughly this way:

1. First, depth bias documentation on Direct3D is wrong. Depth bias is not in 0..16 range, it is in 0..1 range which corresponds to entire range of depth buffer.
2. Back then, our code was always using 16 bit depth buffers, so the equivalent of -1,-1 depth bias in OpenGL was multiplied with something like 1.0/65535.0, and that was fed into Direct3D. Hey, it seemed to work!
3. Later on, the device setup code was modified to do proper format selection, so most often it ended up using 24 bit depth buffer. Of course no one I never modified the depth bias code to account for this change…
4. And it stayed there. And I kept deceiving myself that the content of the users is to blame, and not some stupid code of mine.

It’s good to check your assumptions once in a while.

So yeah, the proper multiplier for depth bias on Direct3D with 24 bit depth buffer should be not 1.0/65535.0, but something like 1.0/(2^24-1). Except that this value is really small, so something like 4.8e-7 should be used instead (see Lengyel’s GDC2007 talk). Oh, but for some reason it’s not really enough in practice, so something like 2.0*4.8e-7 should be used instead (tested so far on GeForce 8600, Radeon HD 3850, Radeon 9600, Intel 945, reference rasterizer). Oh, and the same value should be used even when a 16 bit depth buffer is used; using 1.0/65535.0 multiplier with 16 bit depth buffer produces way too large bias.

With proper bias values the image is good on Direct3D again. Yay for that (fix is coming in Unity 2.1 soon).

…and yes, I know that real men fudge projection matrix instead of using depth bias… someday maybe.

CAsyncMonikerFile is derived from CMonikerFile, which in turn is derived from COleStreamFile. A COleStreamFile object represents a stream of data; a CMonikerFile object uses an IMoniker to obtain the data, and a CAsyncMonikerFile object does so asynchronously.

So yeah, I am dealing with downloading something from the internet inside an ActiveX control that is written in MFC. A seemingly simple task – I give you an URL, you give me back the bytes. But no! That would not be a proper architecture, so instead it has asynchronous monikers which are based on monikers which are based on stream files which use some interfaces and whatnot. And for ActiveX controls the docs suggest using CDataPathProperty or CCachedDataPathProperty, which are abstractions build on top of the above crap. And I don’t even know what “a moniker” is!

Of course all this complexity fails spectacularly in some quite common situations. For example, try downloading something when the web server serves gzip compressed html output. Good luck trying to figure out why everything seemingly works, you are notified of downloading progress, but never get the actual downloaded bytes.

Turns out the solution is to change downloading behaviour of the above pile of abstractions to use “pull data” model, instead of default “push data” model. The default behaviour just seems to be broken (though it is not broken in that pile of abstractions, instead it is broken somewhere deeper in Windows code). Is this mentioned anywhere in the docs? Of course not!

This is pretty much how a code comment looks like for this:

We don’t use CCachedDataPathProperty because it’s awfully slow, doing data reallocations for each 1KB received. For 8MB file it’s 8000 reallocations and 32 GB (!) of data copied for no good reason!

While we’re at it, we don’t use CDataPathProperty either, because it’s a useless wrapper over CAsyncMonikerFile.

Oh, and we don’t use CAsyncMonikerFile either, because it has bugs in VS2003′ MFC where it never notifies the container that it is done with download, making IE still display “X items remaining” indefinitely. Some smart coder was converting information message and returning “out of memory” error if result was NULL, even if input message was NULL (which it often was). So we use our own “fixed” version of CAsyncMonikerFile instead.

Oh MFC, how we love thee.

CAN I HAS MOIRE SHADOWS?

Some N years ago I would have liked a title with “Architect” or “Analyst” or something like that. I would have called myself “developer” instead of “programmer” because hey, a developer thinks up things, whereas a programmer is a mere “code monkey”. More on code monkeys below.

But wait! Back then I also believed that knowing and using Design Patterns is essential for a programmer! In one place when I was interviewing new hires, design pattern knowledge was something I would look for… how stupid! Nowadays my view of patterns is more along the lines of “yeah, whatever”. I don’t exactly think of them as things from hell, but they could have caused more harm than good already.

Back to job titles. Code monkey is actually the key employee. A software product is largely defined by the code, heck, it is code. Sure, it also has the user interface, the fancy icons, the documentation, the website, the support, the roadmap and whatnot, but the code is the product, whereas everything else is more or less addons (possibly excluding UI… UI also defines the product).

Code design? Design patterns? Who cares about that.

It’s the final result that matters. Futurist programming for the win.

On the other hand, Memento Observer is probably very cool.

What kind of filtering does Resize() function use? Nearest-neighbor, bilinear, bicubic, biawesome?

Since then “biawesome” became a local meme at work. Biawesome is awesome on steroids.

Fix FBX animation import crash once more. When exported symbols are not listed for a dylib, it seems to link back to calling executable (?!), making them share function impls with the same name. And because Keyframe is actually different in editor vs ImportFBX, this is wrong. Apparently this is OS X Leopard only, or something. Argh.

The code change in question was just telling the compiler “here’s the list of the functions that are exported from this dynamic library”. The list was already there, just the compiler was never told about existence of it.

The bug manifested itself as a crash when importing animations. But it would not happen when importer was run from a small unit test application. There were no memory corruptions happening, it was not running out of memory, yet the code was crashing with access violation, usually because STL’s vector was returning it’s wrong size (but the actual data of the vector was correct; it was just returning bogus size). And it was doing that only on OS X Leopard, and not on OS X Tiger. Huh?

Turns out what did happen – and I’m not sure if that’s a bug in OS X or a feature – is that the calling application did contain a class called Keyframe. And the shared library (where the crash was happening) also contained a class called Keyframe. But those classes were slightly different; first was 20 bytes in size, and second one was 16 bytes.

Now, somehow when the shared library was calling vector<Keyframe>::size(), the function from the calling application was used. I have no idea at all how or why this was happening, but it sure was! I could see from tracing the assembly code, that it was doing difference of two pointers, and then doing something that for sure was not division by 16.

What was the code doing? Turns out it was calculating division by 20 in a cunning way:

mov  edx,esi   # edx = end()
sub  edx,eax   # edx -= begin()
mov  eax,edx   # eax = edx
sar  eax,0x2   # eax >>= 2
imul eax,eax,0xcccccccd # eax *= 0xcccccccd

In other words, the compiler was replacing division by constant (as used in vector’s size()) by a shift and multiplication with a magic number. You can read more about the technique here or here.

But of course the code above only works if the number was actually divisible by 20; otherwise it returns totally wrong result. This is perfectly fine for computing the difference in two pointers to structures of known size… Except that inside the shared library the Keyframe structures are 16 bytes, and not 20!

So yeah. Watch out for peculiarities of dynamic linking on your platform.

One of our customers found an interesting bug the other day: embedding Unity Web Player into a web page makes some javascript animation libraries not work correctly. For example, script.aculo.us or Dojo Toolkit would stop doing some of their tasks. But only on Windows, and only on some browsers (Firefox and Safari).

Wait a moment… Unity plugin makes nice wobbling web page elements not wobble anymore!? Sounds like an interesting issue…

So I prepared for a debug session and tried the usual “divide by two until you locate the problem” approach.

• Unity Web Player is composed of two parts: a small browser plugin, and the actual “engine” (let’s call it “runtime”). First I change the plugin so that it only loads the data, but never loads or starts the runtime. Everything works. So the problem is not in the plugin. Good.
• Load the runtime and do basic initialization (create child window, load Mono, …), but never actually start playing the content – everything works.
• Load the runtime and fully initialize everything, but never actually start playing the content – the bug appears! By now I know that the problem is somewhere in the initialization.

Initialization reads some settings from the data file, creates some “manager objects” for the runtime, initializes graphics device, loads first game “level” and then the game can play.

What of the above could cause something inside browser’s JavaScript engine stop working? And do that only on Windows, and only on some browsers? My first guess was the most platform-specific part: intialization of the graphics device, which on Windows usually happens to be Direct3D.

So I continued:

• Try using OpenGL instead of Direct3D – everything works. By now it’s confirmed that initializing Direct3D causes something else in the browser not work.
• “A-ha!” moment: tell Direct3D to not change floating point precision (via a create flag). Voilà, everything works!

I don’t know how I actually came up with the idea of testing floating point precision flag. Maybe I remembered some related problems we had a while ago, where Direct3D would cause timing calculations be “off”, if the user’s machine was not rebooted for a couple of weeks or more. That time around we properly changed our timing code to use 64 bit integers, but left Direct3D precision setting intact.

Side note: Intel x86 floating point unit (FPU) can operate in various precision modes, usually 32, 64 or 80 bit. By default Direct3D 9 sets FPU precision to 32 bit (i.e. single precision). Telling D3D to not change FPU settings could lower performance somewhat, but in my tests it did not have any noticeable impact.

So there it was. A debugging session, one line of change in the code, and fancy javascript webpage animations work on Windows in Firefox and Safari. This is coming out in Unity 2.0.2 update soon.

The moral? Something in one place can affect seemingly completely unrelated things in another place!

It’s been an amazing ride so far. I’m working with amazing people, we have (mostly :)) amazing customers, we’ve done three major releases and five minor ones, I’ve seen sales grow from ridiculously low amount to feels good levels, I’ve seen the team expansion, and I took svn revision number 10000 for myself (that happened half a year ago):

> Wohoo! Congratulations on the 10,000th commit Aras! You have won a warm cup of freshly brewed coffee when you come to work! :)

You don’t know the whole exciting story behind this… :)

I had two tiny “cleanups” to two files – removing unused variable from each. Now, I’d so something on my Mac, start compiling, and while it’s compiling I’d check svn log on the windows machine.

As soon as Valdemar checked in revision number 9998, I knew what I had to do. Two trivial svn commits!

Some serious work, as you can see. Oh, some of my work also ended up being actually released, I’m not just sitting there claiming svn revisions for myself. Honestly!

I’ve moving back to Lithuania now, continuing to work on Unity from home. Gonna miss some of the office fun, but oh well. Tradeoffs have to be made.

Let’s see what the coming years will bring. Rock on.

What if the eyes were orthographic?

Think about it…. that would be totally weird!

This is actually me debugging point light shadows (that happen to use depth encoded into RGBA8 cubemaps).

This is what happens when you use a too wide Poisson disc blurring in screen space and no prevention of “shadow leakage” over different depths.

LOL! Internet!