Recently I’ve seen a mention that game engine programming is considered a mysterious, elite, and highly demanding type of work. So let me write up what often actually happens in day to day work. Also about how in many types of tasks, doing the UI is often the hardest one :)

Request: separate texture UV wrapping modes

I saw “could we have separate wrap modes for texture U and V axes?” being requested on the internets. This is not new; we had actually discussed this same thing internally just a few weeks before.

Up until now, in Unity you could only specify one texture coordinate wrapping mode that would apply to both (or in volume texture case, all three) axes.

All the graphics APIs and and GPUs I’ve seen do support separate UV(W) wrap modes, and while it’s not a common use case, there are valid cases where that is useful to have. For example, when using Lat-Long environment maps for reflection probes, it is useful to have Clamp on vertical texture coordinate, but Repeat on the horizontal coordinate (why use lat-long environment maps? because some platforms don’t support cubemap arrays, and yeah I’m looking at you mobile platforms).

So I thought I’d do it as part of Mondays are for Mandatory Fun(tm) thing we have:

How hard could this possibly be?

The change itself is trivial. Instead of having one wrap mode in a sampler descriptor" we need to have three, and set them up to the graphics API acoordingly. Actual platform specific change looks something like this (Metal here, but very similar for any other API). Somewhere where the sampler state is created or set up:

“Oh, but Unity supports, I don’t know, three million graphics APIs? That’s gonna be hard to do?” – turns out, not really. At the time of writing, I had to add code to 11 “platform API abstraction” implementations. Eleven is more than one or two, but doing all that was trivial enough. Even without having compilers/SDKs for at least half of them :)

Real amount of work starts to appear once you try to write down what are “all the things” that need to be done. The “graphics API” bit is just one entry there!

Before doing a task like this, I look for whether that particular area is in a need of some small cleanup or refactor. In this case it was; we were passing all sampler state as separate arguments into platform abstraction functions, instead of something like a “sampler descriptor struct”. It was already cumbersome, and adding separate wrapping modes would not make it better. So first item on the list becomes, “refactor that”.

And then when doing actual changes, I’d keep on noticing “well this should be cleaned up” type of code too, and write that down to the end of the list. None of that is critical for the task at hand, but codebase cleanup does not happen by itself otherwise.

Most of the items on the list are easy enough though. Except… yeah, UI.

User Interface

Ok, so how do you show UI in texture settings for separate wrapping modes?

I looked at what others do, and for example UE4 just shows two dropdowns. This is trivial to implement, but did not feel “quite right”. Afterall, the expected 95% use case is that you’d want to use the same wrapping mode on all axes. Doing this would get you the feature/flexibility (yay!), but a fairly rarely used one that costs an extra row of setting controls, no matter whether you need it or not.

It should be possible to do better.

Try 1: extend wrap mode popup with more options

Today we only support Repeat and Clamp wrapping modes, and absolute majority of textures are non-volume textures. Which means extending to separate UV wrapping modes only needs to add two more entries into a single popup:

That is not too bad. For volume textures, there are three axes to worry about, so the popup becomes a choice of 8 possible options. This is more confusing, but maybe we can sweep it under a “hey this is a super rare case” rug.

A slightly bigger worry is that people are also asking for other coordinate wrapping modes that we have not exposed before (“border” and “mirror”). If/when we add them, the single popup would not be a good solution. The number of entries in in would become too large to be useful.

Try 2: per-axis popups, linked by default

You know that “these fields are linked” widget from image editors?

I thought maybe let’s do that; show one popup per-axis, but by default have them linked together. Here’s how it looks like (using “lock” icon to mean “linked”, because no one painted a proper icon yet):

And then it can be unlinked to select different wrapping modes. For volume textures, it would display three popups, but otherwise function the same.

This almost works fine. The downsides are:

• Still additional visual noise in the settings even if you don’t use the feature, and
• In the image editors, “linked” is mostly used for numeric input fields; linking dropdown controls together is not a familiar UI pattern.

Try 3: one popup with “Per-Axis” choice

Here’s another idea: keep one popup by default, but instead of it having just [Repeat, Clamp] options, make them [Repeat, Clamp, Per-axis]. When per-axis is selected, that rolls out two more popups underneath (or three more, for volume textures):

This one actually feels nice. Yay! And only took three attempts to get right.

Oh, and then some more fumbling around to nicely handle various cases of multiple textures being selected, them all possibly having different settings set.

Doing all that UI related work took me about twice as long as doing everything else combined (and that includes “changes to eleven graphics APIs”). Now of course, I’m not a UI programmer, but still. UI is hard.

That’s it!

So yeah. A super small feature, that ended up probably two full days of work. Majority of that: trying to decide how exactly to present two popup controls. Who would have thunk, eh.

Otherwise, pretty much trivial steps to get there. However this does end up with about a hundred files being changed.

…and that is how “mysterious engine programming” looks like :) Now of course there is plenty of really challenging, “cutting edge knowledge required” type of work, where juggling chainsaws would probably look easy in comparison. But, plenty of “nothing special, just work” type of items too.

Separate texture UV wrapping modes might be coming to a nearby Unity version soon-ish. Thanks to Alexey, Lukas, Shawn, Vlad for UI discussions & suggestions.

Did you know that Google Chrome has a built-in profiler? Some people assume it’s only for profiling “web stuff” like JavaScript execution. But it can also be used as a really nice frontend for your own profiling data.

Colt McAnlis actually wrote about using it for game profiling, in 2012: Using Chrome://tracing to view your inline profiling data. Everything written there still stands, and I’m mostly going to repeat the same things. Just to make this Chrome Tracing thing more widely known :)

Backstory

For a part of our code build system, we have a super simple profiler (called TinyProfiler) that captures scoped sections of the code and measures time spent in them. It was producing a simple HTML+SVG “report file” with a flame graph style visualization:

This gets the job done, but the resulting HTML is not very interactive. You can hover over things and have their time show up in the text box. But no real zooming, panning, search filtering or other niceties that one could expect from a decent profiler frontend UI.

All these things could be implemented… but also, why do that when someone else (Chrome) already wrote a really nice profiler UI?

Using Chrome Tracing

All that is needed to do to use Chrome Tracing view is:

1. Produce a JSON file with the format expected by Chrome,
2. Go to chrome://tracing in Chrome,
3. Click “Load” and open your file, or alternatively drag the file into Chrome,
4. Profit!

The final result looks pretty similar to our custom HTML+SVG thing, as Chrome is also visualizing the profiling data in the flame graph style:

Advantages of doing this:

1. Much better profiling UI: zooming, panning, filtering, statistics, and so on.
2. We no longer need to write profiler UI frontend (no matter how simple) ourselves.
3. 500 lines of code less than what we had before (writing out the JSON file is much simpler than the SVG file).

And some potential disadvantages:

1. No easy way to “double click report file to open it” as far as I can see. Chrome does not have any command line or automated interface to open itself, go to tracing view, and load a file all in one go. So you have to manually do that, which is more clicks.
   • This could be improved by producing HTML file with everything needed, by using trace2html tool from the Chrome Tracing repository. However, that whole repository is ~1GB in size (it has way more stuff in it than just tracing UI), and I’m not too keen on adding a gigabyte of external dependency just for this. Maybe it would be possible to produce a “slimmed down, just the trace2html bits” version of that repository.
2. Added dependency on 3rd party profiling frontend. However:
   • It seems to be fairly stable (everything from Colin’s article in 2012 still works in 2017),
   • The frontend is open source: catapult-project at github,
   • Other people, e.g. Facebook’s Buck build system are also using it.

My take is that the advantages outweigh the disadvantages.

JSON file format

Trace Event Format is really nicely documented, so see there for details on more advanced usage. Basic structure of the JSON file is as follows:

{
"traceEvents": [
{ "pid":1, "tid":1, "ts":87705, "dur":956189, "ph":"X", "name":"Jambase", "args":{ "ms":956.2 } },
{ "pid":1, "tid":1, "ts":128154, "dur":75867, "ph":"X", "name":"SyncTargets", "args":{ "ms":75.9 } },
{ "pid":1, "tid":1, "ts":546867, "dur":121564, "ph":"X", "name":"DoThings", "args":{ "ms":121.6 } }
],
"meta_user": "aras",
"meta_cpu_count": "8"
}

traceEvents are the events that show up in the UI. Anything that is not recognized by the event format is treated as “metadata” that is shown by the UI metadata dialog (in this case, meta_user and meta_cpu_count are metadata). The JSON above looks like this in the Chrome Tracing UI:

Events described in it are the simplest ones, called “Complete Events” (indicated by ph:X). They need a timestamp and duration given in microseconds (ts and dur). The other fields are process ID and thread ID (pid and tid), and a name to display. There’s no need to indicate parent-child relationships between the events; the UI automatically figures that out based on event timings.

Events can also have custom data attached to them (args), which is displayed in the lower pane when an event is selected. One gotcha is that there has to be some custom data in order for the event to be selectable at all. So at least put some dummy data in there.

And basically that’s it for a super simple usage. Check out Trace Event Format for more advanced event types and usage. Happy profiling!

I don’t have any technical insights to share right now, so you’ll have to bear with me blabbering about random stuff!

Unity Work

By now it’s been 11 years at Unity (see 10 Years at Unity a year ago), and the last year has been interesting. Started out as written in that blog post – that I will be mostly doing plumbing & fixing and stuff.

But then at some point we had a mini-hackweek of “what should the future rendering pipeline of Unity should be?”, and it went quite well. So a team with tasks of “build building blocks for future rendering pipelines” and “modernize low level graphics code” was built, see more on Scriptable Render Loops google doc and corresponding github project. I’ve been doing mostly, well, plumbing there, so nothing new on that front :) – but overall this is very exciting, and if done well could be a big improvement in how Unity’s graphics engine works & what people can do with it.

I’ve also been a “lead” of this low level graphics team (“graphics foundation team” as we call it), and (again) realized that I’m the worst lead the world has ever seen. To my fellow colleagues: sorry about that (シ. .)シ So I stopped being a lead, and that is now in the capable hands of @stramit.

Each year I write a “summary of stuff I’ve done”, mostly for myself to see whether my expectations/plans matched reality. This year a big part in the mismatch has been because at start of year I did not know we’d go big into “let’s do scriptable render loops!”, otherwise it went as expected.

Day to day it feels a bit like “yeah, another day of plumbing”, but the summary does seem to indicate that I managed to get a decent amount of feature/improvement work done too. Nothing that would set the world on fire, but not bad either! Though thinking about it, after 2016, maybe the world could do with less things that set it on fire…

My current plan for next year is to continue working on scriptable render loops and whatever low level things they end up needing. But also perhaps try some new area for a bit, something where I’m a newbie and would have to learn some new things. We’ll see!

Oh, and extrapolating from the trend, I should effectively remove close to half a million lines of code this year. My fellow colleagues: brace yourselves :)

Open Source

My github contributions are not terribly impressive, but I managed to do better than in 2015.

• Primarily SMOL-V, a SPIR-V shader compressor for Vulkan. It’s used in Unity now, and as far as I know, also used in two non-Unity game productions. If you use it (or tried it, but it did not work or whatever), I’d be happy to know!
• Small amount of things in GLSL Optimizer and HLSL2GLSL, but with Unity itself mostly moving to a different shader compiler toolchain (HLSLcc at the moment), work there has been winding down. I did not have time or will to go through the PRs & issues there, sorry about that.
• A bunch of random tiny fixes or tweaks submitted to other github projects, nothing of note really.

Writing

I keep on hearing that I should write more, but most of the time don’t feel like I have anything interesting to say. So I keep on posting random lame jokes and stuff on twitter instead.

Nevertheless, the amount of blog posts has been very slowly rising. The drop starting around year 2009 very much coincides with me getting onto twitter. Will try to write more in 2017, but we’ll see how that goes. If you have ideas on what I should write about, let me know!

The amount of readers on the website is basically constant through all the time that I had analytics on it (starting 2013 or so). Which could mean it’s only web crawling bots that read it, for all I know :) There’s an occasional spike, pretty much only because someone posted link on either reddit or hackernews, and the hive mind there decided to pay attention to it for five minutes or whatever.

Giving Back

Financially, I live a very comfortable life now, and that is by no means just an “I made it all by myself” thing. Starting conditions, circumstances, family support, a ton of luck all helped massively.

I try to pay some of that back by sharing knowledge (writing the blog, speaking, answering questions on twitter and ask.fm). This is something, but I could do more!

So this year started doing more of a “throw money towards good causes” thing too, compared to what I did before. Helped some local schools, and several local charities / help funds / patreons / “this is a good project” type of efforts. I’m super lucky that I can afford to do that, looking forward to doing more of it in 2017.

Started work towards installing solar panels on the roof, which should generate about as much electricity as we use up. That’s not exactly under “giving back” section, but feels good to have finally decided to do it. Will see how that goes in our (not that sunny) land.

Other Things

I’m not a twenty-something anymore, and turns out I have to do some sort of extra physical activity besides working with a computer all day long. But OMG it’s so boring, so it takes a massive effort to force myself to do it. I actually stopped going to the gym for half a year, but then my back pains said “ohai! long time no see!”. Ugggh. So towards end of 2016 started doing the gym thing again. I can only force myself to go there twice a week, but that’s enough to keep me from collapsing physically :) So far so good, but still boring.

Rocksmith keeps on being the only “hobby” that I have, and still having loads of fun with it. I probably have played about 200 hours this year.

It feels like I am still improving my guitar playing skills, but I’m not doing enough effort to actually improve. Most of the time I just play through songs in a messy way, without taking time to go through difficult riffs or solos and really nail the things down. So my playing is like: eh, ¯\_(ツ)_/¯. Maybe in 2017 I should try playing with some other people (something I have never done, besides a few very lame attempts in high school), or try taking “real” guitar lessons.

INSIDE and Firewatch are my games of the year. They are both made with Unity too, which does not feel bad at all.

Did family vacations, lazy one in Maldives and a road trip in Spain. Both were nice in their own way, but we all (again) realized that lazy-type vacations are not our style, even if they are nice to have once in a while.

In 2017 I want to spend less time on social media, which lately is mostly a source of really depressing news, and just do more useful or helpful things instead. Wish me luck.

I wrote some tests to verify sorting/batching behavior in rendering code, and they were producing different results on Windows (MSVC) vs Mac (clang). The tests were creating a “random fake scene” with a random number generator, and at first it sounded like our “get random normalized float” function was returning slightly different results between platforms (which would be super weird, as in how come no one noticed this before?!).

So I started digging into random number generator, and the unit tests it has. This is what amazed me.

Here’s one of the unit tests (we use a custom native test framework that started years ago on an old version of UnitTest++):

TEST (Random01_WithSeed_RestoredStateGenerateSameNumbers)
{
	Rand r(1234);
	Random01(r);
	RandState oldState = r.GetState();
	float prev = Random01(r);
	r.SetState(oldState);
	float curr = Random01(r);
	CHECK_EQUAL (curr, prev);
}

Makes sense, right?

Here’s what MSVC 2010 compiles this down into:

push        rbx  
sub         rsp,50h  
mov         qword ptr [rsp+20h],0FFFFFFFFFFFFFFFEh  
	Rand r(1234);
	Random01(r);
	RandState oldState = r.GetState();
	float prev = Random01(r);
movss       xmm0,dword ptr [__real@3f47ac02 (01436078A8h)]  
movss       dword ptr [prev],xmm0  
	r.SetState(oldState);
	float curr = Random01(r);
mov         eax,0BC5448DBh  
shl         eax,0Bh  
xor         eax,0BC5448DBh  
mov         ecx,0CE6F4D86h  
shr         ecx,0Bh  
xor         ecx,eax  
shr         ecx,8  
xor         eax,ecx  
xor         eax,0CE6F4D86h  
and         eax,7FFFFFh  
pxor        xmm0,xmm0  
cvtsi2ss    xmm0,rax  
mulss       xmm0,dword ptr [__real@34000001 (01434CA89Ch)]  
movss       dword ptr [curr],xmm0  
	CHECK_EQUAL (curr, prev);
call        UnitTest::CurrentTest::Details (01420722A0h)
; ...

There’s some bit operations going on (the RNG is Xorshift 128), looks fine on the first glance.

But wait a minute; this seems like it only has code to generate a random number once, whereas the test is supposed to call Random01 three times?!

Turns out the compiler is smart enough to see through some of the calls, folds down all these computations and goes, “yep, so the 2nd call to Random01 will produce 0.779968381 (0x3f47ac02)”. And then it kinda partially does actual computation of the 3rd Random01 call, and eventually checks that the result is the same.

Oh-kay!

Now, what does clang (whatever version Xcode 8.1 on Mac has) do on this same test?

pushq  %rbp
movq   %rsp, %rbp
pushq  %rbx
subq   $0x58, %rsp
movl   $0x3f47ac02, -0xc(%rbp)   ; imm = 0x3F47AC02 
movl   $0x3f47ac02, -0x10(%rbp)  ; imm = 0x3F47AC02 
callq  0x1002b2950               ; UnitTest::CurrentTest::Results at CurrentTest.cpp:7
; ...

Whoa. There’s no code left at all! Everything just became an “is 0x3F47AC02 == 0x3F47AC02” test. It became a compile-time test.

w(☆o◎)w

By the way, the original problem I was looking into? Turns out RNG is fine (phew!). What got me was code in my own test that I should have known better about; it was roughly like this:

transform.SetPosition(Vector3f(Random01(), Random01(), Random01()));

See what’s wrong?

.

.

.

The function argument evaluation order in C/C++ is unspecified.

(╯°□°)╯︵ ┻━┻

Newer languages like C# or Java have guarantees that arguments are evaluated from left to right. Yay sanity.

Recently saw quite some twitter discussions about good & bad interview questions. Here’s a few I found useful.

In general, the most useful questions seem to be fairly open-ended ones, that can either lead to a much larger branching discussions, or don’t even have the “right” answer. Afterall, you mostly don’t want to know the answer (it’s gonna be 42 anyway), but to see the problem solving process and/or evaluate general knowledge and grasp of the applicant.

The examples below are what I have used some times, and are targeted at graphics programmers with some amount of experience.

When a GPU samples a texture, how does it pick which mipmap level to read from?

Ok this one does have the correct answer, but it still can lead to a lot of discussion. The correct answer today is along the lines of:

GPU rasterizes everything in 2x2 fragment blocks, computes horizontal and vertical texture coordinate differences between fragments in the block, and uses magnitudes of the UV differences to pick the mipmap level that most closely matches 1:1 ratio of fragments to texels.

If the applicant does not know the answer, you could try to derive it. “Well ok, if you were building a GPU or writing a software rasterizer, what would you do?”.

Most people initially go for “based on distance to the camera”, or “based on how big the triangle is on screen” and so on. This is a great first step (and was roughly how rasterizers in the really old days worked). You can up the challenge then by throwing in a “but what if UVs are not just computed per-vertex?”. Which of course makes these suggested approaches not suitable anymore, and something else has to be thought up.

Once you get to the 2x2 fragment block solution, more things can be discussed (or just jump here if the applicant already knew the answer). What implications does it have for the programming models, efficiency etc.? Possible things to discuss:

• The 2x2 quad has to execute shader instructions in lockstep, so that UV differences can be computed. Which leads to branching implications, which leads to regular texture samples being disallowed (HLSL) or undefined (GLSL) when used inside dynamic branching code paths.
• Lockstep execution can lead into discussion how the GPUs work in general; what are the wavefronts / warps / SIMDs (or whatever your flavor of API/GPU calls them). How branching works, how temporary variable storage works, how to optimize occuppancy, how latency hiding works etc. Could spend a lot of time discussing this.
• 2x2 quad rasterization means inefficiencies at small triangle sizes (a triangle that covers 1 fragment will still get four fragment shader executions). What implications this has for high geometry density, tessellation, geometric LOD schemes. What implications this has for forward vs deferred shading. What research is done to solve this problem, is the applicant aware of it? What would they do to solve or help with this?

You are designing a lighting system for a game/engine. What would it be?

This one does not even have the “correct” answer. A lighting system could be anything, there’s at least a few dozen commonly used ways to do it, and probably millions of more specialized ways! Lighting encompasses a lot of things – punctual, area, environment light sources, emissive surfaces; realtime and baked illumination components; direct and global illumination; shadows, reflections, volumetrics; tradeoffs between runtime peformance and authoring performance, platform implications, etc. etc. It can be a really long discussion.

Here, you’re interested in several things:

• General thought process and how do they approach open-ended problems. Do they clarify requirements and try to narrow things down? Do they tell what they do know, what they do not know, and what needs further investigation? Do they just present a single favorite technique of theirs and can’t tell any downsides of it?
• Awareness of already existing solutions. Do they know what is out there, and aware of pros & cons of common techniques? Have they tried any of it themselves? How up-to-date is their knowledge?
• Exploring the problem space and making decisions. Is the lighting system for a single very specific game, or does it have to be general and “suitable for anything”? How does that impact the possible choices, and what are consequences of these choices? Likewise, how does choice of hardware platforms, minimum specs, expected complexity of content and all other factors affect the choices?
• Dealing with tradeoffs. Almost every decisions engineers do involve tradeoffs of some kind - by picking one way of doing something versus some other way, you are making a tradeoff. It could be performance, usability, flexibility, platform reach, implementation complexity, workflow impact, amount of learning/teaching that has to be done, and so on. Do they understand the tradeoffs? Are they aware of pros & cons of various techniques, or can they figure them out?

You are implementing a graphics API abstraction for an engine. How would it look like?

Similar to the lighting question above, there’s no single correct answer.

This one tests awareness of current problem space (console graphics APIs, “modern” APIs like DX12/Vulkan/Metal, older APIs like DX11/OpenGL/DX9). What do they like and dislike in the existing graphics APIs (red flag if they “like everything” – ideal APIs do not exist). What would they change, if they could?

And again, tradeoffs. Would they go for power/performance or ease of use? Can you have both (if “yes” - why and how? if “no” - why?). Do they narrow down the requirements of who the abstraction users would be? Would their abstraction work efficiently on underlying graphics APIs that do not closely map to it?

You need to store and render a large city. How would you do it?

This one I haven’t used, but saw someone mention on twitter. Sounds like an excellent question to me, again because it’s very open ended and touches a lot of things.

Authoring, procedural authoring, baking, runtime modification, storage, streaming, spatial data structures, levels of detail, occlusion, rendering, lighting, and so on. Lots and lots of discussion to be had.

Well this is all.

That said, it’s been ages since I took a job interview myself… So I don’t know if these questions are as useful for the applicants as they seem for me :)