In Unity we use Mono to do game code (well, Unity users are doing that, not us). Overall it’s great; it has tons of advantages, loads of awesome and a flying ninja here and there. But no technology is perfect, right?

Edit: Miguel rightly points out in the comments that Mono team is solving or has already solved some of these issues already. In some areas they are moving so fast that we at Unity can’t keep up!

#1: Garbage Collector

Most game developers do not like Garbage Collection (GC) very much. Typically, the more limited/hardcore their target platform is, the more they dislike GC. The reason? Most GC implementations cause rather unpredictable spikes.

Here’s a run of something recorded in the (awesome) Unity 2.6 profiler. Horizontal axis is time, vertical is CPU time spent in that frame:

At the bottom you see dark red thingies appearing once in a while. This is garbage collector kicking in, because some script code is allocating some memory at runtime.

Now of course, it is possible to write your script code so that it does no allocations (or almost no allocations). Preallocate your objects into pools, manually invoke GC when there’s a game situation when a small hickup won’t affect gameplay, etc. In fact, a lot of iPhone games made with Unity do that.

But that kind of side steps the whole advantage of “garbage collector almost frees you from doing memory management”. If you’re not allocating anything anyway, GC could just as well not be there!

A little side story. Me and Unity’s iPhone tech lead ReJ tried to explain what GC is to a non-programmer. Here’s what we came up with:

Garbage Collection is this cleaning service for lazy people. They can just leave any garbage on the floor in their house, and once in a while a garbage guy comes, collects all the garbage and takes it outside. Now, there are some intricacies in the service.

First, you never know when the garbage guy will come. You might be taking a shower, doing a meditation or having some “sexy time” – and it’s in the service agreement that when a garbage guy comes, you have to let him in to do his work.

Second thing is, the garbage guy is usually some homeless drunkard. He smells so bad that when he comes, you have to stop whatever you were doing, go outside and wait until he’s done with the garbage collection. Even your neighbors, who might be doing something entirely else in parallel, actually have to stop and idle while garbage is being collected in your house!

There are variations of this GC service. One variation is called “moving GC”, where the garbage guy also rearranges your furniture while collecting the garbage – he moves it all into one side of your house. This is so that you can buy a bigger piece of furniture, or throw a huge piece of garbage – and there will be enough unused space for you to do that! Of course this way GC process takes somewhat longer, but hey, you get all your stuff nicely packed into one corner.

Can’t you see that this service is the greatest idea of all time?

This is quite a harsh attitude towards GC, and of course it’s exaggerated. But there is some truth to it. So how could GC be fixed?

GC fix #1: more control

More explicit control on when & how long GC runs. I want to say to the garbage guy, “come everyday at 4PM and do your work for 20 minutes”. In the game, I’d want to call GC with an upper time limit, say 1 millisecond for each call, and I would be calling that 30 times per second.

GC fix #2: sometimes I want to clean garbage myself

Inefficiencies and unpredictability of GC cause people to do even more work than a normal, oldskool memory allocation. Why not provide an option to deal with deallocations manually? I.e. a keyword reallynew could allocate an object that is not part of garbage collected world. It would function as a regular .NET object, just it would be user’s responsibility to reallydelete it.

Mono is already extending .NET (see SIMD and continuations). Maybe it makes sense to add some way to bypass garbage collector?

#2: Distribution Size

Using C#/.NET in a game requires having .NET runtime. None of the interesting platforms are guaranteed to have it, and even on Windows you can’t count on it being present. Mono is great here in a sense that it can be used on many more platforms than Microsoft’s own .NET. It’s also great on distribution size, but only if you compare it to Microsoft’s .NET.

In Unity Web Player, we package Mono DLL + mscorlib assembly into something like 1.5 megabytes (after LZMA compression). Which is great compared to 20+ megabytes of .NET runtime, but not that great it you compare it so, say, Lua runtime (which is less than 100 kilobytes).

On some platforms (iPhone, Xbox 360, PS3, …) it’s not possible to generate code at runtime, so Mono’s JIT does not work. All code that’s written in C# has to be precompiled to machine code ahead of time (AOT compilation). This is not a problem per se, but because .NET framework was never designed with small size and few dependencies in mind, doing anything will ultimately pull in a lot of code.

We joke that doing anything in C# will result in an XML parser being included somewhere. This is not that far from the truth; e.g. calling float.ToString() will pull in whole internationalization system, which probably somewhere needs to read some global XML configuration file to figure out whether daylight savings time is active when Eastern European Brazilian Chinese calendar is used.

Size fix #1: custom core .NET libraries?

For game uses, most of “fat” stuff in .NET runtime is not really needed. float.ToString() could just always use period as a decimal separator. Core libraries could consist just of essential collections (list, array, hash table) and maybe a String class, with just essential methods. Maybe it’s worth sacrificing some of the generality of .NET if that could shave off a couple of megabytes from your iPhone game size?

Of course this is very much doable; “all that is needed” ™ is writing custom mscorlib+friends, and telling C# compiler to not ever reference any of the “real” libraries.

Size fix #2: make Mono runtime smaller

Uncompressed Mono DLL in our Windows build is 1.5 megabytes. We have turned off all the easy stuff (profiler, debugger, logging, COM, AOT etc.). But probably some more could be stripped away. Do our games really need multiple AppDomains? Some fancy marshalling? I don’t know, it just feels that 1.5MB is a lot.

#3: Porting to New Platforms

You know this classic: “There’s no portable code. There’s only code that’s been ported.”

Most existing gaming platforms are quite weird. Most upcoming smartphone platforms also are quite weird, each in their own interesting way. Porting a large project like Mono is not easy, especially since parts of it (JIT or AOT engine) highly depend on the platform.

For Unity iPhone, unexpected discovery that it’s not possible to JIT on iPhone made the initial release be delayed by something like 4 months. It did not help that in early iPhone SDK builds JIT was actually possible, and Apple decided to disable runtime generated code later. Making Mono actually work there required significant work both from Mono team and from Unity. We still have one guy working almost exclusively on Mono+iPhone issues!

Of course, maybe all the Mono iPhone work made porting to new platforms easier as a byproduct. But so far we don’t have Mono ported to any other platform, up to production quality. So judging from experience, we now always assume Mono port will be a pain, just because “some nasty surprises will come up” (and they always do).

#4: Small Stuff

There is a ton of small bits where extending .NET would benefit gaming scenarios. For example:

Suppose there is some array on the native engine side; for example vertex positions in a mesh (3xFloat for each vertex). Is it possible to make that piece of memory be represented as a native struct array for .NET side? So that it would not involve any extra memory copies, but N vertices somewhere in memory would look just like Vector3[N] for C#?

On a similar note, having “strided arrays” would be useful. For example, mesh data is often interleaved, so for each vertex there is a position, normal, UVs and so on. It would be cool if in C# position array would still look like Vector3[N], but internally the distance between each element would be larger than 12 bytes required for Vector3.

Where do we go from here?

The above are just random ideas, and I’m not complaining about Mono. It is great! It’s just not perfect. Mono being open source is a very good thing, which means pretty much any interested party can improve it as needed. So rock on.

Ok, now that the controversial bits are done, let’s continue.

Most of this can be (and probably is) wrong, and I haven’t given it enough thought yet. But here’s my thinking about shaders of “regular scene objects”. All of below is about things that need to interact with lighting; I’m not talking about shaders for postprocessing, one-off uses, special effects, GPGPU or kitchen sinks.

Operating on vertex/pixel shader level is a wrong abstraction level

Instead, it should be separated out into “surface shader” (albedo, normal, specularity, …), “lighting model” (Lambertian, Blinn Phong, …) and “light shader” (attenuation, cookies, shadows).

• Probably 90% of the cases would only touch the surface shader (mostly mix textures/colors in various ways), and choose from some precooked lighting models.
• 9% of the cases would tweak the lighting model. Most of the things would settle for “standard” (Blinn-Phong or similar), with some stuff using skin or anisotropic or …
• The “light shader” only needs to be touched once in a blue moon by ninjas. Once the shadowing and attenuation systems are implemented, there’s almost no reason for shader authors to see all the dirty bits.

Yes, current hardware operates on vertex/geometry/pixel shaders, which is a logical thing to do for hardware. After all, these are the primitives it works on when rendering. But those primitives are not the things you work on when authoring how a surface should look or how it should react to a light.

Simple code; no redundant info; sensible defaults

In the ideal world, here’s a simple surface shader (the syntax is deliberately stupid):

Haz Texture;
Albedo = sample Texture;

Or with bump mapping added:

Haz Texture;
Haz NormalMap;
Albedo = sample Texture;
Normal = sample_normal NormalMap;

And this should be all the info you have to provide. This would choose the lighting model based on used things (in this case, Lambertian). It would somehow just work with all kinds of lights, shadows, ambient occlusion and whatnot.

Compare to how much has to be written to implement a simple surface in your current shader technology, so that it would work “with everything”.

From the above shader, proper hardware shaders can be generated for DX9, DX11, DX1337, OpenGL, next-gen and next-next-gen consoles, mobile platforms with capable hardware, etc.

It can be used in accumulative forward rendering, forward rendering with multiple lights per pass, hybrid (light pre-pass / prelight) rendering, deferred rendering etc. Heck, even for a raytracer if you have one at hand.

I want!

Now of course, it won’t be as nice as more complex materials have to be expressed. Some might not even be possible. But shader text complexity should grow with material complexity; and all information that is redundant, implied, inferred or useless should be eliminated. There’s no good reason to stick to conventions and limits of current hardware just because it operates like that.

Shaders must die!

In summary, Vista is not that bad.

Once you get used to changes in Explorer, different skin and so on – it’s actually usable. I think they have made some real improvements in the underlying technology, too bad they managed to “compensate” for all of that by inconsistencies and lack of polish in user interface.

At this point it’s minor quirks in UI that annoy me, but apart from that, Vista is okay. Look:

Who implemented blending of icon overlays and do they still have a job? No sir, that shield icon is not properly blended here!

Who thought it’s a good idea to make the Burn icon bright red? In 6 months, I never used it. Why is it the brightest thing in the whole Explorer window?

Try going one folder up without resorting to this drop down menu. Utilities is the current folder here. And no, there’s no keyboard shortcut for “go up” either (there was in XP, which was perfect).

And of course, the awesome shutdown menu. The two buttons – never used them. What I always use is “Shut Down” from the menu. And let’s not even talk about all the choices in the menu (no, more choices is not always better).

So yeah. It’s not stellar, it has tons of small annoyances (and some large ones – try developing web plugins with UAC on…), but it’s usable. I might have gotten used to it by now, actually.

Back in 2002, I wrote something that would be roughly translated like “C++ amazes me more and more”. In a positive sense! And I was talking about what is Boost.Spirit now.

A reply on local game development forums I wrote today (again, rough translation): “C++ is very hard and quite a horrible language, maybe you should not use it unless there are no alternatives”.

That’s quite a change in attitude we have here!

I feel like much of C++ horrors are a consequence of “it just somehow happened” (the whole template metaprogramming thing) or as a backwards compatibility with C requirement. Or maybe not, but I do agree with what ryg says here. Let’s play the internet memes:

void GUIView::MakeVistaDWMHappyDance()
{
    // Looks like Vista has some bug in DWM. Whenever we maximize or dock
    // a view, we must do something magic, otherwise
    // white stuff appears in place of the view.
    // See http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=4208117&SiteID=1

    bool earlierThanVista = systeminfo::GetOperatingSystemNumeric() < 600;
    if( earlierThanVista )
        return;

    // What seems to work is drawing one pixel via GDI.
    // We draw it at (1,1) with usual background color.
    int grayColor = 0.61f * 255.0f;
    PAINTSTRUCT ps;
    BeginPaint(m_View, &ps);
    SetPixel(ps.hdc, 1, 1, RGB(grayColor,grayColor,grayColor));
    EndPaint(m_View, &ps);
}

I know. Reading from screen when Aero is on is slow, bad and wrong. But then, what do you do? It’s better than users staring an all-white window just because Vista decided to draw it white, no matter what you think you’re drawing into it.

…still, MakeVistaDWMHappyDance is not nearly as cool as

internal interface ICanHazCustomMenu { … }

that Nicholas added a while ago.

• Fixed import progress bar for movies with no audio
• Fixed first context menu click not working on Windows
• Eye dropper backend on Windows
• Export Package actually works on Windows
• Compare Binary works on Windows
• Add checkbox to project wizard to always open it on startup
• F1 in bundled text editor goes to scripting docs for current word
• Fixed q/w/e/r keys in password fields and text areas toggling active Tool on Windows
• Fixed panes not repainting on Windows after some change is done via context menu on them
• …and so on.

Boring tiny little details.

This probably best summarizes where lion’s share of time goes when developing anything. I’m not working on some cool spherical harmonics lightmap compression. Or on cunning ways to encode shadow map information for better filtering. Or on using CUDA to compute something interesting.

In other words, I’m not working on cool technology. Instead I’m adding missing menu items. Fixing obscure corner cases. Fighting inconsistencies in operating system APIs. Spotting misplaced pixels. Adding missing keyboard shortcuts.

Nothing interesting to blog about!

But still, methinks the difference between software that is merely “good” and software that is “great” is in the details. And only in the details.

I’ll just take care of tons of more details. Maybe it will result in something good.

I’m the master of svn deception. I do tons of useless commits just so that the stats look good. Yeah!

…ok, back to work.

Take a quick look at zillions of new posts.

Answer about five questions with “what’s the value of your camera’s near plane?”.

There should be some way to automate all of this. For every 20th question, reply with “increase your near plane!”, or something.

For the sake of the nation,
this operator must die!

Seriously. Suppose there is some class, let’s say ColorRGBAf. That has four floats inside. Now, someone at some point decided to add this operator to it:

operator float* () { /**/ }
operator const float* () const { /**/ }

Probably because it’s easier to pass color to OpenGL this way, or something like that.

This is evil. Like, really evil. Especially if that class did not have comparison operators defined, and some totally unrelated code four years later does:

if (color != oldColor) { /* … */ }

Ouch! Sounds like someone will spend four hours debugging something that looks like an event routing issue that only happens on Windows and only with optimizations on (yes, I just did that…).

What happens here? The compiler takes pointers to two colors and compares the pointers. If for some reason both colors are temporary objects, then it can even happen that both get folded into the same variable/register/whatnot. The pointers are the same. Ouch!

Implicit “nice” operators are just disguised evil. Remove that operator, add something like GetPointer() to class if someone really wants to use that, and better even make the comparison operators private and without implementations. Yes. Much better.

So after some promises, delays and a period of deadly silence, OpenGL 3.0 was released.

Response to it was “interesting“, to say at least. Some part of that response is related to seriously mishandled communication on Khronos part. Some part is because GL 3.0 is not what it was promised to be. Let’s just ignore the communication issue, it does not affect OpenGL itself in a direct way (it affects the developer community though).

By the way, I borrowed part of the post title from a blog post linked from opengl.org. In general, I do not agree with that blog post, but it’s a valid point of view. Unlike some other blog posts linked from opengl.org that are just pure garbage…

I am not sure what are the goals of OpenGL at this point. OpenGL’s current position, as far as games are concerned, seems to be roughly this:

Be the graphics API on various platforms where no alternatives are available.

Why? Because Windows has got D3D, which is far more stable, comes with useful tools, more often updated and actually works for variety of users (I’ll get to this point in a second). Mobile platforms have OpenGL ES, which is decent. All consoles have their own APIs (some of them similar to D3D, none of them similar to GL). So that leaves OpenGL as the choice on OS X, Linux and such. Not because it’s better. Because it’s the only choice.

“Oh, but look, id uses OpenGL! Two other games use OpenGL as well!” Well, good for them. But they are in a different league than “the rest of us”. For some games, driver writers will do whatever it takes to get those games running correct & fast. Surprise surprise, id games fall into this category. For the rest of us – no such luxury. Hey, try talking to your friendly IHV, the most likely answer is “yeah, but are really busy with some high profile games right now, ping us back in two months”. After two months, repeat.

So the rest comes from somone who is not working on the high-profile games that IHVs specially tune drivers to.

If OpenGL’s goals are to stay in this current position, then GL 3.0 is okay. It adds some new features, brings some extensions into core, hey, it even says “it’s quite likely that maybe perhaps someday some of the old cruft in the API will be removed, if we feel like it”. No problem with that.

However, OpenGL is advertised as something different, as if it wants to:

Be the graphics API on various platforms.

Which is quite different from it’s current position. I’m not sure if that’s the goal of OpenGL. Myself, I don’t care about the mythical cross-platform API that would actually work on those different platforms. API is a tool to do stuff; if different platforms have different APIs – no problem with that.

However, if OpenGL wants to achieve this advertised goal, it has to do several things. First and foremost:

Actually work

Stable drivers and runtime. In it’s current state, GL is too complex to implement good quality drivers/runtime. Complexity can be reduced in several ways:

• Cleanup the API. This was what GL 3.0 was supposed to be. Actual 3.0 did not do any of that, instead it just postponed the cleanup “until we feel like it”.
• Share some of the hard work. Why does everyone and their dog have to write GLSL preprocessor, lexer, parser and basic optimizer themselves? Define precompiled shader format, write frontend once, make it open. This would also be actually useful to reduce load times.

GL 3.0 could have done both of the above, instead it did none. It could have cleaned up the API, and provide one platform independent GL 1.x/2.x library that calls into actual 3.0 runtime. All the fixed function, immediate mode, display lists, whatever would be in one nice library. Even existing apps could continue to function transparently this way (with the benefit of actually simpler = more stable drivers).

Support platforms/hardware/features user needs

This is of course dependent on the user in question. For someone like us, we still have to support 10 year old hardware.

D3D9 does a fine job for that (provided you have drivers installed, and DX9 runtime installed – which comes included in XP SP2 and upwards). OpenGL 2.1 and earlier would do a fine job for that, provided it would “actually work” (see above).

If GL 3.0 would be as was originally promised – almost new API, shader model 2.0+ hardware, it would be sort of fine. In our case, that would mean writing and supporting two renderers – “old GL” and “new GL”, where old one would be used on old hardware or old platforms where “new GL” is not available. If the new runtime were much leaner, much more stable and generally nicer, this would not be a big problem.

With actual GL 3.0, in theory one does not have to write two renderers. Minimum hardware level for GL 3.0 is shader model 4+ though. So to support both old hardware/platforms and new hardware/platforms, quite a lot of duplication has to be done. Especially if you intend to go towards proposed “future GL path”, i.e. start dropping deprecated functionality from the codebase. At which point you’ll probably write two separate renderers already. So we’re back to where original GL 3.0 would have been, just without any extra niceness/stability/leanness right now.

Oh, and look at vendor announcements from 2008 OpenGL BOF. NVIDIA: we have almost full drivers now. AMD: we’re committed to having drivers. Intel: look for GL 3.0 on future platforms. In other words, looks like current Intel’s cards won’t ever have GL 3.0 drivers. And in our target market, Intel has the majority of cards.

That sounds very much like “just ignore whole GL 3.0 thing” plan to me.

Be nice

This is a point of far lesser importance than “actually work” and “support what is needed” ones. Having good tools (PIX, …), documentation, code examples etc. is nice. But not much more; being nicest API in the world does not do much if it does not actually work or does not support what you need. Even in this area, actual GL 3.0 is not nice – it’s full of redundancies and crap that goes 15 years back in history.

Summing it up

To me, GL 3.0 looks like a blunder. Instead of fixing the core problems, they just postponed that. Well, Keep up the good work!

I am not sure what shutdown buttons do when they look like this. I guess they are invincible, or something. Ha, I’m your log off button! You can’t kill me!

Yes, one of the updates installed was the ATI driver update, so I guess there’s some glitch somewhere in the driver update that makes some buttons be displayed like this… But hey, this is not some random driver that I found on the net, it’s the one that is officially suggested by Vista’s update!

You know, little things that would seem unimportant, but screams something like “I was made in a hurry by people who don’t really love me”. Aliased shield icon overlays? Check. Horrible screen flickering when logging in or UAC prompt pops up? Check. The infamous Shut Down menu? Check. Awful file copy progress dialogs? Check. Explorer window title bar sometimes displaying green progress bar inside of it for some reason I can never understand? Check. General lack of unified style for UI? Check. The list goes on.

But still, I wonder whether lack of polish is the real problem with Vista. From my point of view, lack of direction or lack of vision seems to be a problem of similar size, if not larger. What is the vision for Vista?

“Security!” is not a vision. However hard it is to make something secure, “more security” is an improvement in one area, and not a vision on what a product should be. And second, “security” does not explain everything else about Vista. At start, it looked like some architecture astronauts had some fancy visions, like “all your filesystem is a database now!”… Well, that did not end up in Vista, and it is something that users genuinely don’t care about.

I might sound like an Apple fanboy (and indeed, OS X grows on you after a while), but when upgrading from OS X 10.4 (Tiger) to 10.5 (Leopard) I had a pretty clear list of what will be more useful to me:

• New version feels faster (on the same machine). I am not sure if it is actually faster; or it’s only a perceived improvement. Maybe they optimized something, maybe they multithreaded something, I don’t really care. It feels faster and smoother. That’s good.
• Quick Look is amazing. A seemingly simple feature – press Space over a file to preview it. With added polish, like when pressing space over multiple images selected, you can go into slideshow mode. Simple, yet highly effective.
• Spotlight (desktop search) that is fast.
• …and so on.

Those are things I, as a user, care about. I want computer to feel faster. I want to instantly preview files. I want to search for something fast.

A filesystem that is a database? I can almost see the regular user salivating over that… Yeah right. Users don’t want a platform, users want useful features.

And this is where Vista fails – it does not have obvious new useful features or improvements. Aside from Direct3D 10 – which I am not using yet – all so called “improvements” just feel like gimmicks.

• It feels slower (I don’t care whether it actually is faster but just feels sluggish). And yes, it feels slower on a quad-core CPU with 4 gigs of RAM and a fast graphics card, so no “Vista runs circles around XP on a new box” please.
• The reorganized menus, title bars and layout of Explorer just scream “I totally don’t understand what users need” at you. Previews are too small to be useable, organization of menus and buttons is horrible, and the constantly fading-in-and-fading-out user interface elements (folders tree view) are just distracting. I dig the new Office 2007 UI and I can see some understanding of users and vision behind it (see Jensen Harris), but Vista’s UI feels like it was designed by a bunch of people who never talked to each other. And it’s not just lack of polish, the “design” of it is wrong.
• The Sidebar? Again, an attempt at doing something that seemed good, but without any understanding. Yes, I know that Apple might have taken the idea and implemented it right, but that does not leave Sidebar as being useful.
• The new skin? Oh come on. How many users did upgrade because window close buttons now glow in red when you hover over them?
• Was there anything else new in Vista? I didn’t notice anything.

So this pretty much sums up my view on Vista. Zero new useful things, many annoyances. Microsoft, here’s you chance to execute it better next time around.

CAsyncMonikerFile is derived from CMonikerFile, which in turn is derived from COleStreamFile. A COleStreamFile object represents a stream of data; a CMonikerFile object uses an IMoniker to obtain the data, and a CAsyncMonikerFile object does so asynchronously.

So yeah, I am dealing with downloading something from the internet inside an ActiveX control that is written in MFC. A seemingly simple task – I give you an URL, you give me back the bytes. But no! That would not be a proper architecture, so instead it has asynchronous monikers which are based on monikers which are based on stream files which use some interfaces and whatnot. And for ActiveX controls the docs suggest using CDataPathProperty or CCachedDataPathProperty, which are abstractions build on top of the above crap. And I don’t even know what “a moniker” is!

Of course all this complexity fails spectacularly in some quite common situations. For example, try downloading something when the web server serves gzip compressed html output. Good luck trying to figure out why everything seemingly works, you are notified of downloading progress, but never get the actual downloaded bytes.

Turns out the solution is to change downloading behaviour of the above pile of abstractions to use “pull data” model, instead of default “push data” model. The default behaviour just seems to be broken (though it is not broken in that pile of abstractions, instead it is broken somewhere deeper in Windows code). Is this mentioned anywhere in the docs? Of course not!

This is pretty much how a code comment looks like for this:

We don’t use CCachedDataPathProperty because it’s awfully slow, doing data reallocations for each 1KB received. For 8MB file it’s 8000 reallocations and 32 GB (!) of data copied for no good reason!

While we’re at it, we don’t use CDataPathProperty either, because it’s a useless wrapper over CAsyncMonikerFile.

Oh, and we don’t use CAsyncMonikerFile either, because it has bugs in VS2003′ MFC where it never notifies the container that it is done with download, making IE still display “X items remaining” indefinitely. Some smart coder was converting information message and returning “out of memory” error if result was NULL, even if input message was NULL (which it often was). So we use our own “fixed” version of CAsyncMonikerFile instead.

Oh MFC, how we love thee.

Some N years ago I would have liked a title with “Architect” or “Analyst” or something like that. I would have called myself “developer” instead of “programmer” because hey, a developer thinks up things, whereas a programmer is a mere “code monkey”. More on code monkeys below.

But wait! Back then I also believed that knowing and using Design Patterns is essential for a programmer! In one place when I was interviewing new hires, design pattern knowledge was something I would look for… how stupid! Nowadays my view of patterns is more along the lines of “yeah, whatever”. I don’t exactly think of them as things from hell, but they could have caused more harm than good already.

Back to job titles. Code monkey is actually the key employee. A software product is largely defined by the code, heck, it is code. Sure, it also has the user interface, the fancy icons, the documentation, the website, the support, the roadmap and whatnot, but the code is the product, whereas everything else is more or less addons (possibly excluding UI… UI also defines the product).

Code design? Design patterns? Who cares about that.

It’s the final result that matters. Futurist programming for the win.

On the other hand, Memento Observer is probably very cool.

It’s probably three frames per second. In the menu!

I did not see the game itself yet, got bored while waiting for the after-menu-but-before-game intro movie to end (it’s not skippable, and it also ran at about three FPS). This is after watching half a dozen obligatory before-menu intro movies at 3 FPS with stuttering sound (“nvidia,vidia,vidia,vidia… the way it’s meant,meant,meant,meant…” – TWIMTBP).

All of this on a half-decent PC, I think – Intel Core 2 Quad, 4GB RAM, Radeon 3850, Windows XP, latest drivers, none of extra stuff running; the PC is able to run other 3D stuff just fine. I’m sure the developers and EA’s testing labs have tested everything extensively, but sometimes something completely random apparently can make things be oh so slow. Oh well. Get back to work.

One of our customers found an interesting bug the other day: embedding Unity Web Player into a web page makes some javascript animation libraries not work correctly. For example, script.aculo.us or Dojo Toolkit would stop doing some of their tasks. But only on Windows, and only on some browsers (Firefox and Safari).

Wait a moment… Unity plugin makes nice wobbling web page elements not wobble anymore!? Sounds like an interesting issue…

So I prepared for a debug session and tried the usual “divide by two until you locate the problem” approach.

• Unity Web Player is composed of two parts: a small browser plugin, and the actual “engine” (let’s call it “runtime”). First I change the plugin so that it only loads the data, but never loads or starts the runtime. Everything works. So the problem is not in the plugin. Good.
• Load the runtime and do basic initialization (create child window, load Mono, …), but never actually start playing the content – everything works.
• Load the runtime and fully initialize everything, but never actually start playing the content – the bug appears! By now I know that the problem is somewhere in the initialization.

Initialization reads some settings from the data file, creates some “manager objects” for the runtime, initializes graphics device, loads first game “level” and then the game can play.

What of the above could cause something inside browser’s JavaScript engine stop working? And do that only on Windows, and only on some browsers? My first guess was the most platform-specific part: intialization of the graphics device, which on Windows usually happens to be Direct3D.

So I continued:

• Try using OpenGL instead of Direct3D – everything works. By now it’s confirmed that initializing Direct3D causes something else in the browser not work.
• “A-ha!” moment: tell Direct3D to not change floating point precision (via a create flag). Voilà, everything works!

I don’t know how I actually came up with the idea of testing floating point precision flag. Maybe I remembered some related problems we had a while ago, where Direct3D would cause timing calculations be “off”, if the user’s machine was not rebooted for a couple of weeks or more. That time around we properly changed our timing code to use 64 bit integers, but left Direct3D precision setting intact.

Side note: Intel x86 floating point unit (FPU) can operate in various precision modes, usually 32, 64 or 80 bit. By default Direct3D 9 sets FPU precision to 32 bit (i.e. single precision). Telling D3D to not change FPU settings could lower performance somewhat, but in my tests it did not have any noticeable impact.

So there it was. A debugging session, one line of change in the code, and fancy javascript webpage animations work on Windows in Firefox and Safari. This is coming out in Unity 2.0.2 update soon.

The moral? Something in one place can affect seemingly completely unrelated things in another place!

Until you hit a graphics card (quite new – from 2006, it can even do pixel shader 2.0) that completely hangs up the machine in one of your unit tests. In fact, in the first unit test, that does almost nothing. Debugging that thing is total awesomeness – try something out, and the machine either hangs up or it does not. Reboot, repeat.

After something like 30 hang-ups I found the cause: you are damned if you set GL_SEPARATE_SPECULAR_COLOR and GL_COLOR_SUM to different values (i.e. use separate specular but don’t turn on color sum). Because, you know, some code was there that did not see a point in changing light mode color control when no lighting was on. So yeah, always set those two in sync. Just to please this card’s drivers.

It’s hard for me to have any faith in driver developers. I know that their job is hard, walking the fine line between correctness and getting decent benchmark scores… But still – hanging up the machine when two OpenGL 1.2 states are set to different values? Would you trust those people to write full fledged compilers?

…this is just me venting out, during the process of chasing a video memory leak for 4 days already. It involves p-buffers, depth textures, shared OpenGL contexts and other delicious things. Still didn’t find the cause, but I’m getting close.

Pbuffer my a**.

The most interesting effects are in fact not achieved by the use of shaders – rather it’s the complex interplay of render-to-textures, postprocessing them, using them somewhere else, postprocessing that, using them somewhere else again, combine the intermediate results in various interesting ways, and finally get something that represents the colors on the screen.

So the effect is much more than the shaders; it’s more like “an orchestration of different stuff”. It’s the shaders, textures, models (often with custom data that is required for the effect to work), specially precomputed textures (more custom data for the effect), multiple render textures to store intermediate results, and finally some sort of “script” that controls the whole process.

Shaders are (relatively) small, isolated thingies that operate on small, isolated pieces of data. So they are only a small part of the whole picture, and in fact often they are the easiest part. Both to the effect author, and even for the “engine” to support.

Adding “shader support” to the engine/toolset is a piece of cake. The hard parts are:

• How shaders integrate with the lighting? Once you start doing shaders, you lose all lighting that used to be computed for you.
• How shaders integrate with various data the mesh may have or it might not? For example, to support per-vertex colors you either have to write a single (sub-optimal) shader that always assumes vertex colors are present, or write multiple shaders: when vertex colors are present, and when they are not.
• How shaders integrate with … lots of other things that were taken for granted in fixed function world. Scaled meshes. Various fog types. Texture transforms. The list goes on.
• How do you expose the rest of the “stuff” that makes interesting effects possible? Shaders alone are nothing. You need render-to-texture with various interesting formats. You need to store custom data in the meshes. You need to store custom data in the textures, in various interesting formats. You need custom cameras to render intermediate results into render textures. You need a way to replace object shaders with some other shaders so that these objects can render these intermediate results. The list goes on.
• Hardware support. Once you enter shader world, you get a totally new set of inconsistencies between operating systems, graphics APIs, hardware vendors, graphics card models and driver versions. And a new bag of driver bugs of course.

So shaders do present three big challenges.

1) Shader explosion. How do you do lighting? Do you say “there shall only ever be a single directional light” (that is what most tech-demos do)? Do you try to support the general case and write some dozen shader combinations? How do you handle all other cases that were “just handled” by fixed function pipeline? Vertex formats, fog, texture transforms, etc. Congratulations, multiply your shader count by 10 or so. Or don’t multiply the shader count, but use static/dynamic branching if you have hardware support for it.

2) Platform differences. How you write shaders in the general case, when some things don’t work on AMD cards, some other things don’t work on NVIDIA cards, yet some other things don’t work in OpenGL, some other things don’t work in Direct3D, and something entirely else does not work on Mac OS X (with different things not working on PPC vs Intel, yay)? Congratulations again, multiply your already large shader count by 10 or so.

3) Shaders are just a small part of the equation. You need the whole infrastructure around them to actually enable those interesting effects.

To return to the original message about the skin rendering demo: no, “just shaders” won’t enable that. You need render textures in several formats (color & depth) at least. And some dozen shaders to do the effect for a single light type, for a single mesh configuration, and for a single hardware platform. Multiply the shader count by, hmm…, hundred or so to cover the general case.

Real life is harder than a tech demo. Real life is more than just the shaders.

(*) I’m taking the message out of the context on purpose. Nothing is wrong with noisecrime’s message, I just want to point out that just exposing shaders does not get you very far.

Pressing Cancel somewhere in there results in this dialog:

Pretty cool, I must say. Some advice I’d like to give for the application developers, if I may:

• Instead of trying to copy Aqua’s GUI, make the application actually do useful things. “I don’t need your stuff, please give me a single partition” is a pretty common use case. And please don’t make error messages like that be the topmost windows (now it stays on top even when I switch to another app).
• Either get someone to write English messages, or leave the message as is and change the caption to “All your base are belong to us”. For such an exciting error message I’d also change the button text to “YES!!!”. Let the user enjoy your software!

From the software manual: “Thanks a lot to you that are use our company’s product”. Gee, thank you that you are use your software on me!

It started out like this: converting some existing code that generates geometry on the fly. It used to generate that into in-memory arrays and then Just Draw Them. Probably not the most optimal solution, but that’s fine. Of course we can optimize that, right?

So with all my knowledge how things used to work in D3D I start “I’ll just do the same in OpenGL” adventure. Create a single big dynamic vertex buffer, a single big dynamic element buffer; update small portions of it with glBufferSubData, “discard” it (=glBufferData with null pointer) when the end is reached, rinse & repeat.

Now, let’s for a moment ignore the fact that updating portions of index buffer does not actually work on Mac OS X… Everything else is fine and it actually works! Except for… it’s quite a lot slower than just doing the old “render from memory” thing. Ok, must be some OS X specific thing… Nope, on a Windows box with GeForce 6800GT it is still slower.

Now, there are three things that could have gone wrong: 1) I did something stupid (quite likely), 2) VBOs for dynamically updated chunks of geometry suck (could be… they don’t have a way to update just one chunk without one extra memory copy at least), 3) both me and VBOs are stupid. If I was me I’d bet on the third option.

What I don’t get is: D3D has had a buffer model that is simple to understand and actually works for, like, 6 years now! Why ARB_vertex_buffer_object guys couldn’t just copy that? The world would be a better place! No, instead they make a way to map only whole buffer; updating chunks is extra memory copy; there are confusing usage parameters (when should I use STREAM and when DYNAMIC?); performance costs are unclear (when is glBufferSubData faster than glMapBuffer?) etc. And in the end when an OpenGL noob like me tries to actually make them work – he can’t! It’s slow! ]]> http://aras-p.info/blog/2007/03/22/arb_vertex_buffer_object-is-stupid/feed/ 0 http://aras-p.info/blog/2006/08/14/a-steam-of-random-things/ http://aras-p.info/blog/2006/08/14/a-steam-of-random-things/#comments Sun, 13 Aug 2006 21:24:00 +0000 Aras Pranckevičius http://aras-p.info/blog/?p=96 Awards: Hey, we‘ve got a “runner up” award in Apple Design Awards 2006, Best Use of Graphics category! Yeah, a runner up is more like “the first of the losers”, but oh well. Got beaten by modo 201, which probably is a fair trade. It’s just that we thought we’d be in Best Developer Tool category, but that is apparently for text editors and scripting languages :)

Demos: In the other news, fellow ReJ with TBL just won Assembly 2006 demo competition with an Amiga demo, putting all PC demos faces’ to dust. Check it out. Art direction over hardware capabilities, one more time.

Drivers: why oh why the graphics drivers must be so bad? The other day I was thinking why can’t they auto-update themselves (with an option to turn it off for corporate users etc.). Now you’ve got a (not so recent!) driver that is able to parse vertex programs wrong, and a user who does not have a clue that he should update it. It’s bad enough to have a bug in the first place, but auto-update at least would fix…

Or you have a driver that says “I’m OpenGL 1.2!” but the 3D texture functions are null. And it’s the most recent driver for a particular graphics card that you can buy today! And it’s not even a hard problem! What the developers are thinking – they go over the required GL 1.2 functionality, see that some is actually missing and decide “ah, screw it, we’ll say it’s 1.2 anyways”?!

I just don’t get it. I could use some enlightenment on this.

Why I’m writing this? Because my head hurts from trying to make the renderer run in one thread, and the containing application in the other. With added spice that most (win32) GUI stuff must happen in one thread, OpenGL contexts can only be active in a single thread, simple functions must be split into complex chains of inter-threading calls, etc. etc. The result is a mess that nobody can understand.

I hope I’m not missing something obvious (well, aside from the suggestion “just don’t use threads”).

I feel kinda sad, even if I blame the management for the failures mostly (we programmers never blame ourselves, you know :)). Yes, I/we could have worked harder, done better etc., but all the way I (and probably ‘we’) felt that the ambitions were too unrealistic^*, the business plans were too non-existant, the management was just a bit too poor, and some forward-thinking was ‘just a bit missing’.

I guess that’s just my attempt to transfer blame to someone else :)

That said, it’s still sad. Oh well, just keep going on.

* Ambitions: “by that time Sony, Nintendo and Konami will be years behind us” doesn’t sound very realistic, does it? :) ]]> http://aras-p.info/blog/2005/09/14/interamotion-closes-offices/feed/ 1 http://aras-p.info/blog/2005/08/30/doing-something/ http://aras-p.info/blog/2005/08/30/doing-something/#comments Tue, 30 Aug 2005 13:09:00 +0000 Aras Pranckevičius http://aras-p.info/blog/?p=62 After taking a break, I already want to do something programming-related in my free time. What should I?

• The standard thing: do a demo. I have some ideas for that, but for various reasons I’m not in the mood right now to start a big/long project.
• Read some books. I’m doing this a bit.
• Learn something. Do something with wxWidgets or some Lua-only programming, but didn’t start anything yet.
• I already should do something related to the next version of Project Hoshimi for Imagine Cup 2006. I’m trying to!

• Software That Rocks award goes to TortoiseSVN and PmWiki. Both are slick, sexy and do not try to annoy you. They actually work (that’s already something!), they do not crash whenever I do just-a-bit-non-usual action, they do not corrupt my files behind my back.
• Software That Sucks award this week goes to Oracle Reports and Designer 10g. Sorry dear Oracle, but the best description I have for those is “Ugh”. For a longer description, just invert everything that’s written in the first point.

See you later!

Yeah, I know, the code is a product of 2 years of wandering in the darkness :) – prototyping, adding something and removing something, and when you constantly have to actually produce “working demos” in short timeframes, it ain’t easy to actually sit down and refactor whole thing. Oh well, the reality…

Still, code with lots of dependencies does suck of course.

On the other hand, I’m pretty happy with the module I’m writing. All the outside just needs one header file that’s right now 76 lines long and that includes only very basic “resource id” header file. The insides of my module also need some of the outside functionality; I’ve written some “outside interface” for that – a header file with something like 50 lines and also absolutely no includes.

That’s what I consider to be a low-dependency interface. That’s good. That doesn’t suck :)

[EDIT] The code I was referring to isn’t so bad (e.g. it really doesn’t “includes everything else”, but “includes quite large amount of stuff”). I’m just somewhat frustrated at the moment, and anything that’s not very ideal in the codebase irritates me. ]]> http://aras-p.info/blog/2005/02/26/code-dependencies/feed/ 5 http://aras-p.info/blog/2005/02/10/a-bunch-of-hacks/ http://aras-p.info/blog/2005/02/10/a-bunch-of-hacks/#comments Thu, 10 Feb 2005 20:24:00 +0000 Aras Pranckevičius http://aras-p.info/blog/?p=10 And I though my own freetime projects were coded like a bunch of hacks, with no clear structure, with lots of code duplication etc… These are nothing compared to the code I now work with!