Screenspace vs. mip-mapping
Just spent half a day debugging this, so here it is for the future reference of the internets.
In a deferred rendering setup (see Game Angst for a good discussion of deferred shading & lighting), lights are applied using data from screen-space buffers. Position, normal and other things are reconstructed from buffers and lighting is computed “in screen space”.
Because each light is applied to a portion of the screen, the pixels it computes can belong to different objects. If in any place of lighting computation you use textures with mipmaps, be careful. Most common use for mipmapped light textures is light “cookies” (aka Gobo).
Let’s say we have a very simple scene with a spot light:
Light’s angular attenuation comes from a texture like this:
If the texture has mipmaps and you sample it using the “obvious” way (e.g. tex2Dproj), you can get something like this:
Black stuff around the sphere is no good! It’s not the infamous half-texel offset in D3D9, not a driver bug, not a shader compiler bug and not the nature trying to prevent you from writing a deferred renderer.
It’s the mipmapping.
Mipmaps of your cookie texture look like this (128×128, 16×16, 8×8, 4×4 shown):
Now, take two adjacent pixels, where one belongs to the edge of the sphere, and the other belongs to the background object (technically you take a 2×2 block of pixels, but just two are enough to illustrate the point). When the light is applied, cookie texture coordinates for those pixels are computed. It can happen that the coordinates are very different, especially when pixels “belong” to entirely different surfaces that are quite far away from each other.
What the GPU does when texture coordinates of adjacent pixels are very different? Chooses a lower mipmap level so that texel to pixel density roughly matches 1:1. On the edges of this “wrong” screenshot, it happens that very small mipmap level is sampled, which is either black or white color (see 4×4 mip level).
What to do here? You could disable mip-mapping (which is not good for performance and not good for image quality). You could drop some smallest mip levels which might be enough and not that bad for performance. Another option is to manually supply LOD level or derivatives to sampling instructions, using something else than cookie texture coordinates. For example, derivative in view space position, or something like that. This might not be possible on lower shader models though.
Getting dx/dy for uv/w, clamping them and feeding them into the tex2D/tex2Dgrad will limit the issue, but is, as you say, not necessarily supported on older hardware. Forcing lod0 always is though :p
Haha, cool!
I had the exact same issue 6 months ago! What I found worked resonably well is to cut off a few of the lowest mip levels. The penalty wasn’t that bad and it (mostly) fixed the issue. It depends what your texture looks like though, in your example you should get away with removing 1 or 2 levels to get the same result as you have in the top picture.
Say hi to Mircea :)
The problem actually occurs not when the texture coordinates are near each other, but when the texture coordinates are far from each other, right? (A simple way to imagine it is to draw a bounding box around adjacent samples and then consider which mipmap texel best approximates that box. If the samples are near each other, it will try to magnify, not minify.) Naturally it’s *very common* for the texture coordinates on unrelated objects to be very far from each other.
I actually first saw a related scenario happen with mipmapped light maps on terrain in 2001 or so… you have a triangle that ought to be in shadow, but it’s at an extreme edge-on angle that forces the mipmap sample to sample from very far outside the triangle, and it suddenly pops bright.
But discontinuous pixel shading is definitely a much more common way to get this effect; it happens with the naive implementation of trilinear virtual texturing pretty much everywhere on page boundaries.
Thanks a lot for sharing this, I’m sure it will save me some time in the future =)
If the incorrect mip selection only happens when there’s a large depth difference, it might be possible to render the light in depth layers, controlled by depth bounds test. That would require rendering the light geometry multiple times but pixel shader work will be unaffected.
@sean: right, I got the near/far texture coordinate stuff backwards. Updated the post, thanks for the correction!
What if you just blur the low mip maps, so the lowest ones would be complete solid color white?
Then in the distance your light will illuminate all pixels it touches. Which can be ok if you render light as tight bounding shape in screen space, but not that ok if you render it as 2D bounding quad.
Slide 32 of the following presentation mentions the same problem, using tex2dlod to manually set the mip-level using the screen-space size of the projected light.
http://cmpmedia.vo.llnwd.net/o1/vault/gdceurope09/slides/TKlajnscek_Programming_BattletestedDeferredRendering.ppt
[...] Pranckevičius has a worthwhile post on deferred rendering and mipmap bugs, along with some good follow-up [...]