Support : How do you remove jaggys ?

[Chris]

Is gamma correct rendering an issue (disregarding the fact that Morrowind specifically doesn't do it)? I thought it was as simple as enabling GL_FRAMEBUFFER_SRGB and then it magically worked when you emit 32-bit floats from the fragment shader.

It's a little more involved than that. You need to load textures as sRGB instead of the normal RGB variants (RGB8, DXT1-RGB, etc)*, to tell the gfx driver to apply the inverse gamma correction when sampling the texture to retrieve linear space color values (on the presumption that textures were authored to display as-is on a monitor, which naturally applies the same/a similar correction). The GL_FRAMEBUFFER_SRGB flag then tells the gfx driver to treat the render surface as linear space, and apply gamma correction when copying to output to display correctly. Mathematically the two processes cancel out when no intermediate processing is done, but any intermediate processing that is done (such as lighting or filtering) works in a linear color space which makes it easier to create more accurate color and lighting ramps.

However, given that a gamma-corrected image has a non-linear intensity ramp, smaller variations are possible in the lower intensity values while higher intensity values have bigger variations (the difference between Red=0 and Red=1 is not the same as the difference between Red=245 and Red=255, for example). As a result, linear 8bpc (bits per color) can't represent certain intensity values that sRGB 8bpc can, and vice-versa. This ultimately reduces the effective bit-depth, causing color-banding. The fix is to use a greater bit-depth for the linear space render surface, which increases the dynamic range and allows those lost sRGB values to be represented again.

* With GL3, you can use sampler objects to toggle sRGB sampling separately from the texture format, so you can load textures as normal RGB color and tell the gfx driver to treat them as sRGB later.

[AnyOldName3]

But once the values are converted to linear RGB, they should be floats anyway, so you shouldn't care. Any render surface or texture that's not the final output or a texture being loaded off disk should just be a float representing a linear RGB value, and you shouldn't do anything with integer linear RGB because the separation of floats is smaller for smaller values, just like is needed.

[Chris]

But once the values are converted to linear RGB, they should be floats anyway, so you shouldn't care. Any render surface or texture that's not the final output or a texture being loaded off disk should just be a float representing a linear RGB value, and you shouldn't do anything with integer linear RGB because the separation of floats is smaller for smaller values, just like is needed.

Yes, which is why you need a separate offscreen renderbuffer, with a higher bit-depth. The render target you get as the window's back-buffer uses the same bit-depth as the front-buffer, which is normally 8bpc integer and is thus unsuitable for linear-space rendering.

The fact that you "write" float values out from pixel shaders is a bit of a lie. It's basically a normalized representation of the output surface depth. Much like how UV coords go from 0.0 to 1.0 regardless of the number of texels, the output goes from 0.0 to 1.0 regardless of the number of color values. 0.0 represents the lowest intensity and 1.0 represents the highest. When you write out 0.5 and 1.0 with an 8bpc integer render target, it will actually write 127 and 255 to the buffer, while with a 10bpc integer render target it would write 511 and 1023. The transients used during shader processing may be actual floats (or doubles), but once it's written out it gets converted to the target surface's bit-depth (which occurs before conversion to sRGB space).

[AnyOldName3]

If the conversion to 8-bit-per-channel integer colours was done before the sRGB gamma conversion with GL_FRAMEBUFFER_SRGB on, then surely I'd be able to detect banding when I take a screenshot of something and look at the values of the colours. My OpenGL experiments never got as far as using an off-screen render target and didn't have banding. It's possible that because I was using OpenTK from C#, it was creating an off-screen buffer and doing some magic there automatically, but that would strike me as bad design because as soon as a user started binding different buffers they'd have no way of knowing which was which.

[Chris]

I can't say how OpenTK works. It may also be that with modern OSs and compositing managers, it allows to create higher-depth (and/or floating-point) GL windows alleviating that problem.