If you draw a diagonal line with square pixels, their hard edges create a jagged ‘staircase’ effect. This ugliness (among other artifacts) is called aliasing. If resolutions were much higher, it wouldn’t be a problem, but until display technology advances, we have to compensate with anti-aliasing.
There are many techniques for anti-aliasing, but supersampling (SSAA) is useful to explain the process. It works by rendering frames at a higher resolution than the display resolution, then squeezing them back down to size. On the previous page, you can see the anti-aliasing effect of downsampling Shadow of Mordor from 5120x2880 to 1440p.
Consider a pixel on a tile roof. It’s orange, and next to it is a pixel representing a cloudy sky, which is light and blueish. Next to each other, they create a hard, jagged transition from roof to sky. But if you render the scene these pixels live in at four times the resolution, that one orange roof pixel becomes four pixels. Some of those pixels are sky-colored and some are roof-colored. If we take the average of all four values, we get something in between. Do that to the whole scene and the transitions become softer.
That's the gist, at least, and while it looks very good, supersampling is extremely computationally expensive. You’re rendering each frame at a resolution two or more times higher than the one you’re playing at—even with our four GTX Titans, trying to run supersamling with a display resolution of 2560x1440 isn't practical. That’s why there are so many more efficient alternatives:
Multisampling (MSAA): Achieves good results, but is much more efficient than SSAA. This is typically the standard, baseline option in games, and it's explained very simply in the video below.
Coverage Sampling (CSAA): Nvidia’s more efficient version of MSAA.
Custom-filter (CFAA): AMD’s more efficient version of MSAA.
Fast Approximate (FXAA): Rather than analyzing the 3D models (i.e. MSAA, which looks at pixels on the edges of polygons), FXAA is a post-processing filter, meaning it applies to the whole scene after it has been rendered, and it's very efficient. It also catches edges inside textures which MSAA misses.
Morphological (MLAA): Available with AMD cards, MLAA also skips the rendering stage and processes the frame, seeking out aliasing and smoothing it. As Nicholas Vining explains: "Morphological anti-aliasing looks at the morphology (read: the patterns) of the jaggies on the edges; for each set of jaggies, it computes a way of removing the aliasing which is pleasing to the eye. It does this by breaking down edges and jaggies into little sets of morphological operators, like Tetris blocks, and then uses a special type of blending for each Tetris block." MLAA can be enabled in the Catalyst control panel.
Temporal (TXAA): Supported on Nvidia's Kepler GPUs, TXAA combines MSAA with other filters, and can help reduce the 'crawling' motion on edges, which looks a bit like marching ants. It cannot, however, remove actual ants from inside your display. You should probably just throw that display out.
As Vining again explains: "The notion here is that we expect frames to look a lot like each other from frame to frame; the user doesn't move that much. Therefore, where things haven't moved that much, we can get extra data from the previous frame and use this to augment the information we have available to anti-alias with."
Multi-Frame (MFAA): Nvidia's latest, exclusive to Maxwell GPUs. Whereas MSAA samples in set patterns, MFAA allows for programmable sample patterns. Nvidia does a good job of explaining MSAA and MFAA simply in the video below.
What do the numbers mean?
Anti-aliasing settings almost always include a series of values: 2x, 4x, 8x, and so on. The numbers refer to the number of color samples being taken, and in general, the higher the number, the more accurate (and computationally expensive) the anti-aliasing will be.
Then there's the special case of the 'Q.' CSAA attempts to achieve a quality better than or equal to MSAA with fewer color samples, so 8xCSAA actually only takes four color samples. The other four are coverage samples, explained here. 8QxCSAA, however, bumps the number of color samples back up to eight for increased accuracy.
I used Batman: Arkham City's benchmarking tool to test a few anti-aliasing methods: MSAA, FXAA, and TXAA. The results show, as expected, that FXAA is the least resource intensive, while MSAA and TXAA cause a significant drop in average framerate over no anti-aliasing.
Batman: Arkham City AA benchmarks (2x Nvidia GTX Titan SLI)
|Avg. FPS||Max FPS||Min FPS|
|FXAA (high)||154||204||60 |
Which AA should I use?
Tough question, and it depends on your GPU and your preference. If I had to order my preference from least-desirable to most-desirable (ignoring Nvidia vs. AMD tech), it would go: FXAA, MLAA, MSAA, CFAA, CSAA, TXAA. In certain games, you might find FXAA or MLAA more pleasing than MSAA—you'll have to try both to know which you prefer. If framerate is an issue, however, the choice is obvious: FXAA is much more efficient. I have not yet been able to test Nvidia's MFAA, but expect updates to this article.
Overriding anti-aliasing settings
Theoretically, in-game graphics options shouldn’t matter. You can just open up the Nvidia or AMD control panel and override their settings—use 32x CSAA if you want. Unfortunately, that’s not really the case. While you can set overrides for any game, I've had little success getting them to work. The two most recent games I tried were Shadow of Mordor and Defense Grid 2, and neither accepted any overrides. The only games out of the 10 or so that I tried that worked were Left 4 Dead 2 and Half-Life 2: Lost Coast.
"Very often when overrides don't work it's due to deferred rendering," says Vining, "which just breaks a lot of common anti-aliasing techniques." Alex Austin also notes that some of his techniques don't work with override settings. So, it's just a matter of testing. Turn off all AA in the in-game options, set the override in your control panel, and hop back in: it should be apparent whether or not it took effect.
I've found that AMD's MLAA works the most often when enabled in the control panel. It's important to note, however, that it's a post-processing filter and applies to everything in the scene. That means it can take care of hard edges within textures, which can be good, but comes with the side-effect that it may also go after desirable edges, such as in text. Notice how it slightly smooths the text in the BioShock Infinite menu, and even goes so far as to smooth my Fraps FPS overlay.