Peter "Durante" Thoman is a PC modder known for the Dark Souls mod DSFix. He's previously written about what optimization really means in games and recently helped develop Xseed's PC port of Trails of Cold Steel 2.
Final Fantasy 15 was released on PC this week, and there are many facets of this release to celebrate: it includes all the standard features expected of a PC game, a wealth of PC-exclusive graphical effects and assets, and even some official modding support. It's clearly a high-effort production. Still, a large part of the conversation online has centered on Final Fantasy 15's use of Denuvo DRM, and a workaround that let people play (and pirate) the game early.
Whenever a game uses Denuvo, some people claim the DRM has a negative performance impact. Others disagree. For me, this conflict invites a more detailed examination. And Final Fantasy 15 provides a great opportunity for that: its free demo executable does not use Denuvo and can run some of the exact same scenes and assets as the full game.
After hours of testing, I found no obvious measurable impact of Denuvo on performance in Final Fantasy 15—but that's not quite the full story.
Methodology
From what is commonly understood about how Denuvo (or any DRM) works, it is exceedingly unlikely for it to affect GPU performance in any way. So to isolate and maximize its potential effects, I chose the settings for all benchmarks that would minimize GPU load while producing the maximum possible CPU load. This was achieved by putting all options at their highest setting, but rendering at 50% of 1280x720 resolution—basically as low as I could go. Nvidia Gameworks settings other than Turf Effects were disabled, as they primarily cause GPU load. This resulted in the test system (an i7 5820k with a GTX1080) being CPU-bound across the board.
Note that this setup inherently creates a worst-case scenario for Denuvo: in a more realistic setup, in most modern games, most gaming PCs are far more likely to be partially or entirely GPU-bound rather than CPU-bound when it comes to performance. This worst-case selection is intentional, as we want to determine the most dramatic possible effect Denuvo might have.
A common, reasonable guess about DRM is that it takes a heavier performance toll on slower CPUs than faster ones. To test this as well, all benchmarks were performed with the game having access to 12, 6, 4, 2 and 1 hardware threads, and at 4.1 GHz as well as 3.1 GHz. Of course, both versions of the game were installed to the same SSD.
Determining reproducible testing locations proved a challenge, as game saves are not portable between the final retail and the demo version. As such, I had to resort to manually recreating the exact location, camera angle, and time of day of each of the three testing scenes in both the retail game and the demo version. The screenshot above shows me manually aligning one of the locations. The performance numbers in that screenshot are meaningless—obviously I did not run two instances of the game concurrently for actual benchmarking.
I ended up testing three scenes:
- Early morning in a field, with a lot of foliage visible (Scene 1)
- The initial location after the intro, where you first start actually playing the game (Scene 2)
- Evening at a resting spot, during cooking (Scene 3)
The results: FPS
Without further ado, let’s look at the numbers that were the reward for all this set-up work.
In the first benchmark scene—which was also the most consistently reproducible—there were absolutely no differences in performance between the release and demo version that can be classified as statistically meaningful. This was the case regardless of the number of cores accessible to the game, and also regardless of CPU clock speed (not pictured, see the full results here).
Scenes 2 and 3 look somewhat more interesting:
In scene 2, the release version is 3.5% slower in the geometric mean across core counts, while in scene 3 the release version is 5.6% faster in the same metric. While these differences are still small, they are sufficiently large and repeatable to qualify as more than measurement errors.
Does this mean that Denuvo slows down the game in scene 2, but then turns around and speeds up scene 3? I very highly doubt it. It seems far more likely that—due to the open world nature of the game and the lack of direct save portability—the benchmark situations are simply not exactly equivalent.
Overall, these results paint a rather clear picture: Denuvo does not affect in-game performance negatively in Final Fantasy 15.
The results: loading times
From what is publicly known about the implementation of Denuvo, its integration likely requires game developers to select a subset of functions to be protected. If this is done well, as it appears to be in Final Fantasy XV, then nothing that is called often during a normal gameplay frame will be affected. What, then, about a commonly CPU-bound, far less frequent, but still fundamental operation—loading?
For these experiments I had to implement a rather exhaustive testing scheme to make them truly repeatable: I always load the same sequence of areas, warm up caches with an initial loading phase, and repeated each measurement at least 3 times (more if there was an outlier). The number I report is the median time it takes from clicking “yes” on the loading prompt until the game is fully faded in at the loaded location.
The chart above shows the loading times with 2, 4 and 6 cores available to the game in the release and demo versions. There is a measurable, though small, difference, and this difference persists across multiple measurements and game restarts. It also occurs similarly in both other scenes, and overall the demo version shows a 6.7% geometric mean improvement in loading times compared to the final release.
Does this mean that Denuvo slows down loading in FFXV? Not necessarily. While both executables are using the same asset base, there might be another code difference. However, unlike the in-game performance analysis, these results do show a consistent albeit minor difference across the tested scenes and hardware configurations.
General CPU performance
Given all this data, it would be a shame not to have a quick look at the CPU performance of the game independently of Denuvo. It’s clear that, at its highest settings, Final Fantasy 15 is a rather demanding game CPU-wise—though not so demanding that it will be noticeable in most normal setups given the degree to which it can put GPU resources to good use. Even at the highest settings, the results show that a modern quad-core CPU is sufficient to maintain 60 fps, and 2 fast cores seem to still be enough to remain above the 30 fps mark. Do note though that I haven’t done enough experiments to see whether this holds for minimum frame rates, or for streaming during fast traversal of the map.
What appears clear beyond any reasonable doubt in all the results is that FF15 has very good parallel scaling. In all tested scenes, its performance scales linearly from 1 to 2 cores, almost linearly to 4, and still rather well to 6 cores. It does not appear to benefit from hardware multithreading, but this is generally true of most games.
Conclusion
I am personally against DRM, since I consider both the preservation and the moddability of games to be incredibly important, particularly on PC. And I have been fortunate to collaborate with a publisher that allows all games I have worked on to be released completely DRM free, so I can put my money where my mouth is.
That said, whether we consider DRM counter-productive or stupid does not have any bearing on a performance assessment. And for FF15, the results of such an assessment—performed as scientifically as possible under the circumstances—are these:
- There is no support in the data for the idea that Denuvo affects in-game performance negatively, even in the worst-case scenario, and regardless of CPU speed.
- Denuvo might increase load times by small factor (about 6.7%) in its implementation in Final Fantasy 15.
Given that Denuvo comes in many different flavours, with continuously updated versions, and that its actual integration process depends on a number of developer decisions, these results do not necessarily hold true with all other games. Nonetheless, I consider it confirmed by this study that a competent implementation of current Denuvo on a high-end game can be used without any meaningful in-game performance impact.
Does that mean you should use it or approve of its use? No, I don't think so, but not because of performance reasons.
Final Fantasy 16 isn't on PC yet, but you can get the full Clive experience in FF14 right now by wearing his clothes and riding his dog, via the new crossover event
Final Fantasy 14: Dawntrail is having server queue issues months before it's even out, as Square Enix's online store crashes under the weight of pre-orders
