Tim Sweeney predicts multithreading and 'digital humans' will push PC technology forward

At the 2017 Game Developer's Conference, I spoke with Tim Sweeney about Microsoft's Universal Windows App platform, which he called "woefully inadequate." Sweeney has been a vocal critic of UWP for the past year, but that's not how he spends most of his time. As the founder of Epic Games and a programmer on the Unreal Engine, he's one of the most important figures in the technology of PC games.

After his keynote on what's new with Unreal Engine 4, I spoke with Sweeney about what he expects will push game system requirements forward, and how the gaming industry can afford to make even more expensive, higher fidelity games.

PC Gamer: With your insight, where do you see CPU utilization for games right now, and where it might be going in the next few years? Over the past 5-7 years games have been very slow demanding more from CPUs. People were still on dual-core CPUs that, if you weren't playing Battlefield or something, were probably still doing okay. The i5-2500k is still giving pretty solid performance in most games. What do you see changing there, if anything?

Tim Sweeney: It's improving quickly. You know, engines took a long time to become scalable to an arbitrary number of cores. The first major step we made in multithreading was happening a separate rendering thread so dual-core CPUs could get full advantage of gameplay and rendering work occurring concurrently. The challenge now is to scale up to a lot more cores. There are going to be eight core consumer CPUs that are incredibly high performance and high capability. 

We've been, over the last three years, building Unreal Engine systems that are more and more scalable to that. For all parts of the engine, all the different tasks and rendering, to scale to multiple CPUs. And loading and other parts of the process. So it's getting better and better, and I think you're going to measure a bigger and bigger difference in overall performance between a 2-core, a 4-core and an 8-core CPU.

So right now in Unreal Engine 4, is the engine capable of taking advantage of a 4-core CPU easily, or 6 cores? And if so, what games are taking advantage of that right now?

A lot of the engine systems now are running as multithreaded tasks for some parts of the animation updates and particle updates, things like that. We're working on multithreading more and more over time. Which is a big task. Multithreading is one of the hardest problems of computer science, especially in software on the scale of millions of lines of code. But it's happening, and I think that you're really going to see it most for games designed for high-end PCs. Games like Paragon with a huge number of characters and effects and realistic animation. Whereas games designed for a fixed platform will tend to take advantage of what's there and not push beyond it.

What do you see as the things that, in the next 2-5 years, are going to be really pushing PC system requirements forward? What's our next Crysis, or things that aren't games like VR?

Well, digital humans are a big, big task that's going to push the hardware very hard. Both for VR and non-VR. For generations game developers have used a fairly low fidelity simulation of human movement. They call it skeletal animation, but it's not a real simulation of a skeleton, it's just a small number of bones that are influencing the skin on a human character. The next step will be a full physics simulation of the entire content of your body. Your actual bones, your tendons, your muscles, your fat. It'll be simulating that accurately so you can see all the close-up detail as a player's muscles bulge or his face responds in a lot of detail. And not just responds according to some pre-recorded animations that are played back, but also responds dynamically.

I think also, the game industry really needs to start adopting the deep learning technology that's so heavily changing image recognition and speech recognition and voice synthesis and all these other things. When you're driving your Tesla you have deep learning algorithms supported by an immense amount of GPU power. They're making decisions about autonomous vehicle driving. And that's awesome, and something the game industry hasn't adopted much so far, but it's exactly the kind of technology we need so that we can record thousands of hours of random live human motion and then be able to project that onto arbitrary situations that happen in games.

So instead of saying, like, 'let's blend the running animation into this walking animation' as we do today, we'll say 'let's bring the character to speed now,' and then the animation system should be able to automatically deduce what physics and body motion should occur as a result, based on the huge data sample from actual humans.

That degree of fidelity brings up a commonly discussed issues in the games industry these days, which is triple-A development getting more and more expensive. I thought the next generation of engines like Unreal 4 being more efficient would lower some of that cost, but it seems like games like Assassin's Creeds and Far Crys and EA's blockbusters are getting more and more expensive.

Well, those games aren't using the Unreal Engine!

That's fair, that's fair. But there are a lot of these big budget games, right, that are ballooning in cost.

Yeah, and I think that's a real problem for the industry. We've put an immense amount of effort into optimizing the Unreal Engine content creation pipeline so you can get as much as possible out of a small art team, a small programming team, a small animation team. And if you compare it to the state of the art 10 years ago, I think we can create 3-4 times more high quality content with the same resources compared to back then. I think a lot of these teams have not adopted modern tools and techniques, so they're spending a vast amount of time in the friction involved in the internal use of their tools.

The challenge with all these new systems that we're trying to build, is we want to be able to build more realistic characters without consuming any more artist time than our current generation, lower fidelity characters, and even bring that down.

Gears of War 4 was on Unreal Engine 4. I know it wasn't an Epic game, but it's a beautiful game, and there were stories around when it came out that it cost Microsoft many many times what it cost to make the first Gears of War.

Oh, sure. You know, the first Gears of War game cost $12 million, which is kind of a ridiculous number if you look at it from the perspective of the industry today. Most triple-A games cost $100 million plus. So I think actually Microsoft's development on Gears of War 4 was pretty efficient and economical compared to the teams at Ubisoft or Activision or EA where they have literally thousands of people working on a game sometimes.