Opinion: The sad state of beta testing
In the beginning, there was the demo. And it was good. It let players take a try-before-you-buy approach to their major purchases, reassuring them that they hadn't dropped $60 on a steaming heap of digital detritus. The demo, though, was also a pain-in-the-ass to dedicate much-needed development resources to - especially before a full release. As such, it was slowly, quietly assassinated. Its spirit, however, lives on in the not-so-subtle art of the open/closed/slightly ajar beta. And that's a problem.
In the past couple weeks, we've seen the Internet very nearly begin a drool flood of Biblical proportions over the beta tests of Diablo III and Battlefield 3. And why not? Select players get to take their favorite videogame juggernauts for a spin before anyone else. When you've coveted these things like water in a desert for ages, one tiny taste goes a long way. So we get Diablo's all-too-brief sample of Act I and Battlefield's one entire map (unless you're willing to wage actual war for a slot on the Caspian Border server), because that's more than enough for players who are starved for anything at all.
Sounds an awful lot like a demo, right? Well, that's because it basically is. Speaking about CSGO's upcoming beta, Valve's Chet Faliszek pinpointed a major part of the problem:
“The PC Beta will be extended longer, because it’s not really a good model for doing betas, real betas, on the console. Other console betas have been more promotional demos. What we’re saying is that we want to do a beta that’s constantly changing and updating based on player feedback.”
Due to the fact that console certification processes are basically slow-mo mazes of red tape, constant updates are out of the question. So for multiplatform games, the beta system's already pretty much broken.
That, however, isn't the only issue. Cynical though it may sound, Faliszek is right: Marketing plays a huge part in these things. That's dangerous, though, because that change in central objective brings a complete 180 in priorities. The testing mentality: "Who cares how buggy it is? Let people teleport the whole game world into the sun for all we care - just so long as they tell us about it." The marketing mentality: "Oh no, no fiery visions of the apocalypse for you. But do pre-polish the beta to a sun-eclipsing sheen. Otherwise, players might think it's - gasp - an incomplete game."
See the problem there? And that's a damn shame, because proper public testing can be hugely helpful in ironing out pesky kinks or even changing the course of entire games. I've spoken with gobs of indie devs who swear by it (Hello, Minecraft), insisting until they're blue in the face that they would have churned out a different, largely inferior game without early feedback from fans.
Instead, though, we get situations like BF3's, where the beta feedback forum has all the organization of a pie-fight/anarchy convention (No search feature? Seriously?) or a plethora of MMOs that have shipped without outside eyes even glimpsing their endgames. Disappointing? You don't even know the half of it.