This viral Chatroulette-style browser game challenges you to figure out if your partner is human or AI, and I keep failing

A lose screen on Human or Not, showing that I guessed my partner was a human incorrectly.

(Image credit: AI21 Labs)

We live in an era of fakes and frauds. With the proliferation of AI, it's harder than ever to figure out what's true and what isn't. Is the Pope dripped out in Balenciaga? Has Tom Cruise met Gorbachev? Am I AI? Are you?

There's only one way to respond to the crushing psychic torture inflicted on us by modernity: Turn it into a game. So here's Human or Not, a kind of Chatroulette-like that challenges you to decide whether your conversation partner is human or AI after a two-minute chat.

Billed as a "part of a larger scientific research project by AI21 Labs," Human or Not actually came out last year before becoming unavailable. Lucky for us, it's now back and taking over most of my Twitter feed as people find themselves thoroughly befuddled by its mix of tricksy large language models (LLM). It uses "Jurassic-2, GPT-4, Claude and Cohere," as part of its mix, per the FAQ.

The game is presented as a "social Turing test," referring to the old evaluation method suggested by Alan Turing himself that was meant to evaluate whether an entity was a machine or a human. It's meant to figure out if a machine can present behaviour indistinguishable from a human. If my games are any indication, they can. But maybe those LLMs aren't really passing the test so much as I'm failing it.

It's disquieting. Based on the conversations I've had, it turns out that I'm remarkably easy to hoodwink when it comes to the ol' Turing test. I regularly find myself confidently declaring my partners to be human before the game pulls the rug out from under me and tells me they were AI. Sometimes the reverse happens, too, which feels more like an indictment of my conversation partners than it does of me, frankly.

It's worth giving a go if you've not quite reached your quota of suffocating existential dread on this fine Friday morning, but do be warned: Some of the humans you encounter will immediately reach for the graphically sexual nuclear option to figure out if you're a robot or not, which might not be your cup of tea. Me? I'm just glad there's one thing left that the bots can't beat us at.

One of Josh's first memories is of playing Quake 2 on the family computer when he was much too young to be doing that, and he's been irreparably game-brained ever since. His writing has been featured in Vice, Fanbyte, and the Financial Times. He'll play pretty much anything, and has written far too much on everything from visual novels to Assassin's Creed. His most profound loves are for CRPGs, immersive sims, and any game whose ambition outstrips its budget. He thinks you're all far too mean about Deus Ex: Invisible War.