"I hate that AI can do this," is my opinion for most every value of "this" that I've seen in the past couple years. It's also what fantasy author Mark Lawrence said after pitting AI writing against flash fiction written by award-winning authors and finding that not only can readers not reliably tell the difference in a blind test, they also preferred the AI-written stories.
Earlier this month, Lawrence composed a quiz on his blog (which you can still take) with eight very short fantasy stories of about 350 words, asking readers to rate each story's quality and whether they believe it was composed by a human or AI.
"This blog post is a genuine attempt to investigate where things stand with AI writing," Lawrence wrote. "It is not beating the drum for AI or advocating its use."
Four stories were written by AI and four by other well-known and award-winning authors. According to Lawrence's results post, the aggregate decision of the 964 voters correctly judged the origin of three stories, got three others wrong, and couldn't meaningfully decide on the remaining two, which is not an inspiring success rate. It's armchair statistics, but that's not a small sample size.
Much as we all want to believe that we can always spot AI writing by wit or by counting em dashes—a premise that forgets that human writers treat dashes like Pringles and should have them taken away by editors who are regrettably also human writers and susceptible to the same siren call of extra clauses—the reality is that we often can't, which sucks.
Not only did the votes place an AI story as the overall top rated, the AI stories on average were rated better than the human authored ones.
These human-authored stories weren't written by just anyone. They were written by Lawrence himself, Janny Wurts, Christian Cameron, and my personal favorite author Robin Hobb. Readers liked AI fiction more than a short story by prolific writer Robin Hobb, creator of the decades-spanning, multi-series Realm of the Elderlings universe. Bummer.
Lawrence does caveat this test in two ways: the contributing authors all write novels and so likely don't consider flash fiction their best skill, and shorter outputs are where AI is most likely to succeed in sounding like a human and retaining internal consistency.
"Should AI generate fiction, imagery, voices etc competing with artists in a number of fields and fooling the public. No, of course not. I hate that idea and most people do too," Lawrence wrote.
"Will it happen? It's already happening. Wherever anyone can circumvent skill and heart and just profiteer off a new technology, they're going to do it."
It is indeed happening. A sci-fi magazine had to halt submissions due to a flood of AI-written entries and authors are self-publishing novels with AI-generated writing in them. So far in consumer art spaces, platforms like Steam and Kindle are relying on creators self-reporting that work has AI-created elements, which just doesn't seem like a sustainable solution.
The short of it is that we can't stop AI-created art from spreading by spotting it in the wild and shaming it out of existence if we still can't reliably identify it. (Or if we're misidentifying it and harassing human-authored work, which happens too.) AI may be at a stumbling point this year, but it's going to take something more rigorous than the honor system and our own unreliable instincts to be sure we're reading work by humans.
