In a dystopian world first, China arrests man for allegedly using AI to create and spread a fake news story

Flag of China — (Image credit: Matthias Hangst (Getty))

The South China Morning Post reports that Chinese police have arrested a man who allegedly used ChatGPT to create a fake news story and spread it online. This may well be the first instance of a person being detained for their use of AI, though it certainly won't be the last.

The man was detained in the northwestern province of Gansu and was identified by the police using only his surname, Hong. In a statement the force said he'd been held for "using artificial intelligence technology to concoct false and untrue information."

It's the first arrest since China introduced new laws to regulate the use of AI and 'deepfake' technology in January. The Administrative Provisions on Deep Synthesis for Internet Information Service target any technologies that generate text, images, audio or video, and explicitly mention deep learning models. It doesn't outlaw the creation of things using these technologies, but does force them to be "clearly labelled" as such.

The police traced the article to a company owned by the suspect, and 10 days after it appeared the police arrested Hong, took his computer, and searched his house. The police statement says Hong confessed to using elements from past trending stories in China to input into ChatGPT, producing various versions of the story quickly, before they were uploaded to the Baijiahao accounts. Hong claimed he'd done it after friends on WeChat told him how to make money from clicks.

Hong's crime is, per the police, "picking quarrels and provoking trouble", which carries a maximum sentence of five years. However authorities can choose to make an example and, given this is a first, Hong may well be in very hot water.

This is all part of a wider and ongoing tech crackdown from Beijing, as the authoritarian Chinese Communist Party seeks to exert its control over emerging tech and how China's population uses it. The Chinese police issued a statement in February warning the public to be wary of rumours created by ChatGPT.

This particular concern, of course, extends far beyond China: The CCP is just one of the first national governments to explicitly tackle it. Both the UK and US governments are belatedly waking up to some of the potential problems with AI, and recently the '"godfather of deep learning" Geoffrey Hinton left his position at Google with some rather dire warnings about how we control this technology (or fail to). People using this stuff to generate fake news was always distressingly predictable but, unfortunately, we're only just getting started.

Rich is a games journalist with 15 years' experience, beginning his career on Edge magazine before working for a wide range of outlets, including Ars Technica, Eurogamer, GamesRadar+, Gamespot, the Guardian, IGN, the New Statesman, Polygon, and Vice. He was the editor of Kotaku UK, the UK arm of Kotaku, for three years before joining PC Gamer. He is the author of a Brief History of Video Games, a full history of the medium, which the Midwest Book Review described as "[a] must-read for serious minded game historians and curious video game connoisseurs alike."