Anthropic says it has identified thousands of 'fraudulent accounts' taking Claude and 'extracting its capabilities to train and improve their own models'

CHONGQING, CHINA - DECEMBER 29: In this photo illustration, a person holds a smartphone displaying the logo of “Claude,” an AI language model by Anthropic, with the company’s logo visible in the background, illustrating the rapid development and adoption of generative AI technologies, on December 29, 2024 in Chongqing, China. Artificial Intelligence (AI) has become a cornerstone of China’s strategic ambitions, with the government aiming to establish the country as a global leader in AI by 2030.
(Image credit: Cheng Xin via Getty Images)

The question of what data AI models are trained on, and the legitimacy of that data, is a thorny one. Anthropic found itself defending its use of copyrighted material to train its Claude AI in the US last year, a case that eventually resulted in a ruling that its copyrighted scraping fell under fair use privileges.

However, the company eventually agreed to pay a $1.5 billion settlement in regards to claims that it pirated copies of several author's works. I mention this, because Anthropic has recently taken to X to complain about "industrial-scale distillation attacks" on Claude, perpetrated by what it says are over "24,000 fraudulent accounts" that have generated over 16 million exchanges with the AI chatbot, thereby "extracting its capabilities to train and improve their own model."

Which, as far as Anthropic is concerned, really isn't on. It identifies DeepSeek, Moonshot AI, and MiniMax as the perpetrators of the attacks, and while it says that "distillation can be legitimate", it also declares: "Foreign labs that illicitly distil American models can remove safeguards, feeding model capabilities into their own military, intelligence, and surveillance systems."

In a further post, Anthropic says: "These attacks are growing in intensity and sophistication. Addressing them will require rapid, coordinated action among industry players, policymakers, and the broader AI community," before linking out to a news post on the topic.

The post goes into further detail regarding the discovery of the attacks, and also says that Anthropic was able to attribute "each campaign to a specific lab with high confidence through IP address correlation, request metadata, infrastructure indicators, and in some cases corroboration from industry partners."

Which, as X user AntonLaVay points out, sounds like Anthropic loudly declaring that it can de-anonymize its users with relative ease. That's perhaps a privacy-related point for another day.

In the meantime, though, it seems that while Anthropic is fine with training its own models on copyrighted data, other companies using Anthropic's work to train their own models is a serious problem.

And while the foreign military angle is certainly an interesting one, I've got a feeling it might not engender the same sort of sympathy as that given to private individuals who claim to have had their work incorporated into the Claude AI behemoth. Just a thought.

Secretlab Titan Evo gaming chair in Royal colouring, on a white background
Best PC gaming kit 2026

1. Best gaming chair: Secretlab Titan Evo

2. Best gaming desk: Secretlab Magnus Pro XL

3. Best gaming headset: Razer BlackShark V3

4. Best gaming keyboard: Asus ROG Strix Scope II 96 Wireless

5. Best gaming mouse: Razer DeathAdder V4 Pro

6. Best PC controller: GameSir G7 Pro

7. Best steering wheel: Logitech G Pro Racing Wheel

8. Best microphone: Shure MV6 USB Gaming Microphone

9. Best webcam: Elgato Facecam MK.2


👉Check out our list of guides👈

Andy Edser
Hardware Writer

Andy built his first gaming PC at the tender age of 12, when IDE cables were a thing and high resolution wasn't—and he hasn't stopped since. Now working as a hardware writer for PC Gamer, Andy spends his time jumping around the world attending product launches and trade shows, all the while reviewing every bit of PC gaming hardware he can get his hands on. You name it, if it's interesting hardware he'll write words about it, with opinions and everything.

You must confirm your public display name before commenting

Please logout and then login again, you will then be prompted to enter your display name.