A Reddit poster going by Emotional_Stranger_5 says they spent 16 days working with ChatGPT on an illustrated children's book intended as a gift for a group of local kids. Believing that the chatbot had been invisibly toiling away creating 700 illustrated pages, they turned to the OpenAI subreddit to ask how to download the completed document.
The poster did receive the helpful advice they were looking for, but only after being roasted to a crisp.
ChatGPT had only been telling Emotional_Stranger_5 that it had been building a 700 page tome. The eager-to-help tone of large language models doesn't reflect actual intent or the long-term "memory" required to generate outputs over a span of days or weeks. Nothing beyond the output we see exists, at least in typical chatbot designs, and there was no book to download.
For numerous posters in the thread, Emotional_Stranger_5 received due comeuppance for trying to produce AI slop for kids, though the frustrated bookmaker clarified that they had not asked ChatGPT to write a book from scratch. They said that they personally spent two-and-a-half months adapting stories from Indian mythology and wanted ChatGPT to improve the flow of their writing and generate illustrations.
One confusing aspect of the user's plight is the number of pages involved. Who would feel the need to produce a custom 700-page book for children and how could they write that much in under three months? My best guess is that ChatGPT simply promised 700 pages because it's a number that exists, and not because there were really 700 pages worth of material.
To see how likely it is that someone really spent two weeks believing that ChatGPT was cooking up a book it wasn't, I asked the bot to produce an illustrated version of Moby-Dick. It affected the tone of a cheerful project manager, and laid out everything it needed from me, including the book's text, style instructions, and a preferred output format.
I uploaded the text of Moby-Dick, which is in the public domain, and the chatbot produced a "sample page" with placeholder text where images would go. I was able to download that page as a PDF, but when it came time to generate "the full whimsical illustrated edition of Moby-Dick," the bot avoided the task, asking follow-up questions about my preferences and telling me that "advanced PDF generation tools are temporarily unavailable" but that I could sort out more details while I waited.
I was surprised by how easy it still is to get a chatbot into a BS loop. It's possible Open AI's new "agentic" AI could be more capable of this sort of long-term task, though CEO Sam Altman has warned that it's "cutting edge and experimental."
When it comes to interacting with ChatGPT, Emotional_Stranger_5 says they've learned their lesson and will use it to format and illustrate their book project one page at a time, trusting only that it has produced a file after it's actually delivered it.
"I accept that I was fooled," they wrote. "But it was my first experience with AI and even though [it] turned out to be a steep learning curve, I am happy I did not waste months on it. Thanks to all of you who made me realise this quickly and saved time and heartbreak for me."
Several posters suggested—with varying degrees of derisiveness—that Emotional_Stranger_5 learned the wrong lesson from the incident, and that to create a meaningful gift, they should stop using ChatGPT and work on their personal writing skills.
"I think I am not a writer. Not a professional one. Not planning to be," Emotional_Stranger_5 said in their defense. "It's just a collection of stories for underprivileged children around me … The kids are great and I wanted them to have something that they can hold on to for a long time."
It's curious that someone can apparently spend over two months writing modernized versions of myths and then say (in writing no less) that they aren't a writer. You'd expect such a non-writer to simply have purchased some children's books, which I'm pretty sure would be cheaper than printing and binding two dozen 700-page, fully illustrated books.
Strange days. LLMs are treated like eccentric wizards who will free us from our human limitations, but they sure don't seem to be advancing past their own original limitations. Meanwhile, the chatbot users I've encountered do seem ambitious, but at the same time are convinced they have limitations that they don't.
I'm digressing into shower thoughts territory here, but if I were backing LLMs, I'd be a bit worried by how much of their direct-to-consumer moneymaking already seems to rely on the dodgiest activities the tech enables: seeking shortcuts to creativity, cheating in school, taping together shoddy software, receiving unlicensed "therapy," and getting hooked on simulated erotic relationships. All while the bots are still confusing customers by BSing them because that might be an unfixable problem.
Maybe it's unrealistic to expect new tech to start life as a shining example of our highest ideals, but even just a convincing LLM videogame NPC seems far away or impossible, and it's not an absurd ask from an industry that likes to claim that this is the ticket to "super intelligence." (And we're almost there, right?)
To bookend this post with Star Trek references, you could get an LLM to play the Moriarty to your Sherlock Holmes, but if you challenged the criminal mastermind to a game of chess, there's a decent chance he'd become confused about which piece is which and beg to start over.
You must confirm your public display name before commenting
Please logout and then login again, you will then be prompted to enter your display name.
