These scripts use AI to turn Minecraft worlds into adorable picturesque scenes

Audio player loading…

It was only a matter of time before someone combined Stable Diffusion AI image generator and Minecraft command-line. This is probably one of the best uses of AI image generators I've seen so far, essentially using a combination of Minecraft screenshots and text prompts to create highly detailed images each time the user makes changes to their design within the Minecraft game world.

Sean Simon, more commonly known as ThoseSixFaces (or Lozmosis on Reddit (opens in new tab)), posted the above video yesterday showing off a sped-up version of their scripts running live as they messed around in-game. 

It works by taking screenshots from a specific angle as the game is played, resizing or cropping them to 512x512 pixels and saving them to a folder. Each frame is generated on "a fixed interval, or whenever the player places/removes a block." Using a version of InvokeAI (opens in new tab) that Sean modified, the model then goes about "targeting the latest image in the folder and outputting the Stable Diffusion image." Another script then displays the output image using tkinter (opens in new tab)

Stable Diffusion (opens in new tab) is the main thing translating each frame into gorgeous AI generated images, however, which Sean explains "denoises on a random value between 0.47-0.5." So although Sean is able to use prompts to push the AI in the preferred direction using image2image (opens in new tab), there's still a little leeway in how random the generated results can be. 

As it stands right now, the process involves running several scripts, though Sean is "keen to compress this all into a single script."

Your next upgrade

(Image credit: Future)

Best CPU for gaming (opens in new tab): The top chips from Intel and AMD
Best gaming motherboard (opens in new tab): The right boards
Best graphics card (opens in new tab): Your perfect pixel-pusher awaits
Best SSD for gaming (opens in new tab): Get into the game ahead of the rest

It's a fascinating project, and one my art teachers from college would be very excited to see working; they seemed to have an obsession with translating work into different mediums, then translating it back. On that note, one Reddit commenter jokingly suggests a reverse process to Sean, in which the AI would "turn arts into minecrafts."

Sean responds with, "I've started cooking something up," so it may not be too long before we see Minecraft worlds being generated using AI image prompts. Or indeed Minecraft worlds being translated into images, then back into Minecraft worlds. 

Whew, what a world we live in.

Sadly, Sean's scripts are not yet available to download from Github, but do keep an eye on their Twitter (opens in new tab) for updates.

Now, we've seen Stable Diffusion being used in VR (opens in new tab), and the CEO of Discord image bot Midjourney speculates that future consoles will have a "giant AI chip and all the games will be dreams (opens in new tab)." This Minecraft project is yet another addition to the exciting, AI-focussed narrative springing up in the gaming sphere right now.

Katie Wickens
Hardware Writer

Screw sports, Katie would rather watch Intel, AMD and Nvidia go at it. Having been obsessed with computers and graphics for three long decades, she took Game Art and Design up to Masters level at uni, and has been demystifying tech and science—rather sarcastically—for two years since. She can be found admiring AI advancements, scrambling for scintillating Raspberry Pi projects, preaching cybersecurity awareness, sighing over semiconductors, and gawping at the latest GPU upgrades. She's been heading the PCG Steam Deck content hike, while waiting patiently for her chance to upload her consciousness into the cloud.