These scripts use AI to turn Minecraft worlds into adorable picturesque scenes

It was only a matter of time before someone combined Stable Diffusion AI image generator and Minecraft command-line. This is probably one of the best uses of AI image generators I've seen so far, essentially using a combination of Minecraft screenshots and text prompts to create highly detailed images each time the user makes changes to their design within the Minecraft game world.

Sean Simon, more commonly known as ThoseSixFaces (or Lozmosis on Reddit), posted the above video yesterday showing off a sped-up version of their scripts running live as they messed around in-game. 

It works by taking screenshots from a specific angle as the game is played, resizing or cropping them to 512x512 pixels and saving them to a folder. Each frame is generated on "a fixed interval, or whenever the player places/removes a block." Using a version of InvokeAI that Sean modified, the model then goes about "targeting the latest image in the folder and outputting the Stable Diffusion image." Another script then displays the output image using tkinter

Stable Diffusion is the main thing translating each frame into gorgeous AI generated images, however, which Sean explains "denoises on a random value between 0.47-0.5." So although Sean is able to use prompts to push the AI in the preferred direction using image2image, there's still a little leeway in how random the generated results can be. 

As it stands right now, the process involves running several scripts, though Sean is "keen to compress this all into a single script."

Your next upgrade

(Image credit: Future)

Best CPU for gaming: The top chips from Intel and AMD
Best gaming motherboard: The right boards
Best graphics card: Your perfect pixel-pusher awaits
Best SSD for gaming: Get into the game ahead of the rest

It's a fascinating project, and one my art teachers from college would be very excited to see working; they seemed to have an obsession with translating work into different mediums, then translating it back. On that note, one Reddit commenter jokingly suggests a reverse process to Sean, in which the AI would "turn arts into minecrafts."

Sean responds with, "I've started cooking something up," so it may not be too long before we see Minecraft worlds being generated using AI image prompts. Or indeed Minecraft worlds being translated into images, then back into Minecraft worlds. 

Whew, what a world we live in.

Sadly, Sean's scripts are not yet available to download from Github, but do keep an eye on their Twitter for updates.

Now, we've seen Stable Diffusion being used in VR, and the CEO of Discord image bot Midjourney speculates that future consoles will have a "giant AI chip and all the games will be dreams." This Minecraft project is yet another addition to the exciting, AI-focussed narrative springing up in the gaming sphere right now.


Minecraft potions: Recipes and brewing guide
Minecraft enchantments: Magical reference list
Minecraft villagers: All the jobs and trades
Minecraft realms: How to start your own server

Katie Wickens
Hardware Writer

Screw sports, Katie would rather watch Intel, AMD and Nvidia go at it. Having been obsessed with computers and graphics for three long decades, she took Game Art and Design up to Masters level at uni, and has been demystifying tech and science—rather sarcastically—for three years since. She can be found admiring AI advancements, scrambling for scintillating Raspberry Pi projects, preaching cybersecurity awareness, sighing over semiconductors, and gawping at the latest GPU upgrades. She's been heading the PCG Steam Deck content hike, while waiting patiently for her chance to upload her consciousness into the cloud.