This AI controlled Lego sorting machine is what my childhood dreams were made of

Daniel Wests Raspberry Pi powered AI Lego sorting machine
(Image credit: Daniel West)

Sick of sorting bricks? So was 25-year-old Daniel West, until his skill and determination brought him to build the incredible "Universal Lego Sorter". Made from 10,000+ Lego bricks itself, along with a bunch of other mechanical parts, the machine utilises deep learning AI to sort "any brick that has ever been produced" into one of 18 output buckets, at a rate of one brick every two seconds.

While it's not the first automated Lego sorter, or even the first to use AI, West's project is distinct in that the AI has been trained using 3D renders of Lego parts. So even if it's never seen a part in real life, it is still able to recognise and categorise it.

Here's how it works.

First, add Lego! The input bucket can hold a large volume of Lego ready to be fed into the mechanism. Once its been fed, the pieces are separated by a series of belts that push lines of parts onto the "vibration feeder." This segment prevents parts from sitting on top of one another, moving them steadily along to the next section.

Once separated, the parts are dropped one by one into the enclosed, brilliantly lit, white-bricked scanner section, where a Raspberry Pi Camera Module V2 records streaming video footage of each part. The footage is processed into a series of images by the Raspberry Pi 3 Model B+, and these images are finally relayed to a more powerful computer for categorisation. 

Okay, so this means the process is not fully powered by the Raspberry Pi, but you try running a neural network on Raspberry Pi. Chatting to PC Gamer, West admitted the use of an RTX 2060 laptop was a little overkill and noted "it would probably be possible to use something like the Jetson Nano to run the machine entirely self-contained."

The AI algorithm we're talking about is the go-to for sorting machines. It utilises a convolutional neural network, basically meaning it's a super powerful tool for the recognition and classification of objects. And having been trained with 3D models of Lego bricks, as long as there exists a 3D model of the brick it's being shown, it should be able to classify it. No problem. 

Just don't go feeding it lint balls, chewing gum wrappers, or more of the usual crap you find in a Lego hoard—who knows how it would react.

West goes into more detail here about the process of teaching the AI.

The RC buggy motor used for the vibration feeder was probably to most awkward part to get hold of, West tells us, everything else was pretty commonly available. And it sure looks like he had no shortage of Lego parts lying around to build with.

Virtual reality

(Image credit: Valve)

Best VR headset: which kit should you choose?
Best graphics card: you need serious GPU power for VR
Best gaming laptop: don't get tied to your desktop in VR

The project took around three years to finalise, and wasn't without its hurdles. West recalls the "reality gap" issue that surrounds using synthetic, fake data such as 3D models to train machines. "It ends up performing terribly in real world conditions, much worse than you'd expect," he explains, "it took years of failed experiments before I figured out how to crack it." 

He admits he was even ready to abandon the project at one point, but we're all really glad he didn't. Just goes to show where a little perseverance can get you.

Next on the agenda for West, is "more sorter-related work I can't talk about (yet)," and its likely he'll be stepping back into the game development fray soon, too. In fact, we've written about West's game development exploits previously on PC Gamer, so take a look if you want more of a peek into the eclectic mind we're dealing with here.

As always, keep building.

Katie Wickens
Hardware Writer

Screw sports, Katie would rather watch Intel, AMD and Nvidia go at it. Having been obsessed with computers and graphics for three long decades, she took Game Art and Design up to Masters level at uni, and has been demystifying tech and science—rather sarcastically—for three years since. She can be found admiring AI advancements, scrambling for scintillating Raspberry Pi projects, preaching cybersecurity awareness, sighing over semiconductors, and gawping at the latest GPU upgrades. She's been heading the PCG Steam Deck content hike, while waiting patiently for her chance to upload her consciousness into the cloud.