This algorithm picks out Steam's best hidden gems

Wuppo, the most overlooked "hidden gem" on Steam.

It's easy to get lost in a crowd of 15,000. That's life on Steam in 2017, with around 500 new games streaming in every month. These days, I tend to view a game's review count as a vague barometer of its success. Lot of reviews? It probably has a decent number of owners! One or two reviews? Well, that's when I start to picture the poor wallflower in the corner of the party.

You can see how this problem might snowball: I don't know exactly how Steam's visibility algorithms work, but the more reviews a game has the more attention it likely gets, thus garnering more reviews, while overlooked games remain overlooked. And that's why I'm especially interested in this algorithm, posted on Github, that ferrets out Steam's "hidden gems" using a bit of smart math.

The code here is written by NeoGAF member Wok, who describes it like so: "the score of a game is defined as the product of a quality measure (its Wilson score) and a decreasing function of a popularity measure (its players total forever). The quality measure comes from SteamDB and the popularity measure comes from SteamSpy API."

Let's break that down a bit. The Wilson Score is a statistical calculation that gives a lower bound of the confidence interval, so for example, "We are 95 percent confident that the actual score is at least equal to the Wilson Score." The more ratings a game has, the narrower the confidence interval becomes. Or in English, if a game only has a few ratings, even if they're all positive, we can't really trust that number. So 100 percent positive with only five ratings would be a Wilson score of 0.5655, roughly the same as a game with 1000 ratings but only 60 percent positive, or 57 percent positive out of 10,000 ratings. The more ratings a game has, the more 'accurate' the overall Steam score becomes, and the closer it is to the Wilson Score.

But what about that 'hidden' part? This is the opposite of the Wilson Score in a sense, where the more players a game has the less hidden it becomes. Wok investigated a few options for how to best find hidden games, eventually settling on a value of pow(10, 6.45), or about 2.8 million for alpha, with the popularity score being alpha / (alpha + total_players). So with only a few thousand players, the popularity score will be closer to 1.0, while 2.8 million or more players will result in a score of 0.5 or lower. Thus a game with tons and tons of positive reviews like, say, Half-Life 2, doesn't make it anywhere near the top of the list. 

Caves of Qud, a roguelike that deserves more love, according to the hidden gems algorithm.

There's one pretty obvious issue with this way of finding "hidden gems" on Steam, and that's the "gem" bit. Wok's code relies on the quality rating for a game pulled from SteamDB, which in turn comes from the ratio of positive-to-negative reviews on Steam. If a game has no reviews, it's not going to make the list. So Steam's most hidden games will remain hidden; it's up to word-of-mouth to spread the love for those. There's another great NeoGAF thread for just that, with hundreds of posts linking to some games you've surely heard of and plenty I guarantee you haven't, most of which are on sale this week.

Still, Wok's algorithm turns up an interesting collection of games. The top score goes to a 2D action-RPG (at least, I think that's what it is) called Wuppo, released last September. Others include visual novel The House in Fata Morgana, adventure game Rakuen, which I've heard some buzz about, and retro-looking roguelike Caves of Qud which I love for its title font alone.

Compared to Steam's most ignored games, these games are all basking in the glory of dozens or hundreds of reviews. But in a crowd of 15,000, that still safely qualifies as overlooked. Another Steam curator took a similar approach and put together his own list of hidden gems, and you can see the top 250 here. You'll see a lot of crossover with Wok's list—this one also uses a Wilson score—but some changes in parameters put games at pretty different spots in the ranking.

Wok actually applied his algorithm to every single game on Steam (although there's a discrepancy of a couple thousand games, which we can maybe chalk up to DLC or expansions). Obviously, the further you scroll, the less the metric applies. Games near the bottom of the list aren't necessarily bad; they simply don't have enough reviews—or too many reviews—to yield a higher score.

If you're interested in looking into some of Steam's top-rated, least-played games, here's the top 100 from Wok's list. And if you're mathematically minded, check out the entry on github to dive into the code.

  1. Wuppo
  2. TY the Tasmanian Tiger
  3. The House in Fata Morgana
  4. Hidden Folks
  5. Everyday Genius: SquareLogic
  6. fault - milestone two side:above
  7. Castle Must Be Mine
  8. STEINS;GATE
  9. planetarian HD
  10. OneShot
  11. tayutama 2 -you're the only one-
  12. Glass Masquerade
  13. Paradigm
  14. CLANNAD
  15. planetarian ~the reverie of a little planet~
  16. Angels with Scaly Wings
  17. Lazerbait
  18. Rakuen
  19. Caves of Qud
  20. Slayaway Camp
  21. Paint it Back
  22. Contradiction - the all-video murder mystery adventure
  23. Voodoo Vince: Remastered
  24. Choice of Robots
  25. Umineko
  26. CrossCode
  27. Higurashi When They Cry Hou - Ch.2 Watanagashi
  28. Smashbox Arena
  29. Racket: Nx
  30. Highway Blossoms
  31. Blackwell Epiphany
  32. Zombie Estate 2
  33. NEKO-NIN exHeart
  34. The Room Two
  35. Doom & Destiny
  36. The Legend of Heroes: Trails in the Sky SC
  37. Primordia
  38. Rabi-Ribi
  39. Escape Goat
  40. Hustle Cat
  41. klocki
  42. NEKOPARA Vol. 3
  43. Redirection
  44. Princess Remedy in a World of Hurt
  45. Sally Face
  46. Orbox C
  47. Orbt XL
  48. Megadimension Neptunia VII
  49. BLUE REVOLVER
  50. Flowers -Le volume sur printemps-
  51. Blockwick 2
  52. Dungeon Warfare
  53. Kindergarten
  54. Quell
  55. Mini Ghost
  56. Green Elephant 2D
  57. The Tale of a Common Man
  58. Harmonia
  59. Linelight
  60. Mushihimesama
  61. Escape Goat 2
  62. Airships: Conquer the Skies
  63. Pajama Sam in No Need to Hide When It's Dark Outside
  64. Monolith
  65. Master of Orion 2
  66. DoDonPachi Resurrection
  67. Tomoyo After ~It's a Wonderful Life~ English Edition
  68. IronWolf VR
  69. The 9th Day
  70. SHENZHEN I/O
  71. Deathsmiles
  72. WILL: A Wonderful World
  73. LiEat
  74. Danganronpa 2: Goodbye Despair
  75. Life Goes On
  76. Muv-Luv
  77. Narcissu 10th Anniversary Anthology Project
  78. Zenge
  79. Kingdom Rush Frontiers
  80. Hidden in Plain Sight
  81. ACE Academy
  82. Enigmatis 2: The Mists of Ravenwood
  83. Waltz of the Wizard
  84. UnReal World
  85. Lucy -The Eternity She Wished For-
  86. G-senjou no Maou - The Devil on G-String
  87. Impossible Creatures
  88. SEUM: Speedrunners from Hell
  89. Tetrobot and Co.
  90. Pajama Sam 3: You Are What You Eat From Your Head To Your Feet
  91. Higurashi When They Cry Hou - Ch.3 Tatarigoroshi
  92. Emily is Away Too
  93. VA-11 Hall-A: Cyberpunk Bartender Action
  94. Hexcells Infinite
  95. Climbey
  96. TheWaveVR
  97. What Remains of Edith Finch
  98. Geneshift
  99. Kindred Spirits on the Roof
  100. Supercharged Robot VULKAISER