Or, you can try to fudge it. That's what YouTuber LowSpecGamer did, setting out to play some of the best VR games for as little money as he could. For starters, he swerved the Rift and the HTC Vive to pick up a refurbished Windows Mixed Reality headset for $160. Then, he assembled his PC, combining parts he'd picked up for cheap with some that viewers had donated. At its core was an i5-2500K, and it had 8GB of RAM.
The graphics card was where it got really interesting. He knew he could get games working with the GTX 1050 Ti he had in his laptop, so he dialed it all the way back to the GT 1030, the least powerful card in NVidia's GeForce 10 series.
By playing around with Windows settings, he managed to get Skyrim VR running with the system. I mean, it doesn't look great, but it's running nonetheless.
I'd recommend watching the whole video, in which he tries other games—including Superhot VR and Fallout 4 VR—with a variety of graphics cards. It didn't make me want to rush out and immediately recreate his setup, but it's interesting to see how far a limited budget will take you.