December 12, 2018
991 words, 5 min read

Introduction

After I had visited with the VR startup, StoryUP, I was inspired to try my hand at developing VR applications that utilized the brain as an input. While we had spent our first 8 weeks of our Computer Graphics course studying OpenGL, we had decided to spend our second 8 weeks studying and practicing VR development, and working with EEG devices for gameplay. We wanted to do desktop VR development instead of mobile, since we believed that not only would it be more challenging, but we wanted to be able to work with 6DoF instead of 3DoF. To this effect we layed out a plan to develop 3 games.

The first game, ZombieDefender, was an arena style game that infinitely spawned zombies to be slain. This project helped us with coming to grips with VR mechanics, and getting familiar with developing for Oculus Rift using the SteamVR plugin. Our second project, Mystic Arts 101, focused on learning to work with the EEG headband made by Neurosky called the Mindwave. Mystic Arts 101 was a simple game where the user would focus to levitate a ball, then using the VR controllers, they could draw the ball towards or away from themselves to guide the ball into the goal. Finally, we pulled both of these projects together to build our third and final project, WayFinder, which is a VR escape the room style of game that uses EEG to assist in solving puzzles.

The Deets

WayFinder consists of the following mechanics: instant teleportation locomotion, grabbing, throwing, and measuring levels of attention. There are 6 scenes total for WayFinder. A title screen, a starting room, 3 puzzles, and the end room.

The user will begin at the title screen, surrounded by a mountainous region, created using Unity’s terrain modeling feature. Pressing the trigger will begin the game. In the start scene, the player will be able to move towards one of three teleport pads, which will then take them to one of the puzzles.

The first puzzle room was developed by Richard and is designed to make a candle glow brighter the more you concentrate. Once the concentration levels are high enough, a trigger activates that allows you to collect a crystal. You have to collect the correct crystal in order to activate the teleport pad to go to the next room.

The second puzzle was made by Kyle. It features posters that have medals hidden behind them. There are video game riddles around the room, and you must solve the riddle and select the associated poster. The player must also focus in order to make the posters reveal their medals to be grabbed. Get all three riddles correct without messing up and you’ll be able to continue onto the next level. If you do mess up, all the posters reset.

The final puzzle, was made by myself. It has moving targets and javelins in the room. The player must hit all three targets to activate the teleport pad. This can be done without the EEG, but raising your level of concentration will slow down the moving targets and make them easier to hit.

Finally, there is the end scene. The end scene is much like the start scene, you find yourself outside of the maze, back in the mountainous region, with a sunset on the horizon. There is a teleport pad that will take you back to the start scene.

The Specs

Our setup included an Oculus Rift, Neurosky Mindwave Mobile, a generic Bluetooth adapter, AMD FX-8300, 8GB Radeon R9-390 GPU, 8GB DDR3 RAM, and Windows 10 Home Edition. The assets we used for the rooms all came from BitGem. This includes the textures for the walls and floor, the pillars, arches, plinth, and candles. We used Unity 2018.2.12 for development in conjunction with the SteamVR Plugin and Neurosky ThinkGear Connector. The SteamVR Plugin gave us the library for accessing the button inputs of VR controllers and came with prefabs for handling our VR camera. The ThinkGear Connector controls the socket for our Bluetooth connection to the EEG headband to fetch its attention/meditation values. The models for the teleport pad, javelins, and javelin stand were created by myself using TinkerCAD.

Final Remarks

While the potential for VR and EEG being used together is pretty great, I don’t believe that the Mindwave is suitable for the endeavor, or at least not when utilising the ThinkGear Connector. While our games did work, the framerates would drop drastically every time we turned Neurosky’s ThinkGear Connector on, to the point that the game would give the unprepared soul motion sickness. The connector would either need to be heavily modified to not be so resource intensive, or a new EEG headband would be recommended. From my visit with StoryUP, I’ve found that the Muse is a good alternative, and has a sleeker form factor. The Neurosky Mindwave consists of a single EEG sensor and an ear clip that is used to drown out white noise and costed me $85. The Muse doesn’t have the awkward ear clip, consists of 5 EEG sensors, but costs $200.

I can see EEG being used in VR more in the future, and perhaps even being built into the headsets. As we’ve shown, the extra input can be utilized for controlling object transparency, brightness, and movement. Another idea we had was to use it to convert text of one language into another. It would also be interesting to experiment with altering game mechanics dependent on measured stress/relaxation levels or attempting to predict a user's next action to activate some sort of assistance mechanism (like SAO's sword skill system). I’m certain there are many applications that can be spawned from this technology and I'm excited to see what the future yields.

The code for this project, as well as the previous two projects, can be found on GitHub.
Repository for the CS375 Fall 2018 group: github.com/RedHawkVR
Repository for the WayFinder final project: github.com/RedHawkVR/WayFinder