Arvutigraafika projekt - Kursused - Arvutiteaduse instituut

White Noise Environment in VR

Kristjan Kitse

This project is part of my bachelor thesis to make a VR environment where it's possible to make people hallucinate. For this I'm using Unreal Game Engine 4.

Project plan

I plan to construct a simple VR environment where different everyday objects can be hidden in plain sight, so the player has to look for objects by moving around slightly. On some occasions, there actually will be no object present, but the environment should make the player wrongly think (hallucinate) that they see some object anyway.

For this it's needed to:

Create a noisy shader for VR that is static when the player is not moving, but dynamic when the player is moving (or vice versa)
Allow for different levels of randomness in the shader (for finding the threshold where it's easiest to induce hallucinations in people, this could be either very small noise or big "ink blobs")
Make objects appear gradually (either by changing their opacity, level of randomness in the shader or something else completely).
Recognise the moment when player sees some object.
Create basic UI.

The inspiration behind the experience is the fact that human brains are excellent at finding patterns. A nice example is hearing the song "White Christmas" hidden in random white noise when instructed to do so, even though no song is actually present. Can we do the same with visual stereoscopic perception?

Foreseen difficulties:

This will be my first time using Unreal Engine, which might cause some difficulties in the development process.
Find some useful information from tracking the player's headset. (Recognise if test subject sees something)

Milestone 1 (09.03)

Learn the ins and outs of Unreal Game Engine.
Start reading "Unreal Engine VR Cookbook: Developing Virtual Reality with UE4".
Create initial test environment.

Attach:material1.mp4

Attach:matvid2.mp4

You can add development notes here, or remarks on the progress / result. Screenshots and videos are always good!

Milestone 2 (23.03)

Experiment with different noise algorithms and find out what works best
Read Chapter 8 (Motion Controller Interaction) from "Unreal Engine VR Cookbook: Developing Virtual Reality with UE4"
~~Develop initial functionality for using hand controllers.~~

Milestone 3 (06.04)

Develop initial functionality for using hand controllers.
Watch Unreal Game Engine live training video https://www.youtube.com/watch?time_continue=79&v=CWzPP5FAYAg and try to experiment with this Volume Modelling Plugin.

I tried to get to know the fundamentals of using hand controllers and created this simple blueprint that basically stores the number of clicks the subject does with his/her hand controller while in the game.

I also tried out the plugin that enables to create cool foggy environments, but unfortunately when going inside the fog, everything gets dark and also when I was testing it in VR, it got very laggy.

Attach:fog.mp4

Milestone 4 (20.04)

Develop initial scenario, where different objects come towards the player and player has to identify them as soon as he can and push the trigger on hand controller.

I developed first very basic scenario where experiment supervisor can send 3 different objects (chair, rock, table) towards the subject. Objects are slowly rotating at the same time, so it would be easier to recognise them.

Keyboard keys 1, 2 and 3 are used for moving the objects.

Timeline that controlls the movement of chair.

Supervisor can adjust the brightness of the environment with up and down keys.

m and n keys are used for temporarily hiding the objects and for reseting the position of the object.

when subject presses the trigger of supervisor presses the left mouse button, then the distance between the player and the closest object is displayed.

Milestone 5 (04.05)

More objects
Save the data
Automatic object appearance

With the help of Madis Vasser, 8 new objects were created in Blender. All of the objects are type of chairs that test subject has to recognize and describe.

Also, a very basic way of saving data was implemented with the help of "Victory Plugin".

Basically, the game asks test subjects name in the beginning and when test subject completes the given scenario then program outputs simple text file named after the player with time spent doing the experiment.

The scenario changed also. Objects are not coming towards the player anymore. Instead, objects are just appearing in the close proximity of the player one by one.

Attach:testvid.mp4