Face the Wind

This small project was made for the Tangible and Embodied Interactions course at WPI in spring of 2021. It invites the user to look around a "mirror" screen with pinwheels set in it and blow at them (or make any sound picked up by the microphone) to make them spin.

The Unity project uses the ML-Agents plugin to figure out which way the on-screen face is looking. Training was performed using behavioral cloning, GAIL, and good ol' reinforcement learning.

Make sure you have both your webcam and microphone on in order to use it!

Since it's just a simple prototype, please use Alt+F4 to close it when you're done.

SEE IT IN ACTION: 

USER EVALUATIONS: Evaluations were performed by providing users with the project and asking them to open it and share their their screens via Discord. Users were instructed to continue humming a low note while looking at each of the 6 pinwheels for 10-second intervals each, with the order of the pinwheels randomized for each testing round. Users were told to take a 2-second breath while moving their head to face the next pinwheel after each 10-second interval. Each user performed 4 rounds of testing (~5 minutes in total) as the evaluator timed the number of seconds where the program had the correct pinwheel activated.

4 users (outside of the creator) tested the program with mixed levels of success. Correct pinwheel activation ranged from 15% to 70%. Since training was performed with a Caucasian male with facial hair, the program was found to be less likely to work correctly with a great majority of the population, including, but not limited to, persons of African, Indian, and Middle Eastern descent, as well as feminine-presenting persons. Unfortunately, this is an artifact of the COVID-19 pandemic and the resulting inability to train the project with a wider amount of people due to social distancing restrictions. Additionally, higher levels of detail in the person's background environment were seen to affect the evaluation results negatively.