Project Description

Augmented Cognition is a field of study focused on improving memory and decision making by using computing to overcome limits, weaknesses, and biases in cognition. DARPA opened a large research project to investigate techniques that could be used to apply this ideal to the military.
VR Tracked Head-mount and GunHoneywell received a grant to study how biometric measurement and smart computing could improve the performance of a dismounted soldier in the field. The first phase of their trials sought to determine what information was gatherable with different types of sensing and show that there are situations where we could use this to get a performance improvement at a potential task. To simplify the experiment, it was to be conducted by Stage3 Research in our wireless virtual reality lab.

The experiment that was designed involved a participant needing to distinguish enemies from allies, shoot the enemies, and keep counts of both as well as additional friends and enemies reported to them over a radio. According to our collaborators in cognitive psychology, the shooting and the counting require separate parts of the brain that are capable of working at the same time. However, listing to the radio requires the same cognitive function as the counting of people you see and will make the two difficult if not impossible to do at the same time.

EEG SensorsIn addition to the VR glasses and gun required to perform the task in virtual reality, the participant would also wear a EEG for measuring brain activity and an ECG for measuring heart rate. Initial readings gathered during pilot trials of the task were used to develop software that could recognize in real-time based on these reading whether the person was engaged in the primary task of shooting and counting or not. We used this information to store any radio messages that occurred while they were busy and only played them once the participant was not busy identifying and shooting enemies. The VR simulation was split up into periods with enemies walking by and periods with no activity. Radio messages were scattered throughout both types of periods. At the end of each period we would ask participants to recall how many allies and enemies they'd seen and how many allies and enemies they heard about on the radio. For each participant we did three trials with radio messages played as soon as they appeared and three trials where the radio messages were deferred if the sensors believed they were busy fighting.

My Role

When I came onto this project, the task had been decided upon but not fully implemented. It was my job to take a prototype of the task and finish adding the scheduling and logging of all of the enemies, allies, and radio messages. I also added the ability to defer the radio messages, either during known activity periods (for the pilot) or based upon the output from the biometric filter.

I was also the sole lab technician. I had to set up and operate both the motion capture equipment and the biometric sensors. It was my job to recruit and schedule subjects. For each subject I needed to go through the experimental procedure including explaining the process to them; outfitting them with EEG, ECG, and VR equipment; training them on the task; and recording results during their experimental trials. I had to tabulate all the results and report back to Honeywell. Finally, I had to give part of the final presentation to our DARPA funders.