Detecting People’s Workload

Problem Description

We treat our computers as collaborators in our work, but they don’t reciprocate. They misinterpret our intentions, interrupt us when we’re focused, and completely ignore our emotional needs. While we’ve learned to live with computer annoyances, it turns out that these miscommunications not only hurt our work performance, but increase our stress and anxiety levels. How can we make a better, more personally attentive computer?

My research uses sensors to pay attention to your brain and body signals in order to understand when you are bored or overloaded or busy or highly-focused (for example). Then, the computer can respond in a way that better matches your natural expectations of etiquette or social norms.

The overarching goal of this project is to construct a biocybernetic loop – a system that collects sensor data, associates it with a user state, and signals an application of its state prediction. This system will eventually be used to create a new kind of collaborative computer application that will improve our decision making by delivering information at the moments that are best for us.

Project Goals

Example of a successful scenario: Someone puts on a heart-rate monitor and brain sensor. For 5 minutes, they calibrate the system. During calibration, they alternate between doing a very difficult task (high workload) and a very easy task (low workload). Using the sensor data, the system builds a model of high workload periods and low workload periods. After calibration is completed, when the user does ANY task (for example, takes an exam), they system continuously outputs predictions of whether they are experiencing high or low workload.

Important!: The goal of the project IS NOT to determine the best signal analysis or machine-learning algorithms. Instead, it should focus on building a system with flexible and extensible components – allowing a user to insert (or swap in/out) their own signal analysis or sensors or machine learning algorithms. Using a simple library for testing and demonstration is sufficient.

Constraints

Speed: the program must run quickly enough to be able to process incoming physiological sensor data and output a prediction in real-time.

Criteria

• Maximize flexibility for choosing different combinations of sensors to input into the system.
• Maximize flexibility in choosing signal analysis and machine learning (preprocessing) algorithms.
• Minimize the processing time of the system to allow for timely predictions.
• Provide meaningful output of the data collected. At a minimum, log files, but more expressive output (such as a visualization) is also possible.
• Communicate predictions to other applications (for example, through the network)

Resources

• Brain and body sensors in order to test the system. I will purchase and provide sensors for the team. This may include a brain sensing headband or a heart rate monitor, for example.
• Examples (possibly code) of other biocybernetic loops. There are often serious drawbacks to their approaches (there are many interesting design decisions!), but it may serve as a nice reference point. 
• Consultations with graduate students who are thinking about similar problems.

Licensing

• Open-source software under the MIT License. Preferably on Github.

Impact

The system will be used as the backbone for a series of research applications that will be built over the next couple of years. These applications will experiment with the idea of designing computers that deeply personalize our engagement with information, and even improve the decisions we make.

Point of Contact

Prof. Evan Peck, Department of Computer Science