Capturing Emotional Reactions to Data Visualization
Background
There exists a technology that can infer a user’s emotion from their facial expressions which it detects from web cam input. By utilizing this tool, you can capture user’s emotion at any time. The idea proposed is to capture the user’s emotion as they see different data visualizations, but this could be applied to multiple situations. That data could be saved for that user and used in the future to show them things that evoke a certain emotion. Tailoring content based on a user’s reaction could be a very powerful tool for content creators. This would create a build-your-own adventure but done subconsciously. If the user isn’t aware that their facial emotions are being monitored, then there isn’t any bias in what emotion is recorded.
Executive Summary
I propose that we make a browser extension and API that utilizes the webcam technology. The browser extension could be a new extension or building off of Jordan Sechler’s research project. This would enable us to collect more information about visualizations for the logged in user. The data would be sent to a server to be stored in a database and then accessed by the API. The API could then be a way for permitted web pages to access this data about visualization and use that to customize their pages accordings to your likes/browsing habits. The API would allow requests for specific visualizations, certain kinds of visualizations, or for all visualizations. The data on the requested visualizations would be provided and the web site could use it as it pleases. There would be a way when visiting new sites for the user to approve them to use their visualization history and reactions. Researchers could ask users or groups of users for their data with a broadcasting permission function. Extra information could also be asked for and any information contained in the user profile could be used by researchers if given permission.
Viability Analysis
I believe that this project is relatively viable. With Jordan’s work, a lot is completed so that the database of visualizations doesn’t need to be created but instead added onto. The information about a person’s reaction would be fine to add onto the database for a certain visualization. The hard part is telling when the user is looking at the visualization or something else on the screen. Triggering the extension to capture the user’s current would need to be dependent if the visualization is what the user is looking at. I remember another senior design project creating eye tracking software which could tell where the user is looking. In combination with that, the program could then tell if the user is looking at the visualization. While in theory this could work, it seems like a lot of moving parts and could be pretty faulty.
Risks and Rewards
A huge risk here is the privacy concern. People don’t want information collected about them and they especially don’t want their webcam on all the time. This would be very hard to have people install on their own. In a research setting, it would be fine if the participants are voluntarily participating in some activity. The reward is that a lot of unbiased information could be captured using a system like this.
Closing
Having all this information for website creators and researchers would enable a lot of personalized pages and understanding of visualizations. It could be a new dimension in studying visualizations and web pages.
Print 
PDF