EMOTIONAL SURVEILLANCE AND THE CITY
MARK ANDREJEVIC
The spaces we move through are becoming increasingly “sensorised” to keep track of our movements, our actions, and even our moods. The model for this transformation of our physical environment is a familiar one: the internet. It has become a matter of common knowledge that online “spaces” are supported by an economic model based on comprehensive data collection. Google can sort through every search term we’ve ever entered into its browser; Facebook tracks our every action online—even collecting information about links we may have hovered over but not clicked on (Grothaus, 2018). The goal of so-called “augmented” reality is to provide an interactive overlay for physical space that does the same thing: promising added convenience while simultaneously tracking our movements, activities, and interactions. Accessed by a portable device or even interactive glasses (like Google’s ill-fated “Glass”), augmented reality would allow us to see maps, special offers, and even personalised messages superimposed on physical reality. A glance at a local restaurant through our phone camera or our augmented reality headset might provide us with information about the day’s specials, a full menu, reviews, and instructions for making an online reservation. At the same time, the platform would track detailed information about our movements through the course of the day, the food we ordered, and more.
While augmented reality is still a long way off—not least because it seems unlikely everyone will want to wear interactive glasses—there have been a number of recent initiatives to render physical space more interactive. These range from being anonymous to being able to fully identify us: sensors that count the number of people who move through a space, doors that open in response to our approach, cameras that collect our license plate number as we drive through the city, and, more recently, facial recognition cameras that capture our images and match them to identifying information. Some of these technologies are being linked together—such as an automated door for a convenience store that will only open once it has scanned the customer’s face to see whether it matches a database of known shoplifters. If the system finds a match, the automatic door will remain locked (Hellman, 2019).
The response to the pandemic has resulted in a range of “smart” sensors designed to detect symptoms of illness (remote body temperature monitoring), to enforce quarantine, and even to determine whether people are social distancing or wearing their face masks properly (NEC, 2020).
Perhaps not surprisingly, one of the frontiers of remote monitoring—both online and off—is that of “sentiment analysis”—or emotion detection: the ability to “read” your mood from your face, your posture, and your movements. Marketers are, of course, interested in this information because of the way it might help influence sales—but employers, security officials, and others have noted the potential benefits of being able to track people’s moods. Perhaps, for example, mood might be correlated with someone’s propensity to engage in a criminal act or start a fight. Alternatively, mood monitoring might be used as a management tool to increase worker productivity (and lower costs). As one Human Resources newsletter puts it, mood tracking systems, “allow employees to log their daily states of mind to help reduce stress — which in turn helps employers create a more resilient workforce” (Shatte, 2018). The response to stress detection can even be automated, thanks to one company that connects stressed-out workers with a “chatbot” to lead them through meditation exercises.
The focus on mood is perhaps not surprising, given the new technological capabilities of sensors, and the attempt to “cut through the information clutter”. Advertisers who want to make their messaging count are lured by the promise of being able to read customers’ responses—and to influence it. Security officials view mood detection as a way of seeing beyond the “outward appearance” to an underlying state that might reflect an intention to do harm. In the era of social media and information overload we are prone to imagine that the content of information does not matter as much as the emotional response it triggers. What counts, in the model, is not what we know but what we feel—and the promise of the emerging sensor technology is to track, measure, and monitor these feelings.
Surely there are potential benefits to mood recognition technology. We might welcome Spotify automatically playing music that fits how we are feeling, for example. As in the case of the rest of the online economy, however, the development of mood monitoring platforms promises places detailed information about our emotional states (and the way these influence our behaviour) in centralized databases that are non-transparent to users who will have no idea how the information is used: whether to deny them a job, serve them a manipulative advertisement, deny them access, or even to detain and interrogate them. We therefore need to ask the same question about these inferential infrastructures as we do about other forms of data collection: who is collecting this data, and for what purposes? We need to decide in advance which purposes we deem acceptable, and which should be limited or banned outright. As the sensors around us continue to develop, the time to ask these questions is now.
References:
Grothaus, Michael (2018). Facebook confirms it tracks your mouse movements on
the screen. FastCompany, June 13. https://www.fastcompany.com/40584539/facebook-confirms-it-tracks-your-mouse-movements-on-the-screen.
Hellmann, Melissa (2019). When convenience meets surveillance: AI at the corner
store. The Seattle Times, June 30. Available online at: https://www.seattletimes.com/business/technology/when-convenience-meets-surveillance-ai-at-the-corner-store/
NEC (2020) A futuristic workplace, reimagined by NEC I:Delight. NEC.
https://www.nec.com/en/global/delight/ (accessed 25 August, 2020).
Shatte, Andrew (2018). Forget employee fitness logging: Mood tracking is the latest
HR tech trend. MeQuilibrium: Employee Benefit News, October 16.
https://www.mequilibrium.com/resources/forget-employee-fitness-logging-mood-tracking-is-the-latest-hr-tech-trend/