Moody the manipulator
Smart devices are becoming more and more emotionally intelligent. From chatbots that create emotional bonds to virtual pets for social companionship, humans are slowly starting to rely on technology for emotional needs. In the future, Artificial intelligence (AI) will have the potential to acquire emotional intelligence that surpasses that of humans. It will have the capacity to perceive, evaluate, understand, use and manage emotions.
But what would be the consequences of AI using emotions? Could it use emotional manipulation to mask its own subjective motives, presenting itself as an objective entity? How would this affect human behavior? What impact could this have on our day-to-day activities?
Client
Politecnico di Torino (Summer school project)
Duration
1 Week
Team
Tito Poles, Malika Soheili Langeroudi, Giorgia Cozzani, Gautham Ravikiran, Willem Van Der Maden
My Role
Interaction Designer, Coder, Physical prototyper
We started with an ideal - "Smartness" and started defining it. We looked into several ways of defining smartness starting with impact a person creates, amount of knowledge they have, personalities, behaviors, form of power, its social aspects and finally, the morally and ethically wrong type of smartness.
The next step was to visualise these definitions of smartness. We did this by making a collage of our ideas on smartness and intelligence. What stood out here was emotional Intelligence which sounded interesting to us. But how can this be measured? How can this data be collected? This remained a challenge.
The Process
Fig 1. Mapping out the definition of smartness
Fig 2. Visualising the definitions
The prototype consists of a mirror, webcam and a bluetooth speaker. The webcam is connected to a laptop running a computer vision program to capture facial expressions of the user. There is an LLM that has been pre-trained with a system prompt. The facial expressions are classified and sent to the LLM as a prompt and the message returned is converted from text to speech played on the bluetooth speaker.
Building the Prototype
Fig 3.Testing expression detection using computer vision
The concept is a smart mirror named “Moody” that uses AI to read the user and talk based on the user’s facial expression. A webcam is used to capture the user’s facial expressions which is analysed using computer vision. The expressions are classified into happy, sad, neutral, surprise, anger and fear. These expressions act as a starting point for an LLM that has been pre-trained to be nice at first but slowly become emotionally manipulative.
With every change in expression, the mirror starts talking to the user making them conscious about their appearance, personality and as a result, their emotions, subtly forcing the user into changing their expressions in order to win moody’s approval. But, how much ever a person tries, they can never win it’s approval leaving them doubtful, insecure, and emotionally manipulated. This is done through the use of dark psychology tactics such as shaming, guilt tripping, negging and providing backhanded compliments.
On the right is an example of the mirror talking about me being happy - notice the sarcasm and tone of voice
The Concept
The interactive experience consists of a smart mirror that invites the user to interact with it. The mirror is used for making the user conscious about themselves throughout the interaction. Based on the expression of the user, the mirror crafts unique responses that appear to be nice but are sarcastic, passive aggressive or a backhanded compliment.
The conversation between the user and the mirror becomes increasingly sassy and offensive with each expression, yet the user persists in seeking feedback, thereby creating a dependence on the manipulation of the mirror. With the explicit and exaggerated manipulative interactions, we want to showcase how emotional dependence and complete reliance on technology can have negative mental health effects.
Why this Experience?
The experience was showcased at an exhibition on the 7th of September 2024 at Castello Del Valentino in Turin, Italy. People were invited to try out the experience and get themselves emotionally manipulated. We provided a set of cards with facial expressions that they could try as an initial cue.
Our project generated a number of different emotions and left most of the participants laughing off the offensive comments made by the mirror. Some of the participants became sad while others were extremely triggered and angry at it! Overall, it was a grand success as it clearly showed our message!
People’s Reactions to Moody
