Periodic Reporting for period 5 - EMTECH (Emotional Machines:The Technological Transformation of Intimacy in Japan)
Reporting period: 2023-08-01 to 2024-07-31
EMTECH as an interdisciplinary collaboration between the humanities (cultural and literary studies, Japanese studies), social sciences (anthropology), the affective sciences as well as the “hard” sciences (robotics, affective computing) examined the relationship between aesthetic works on human–machine interaction (HRI) in Japan and advances in artificial emotional intelligence. The project's ultimate aim was to examine how humans create affective bonds with emotionally intelligent machines and determine the impact this has on social issues in private and public spheres in everyday life in Japan. EMTECH produced empirical evidence of how software that has the capacity to read affective feedback in humans is not only reproducing traditional Japanese culture but in fact transforms it by offering novel strategies for coping with stress, loneliness, and the loss of traditional forms of intimacy. By monitoring the effects of the recent introduction of machines equipped with emotionally sensitive software into Japanese homes, EMTECH documented the technological transformation of intimacy within sites such as the everyday domestic spaces of the new Japanese family. Overall, EMTECH has examined the feedback loops of culture, new technologies and society. An important contribution by EMTECH was its initiation of a closer cooperation between European and Japanese research communities. On EMTECH's three international conferences the project gathered researchers, roboticists and artists and initiated fruitful dialogues about the influence of HRI upon society and roboticists’ responsibility for their own inventions.
(A) To investigate novel depictions of HRI in Japanese culture, EMTECH collected and compiled an extensive body of literary texts, anime, manga and games released since the 1980s. Three major narratives were identified: machines as partner substitutes, friends and helpers; machines destroying mankind and taking power; and finally, humans and machines coexisting or peacefully merging. In Section A, EMTECH also used affect-sensing devices to gain insight into the emotional status (affects and feelings) of test subjects. EMTECH has also focused on interviews with roboticists and tech companies, to ask directly which narratives inspired them to build emotionally intelligent machines.
(B) EMTECH further conducted an extensive anthropological study of Japanese romance gaming culture (digital boyfriends, holographic spouses, AR/VR partners) and of the spread of affective devices in Japan (matrimonial intercession robots, match-making software). During fieldwork, EMTECH 1. established contact with romance device users and became familiar with their habits; 2. recruited a pool of participants for a longitudinal follow-up study of the players’ community and individual practices; and 3. started in-depth fieldwork research on the manufacturers and service providers of emotional commodities. EMTECH met with famous romance app makers (Voltage, Cybird, Idea Factory) to conduct interviews with scenario writers, producers and sales representatives (on games’ history, business model, settings, characters, plotlines, design, and genre). Since April 2019, EMTECH has been collecting data on ‘otome’ games’ narrative structures and mechanisms and started game-based surveys, while transcribing, processing and analyzing the aforementioned interviews made in Japan, which led to the publication of several articles in rank A peer-reviewed journals and to the presentation of research at international conferences, seminars and public congresses. EMTECH also shared research insights with the wider public via newspapers (New York Times, Die Zeit or Tagesspiegel) radio and TV.
(C) EMTECH conducted long-term ethnographic studies of families living with companion robots. Despite the challenging conditions during the pandemic in Japan a Pepper robot, the conversation robot Romi and the Gatebox hologram Azuma Hikari were lent to families for several weeks each. Semi-structured interviews were conducted on a weekly basis to determine how the machines were perceived in the households. Analysis of the data generated during these ethnographic studies allowed EMTECH to gain valuable insights regarding HRI bonding processes. EMTECH identified different strategies to cope with the idiosyncrasies/limitations of the robot, ranging from a loss of acceptance to increasing humanization of the machine.
EMTECH explored what we can learn about human emotions and the capacity of humans to create affective bonds through new technologies. The team showed that new technologies expand people's capacity to form new types of affective bonds, beyond traditional social micro- and macrostructures. This is also linked to the question as to what degree new technologies have the potential for helping people escape from their loneliness or emotional distress. One major research question was whether bonding with a machine resulted in people dissociating from other humans, or if, on the contrary, those emotional technologies acted as a bridge towards new attachments with other people, since the particular individual might learn to open up again and be emotionally vulnerable thanks to previously having established a positive bond with a machine. So essentially, EMTECH focus has not been on machines, but always on humans, their desires, their hopes and fears, and how and whether new technologies can help us to better understand our identity and emotional world.
The companies that build emotional machines have promised that their machines can satisfy the human need for emotional closeness. As the field research has shown, the machines available on present-day cannot fulfill the promises that machines can better understand human emotions and replace them as partners, which is also a consequence of their undeveloped language skills. The main model for HRI in cultral narratives is “the machine as a partner substitute”, whereby at the end of this process the machine would ironically hardly differ from humans.
(B) EMTECH further conducted an extensive anthropological study of Japanese romance gaming culture (digital boyfriends, holographic spouses, AR/VR partners) and of the spread of affective devices in Japan (matrimonial intercession robots, match-making software). During fieldwork, EMTECH 1. established contact with romance device users and became familiar with their habits; 2. recruited a pool of participants for a longitudinal follow-up study of the players’ community and individual practices; and 3. started in-depth fieldwork research on the manufacturers and service providers of emotional commodities. EMTECH met with famous romance app makers (Voltage, Cybird, Idea Factory) to conduct interviews with scenario writers, producers and sales representatives (on games’ history, business model, settings, characters, plotlines, design, and genre). Since April 2019, EMTECH has been collecting data on ‘otome’ games’ narrative structures and mechanisms and started game-based surveys, while transcribing, processing and analyzing the aforementioned interviews made in Japan, which led to the publication of several articles in rank A peer-reviewed journals and to the presentation of research at international conferences, seminars and public congresses. EMTECH also shared research insights with the wider public via newspapers (New York Times, Die Zeit or Tagesspiegel) radio and TV.
(C) EMTECH conducted long-term ethnographic studies of families living with companion robots. Despite the challenging conditions during the pandemic in Japan a Pepper robot, the conversation robot Romi and the Gatebox hologram Azuma Hikari were lent to families for several weeks each. Semi-structured interviews were conducted on a weekly basis to determine how the machines were perceived in the households. Analysis of the data generated during these ethnographic studies allowed EMTECH to gain valuable insights regarding HRI bonding processes. EMTECH identified different strategies to cope with the idiosyncrasies/limitations of the robot, ranging from a loss of acceptance to increasing humanization of the machine.
EMTECH explored what we can learn about human emotions and the capacity of humans to create affective bonds through new technologies. The team showed that new technologies expand people's capacity to form new types of affective bonds, beyond traditional social micro- and macrostructures. This is also linked to the question as to what degree new technologies have the potential for helping people escape from their loneliness or emotional distress. One major research question was whether bonding with a machine resulted in people dissociating from other humans, or if, on the contrary, those emotional technologies acted as a bridge towards new attachments with other people, since the particular individual might learn to open up again and be emotionally vulnerable thanks to previously having established a positive bond with a machine. So essentially, EMTECH focus has not been on machines, but always on humans, their desires, their hopes and fears, and how and whether new technologies can help us to better understand our identity and emotional world.
The companies that build emotional machines have promised that their machines can satisfy the human need for emotional closeness. As the field research has shown, the machines available on present-day cannot fulfill the promises that machines can better understand human emotions and replace them as partners, which is also a consequence of their undeveloped language skills. The main model for HRI in cultral narratives is “the machine as a partner substitute”, whereby at the end of this process the machine would ironically hardly differ from humans.
In terms of gender, in both fiction and the actual manufacturing of machines, the opportunity is missed to overcome gender clichés and invent an existence beyond male and female stereotypes or, in fact, beyond gender at all. On the contrary, the narratives and production reveal gender clichés that were thought to be outdated, reproducing and reinforcing them. This became apparent not only in the “feminine” depiction of the machine-girls in anime, presented in a sexualized way, but also from the case study of Azuma Hikari from Gatebox. Azuma corresponds to stereotypical male fantasies of a young, cute and submissive “pixie dream girl” who refers to her male owner as “master”.
Overall, manufacturers are constantly trying to optimize machines so that they can meet human needs. However, towards the end of the project, EMTECH identified yet another trend that goes in the opposite direction, namely the so-called “weak robots” (yowai robotto) by Okada Michio. These are robots that have been intentionally designed with cognitive, motor and physical weaknesses. Weak robots seem to trigger a particularly strong emotional bond with humans. The PI of EMTECH is planning to conduct further research on the development of connections between humans and machines using the concept of “weakness”.
Overall, manufacturers are constantly trying to optimize machines so that they can meet human needs. However, towards the end of the project, EMTECH identified yet another trend that goes in the opposite direction, namely the so-called “weak robots” (yowai robotto) by Okada Michio. These are robots that have been intentionally designed with cognitive, motor and physical weaknesses. Weak robots seem to trigger a particularly strong emotional bond with humans. The PI of EMTECH is planning to conduct further research on the development of connections between humans and machines using the concept of “weakness”.