HD video, sound, 22:56 min
Introduced by Felice Moramarco
Year: 2019
Emilia Tapprest’s speculative fiction film Sonzai Zone portrays Yún and Souvd’s encounter in a near-future where social interactions are largely based on the affective experience of human presence, known as Sonzai-kan. Shifting between XR games, immersion arcades and spatial home displays, their insidiously orchestrated relationship escalates into extreme idealisation. Meanwhile, Souvd’s ex-girlfriend Ntzumi launches into an undercover investigation.
Felice Moramarco: The title of your film refers to the intangible and deterritorialised spatial dimension in which the narrative develops—can you tell us what Sonzai Zone is?
Emilia Tapprest: Sonzai Zone is the spatiotemporal dimension of perceiving another person’s presence as an affective atmosphere around you, even if that person is physically somewhere else. The name comes from the concept of Sonzai-kan, which I came across during my exchange programme in Japan in 2015. In Japanese culture, Sonzai-kan refers to the affective experience of human presence, which inspired my research on nonverbal communication systems. Around the same time, I discovered physiological sensors and calm interfaces which could be used to mediate a sense of presence and nonverbal togetherness over distance. Interestingly, I also found out that researchers in the field of Human Computer Interaction (HCI) had worked in this direction since the late 1990s, although the knowledge derived from these experiments was never fully developed and the prototype interfaces remained in the research labs. The film builds on this alternative media paradigm as a new socio-cultural norm: What if our traditional communication channels could transmit subtle, nonverbal cues about our real-time being? What if interpersonal communication happens not only through individual devices, but also through expanded affective fields, i.e., Sonzai Zones?
FM: This idea of affective atmospheres seems to be central to your practice and I think that it also gives a sense of the mood that pervades your films—can you elaborate a bit more on it?
ET: With this expression I refer to embodied affective states which constitute that pre-individual level from which the subject emerges as an individual entity. Similarly to affects, an atmosphere is something that sets a certain undertone to a space, situation or person, even if it is difficult to pin down. This could be partly explained with the idea that atmospheres are half-entities: they are prompted by actual properties of the world, while also being determined by the pre-dispositions of different subjects. In this way, affective atmospheres provide an interesting link between the material fabric of a world and its affective implications, as well as allowing us to explore how ideological underpinnings behind different power structures reflect on affective experiences of individuals.
FM: The central engine that activates Sonzai Zone’s narrative is a speculative communication system called Sensum. This technology is so convincingly represented in the film that I imagine it has been the result of consistent research—can you outline how you envisioned its functioning?
ET: Sensum is a speculative communication platform that recreates a sense of co-presence and psychological closeness through different types of spatial interfaces. While current social media rely on figurative contents on screens, which require focused attention, Sensum works by mediating users’ multiple types of bodily data, which generate presence information experienced as expanded peripheral activity patterns stretched over time. In more practical terms, users’ ambient representations are built from different sensors’ inputs, such as movement in a particular room, variations of a brainwave frequency, and intentional touching of a surface as an affective gesture. This information is then transduced to multimodal feedback through ambient light, sound, kinetic movement, or tactile stimulation.
Before developing the idea of Sensum as it is represented in the film, I built a working prototype that I tested on my family for a few days. Surprisingly, I realised that even a simple system based on a single input/output (real-time pulse manifesting in the dimming of a light display) can create a very visceral experience of someone else’s presence in space. Yet studies in HCI suggest that when more channels are added to such a system, the presence of the other becomes even more salient. Furthermore, over time, family, friends, and strangers using such interfaces can learn how to intuitively interpret each other’s mediated activity in an increasingly nuanced way and develop their own communication habits.
FM: And the whole relationship between the two main characters of the film, Souvd and Yún, develops through this platform, how is it articulated?
ET: The film portrays an encounter between two characters, Souvd and Yún, who live on different continents. In the current media landscape, they probably wouldn’t have stumbled into each other because they don’t share similar interests or common connections. Yet, on one of Sensum’s arenas Proxy Bay, an XR-mediated stream of presence proxies, they are drawn into each other’s affective atmospheres. They have a profound meeting through their local immersion arcade and decide to establish a connection between their homes. From then on, their long-distance relationship resembles a bit like living together without visually seeing each other: sometimes the person is there, sometimes not. Their moods shift and sometimes their behaviours do not seem to have a clear explanation.
FM: The mediation of affects and the creation of affective atmospheres entails a prelinguistic dimension where agents are no longer individuals but pre- or post-individuals—an “anthropological shift” personified by the characters of your film, who are “converted into background noise”, as one of them states.
ET: A way to describe this shift is to think of names of agents as verbs instead of as nouns. What if your name does not point to you as a defined entity, but as a way of existing? During the film’s research phase, I came across Vilem Flusser’s essay Crisis of Linearity (1988), which resonated a lot with this approach. Flusser writes that while written language brought about linear process-oriented thinking, software challenges our subjective space-time experience, resulting in a rhizomatic state in which “we no longer imagine that we contain some solid kernel (some kind of ‘identity’, an ‘I’, a ‘spirit’ or a ‘soul'), but rather that we are immersed in a collective psychic field, from which we emerge like temporary bubbles, acquire some information, process, share, to submerge again.”
Reflecting this shift, the characters in the film’s storyworld are socially aware that while using Sensum they are not looking at their friends’ latest updates or content archive but are actually experiencing peripheral cues of their states of being. At the same time, the concept of social presence is not just about sensing the existence of the other, but it also pertains to a sense of mutual awareness and significance of a social encounter. It is like making eye contact or lightly touching hands.
FM: The control of affects has always been one of the main concerns of information technologies and the economies developed around them, as Gilles Deleuze already argued in 1992 in his well-known article “Postscript on Societies of Control.” This objective is bluntly exposed also by Sensum’s executive, when he says that “the system wants to make you disclose.” Have you thought about what could be the political implications of a technology that has so efficiently developed a system to control people’s affects?
ET: Yes, the film places these communication technologies in a surveillance capitalist paradigm which controls affects to generate profit. Although the scenario represented in the film is completely fictional, it reflects on an even higher level the increasingly pervasiveness of current communication platforms to collect behavioural data. Yet the executive describes the same logic with which we are confronted in our actual communication platforms, where privacy is in inverse proportion to the quality of tailored experience that platform can offer. In Sonzai Zone, users decide how extensively they are willing to mediate their embodied being. The assumption is that the closer you want to feel to someone, the more information you are willing to share through the interface, thus allowing the platform to collect your data. Furthermore, Sensum does not only mediate sensory information, it also actively learns about the user’s reactions to different real time stimuli and makes adjustments accordingly. So the medium emerges as a series of curated experiences drawing from a vastness of subtle information that is processed via algorithmic black box, presented in a calm and non-invasive fashion, made to be absorbed subconsciously. In this way, ambient media intensifies our already existing dependence on AI to filter and find information for us, while its functioning becomes ever more obscure. However, ambient communication technologies could be just as well run in a decentralized and cooperative way, where users control their own information.
FM: Another aspect outlined by the mysterious executive is that this technology does not simply control affects but replaces them with “simulating activity patterns.” This statement offers a very interesting key of interpretation to understand the narrative of the film—how does this element of simulation affects characters’ relationships and lived experience?
ET: Yún and Souvd meet through Sensum not by coincidence, but because of the likelihood of their affective atmospheres to match. The more they spend time together and interact through the platform, the better their presence proxies—made from a combination of physiological activity patterns—can be enhanced to have a compelling effect on each other. For example, strong emotional peaks might be evened out for someone who cannot handle volatile behaviour, or the system could introduce some simulated micro behaviours in order to intensify a specific feeling. As a result, Sonzai media may amplify idealization practices that are very common in long-distance relationships and the fact that Yún and Souvd have never met in the actual world makes them even more prone for projecting idealized or imaginary attributes on each other.
FM: A very interesting element emerging from Sonzai Zone is the radical cosmopolitanism made possible by affective technologies, as they are able to overcome both geographical and linguistic barriers—do you think there is a margin to speculate on possible modes of coexistence enabled by these technologies beyond the control of capitalist exploitation?
ET: In the process of making Sonzai Zone, I indeed realised that in another political and economic environment the same technological principle might lead to an entirely different outcome. This is precisely what motivates my ongoing project, Zhōuwéi Network, in which I want to zoom out and focus not on a specific technology per se but on different socio-political and economic paradigms. The project takes place in the same narrative universe, but five years later, in 2041, and juxtaposes three different society models: Dolphin Waves, a corporate playbour & wellness hub in New York City; Dragonfly, a technocratic safety state in the Netherlands; and Project Gecko, a network of decentralised liquid democracies scattered in different locations around the world.
Also—and this has been an important input from Victor Evink, with whom I am currently collaborating—all three societies are ambitopian: they are beyond the dystopia/utopia binary. Here Sonzai media are very much present but in a more implicit way. The affective technology and ubiquitous sensing that Sensum uses to manipulate and exploit people’s intimate relationships is repurposed for other means: for example, in Project Gecko, it’s used as a basis for cryptography and digital peer-to-peer interactions. Or in Dragonfly, the zone of presence is something which AI mentors employ to coach people to form healthy communal relationships in the physical world, with the ultimate purpose of engineering societal harmony and stability.
Credits
Emilia Tapprest
Sonzai Zone (2019)
22:56 min, QHD
Souvd: Dasha Golova
Yún: Taro Yamada
Nzumi: Annamaria Merkel
Narration: Tzuyu Shen, Dasha Golova
Costumes: Dasha Golova, Walter Götsch
Conceptual co-development: Victor Evink
Production: NVISIBLE.STUDIO
Soundtracks: David Stevens, ssaliva, Olivier Demeaux, Alex Ortiga, SHXCXCHCXSH, Leevisa
Special thanks: Rob Schröder, Daniel van der Velden, Shadow Channel
Introduced by Felice Moramarco
HD video, sound, 22:56 min
Year: 2019
Emilia Tapprest’s speculative fiction film Sonzai Zone portrays Yún and Souvd’s encounter in a near-future where social interactions are largely based on the affective experience of human presence, known as Sonzai-kan. Shifting between XR games, immersion arcades and spatial home displays, their insidiously orchestrated relationship escalates into extreme idealisation. Meanwhile, Souvd’s ex-girlfriend Ntzumi launches into an undercover investigation.
Felice Moramarco: The title of your film refers to the intangible and deterritorialised spatial dimension in which the narrative develops—can you tell us what Sonzai Zone is?
Emilia Tapprest: Sonzai Zone is the spatiotemporal dimension of perceiving another person’s presence as an affective atmosphere around you, even if that person is physically somewhere else. The name comes from the concept of Sonzai-kan, which I came across during my exchange programme in Japan in 2015. In Japanese culture, Sonzai-kan refers to the affective experience of human presence, which inspired my research on nonverbal communication systems. Around the same time, I discovered physiological sensors and calm interfaces which could be used to mediate a sense of presence and nonverbal togetherness over distance. Interestingly, I also found out that researchers in the field of Human Computer Interaction (HCI) had worked in this direction since the late 1990s, although the knowledge derived from these experiments was never fully developed and the prototype interfaces remained in the research labs. The film builds on this alternative media paradigm as a new socio-cultural norm: What if our traditional communication channels could transmit subtle, nonverbal cues about our real-time being? What if interpersonal communication happens not only through individual devices, but also through expanded affective fields, i.e., Sonzai Zones?
FM: This idea of affective atmospheres seems to be central to your practice and I think that it also gives a sense of the mood that pervades your films—can you elaborate a bit more on it?
ET: With this expression I refer to embodied affective states which constitute that pre-individual level from which the subject emerges as an individual entity. Similarly to affects, an atmosphere is something that sets a certain undertone to a space, situation or person, even if it is difficult to pin down. This could be partly explained with the idea that atmospheres are half-entities: they are prompted by actual properties of the world, while also being determined by the pre-dispositions of different subjects. In this way, affective atmospheres provide an interesting link between the material fabric of a world and its affective implications, as well as allowing us to explore how ideological underpinnings behind different power structures reflect on affective experiences of individuals.
FM: The central engine that activates Sonzai Zone’s narrative is a speculative communication system called Sensum. This technology is so convincingly represented in the film that I imagine it has been the result of consistent research—can you outline how you envisioned its functioning?
ET: Sensum is a speculative communication platform that recreates a sense of co-presence and psychological closeness through different types of spatial interfaces. While current social media rely on figurative contents on screens, which require focused attention, Sensum works by mediating users’ multiple types of bodily data, which generate presence information experienced as expanded peripheral activity patterns stretched over time. In more practical terms, users’ ambient representations are built from different sensors’ inputs, such as movement in a particular room, variations of a brainwave frequency, and intentional touching of a surface as an affective gesture. This information is then transduced to multimodal feedback through ambient light, sound, kinetic movement, or tactile stimulation.
Before developing the idea of Sensum as it is represented in the film, I built a working prototype that I tested on my family for a few days. Surprisingly, I realised that even a simple system based on a single input/output (real-time pulse manifesting in the dimming of a light display) can create a very visceral experience of someone else’s presence in space. Yet studies in HCI suggest that when more channels are added to such a system, the presence of the other becomes even more salient. Furthermore, over time, family, friends, and strangers using such interfaces can learn how to intuitively interpret each other’s mediated activity in an increasingly nuanced way and develop their own communication habits.
FM: And the whole relationship between the two main characters of the film, Souvd and Yún, develops through this platform, how is it articulated?
ET: The film portrays an encounter between two characters, Souvd and Yún, who live on different continents. In the current media landscape, they probably wouldn’t have stumbled into each other because they don’t share similar interests or common connections. Yet, on one of Sensum’s arenas Proxy Bay, an XR-mediated stream of presence proxies, they are drawn into each other’s affective atmospheres. They have a profound meeting through their local immersion arcade and decide to establish a connection between their homes. From then on, their long-distance relationship resembles a bit like living together without visually seeing each other: sometimes the person is there, sometimes not. Their moods shift and sometimes their behaviours do not seem to have a clear explanation.
FM: The mediation of affects and the creation of affective atmospheres entails a prelinguistic dimension where agents are no longer individuals but pre- or post-individuals—an “anthropological shift” personified by the characters of your film, who are “converted into background noise”, as one of them states.
ET: A way to describe this shift is to think of names of agents as verbs instead of as nouns. What if your name does not point to you as a defined entity, but as a way of existing? During the film’s research phase, I came across Vilem Flusser’s essay Crisis of Linearity (1988), which resonated a lot with this approach. Flusser writes that while written language brought about linear process-oriented thinking, software challenges our subjective space-time experience, resulting in a rhizomatic state in which “we no longer imagine that we contain some solid kernel (some kind of ‘identity’, an ‘I’, a ‘spirit’ or a ‘soul'), but rather that we are immersed in a collective psychic field, from which we emerge like temporary bubbles, acquire some information, process, share, to submerge again.”
Reflecting this shift, the characters in the film’s storyworld are socially aware that while using Sensum they are not looking at their friends’ latest updates or content archive but are actually experiencing peripheral cues of their states of being. At the same time, the concept of social presence is not just about sensing the existence of the other, but it also pertains to a sense of mutual awareness and significance of a social encounter. It is like making eye contact or lightly touching hands.
FM: The control of affects has always been one of the main concerns of information technologies and the economies developed around them, as Gilles Deleuze already argued in 1992 in his well-known article “Postscript on Societies of Control.” This objective is bluntly exposed also by Sensum’s executive, when he says that “the system wants to make you disclose.” Have you thought about what could be the political implications of a technology that has so efficiently developed a system to control people’s affects?
ET: Yes, the film places these communication technologies in a surveillance capitalist paradigm which controls affects to generate profit. Although the scenario represented in the film is completely fictional, it reflects on an even higher level the increasingly pervasiveness of current communication platforms to collect behavioural data. Yet the executive describes the same logic with which we are confronted in our actual communication platforms, where privacy is in inverse proportion to the quality of tailored experience that platform can offer. In Sonzai Zone, users decide how extensively they are willing to mediate their embodied being. The assumption is that the closer you want to feel to someone, the more information you are willing to share through the interface, thus allowing the platform to collect your data. Furthermore, Sensum does not only mediate sensory information, it also actively learns about the user’s reactions to different real time stimuli and makes adjustments accordingly. So the medium emerges as a series of curated experiences drawing from a vastness of subtle information that is processed via algorithmic black box, presented in a calm and non-invasive fashion, made to be absorbed subconsciously. In this way, ambient media intensifies our already existing dependence on AI to filter and find information for us, while its functioning becomes ever more obscure. However, ambient communication technologies could be just as well run in a decentralized and cooperative way, where users control their own information.
FM: Another aspect outlined by the mysterious executive is that this technology does not simply control affects but replaces them with “simulating activity patterns.” This statement offers a very interesting key of interpretation to understand the narrative of the film—how does this element of simulation affects characters’ relationships and lived experience?
ET: Yún and Souvd meet through Sensum not by coincidence, but because of the likelihood of their affective atmospheres to match. The more they spend time together and interact through the platform, the better their presence proxies—made from a combination of physiological activity patterns—can be enhanced to have a compelling effect on each other. For example, strong emotional peaks might be evened out for someone who cannot handle volatile behaviour, or the system could introduce some simulated micro behaviours in order to intensify a specific feeling. As a result, Sonzai media may amplify idealization practices that are very common in long-distance relationships and the fact that Yún and Souvd have never met in the actual world makes them even more prone for projecting idealized or imaginary attributes on each other.
FM: A very interesting element emerging from Sonzai Zone is the radical cosmopolitanism made possible by affective technologies, as they are able to overcome both geographical and linguistic barriers—do you think there is a margin to speculate on possible modes of coexistence enabled by these technologies beyond the control of capitalist exploitation?
ET: In the process of making Sonzai Zone, I indeed realised that in another political and economic environment the same technological principle might lead to an entirely different outcome. This is precisely what motivates my ongoing project, Zhōuwéi Network, in which I want to zoom out and focus not on a specific technology per se but on different socio-political and economic paradigms. The project takes place in the same narrative universe, but five years later, in 2041, and juxtaposes three different society models: Dolphin Waves, a corporate playbour & wellness hub in New York City; Dragonfly, a technocratic safety state in the Netherlands; and Project Gecko, a network of decentralised liquid democracies scattered in different locations around the world.
Also—and this has been an important input from Victor Evink, with whom I am currently collaborating—all three societies are ambitopian: they are beyond the dystopia/utopia binary. Here Sonzai media are very much present but in a more implicit way. The affective technology and ubiquitous sensing that Sensum uses to manipulate and exploit people’s intimate relationships is repurposed for other means: for example, in Project Gecko, it’s used as a basis for cryptography and digital peer-to-peer interactions. Or in Dragonfly, the zone of presence is something which AI mentors employ to coach people to form healthy communal relationships in the physical world, with the ultimate purpose of engineering societal harmony and stability.
Credits
Emilia Tapprest
Sonzai Zone (2019)
22:56 min, QHD
Souvd: Dasha Golova
Yún: Taro Yamada
Nzumi: Annamaria Merkel
Narration: Tzuyu Shen, Dasha Golova
Costumes: Dasha Golova, Walter Götsch
Conceptual co-development: Victor Evink
Production: NVISIBLE.STUDIO
Soundtracks: David Stevens, ssaliva, Olivier Demeaux, Alex Ortiga, SHXCXCHCXSH, Leevisa
Special thanks: Rob Schröder, Daniel van der Velden, Shadow Channel